fix(agent): a MoA preset in the fallback chain activates the preset, not the aggregator alone

`try_activate_fallback` and the init-time fallback walk in `agent_init._routed_client_kwargs`
handed `provider: moa` entries to `resolve_provider_client`, whose MoA chokepoint unwraps the
preset to the aggregator's real HTTP client. Both then installed that client under the virtual
identity: the runtime path kept `model=<preset>`/`provider=moa` on an `api.x.ai` client (the
preset name went out as the model id → 404, #112525) and every `provider == "moa"` guard and
key — stale timeout, cache_ttl, pricing, credential pool, and the client-rebuild branches in
`_replace_primary_openai_client` / `_create_openai_client` — saw "moa" for a native client, so
the next credential rotation or dead-connection cleanup swapped the MoA facade in anyway
(#112623). The init-time path had the mirror image: `model=<aggregator slug>`, the aggregator's
base_url/api_key in `_client_kwargs`, facade client.

WHY bind the facade instead of rewriting the identity to the aggregator: `provider: moa,
model: <preset>` means "run this preset" at every other entry point (config, `/model
<preset> --provider moa`, picker, gateway `/moa`), and the chokepoint unwrap exists for aux
tasks that skip the fan-out by design. A fallback entry is the acting model, so it gets the same
semantics and the same four pins as `agent_init` / `switch_model` / `restore_primary_runtime`
(`api_mode=chat_completions`, `base_url=moa://local`, empty client kwargs, facade client).
With identity and client consistent, the existing `provider == "moa"` guards are correct as
written and no re-keying on client identity is needed. The chokepoint call stays as the
preflight: an unresolvable preset or an aggregator without credentials still skips the entry
with the "provider not configured" warning.

`moa_loop.bind_moa_runtime` is the single binder used by init, switch and both fallback paths
so the pins cannot drift again.

Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
Co-authored-by: Zheqing Zeng <26277217+zengzheqing@users.noreply.github.com>
Co-authored-by: fangliquan <fangliquan@qq.com>
This commit is contained in:
teknium1
2026-09-16 10:10:43 -07:00
committed by Teknium
parent d6d6565d90
commit c810640a8a
7 changed files with 132 additions and 31 deletions

View File

@@ -759,9 +759,7 @@ def _init_anthropic_client(agent, api_key, base_url, _provider_timeout):
def _init_moa_client(agent, api_key):
"""provider == "moa": virtual Mixture-of-Agents facade, no real HTTP client."""
from agent.moa_loop import build_moa_facade
agent.api_mode = "chat_completions"
from agent.moa_loop import bind_moa_runtime
# build_moa_facade relays "moa.*" events through tool_progress_callback so every surface
# shows each reference's answer before the aggregator acts. Display-only; shared with
# fallback-restore so a restored facade keeps emitting.
@@ -771,10 +769,7 @@ def _init_moa_client(agent, api_key):
# facade emits "moa.reference", "moa.progress", "moa.phase", and "moa.aggregating" events, forwarded
# through the same callback the tool lifecycle uses. Best-effort and cache-safe — display-only events,
# they never touch the message history. See #53802.
agent.client = build_moa_facade(agent, agent.model)
agent._client_kwargs = {}
agent.api_key = api_key or "moa-virtual-provider"
agent.base_url = "moa://local"
bind_moa_runtime(agent, agent.model, api_key)
if not agent.quiet_mode:
print(f"🤖 AI Agent initialized with MoA preset: {agent.model}")
@@ -822,11 +817,12 @@ def _explicit_client_kwargs(agent, api_key, base_url, _provider_timeout) -> Dict
return client_kwargs
def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Dict[str, Any]:
def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Optional[Dict[str, Any]]:
"""OpenAI-client kwargs via the centralized provider router (no explicit creds).
Falls through to the init-time fallback chain, then raises with the missing-key /
no-provider diagnostic.
no-provider diagnostic. ``None`` when the chain landed on a MoA preset: the facade is
already bound and there is no OpenAI client to construct.
"""
from agent.auxiliary_client import resolve_provider_client
_routed_client, _ = resolve_provider_client(
@@ -856,9 +852,16 @@ def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Dict[str,
logger.debug("Init-time fallback entry %s failed: %s", _fb.get("provider"), _fb_exc)
continue
if _fb_client is not None:
agent._fallback_activated = True
if str(_fb["provider"]).strip().lower() == "moa":
# The chokepoint handed back the preset's aggregator client, which only proves the
# preset resolves and its aggregator has credentials. A MoA entry means the preset
# itself (same as ``provider: moa`` in config), so bind the facade, not the aggregator.
from agent.moa_loop import bind_moa_runtime
bind_moa_runtime(agent, _fb["model"])
return None
agent.provider = _fb["provider"]
agent.model = _fb_model or _fb["model"]
agent._fallback_activated = True
return _client_kwargs_from_routed(_fb_client, _provider_timeout)
if _explicit and _explicit not in {"auto", "openrouter", "custom"}:
# Explicit non-OpenRouter provider with no creds and no usable fallback: fail fast.
@@ -921,6 +924,10 @@ def _init_openai_client(agent, api_key, base_url, fallback_model, _provider_time
client_kwargs = _explicit_client_kwargs(agent, api_key, base_url, _provider_timeout)
else:
client_kwargs = _routed_client_kwargs(agent, fallback_model, _provider_timeout)
if client_kwargs is None: # init-time fallback bound the MoA facade
if not agent.quiet_mode:
print(f"🤖 AI Agent initialized with MoA preset: {agent.model}")
return
from hermes_cli.providers import is_actual_route
if is_actual_route(agent.provider, client_kwargs.get("base_url", "")):
agent.api_mode = "chat_completions"

View File

@@ -1892,15 +1892,11 @@ def _resolve_switch_destination(agent, new_model, new_provider, base_url, api_mo
def _build_switched_client(agent, new_provider, api_key, base_url, api_mode, new_norm) -> None:
"""Build the client for the switched-to destination (MoA facade / native Anthropic / OpenAI wire)."""
if new_norm == "moa":
from agent.moa_loop import build_moa_facade
from agent.moa_loop import bind_moa_runtime
# MoA speaks only chat.completions via the MoAClient facade; the aggregator's real transport
# is applied inside the fan-out. Pin api_mode so the loop never dispatches
# client.responses.create against the facade (matches agent_init.py).
agent.api_mode = "chat_completions"
agent.api_key = api_key or "moa-virtual-provider"
agent.base_url = "moa://local"
agent._client_kwargs = {}
agent.client = build_moa_facade(agent, agent.model)
# is applied inside the fan-out. The binder pins api_mode so the loop never dispatches
# client.responses.create against the facade (same pins as agent_init / fallback).
bind_moa_runtime(agent, agent.model, api_key)
return
if new_provider == "bedrock" and api_mode in ("anthropic_messages", "bedrock_converse"):
# Non-Mantle Bedrock wires authenticate through boto3, never through the generic

View File

@@ -1912,18 +1912,28 @@ def try_activate_fallback(agent, reason: "FailoverReason | None" = None) -> bool
logger.warning("Fallback to %s failed: provider not configured", fb_provider)
unavailable.add(fb_key)
continue
try:
from hermes_cli.model_normalize import normalize_model_for_provider
fb_model = normalize_model_for_provider(fb_model, fb_provider)
except Exception as _norm_err:
logger.warning("Could not normalize fallback model %r for provider %r: %s", fb_model, fb_provider, _norm_err)
if fb_provider == "moa":
# A MoA entry means the preset itself, exactly like ``provider: moa`` in config or
# ``/model <preset> --provider moa``. The chokepoint's client is the preset's
# aggregator: it only proves the preset resolves and the aggregator has credentials.
# Installing it as the acting client with the virtual identity is a hybrid nobody
# handles (#112525: preset name sent as model id → 404; #112623: every
# ``provider == "moa"`` guard and key misfires and the next rebuild swaps in the
# facade anyway). Bind the facade with the same pins every other MoA build site uses.
fb_base_url, fb_api_mode = "moa://local", "chat_completions"
else:
try:
from hermes_cli.model_normalize import normalize_model_for_provider
fb_model = normalize_model_for_provider(fb_model, fb_provider)
except Exception as _norm_err:
logger.warning("Could not normalize fallback model %r for provider %r: %s", fb_model, fb_provider, _norm_err)
fb_base_url = str(fb_client.base_url)
from hermes_cli.providers import is_actual_route
if is_actual_route(fb_provider, fb_base_url):
fb_api_mode = "chat_completions"
elif not fb_api_mode_explicit and fb_api_mode == "chat_completions":
fb_api_mode = _fallback_api_mode_resolved(agent, fb_provider, fb_model, fb_base_url)
fb_base_url = str(fb_client.base_url)
from hermes_cli.providers import is_actual_route
if is_actual_route(fb_provider, fb_base_url):
fb_api_mode = "chat_completions"
elif not fb_api_mode_explicit and fb_api_mode == "chat_completions":
fb_api_mode = _fallback_api_mode_resolved(agent, fb_provider, fb_model, fb_base_url)
old_model, old_provider, old_base_url = agent.model, agent.provider, agent.base_url
@@ -1940,8 +1950,12 @@ def try_activate_fallback(agent, reason: "FailoverReason | None" = None) -> bool
agent._fallback_activated = True
_rebind_fallback_credential_pool(agent, fb_provider, fb_model)
from agent.client_lifecycle import _swap_fallback_clients
_swap_fallback_clients(agent, fb_client, fb_provider, fb_model, fb_base_url, fb_api_mode)
if fb_provider == "moa":
from agent.moa_loop import bind_moa_runtime
bind_moa_runtime(agent, fb_model)
else:
from agent.client_lifecycle import _swap_fallback_clients
_swap_fallback_clients(agent, fb_client, fb_provider, fb_model, fb_base_url, fb_api_mode)
from agent.agent_runtime_helpers import sync_credential_pool_entry_id
sync_credential_pool_entry_id(agent)

View File

@@ -1392,3 +1392,20 @@ def build_moa_facade(agent, preset_name: Any = None) -> MoAClient:
resolved_preset = "default"
# ``agent`` lets the fan-out wait be aborted on a user interrupt.
return MoAClient(resolved_preset, reference_callback=_moa_reference_relay, agent=agent)
def bind_moa_runtime(agent, preset_name: Any, api_key: Any = None) -> None:
"""Make ``agent`` act as the MoA preset: pin the virtual runtime fields and install the facade.
Every site that puts an agent onto ``provider: moa`` (init, ``/model`` switch, fallback
activation) must pin the same fields — the facade speaks only chat.completions, has no HTTP
endpoint and no OpenAI client kwargs — or the next dispatch/rebuild reaches a real wire with a
virtual identity (``moa://local`` 404, or the preset name sent as a model id).
"""
agent.model = str(preset_name or "default")
agent.provider = agent.requested_provider = "moa"
agent.api_mode = "chat_completions"
agent.api_key = api_key or "moa-virtual-provider"
agent.base_url = "moa://local"
agent._client_kwargs = {}
agent.client = build_moa_facade(agent, agent.model)

View File

@@ -510,3 +510,68 @@ class TestFallbackExtraBodyReResolution:
agent.request_overrides["temperature"] = 0.2
self._activate(agent)
assert agent.request_overrides.get("temperature") == 0.2
# ── MoA preset as a fallback entry (#112525, #112623) ─────────────────────
def _write_moa_home(tmp_path, monkeypatch):
"""Real config.yaml with a MoA preset under a temp HERMES_HOME (genuine preset resolution)."""
import yaml
home = tmp_path / ".hermes"
home.mkdir(exist_ok=True)
(home / "config.yaml").write_text(yaml.safe_dump({
"moa": {"default_preset": "default", "presets": {"default": {
"enabled": True,
"reference_models": [{"provider": "xai", "model": "grok-4-fast"}],
"aggregator": {"provider": "xai", "model": "grok-4.6"},
}}},
}))
monkeypatch.setenv("HERMES_HOME", str(home))
return home
def _assert_bound_to_moa_preset(agent, preset="default"):
from agent.conversation_loop import _moa_client_consumes_prepared_request
assert (agent.provider, agent.requested_provider, agent.model) == ("moa", "moa", preset)
assert (agent.base_url, agent.api_mode) == ("moa://local", "chat_completions")
assert agent._client_kwargs == {}
assert _moa_client_consumes_prepared_request(agent.client)
class TestMoaPresetFallback:
def test_runtime_fallback_to_moa_preset_binds_the_facade(self, tmp_path, monkeypatch):
"""#112525 / #112623: a ``{provider: moa, model: <preset>}`` fallback entry activates the
preset (facade, ``moa://local``), never the aggregator's HTTP client wearing the virtual
identity (preset name on the aggregator wire → 404; ``provider == "moa"`` guards misfire)."""
_write_moa_home(tmp_path, monkeypatch)
agent = _make_agent(fallback_model={"provider": "moa", "model": "default"})
aggregator_client = _mock_client(base_url="https://api.x.ai/v1/", api_key="xai-key")
with patch("agent.auxiliary_client.resolve_provider_client",
return_value=(aggregator_client, "grok-4.6")):
assert agent._try_activate_fallback() is True
_assert_bound_to_moa_preset(agent)
assert agent.client is not aggregator_client
assert agent._provider_fallback_route == ("default", "moa")
def test_init_time_fallback_to_moa_preset_binds_the_facade(self, tmp_path, monkeypatch):
"""Primary without credentials at init walks the chain: a MoA entry lands on the preset
with virtual pins, not on the aggregator slug with the aggregator's kwargs."""
_write_moa_home(tmp_path, monkeypatch)
aggregator_client = _mock_client(base_url="https://api.x.ai/v1/", api_key="xai-key")
def _route(provider, model=None, **_kw):
return (aggregator_client, "grok-4.6") if provider == "moa" else (None, None)
with (
patch("model_tools.get_tool_definitions", return_value=[]),
patch("model_tools.check_toolset_requirements", return_value={}),
patch("agent.auxiliary_client.resolve_provider_client", side_effect=_route),
):
agent = AIAgent(model="anthropic/claude-sonnet-4.5", provider="openrouter",
quiet_mode=True, skip_context_files=True, skip_memory=True,
fallback_model={"provider": "moa", "model": "default"})
_assert_bound_to_moa_preset(agent)
assert agent._fallback_activated is True

View File

@@ -95,6 +95,7 @@ OpenAI-compatible base URL continues to use the compatible client instead.
| LM Studio (local) | `lmstudio` | `LM_API_KEY` (or none for local) + `LM_BASE_URL` |
| Hugging Face | `huggingface` | `HF_TOKEN` |
| Custom endpoint | `custom` | `base_url` + `key_env` (see below) |
| Mixture of Agents preset | `moa` (`model` = preset name) | A configured MoA preset whose aggregator has credentials — the fallback runs the whole preset (references + aggregator), not the aggregator alone |
### Custom Endpoint Fallback

View File

@@ -244,3 +244,4 @@ So MoA does not sacrifice prompt caching on either call type. Its only real cost
- A preset's aggregator cannot be another MoA preset. Recursive MoA trees are intentionally blocked.
- Credential failures on one reference model do not abort the turn. Hermes includes the failure in the reference context and continues with whatever models returned.
- MoA increases model-call count. A single model iteration can involve multiple reference calls plus the aggregator call.
- A preset can be a fallback entry (`fallback_providers: [{provider: moa, model: <preset>}]`). When the primary fails, Hermes activates the preset itself — references and aggregator, with `moa://local` as the virtual endpoint — the same way `/model <preset> --provider moa` does. The entry is skipped when the preset does not resolve or its aggregator has no credentials.