fix(agent): a MoA preset in the fallback chain activates the preset, not the aggregator alone
`try_activate_fallback` and the init-time fallback walk in `agent_init._routed_client_kwargs` handed `provider: moa` entries to `resolve_provider_client`, whose MoA chokepoint unwraps the preset to the aggregator's real HTTP client. Both then installed that client under the virtual identity: the runtime path kept `model=<preset>`/`provider=moa` on an `api.x.ai` client (the preset name went out as the model id → 404, #112525) and every `provider == "moa"` guard and key — stale timeout, cache_ttl, pricing, credential pool, and the client-rebuild branches in `_replace_primary_openai_client` / `_create_openai_client` — saw "moa" for a native client, so the next credential rotation or dead-connection cleanup swapped the MoA facade in anyway (#112623). The init-time path had the mirror image: `model=<aggregator slug>`, the aggregator's base_url/api_key in `_client_kwargs`, facade client. WHY bind the facade instead of rewriting the identity to the aggregator: `provider: moa, model: <preset>` means "run this preset" at every other entry point (config, `/model <preset> --provider moa`, picker, gateway `/moa`), and the chokepoint unwrap exists for aux tasks that skip the fan-out by design. A fallback entry is the acting model, so it gets the same semantics and the same four pins as `agent_init` / `switch_model` / `restore_primary_runtime` (`api_mode=chat_completions`, `base_url=moa://local`, empty client kwargs, facade client). With identity and client consistent, the existing `provider == "moa"` guards are correct as written and no re-keying on client identity is needed. The chokepoint call stays as the preflight: an unresolvable preset or an aggregator without credentials still skips the entry with the "provider not configured" warning. `moa_loop.bind_moa_runtime` is the single binder used by init, switch and both fallback paths so the pins cannot drift again. Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com> Co-authored-by: Zheqing Zeng <26277217+zengzheqing@users.noreply.github.com> Co-authored-by: fangliquan <fangliquan@qq.com>
This commit is contained in:
@@ -759,9 +759,7 @@ def _init_anthropic_client(agent, api_key, base_url, _provider_timeout):
|
||||
|
||||
def _init_moa_client(agent, api_key):
|
||||
"""provider == "moa": virtual Mixture-of-Agents facade, no real HTTP client."""
|
||||
from agent.moa_loop import build_moa_facade
|
||||
agent.api_mode = "chat_completions"
|
||||
|
||||
from agent.moa_loop import bind_moa_runtime
|
||||
# build_moa_facade relays "moa.*" events through tool_progress_callback so every surface
|
||||
# shows each reference's answer before the aggregator acts. Display-only; shared with
|
||||
# fallback-restore so a restored facade keeps emitting.
|
||||
@@ -771,10 +769,7 @@ def _init_moa_client(agent, api_key):
|
||||
# facade emits "moa.reference", "moa.progress", "moa.phase", and "moa.aggregating" events, forwarded
|
||||
# through the same callback the tool lifecycle uses. Best-effort and cache-safe — display-only events,
|
||||
# they never touch the message history. See #53802.
|
||||
agent.client = build_moa_facade(agent, agent.model)
|
||||
agent._client_kwargs = {}
|
||||
agent.api_key = api_key or "moa-virtual-provider"
|
||||
agent.base_url = "moa://local"
|
||||
bind_moa_runtime(agent, agent.model, api_key)
|
||||
if not agent.quiet_mode:
|
||||
print(f"🤖 AI Agent initialized with MoA preset: {agent.model}")
|
||||
|
||||
@@ -822,11 +817,12 @@ def _explicit_client_kwargs(agent, api_key, base_url, _provider_timeout) -> Dict
|
||||
return client_kwargs
|
||||
|
||||
|
||||
def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Dict[str, Any]:
|
||||
def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Optional[Dict[str, Any]]:
|
||||
"""OpenAI-client kwargs via the centralized provider router (no explicit creds).
|
||||
|
||||
Falls through to the init-time fallback chain, then raises with the missing-key /
|
||||
no-provider diagnostic.
|
||||
no-provider diagnostic. ``None`` when the chain landed on a MoA preset: the facade is
|
||||
already bound and there is no OpenAI client to construct.
|
||||
"""
|
||||
from agent.auxiliary_client import resolve_provider_client
|
||||
_routed_client, _ = resolve_provider_client(
|
||||
@@ -856,9 +852,16 @@ def _routed_client_kwargs(agent, fallback_model, _provider_timeout) -> Dict[str,
|
||||
logger.debug("Init-time fallback entry %s failed: %s", _fb.get("provider"), _fb_exc)
|
||||
continue
|
||||
if _fb_client is not None:
|
||||
agent._fallback_activated = True
|
||||
if str(_fb["provider"]).strip().lower() == "moa":
|
||||
# The chokepoint handed back the preset's aggregator client, which only proves the
|
||||
# preset resolves and its aggregator has credentials. A MoA entry means the preset
|
||||
# itself (same as ``provider: moa`` in config), so bind the facade, not the aggregator.
|
||||
from agent.moa_loop import bind_moa_runtime
|
||||
bind_moa_runtime(agent, _fb["model"])
|
||||
return None
|
||||
agent.provider = _fb["provider"]
|
||||
agent.model = _fb_model or _fb["model"]
|
||||
agent._fallback_activated = True
|
||||
return _client_kwargs_from_routed(_fb_client, _provider_timeout)
|
||||
if _explicit and _explicit not in {"auto", "openrouter", "custom"}:
|
||||
# Explicit non-OpenRouter provider with no creds and no usable fallback: fail fast.
|
||||
@@ -921,6 +924,10 @@ def _init_openai_client(agent, api_key, base_url, fallback_model, _provider_time
|
||||
client_kwargs = _explicit_client_kwargs(agent, api_key, base_url, _provider_timeout)
|
||||
else:
|
||||
client_kwargs = _routed_client_kwargs(agent, fallback_model, _provider_timeout)
|
||||
if client_kwargs is None: # init-time fallback bound the MoA facade
|
||||
if not agent.quiet_mode:
|
||||
print(f"🤖 AI Agent initialized with MoA preset: {agent.model}")
|
||||
return
|
||||
from hermes_cli.providers import is_actual_route
|
||||
if is_actual_route(agent.provider, client_kwargs.get("base_url", "")):
|
||||
agent.api_mode = "chat_completions"
|
||||
|
||||
@@ -1892,15 +1892,11 @@ def _resolve_switch_destination(agent, new_model, new_provider, base_url, api_mo
|
||||
def _build_switched_client(agent, new_provider, api_key, base_url, api_mode, new_norm) -> None:
|
||||
"""Build the client for the switched-to destination (MoA facade / native Anthropic / OpenAI wire)."""
|
||||
if new_norm == "moa":
|
||||
from agent.moa_loop import build_moa_facade
|
||||
from agent.moa_loop import bind_moa_runtime
|
||||
# MoA speaks only chat.completions via the MoAClient facade; the aggregator's real transport
|
||||
# is applied inside the fan-out. Pin api_mode so the loop never dispatches
|
||||
# client.responses.create against the facade (matches agent_init.py).
|
||||
agent.api_mode = "chat_completions"
|
||||
agent.api_key = api_key or "moa-virtual-provider"
|
||||
agent.base_url = "moa://local"
|
||||
agent._client_kwargs = {}
|
||||
agent.client = build_moa_facade(agent, agent.model)
|
||||
# is applied inside the fan-out. The binder pins api_mode so the loop never dispatches
|
||||
# client.responses.create against the facade (same pins as agent_init / fallback).
|
||||
bind_moa_runtime(agent, agent.model, api_key)
|
||||
return
|
||||
if new_provider == "bedrock" and api_mode in ("anthropic_messages", "bedrock_converse"):
|
||||
# Non-Mantle Bedrock wires authenticate through boto3, never through the generic
|
||||
|
||||
@@ -1912,18 +1912,28 @@ def try_activate_fallback(agent, reason: "FailoverReason | None" = None) -> bool
|
||||
logger.warning("Fallback to %s failed: provider not configured", fb_provider)
|
||||
unavailable.add(fb_key)
|
||||
continue
|
||||
try:
|
||||
from hermes_cli.model_normalize import normalize_model_for_provider
|
||||
fb_model = normalize_model_for_provider(fb_model, fb_provider)
|
||||
except Exception as _norm_err:
|
||||
logger.warning("Could not normalize fallback model %r for provider %r: %s", fb_model, fb_provider, _norm_err)
|
||||
if fb_provider == "moa":
|
||||
# A MoA entry means the preset itself, exactly like ``provider: moa`` in config or
|
||||
# ``/model <preset> --provider moa``. The chokepoint's client is the preset's
|
||||
# aggregator: it only proves the preset resolves and the aggregator has credentials.
|
||||
# Installing it as the acting client with the virtual identity is a hybrid nobody
|
||||
# handles (#112525: preset name sent as model id → 404; #112623: every
|
||||
# ``provider == "moa"`` guard and key misfires and the next rebuild swaps in the
|
||||
# facade anyway). Bind the facade with the same pins every other MoA build site uses.
|
||||
fb_base_url, fb_api_mode = "moa://local", "chat_completions"
|
||||
else:
|
||||
try:
|
||||
from hermes_cli.model_normalize import normalize_model_for_provider
|
||||
fb_model = normalize_model_for_provider(fb_model, fb_provider)
|
||||
except Exception as _norm_err:
|
||||
logger.warning("Could not normalize fallback model %r for provider %r: %s", fb_model, fb_provider, _norm_err)
|
||||
|
||||
fb_base_url = str(fb_client.base_url)
|
||||
from hermes_cli.providers import is_actual_route
|
||||
if is_actual_route(fb_provider, fb_base_url):
|
||||
fb_api_mode = "chat_completions"
|
||||
elif not fb_api_mode_explicit and fb_api_mode == "chat_completions":
|
||||
fb_api_mode = _fallback_api_mode_resolved(agent, fb_provider, fb_model, fb_base_url)
|
||||
fb_base_url = str(fb_client.base_url)
|
||||
from hermes_cli.providers import is_actual_route
|
||||
if is_actual_route(fb_provider, fb_base_url):
|
||||
fb_api_mode = "chat_completions"
|
||||
elif not fb_api_mode_explicit and fb_api_mode == "chat_completions":
|
||||
fb_api_mode = _fallback_api_mode_resolved(agent, fb_provider, fb_model, fb_base_url)
|
||||
|
||||
old_model, old_provider, old_base_url = agent.model, agent.provider, agent.base_url
|
||||
|
||||
@@ -1940,8 +1950,12 @@ def try_activate_fallback(agent, reason: "FailoverReason | None" = None) -> bool
|
||||
agent._fallback_activated = True
|
||||
|
||||
_rebind_fallback_credential_pool(agent, fb_provider, fb_model)
|
||||
from agent.client_lifecycle import _swap_fallback_clients
|
||||
_swap_fallback_clients(agent, fb_client, fb_provider, fb_model, fb_base_url, fb_api_mode)
|
||||
if fb_provider == "moa":
|
||||
from agent.moa_loop import bind_moa_runtime
|
||||
bind_moa_runtime(agent, fb_model)
|
||||
else:
|
||||
from agent.client_lifecycle import _swap_fallback_clients
|
||||
_swap_fallback_clients(agent, fb_client, fb_provider, fb_model, fb_base_url, fb_api_mode)
|
||||
|
||||
from agent.agent_runtime_helpers import sync_credential_pool_entry_id
|
||||
sync_credential_pool_entry_id(agent)
|
||||
|
||||
@@ -1392,3 +1392,20 @@ def build_moa_facade(agent, preset_name: Any = None) -> MoAClient:
|
||||
resolved_preset = "default"
|
||||
# ``agent`` lets the fan-out wait be aborted on a user interrupt.
|
||||
return MoAClient(resolved_preset, reference_callback=_moa_reference_relay, agent=agent)
|
||||
|
||||
|
||||
def bind_moa_runtime(agent, preset_name: Any, api_key: Any = None) -> None:
|
||||
"""Make ``agent`` act as the MoA preset: pin the virtual runtime fields and install the facade.
|
||||
|
||||
Every site that puts an agent onto ``provider: moa`` (init, ``/model`` switch, fallback
|
||||
activation) must pin the same fields — the facade speaks only chat.completions, has no HTTP
|
||||
endpoint and no OpenAI client kwargs — or the next dispatch/rebuild reaches a real wire with a
|
||||
virtual identity (``moa://local`` 404, or the preset name sent as a model id).
|
||||
"""
|
||||
agent.model = str(preset_name or "default")
|
||||
agent.provider = agent.requested_provider = "moa"
|
||||
agent.api_mode = "chat_completions"
|
||||
agent.api_key = api_key or "moa-virtual-provider"
|
||||
agent.base_url = "moa://local"
|
||||
agent._client_kwargs = {}
|
||||
agent.client = build_moa_facade(agent, agent.model)
|
||||
|
||||
@@ -510,3 +510,68 @@ class TestFallbackExtraBodyReResolution:
|
||||
agent.request_overrides["temperature"] = 0.2
|
||||
self._activate(agent)
|
||||
assert agent.request_overrides.get("temperature") == 0.2
|
||||
|
||||
|
||||
# ── MoA preset as a fallback entry (#112525, #112623) ─────────────────────
|
||||
|
||||
|
||||
def _write_moa_home(tmp_path, monkeypatch):
|
||||
"""Real config.yaml with a MoA preset under a temp HERMES_HOME (genuine preset resolution)."""
|
||||
import yaml
|
||||
|
||||
home = tmp_path / ".hermes"
|
||||
home.mkdir(exist_ok=True)
|
||||
(home / "config.yaml").write_text(yaml.safe_dump({
|
||||
"moa": {"default_preset": "default", "presets": {"default": {
|
||||
"enabled": True,
|
||||
"reference_models": [{"provider": "xai", "model": "grok-4-fast"}],
|
||||
"aggregator": {"provider": "xai", "model": "grok-4.6"},
|
||||
}}},
|
||||
}))
|
||||
monkeypatch.setenv("HERMES_HOME", str(home))
|
||||
return home
|
||||
|
||||
|
||||
def _assert_bound_to_moa_preset(agent, preset="default"):
|
||||
from agent.conversation_loop import _moa_client_consumes_prepared_request
|
||||
|
||||
assert (agent.provider, agent.requested_provider, agent.model) == ("moa", "moa", preset)
|
||||
assert (agent.base_url, agent.api_mode) == ("moa://local", "chat_completions")
|
||||
assert agent._client_kwargs == {}
|
||||
assert _moa_client_consumes_prepared_request(agent.client)
|
||||
|
||||
|
||||
class TestMoaPresetFallback:
|
||||
def test_runtime_fallback_to_moa_preset_binds_the_facade(self, tmp_path, monkeypatch):
|
||||
"""#112525 / #112623: a ``{provider: moa, model: <preset>}`` fallback entry activates the
|
||||
preset (facade, ``moa://local``), never the aggregator's HTTP client wearing the virtual
|
||||
identity (preset name on the aggregator wire → 404; ``provider == "moa"`` guards misfire)."""
|
||||
_write_moa_home(tmp_path, monkeypatch)
|
||||
agent = _make_agent(fallback_model={"provider": "moa", "model": "default"})
|
||||
aggregator_client = _mock_client(base_url="https://api.x.ai/v1/", api_key="xai-key")
|
||||
with patch("agent.auxiliary_client.resolve_provider_client",
|
||||
return_value=(aggregator_client, "grok-4.6")):
|
||||
assert agent._try_activate_fallback() is True
|
||||
_assert_bound_to_moa_preset(agent)
|
||||
assert agent.client is not aggregator_client
|
||||
assert agent._provider_fallback_route == ("default", "moa")
|
||||
|
||||
def test_init_time_fallback_to_moa_preset_binds_the_facade(self, tmp_path, monkeypatch):
|
||||
"""Primary without credentials at init walks the chain: a MoA entry lands on the preset
|
||||
with virtual pins, not on the aggregator slug with the aggregator's kwargs."""
|
||||
_write_moa_home(tmp_path, monkeypatch)
|
||||
aggregator_client = _mock_client(base_url="https://api.x.ai/v1/", api_key="xai-key")
|
||||
|
||||
def _route(provider, model=None, **_kw):
|
||||
return (aggregator_client, "grok-4.6") if provider == "moa" else (None, None)
|
||||
|
||||
with (
|
||||
patch("model_tools.get_tool_definitions", return_value=[]),
|
||||
patch("model_tools.check_toolset_requirements", return_value={}),
|
||||
patch("agent.auxiliary_client.resolve_provider_client", side_effect=_route),
|
||||
):
|
||||
agent = AIAgent(model="anthropic/claude-sonnet-4.5", provider="openrouter",
|
||||
quiet_mode=True, skip_context_files=True, skip_memory=True,
|
||||
fallback_model={"provider": "moa", "model": "default"})
|
||||
_assert_bound_to_moa_preset(agent)
|
||||
assert agent._fallback_activated is True
|
||||
|
||||
@@ -95,6 +95,7 @@ OpenAI-compatible base URL continues to use the compatible client instead.
|
||||
| LM Studio (local) | `lmstudio` | `LM_API_KEY` (or none for local) + `LM_BASE_URL` |
|
||||
| Hugging Face | `huggingface` | `HF_TOKEN` |
|
||||
| Custom endpoint | `custom` | `base_url` + `key_env` (see below) |
|
||||
| Mixture of Agents preset | `moa` (`model` = preset name) | A configured MoA preset whose aggregator has credentials — the fallback runs the whole preset (references + aggregator), not the aggregator alone |
|
||||
|
||||
### Custom Endpoint Fallback
|
||||
|
||||
|
||||
@@ -244,3 +244,4 @@ So MoA does not sacrifice prompt caching on either call type. Its only real cost
|
||||
- A preset's aggregator cannot be another MoA preset. Recursive MoA trees are intentionally blocked.
|
||||
- Credential failures on one reference model do not abort the turn. Hermes includes the failure in the reference context and continues with whatever models returned.
|
||||
- MoA increases model-call count. A single model iteration can involve multiple reference calls plus the aggregator call.
|
||||
- A preset can be a fallback entry (`fallback_providers: [{provider: moa, model: <preset>}]`). When the primary fails, Hermes activates the preset itself — references and aggregator, with `moa://local` as the virtual endpoint — the same way `/model <preset> --provider moa` does. The entry is skipped when the preset does not resolve or its aggregator has no credentials.
|
||||
|
||||
Reference in New Issue
Block a user