With `fanout: user_turn` (and off-cadence `every_n` iterations) the advisors run
once per user turn and their guidance is replayed verbatim into every later
iteration of that turn. The block reads as fresh instruction, so an advisor that
proposes a tool call keeps proposing it after the acting model already ran it and
has the result in the transcript.
Observed with the clarify tool: the card was answered, the next iteration replayed
the same guidance, the model issued a second identical card, the user dismissed it,
and the turn then held two contradicting results for one question (an answer and an
empty skip). The following aggregation degenerated into a repetition loop until it
hit the output cap.
- `_STALE_GUIDANCE_NOTE` is appended when cached guidance is reused on an iteration
that already has tool activity since the last real user turn. The advice text is
handed over unchanged; only the framing says it may be out of date.
- The advisor system prompt now rules out emitting a tool call or a JSON tool-call
object. Advisors hold no tools, and a tool-call object in advisory text is what
the aggregator replays.
No change to fanout cadence, caching or accounting. The cadence test that pinned
byte-identical reuse now asserts the advice text is reused and carries the marker.
Signed-off-by: Moep90 <3042152+Moep90@users.noreply.github.com>
(cherry picked from commit 343787f287ad9915345090fba351df5ffa758c9e)
_resolve_review_runtime swallowed every resolver error at DEBUG and returned
the parent runtime, so a dead auxiliary.background_review block ran reviews on
the main model indefinitely with nothing in agent.log and nothing on screen
(#116055: the configured model appeared zero times in session_model_usage).
The fallback now logs a WARNING naming the provider, model and reason on every
review and pushes the same message once per agent through _emit_warning — the
rail every surface (CLI, TUI/Desktop, gateway) already renders for the
reasoning_effort notice. curator and the MoA slot resolver had the identical
debug-only swallow; both are WARNING now.
The aggregator request deliberately ends `user(task), user(guidance)` on
iteration 1 of every turn (#113175) so the whole prefix stays byte-stable
for the provider prompt cache. Strict-alternation chat templates
(llama.cpp / vLLM Jinja templates, Mistral, some OpenRouter routes) 400 on
that adjacency ("Conversation roles must alternate ..."), and the turn
failed. Merging proactively for everyone was declined because it brings
back the byte divergence #113175/#113784 removed and only moves the 400.
Reactive, destination-scoped recovery instead:
- error_classifier: new `FailoverReason.role_alternation` for the vendor
alternation wordings (checked before the request-validation table since
the body also carries `invalid_request_error`); same abort+fallback hints
as format_error so non-MoA consumers behave exactly as before.
- moa_alternation (new sibling): `merge_same_role_messages` (reuses the
loop's `_merge_user_content`), `destination_key` (base_url|provider,
model), `is_role_alternation_rejection`.
- moa_loop._call_prepared_aggregator: on that 400, retry ONCE with the
adjacent user turns merged, remember the destination on the facade for
the session so later iterations pre-merge, never touch destinations that
accepted the split shape. The trace records the messages actually sent.
- docs: caching section explains the reactive merge.
Live loopback (real call_llm -> SDK -> HTTP stub that 400s on same-role
adjacency): before, iteration 1 fails with BadRequestError after 1 request;
after, 2 requests (split -> 400 -> merged -> 200), next turn pre-merged in
1 request; the accepting-stub control sends byte-identical requests.
Fixes#112358
_concat_content joined two string user contents into one "a\nb" string when
_merge_consecutive_roles collapsed adjacent user turns for the Anthropic
Messages wire. For a MoA aggregator on that wire, iteration 1 of a turn ends
[user(task), user(guidance)] and was sent as user("task\n<guidance>"), while
iteration 2 replays user("task") alone, so the prompt-cache prefix diverged
at the first user block and the #112358 collapse persisted there (the
first-pass fix in #113175 only covered the OpenAI-compatible wire).
Merged turns are now always a block list with each side's blocks intact (a
string becomes one text block), matching what the list+list and list+str
shapes already did. The task block is byte-identical to the standalone turn
later iterations replay, a cache_control marker on it stays put, and the
guidance follows as its own text block. Assistant merges are unaffected
(assistant content is already a block list).
Bedrock Converse and native Gemini already merge at block/part granularity,
so the docs' Anthropic/Converse/Gemini fold caveat is replaced with the
accurate statement and the byte-identical-extension claim no longer needs
the OpenAI-compatible scope.
Part of #112358
Review follow-up for #112358: the Anthropic Messages / Converse / native
Gemini converters merge adjacent user turns, so on those wires the
guidance is still folded into user(task) on iteration 1 and the prefix
diverges at the task block; the docs claimed a byte-identical extension
for every request. Also name the trade-off in the attach docstring (two
adjacent user turns on the OpenAI-compatible wire) and stop calling the
reference cost "per iteration" — with the default fanout references run
once per user turn.
Attach the per-turn reference guidance as its own trailing user message
instead of merging it into a trailing user turn. Iteration 1 of every user
turn ends on user(task), so the merged user(task + guidance) byte-differed
from the user(task) that iteration 2+ replayed: the provider prefix cache
collapsed to the system prompt + tool schemas on the second call of every
turn (#112358 — cachedTokens pinned at the system prompt while uncached
input grew 14k -> 214k on a Nous/OpenAI aggregator).
peel_reference_guidance shrinks to the one attach shape (plain string, or
its cache-decorated single text part) so a breakpoint never lands on the
guidance. Converters that require strict alternation (Anthropic Messages,
Converse, native Gemini) already merge adjacent same-role turns.
Live wire probe (loopback server, two user turns x two iterations, real
MoA facade -> call_llm(nous)): before, request i+1 diverged from request i
at messages[1] (user task) on the second iteration of each turn; after,
every request's messages[:-1] is a byte prefix of the next. The same probe
shows the aggregator already carries the main loop's sticky session_id +
tags on the nous provider, so the routing hypothesis is not a Hermes gap.
`try_activate_fallback` and the init-time fallback walk in `agent_init._routed_client_kwargs`
handed `provider: moa` entries to `resolve_provider_client`, whose MoA chokepoint unwraps the
preset to the aggregator's real HTTP client. Both then installed that client under the virtual
identity: the runtime path kept `model=<preset>`/`provider=moa` on an `api.x.ai` client (the
preset name went out as the model id → 404, #112525) and every `provider == "moa"` guard and
key — stale timeout, cache_ttl, pricing, credential pool, and the client-rebuild branches in
`_replace_primary_openai_client` / `_create_openai_client` — saw "moa" for a native client, so
the next credential rotation or dead-connection cleanup swapped the MoA facade in anyway
(#112623). The init-time path had the mirror image: `model=<aggregator slug>`, the aggregator's
base_url/api_key in `_client_kwargs`, facade client.
WHY bind the facade instead of rewriting the identity to the aggregator: `provider: moa,
model: <preset>` means "run this preset" at every other entry point (config, `/model
<preset> --provider moa`, picker, gateway `/moa`), and the chokepoint unwrap exists for aux
tasks that skip the fan-out by design. A fallback entry is the acting model, so it gets the same
semantics and the same four pins as `agent_init` / `switch_model` / `restore_primary_runtime`
(`api_mode=chat_completions`, `base_url=moa://local`, empty client kwargs, facade client).
With identity and client consistent, the existing `provider == "moa"` guards are correct as
written and no re-keying on client identity is needed. The chokepoint call stays as the
preflight: an unresolvable preset or an aggregator without credentials still skips the entry
with the "provider not configured" warning.
`moa_loop.bind_moa_runtime` is the single binder used by init, switch and both fallback paths
so the pins cannot drift again.
Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
Co-authored-by: Zheqing Zeng <26277217+zengzheqing@users.noreply.github.com>
Co-authored-by: fangliquan <fangliquan@qq.com>
The cherry-picked commit extends the config/profile/MCP/managed-scope/completer/
OAuth/skills-manifest signatures. This commit finishes the class and trims it:
- `file_signature()` lives in `utils.py` next to the other stat/metadata helpers
instead of `hermes_cli.managed_scope` (gateway/ and agent/ callers no longer
reach into the managed-scope module for a generic stat helper).
- `hermes_cli/config_effective.py` was left comparing 2-/4-wide prefixes against
the widened `_RAW_CONFIG_CACHE` / `_load_config_cache_sig` records, so
`load_user_config_effective()` re-parsed on every call (3 parses for 3 calls on
an unchanged file, 1 before); index by the new widths.
- Sibling caches keyed on the same (mtime, size) shape and reading the SAME files
now use the helper: `load_env()` memo, `agent/skill_utils` raw-config and
external-dirs caches, `hermes_cli/model_switch` alias identity, `agent/moa_loop`
preset stamp, `hermes_cli/auth` global auth-store memo.
- Tests trimmed to one invariant each (pinned-mtime replacement invalidates; an
unchanged file still hits), both red on origin/main.
Left alone on purpose: `tools/registry.py`, `tools/skills_tool_dedup.py`,
`gateway/status.py`, `hermes_cli/banner.py`, `hermes_cli/main.py`,
`hermes_cli/session_recovery.py` — those fingerprint source files, PID/lock files
or write to persisted on-disk caches shared across processes, where an inode/ctime
key would churn on every checkout/copy rather than catch a replaced config.
Same class as the resolve_nous_access_token memo: three more process-wide
memos carried a credential resolved under one profile's HERMES_HOME override
into another profile's turn for their TTL.
- hermes_cli/nous_billing.py::_token_cache (30s (token, base) memo for the
charge poll loop) was a single unkeyed slot -> dict keyed by
hermes_home_key(); invalidate_cached_token() clears the dict.
- agent/moa_loop.py::_runtime_cache carried api_key/base_url/api_mode keyed
only (provider, model) for 5 min -> (hermes_home_key(), provider, model).
- agent/auxiliary_client.py::_client_cache_key had no profile component, so
callers that omit api_key (pool / Nous auth.json paths) could be handed a
client built with another profile's bearer -> hermes_home_key() leads the key.
WHY hermes_home_key(): it reads the per-turn HERMES_HOME override the
multiplex gateway sets (falling back to the env var), and it is
symlink-stable, so the memo key is exactly the credential home the
resolution itself read from. Profiles stay independent islands; the
default-profile process env never leaks into a secondary's turn.
Tests: one invariant per site, proven red on origin/main.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
LiteLLM OpenAI->Anthropic translation copies tool-message content parts
verbatim, so the envelope-layout part-level cache_control landed at
tool_result.content[0] - a placement the Anthropic Messages schema rejects
with a non-retryable HTTP 400 that killed the whole turn (any tool-using
cron/session on a LiteLLM-fronted Anthropic route).
New envelope_tool_part_cache_markers_supported() predicate (keyed on the
existing _is_litellm_route token matcher) threads a tool_part_markers flag
through build_prompt_cache_plan / apply_anthropic_cache_control and all
four decoration sites (main loop x2, destination replan, MoA). On LiteLLM
routes role:tool messages carry no markers and the breakpoint budget
reallocates to the nearest eligible message; OpenRouter/Nous Portal keep
the part-level form they honor, native Anthropic layout unchanged.
The diff-apply salvage introduced stale-base revert hunks — the PR was 1246
commits behind main, and its diff for conversation_loop.py and moa_loop.py
silently dropped symbols added after the PR's base (e.g.
_CODEX_ACK_CONTINUATION_NUDGE, _INTERRUPT_SCAFFOLD_MARKER, cache_ttl plumbing,
finalize_turn import, _restore_user_after_reference_handoff).
Restored both files to origin/main and re-applied only the PR's additive
changes: _moa_reference_metrics_for_hook, _system_prompt_for_hooks, the
system_prompt= and moa_references= hook kwargs, _last_reference_metrics
attribute and accessors, and the slot_metrics population in the fan-out path.
Fixes CI ImportError: cannot import name '_CODEX_ACK_CONTINUATION_NUDGE' from
'agent.conversation_loop'.
Follow-up fixes from /hermes-pr-review + /simplify-code on PR #83437:
1. Replace _redact_secrets with agent.redact.redact_sensitive_text(force=True)
— the plugin's 11-pattern list was a strict subset of the 50+ patterns in
agent/redact.py. Secrets like Stripe keys, Google API keys, GitLab tokens,
HuggingFace tokens, DB connection strings, and Telegram bot tokens would
all leak through the plugin's list but are caught by the existing redactor.
Added pk-lf- (Langfuse public key) to _PREFIX_PATTERNS in agent/redact.py.
2. Remove dead 'not isinstance(client, object)' check in on_session_finalize —
always False for any Python value.
3. Fix MoAClient.last_reference_metrics() to call the public
self.chat.completions.last_reference_metrics() instead of reaching into
the private _last_reference_metrics attribute via getattr.
4. Deduplicate _coerce_request_messages call in on_pre_llm_request — pass
pre_coerced=input_messages to _messages_for_langfuse_input to avoid
double-coercion + double _capture_content serialization per API request.
5. Add HERMES_LANGFUSE_CAPTURE to OPTIONAL_ENV_VARS in hermes_cli/config.py
for consistency with the other HERMES_LANGFUSE_* env vars.
6. Fix test_sanitized_mode_redacts_secrets test data — the old samples
('sk-abc...1234', 'sk-ant...1234', 'Authorization: Bearer ***') were too
short to match the regex thresholds and never actually tested redaction.
Updated to realistic-length secrets and changed assertions to check that
the output differs from input (redact_sensitive_text masks rather than
inserting the literal string 'REDACTED').
Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).
Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.
Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.
Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.
New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.
Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.
Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.
Closes#29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
Follow-ups on the salvaged #84782 (webtecnica):
1. conversation_loop.py: the empty-response fallback site sits directly
in the OUTER iteration loop, not the retry loop. The salvaged commit's
`break` there exited the conversation loop and ended the turn without
ever calling the just-activated fallback (caught by CI:
test_empty_response_triggers_fallback_provider). Restored `continue`
(which already re-runs the pre-API preflight at the top of the next
outer iteration) while keeping the `_preflight_compression_blocked`
reset. The other 9 sites are inside the retry loop, where `break` to
the restart_with_rebuilt_messages handler is correct.
2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
retry-loop sites must break, outer-loop sites must continue (the old
assertion pinned the bug in (1)). Mutation-checked both directions.
3. test_failover_identity.py: added `model` to the SimpleNamespace agent
fixture — _redecorate_prompt_cache_for_provider now reads agent.model
for the per-destination TTL clamp (2 CI failures).
4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
in prompt_caching and anthropic_prompt_cache_policy imports it, so the
cache-policy opt-in and the TTL clamp can never desync.
5. auxiliary_client.py: threaded the configured tier into
_replan_synchronous_cache_sections via new configured_cache_ttl()
(no live agent on that path) — the aux half of #84733's report also
stopped regressing 1h to 5m. Guarded by
TestAuxFallbackReplanThreadsTtl (mutation-checked).
6. Dropped the redundant `or "5m"` at the two threaded call sites —
effective_cache_ttl already resolves None to "5m", and the `or`
masked the cache-disabled (None) semantics.
Adds a 'council' synthesis style to MoA (per preset via synthesis_style,
one-shot via the new /council command on CLI + gateway). Reference models
answer independently; the aggregator chairs the deliberation and produces
a user-facing report of consensus, per-model disagreements (with the
differing assumptions behind them), unique contributions, and a
recommendation with an explicit confidence level.
Inspired by Perplexity's Model Council rollout to Perplexity Computer
(changelog 08/04/26): pick a board of 2-8 models, run them independently,
synthesize where they agree/disagree and what each uniquely surfaces.
Follow-ups from review of #76113:
- Extract cache_ttl_means_disabled() as the single disable-synonym
predicate; agent_init and prompt_caching_disabled_from_config both use
it so the two detection sites can no longer drift (drift would recreate
the #76085 bug class).
- Mirror _run_reference's not-None injection guard in
aggregate_moa_context (stamping None was a harmless no-op copy).
- Replace a vacuous trailing test assertion with the intended
input-non-mutation check; drop a stray blank line.
- Add a predicate-parity regression test (unknown TTL values keep
caching enabled, matching historical agent_init semantics).
Absorb the useful deltas from the parallel #76121 approach: a single
blank_cache_policy_stub factory so _cache_disabled cannot be left off
hand-rolled SimpleNamespaces, and pin the live agent disable onto MoA
advisor fan-out and one-shot aggregate_moa_context decoration so those
paths track conversation state rather than a fresh config re-read.
Keeps the earlier tri-state prepared-aggregator no-agent fix. Adds
factory and synthesis/advisor regressions.
Coordinates with #76121 / #76085.
Co-authored-by: JoaoMarcos44 <87440198+JoaoMarcos44@users.noreply.github.com>
Prepared-aggregator facades built via __new__ lack _agent. Accessing
self._agent raised inside the planner try and bool-coercion of a missing
snapshot forced False, suppressing config fallback for cache_ttl=off.
Pass a tri-state value and add a no-agent/config-off regression.
Blank SimpleNamespace stubs used by MoA decoration and
plan_cache_sections_for_destination never set _cache_disabled, so
anthropic_prompt_cache_policy re-injected cache_control markers after
operators turned caching off. Stamp the disable onto those stubs from
an explicit flag or the live config, and pass the agent flag from the
MoA aggregator path.
Fixes#76085
Three copies of the same logic landed with #76032:
- MoA's _call_prepared_aggregator and auxiliary_client's
_replan_synchronous_cache_sections both implemented stub → policy →
strip → plan for a resolved destination. Extract
plan_cache_sections_for_destination() into agent_runtime_helpers (which
already owns the policy functions) and route both through it. Also
removes a redundant full-transcript deepcopy+strip per request (the
caller pre-stripped what build_prompt_cache_plan strips again).
- The fallback_chain[N] label regex + chain-entry lookup lived in
_fallback_entry_timeout AND _fallback_destination. Extract
_fallback_chain_entry() and reuse.
MoA's cache-plan failure log is promoted debug → warning: the call-block
site skips MoA, so this block is the aggregator's only decoration path —
a silent failure ships an undecorated request (the 0%-cache MoA bug class).
Behavior-preserving; 195 targeted tests green.
Follow-up to #76032 (#20880).
build_moa_facade() reused agent.model as the preset name; a session
that had drifted to a fallback model crashed on restore with
MoAPresetNotFoundError. Validate the resolved preset against the
configured presets and fall back to the default preset.
Salvaged from PR #74903 by @liusencomic-cyber.
Convert a completed MoA aggregator response into one valid Chat
Completions delta chunk at the MoA facade boundary, normalize completed
message.tool_calls into indexed stream deltas, and classify these local
MoA adapter-shape errors as non-fallback format errors so a local
compatibility bug cannot silently drift the user's MoA route to a
single model (#55933 follow-up).
Salvaged from PR #74903 by @liusencomic-cyber.
_peel_moa_guidance hand-implemented the inverse of moa_loop's
_attach_reference_guidance from a different module — a drifting separator
or shape would make the peel silently no-op and put the last cache
breakpoint on the turn-varying guidance block (the #72626 bug class).
Move the inverse into moa_loop.peel_reference_guidance directly adjacent
to the attach, keep a thin wrapper in conversation_loop, and pin the
contract with a round-trip test over all three attach shapes.
Also fix the empty-list residue: peeling a guidance-only content part now
drops the whole message (mirroring the appended-user-message shape)
instead of leaving an empty-content user turn behind.
Flips the default fan-out cadence from per_iteration (advisors re-run on
every tool iteration, multiplying advisor spend by tool-loop depth) to
user_turn (advisors run once on the first message of each user turn; the
acting aggregator works the rest of the tool loop with that turn's
advice). Until per-mode benchmarks justify a costlier default, MoA
defaults to the cheapest, lowest-impact cadence (#67199).
One default for everyone — no split legacy/new-preset semantics; presets
that want per-step advising set fanout: per_iteration explicitly. All
three modes (user_turn / per_iteration / every_n:N) remain selectable;
every_n:1 still collapses to per_iteration (semantic identity), while
unparseable values now fall to user_turn (the default).
Docs updated with a default-change note; the per-iteration rerun test
pins its mode explicitly.
Co-authored-by: skyer-flyyy <188930297+skyer-flyyy@users.noreply.github.com>
_render_tool_calls only handled dict-shaped entries; a SimpleNamespace-
shaped tool_call (SDK-style stream-stitched responses) rendered as
'[called tool: tool]', silently losing the function name and arguments
from the advisory view. Handle both shapes (including a namespace-shaped
nested function inside a dict entry).
One-hunk hardening salvaged from closed#59712.
Co-authored-by: SquabbyZ <601709253@qq.com>
Reference models may have a smaller context window than the aggregator
(e.g. kimi-k2.7-code @ 262K advising a glm-5.2 @ 1M conversation).
Without context-length protection, a reference whose window is exceeded
gets a hard HTTP 400 from the provider, which _run_reference's
try/except silently converts to a [failed: …] note — the MoA turn
silently degrades to fewer references (#60345).
Redesigned implementation of #60387:
- Estimate AFTER the advisory system prompt is prepended, so the
request that is actually sent is what gets budgeted.
- Reserve output headroom: the preset's reference_max_tokens when set,
else an 8192-token constant, plus a 10% estimator-error fraction.
- Trim on advisory-view boundaries (text-only user/assistant turns; no
tool-result frames to orphan), preserving the system prompt, the
user-first invariant after every pop (never assistant-first), and the
trailing synthetic user turn.
- Cache get_model_context_length per (provider, model) in a per-fan-out
dict shared across the worker threads, so a turn resolves each
window once instead of probing metadata sources
per-reference-per-iteration (failures are cached too).
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Follow-ups for salvaged #56344:
- A reference that completes between the interrupt check and the reap
keeps its REAL output and accounting (the provider call billed) instead
of being zeroed with a placeholder.
- A reference still in flight at interrupt time gets a placeholder in the
results, but its future now carries a done-callback that folds the
eventual real usage/cost into the facade's pending accounting
(late_accounting_sink -> _record_late_reference_accounting), so billed
spend is never silently dropped. Pending totals are folded (not
overwritten) and guarded by a lock since done-callbacks fire on
executor worker threads.
- Interrupted placeholder results are no longer written into the facade's
turn-scoped reference cache: a cache HIT never re-runs references, so
caching a partial snapshot would replay '[skipped: interrupted by
user]' notes for the rest of the turn. The cache is left empty and the
next create() re-runs the fan-out.
agent/tool_executor.py's concurrent tool batch checks agent._interrupt_requested
and aborts the wait early; agent/moa_loop.py's _run_references_parallel had
no equivalent, so a MoA-enabled turn blocked on ThreadPoolExecutor.result()
until every reference model finished or hit its own individual
auxiliary.moa_reference timeout -- there was no way for the user to abort a
live turn mid-fanout.
Thread an optional `agent` parameter through aggregate_moa_context ->
_run_references_parallel (used when MoA references run alongside the main
model) and MoAClient/MoAChatCompletions (used when the MoA preset itself is
the acting model), then poll concurrent.futures.wait() in
_REFERENCE_POLL_INTERVAL_S slices instead of blocking on future.result() per
reference, checking agent._interrupt_requested each cycle.
Deliberately scoped to interrupt/cancel only -- no new or changed timeout
value, so this doesn't overlap open PRs #53784/#53875 (which lower the
per-reference timeout default but don't add interrupt support). `agent` is
optional and defaults to None, so any caller that doesn't pass it keeps
today's uninterruptible blocking behavior unchanged.
Extends the all-references-failed short-circuit (#56975) to the
persistent `provider: moa` facade path: MoAChatCompletions.create()
previously attached 'use the reference responses below' guidance built
entirely from failure sentinels and called the aggregator with it. Now
an all-failed turn attaches either the sanitized unavailability notice
(loud policy) or nothing (silent policy), and the aggregator — which IS
the acting model — simply acts alone. Advisor accounting for the failed
fan-out is still recorded.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
When every MoA reference model returns a failure (HTTP error, timeout,
etc.) or is skipped by the recursion guard, the one-shot aggregator
synthesis call is now skipped entirely. Previously it would try to
synthesise a wall of failure sentinels, which could block for the full
provider timeout (observed ~6 min on SenseNova) before returning a
non-retryable error that left the session hanging.
The early return carries the sanitized unavailability notice (never raw
provider error text, per the failed-reference containment) so the main
agent loop can still act in single-model mode.
Salvaged from #56975, reworked atop the _is_failed_reference helpers.
Follow-ups for salvaged #53784:
- reference_timeout now defaults to None = no per-preset override, so the
reference fan-out inherits auxiliary.moa_reference.timeout (900s default)
via call_llm's own per-task timeout resolution. The PR's 30.0s default
would have cut off long-thinking advisors mid-response, and its 300s max
cap capped legitimate explicit values — both removed. Explicit per-preset
values are still honored as-is.
- _is_failed_reference also treats '[skipped: …]' recursion-guard notes as
internal sentinels, keeping them out of both aggregator prompts.
- Dashboard/desktop TS types updated to number | null; web_server validator
accepts null/empty as 'inherit'.
