Review follow-up (egilewski): the previous commit only hedged the guidance
text; the exact Anthropic 400 was still classified, persisted, and surfaced
as confirmed billing exhaustion. Carry the ambiguity all the way through:
- agent/error_classifier.py: 'out of extra usage' matches on the 400 and
status-less paths now attach error_context {billing_unverified,
possible_content_filter}. Reason stays FailoverReason.billing (rotation +
fallback remain the right recovery either way); ClassifiedError grows a
billing_unverified property.
- agent/credential_pool.py: new FAILURE_REASON_BILLING_UNVERIFIED. An
unverified billing exhaustion gets the short transient cooldown instead of
the one-hour bench, regardless of pool size: a content-filter rejection
leaves the credential healthy and fails identically on every key, and the
hour-long sole-credential latch is what replayed the stored error and made
real fixes look ineffective. A true 402 keeps the full bench. The marker
persists with the entry so a restart cannot upgrade it back to a bench.
- agent/agent_runtime_helpers.py + run_agent.py: recover_with_credential_pool
threads billing_unverified and hands the pool 'billing_unverified' as the
persisted failure_reason.
- agent/conversation_loop.py: the fallback-switch status, max-retries status,
terminal label, and both structured terminal results hedge when the verdict
is unverified. New _billing_terminal_label + _billing_failure_result build
the returned terminal response in one place; the result dict now carries
billing_unverified and the billing_block gains 'unverified': true. The
confirmed-billing path (a real 402 or an API-key credit depletion) keeps
the original assertive wording, so the caveat no longer dilutes it.
Regression tests: classifier marking (400 + status-less + unambiguous-body
negative), pool cooldown TTLs + persistence round-trip, pool failure_reason
plumbing, and the returned terminal response for both unverified and
confirmed verdicts.
Note: tests/agent/test_credential_pool_routing.py::TestFailureAttribution::
test_unmatched_key_does_not_retry_only_pool_entry fails identically on
current main without this change (pre-existing, unrelated).
On an Anthropic subscription OAuth credential, every request failed with
HTTP 400 "You're out of extra usage. Add more at claude.ai/settings/usage".
That is not a billing condition: Anthropic's server-side content filter rejects
the first sentence of Hermes' own built-in SKILLS_GUIDANCE prompt, and the
rejection is surfaced with a billing-shaped message. Because the message points
at the usage settings page, it reliably sends people to buy quota they do not
need — the reporter lost three debugging sessions to it.
Bisected against the live API with the real 71,721-char assembled prompt: the
first SKILLS_GUIDANCE sentence alone reproduces the 400 and removing it alone
clears it. Size was ruled out (20 KB of unrelated filler returns 200) and so was
the system[0] identity gate (that returns 429, a different failure).
Three changes, all serving the same outcome — a subscription user can no longer
be misdirected by this 400:
- agent/prompt_builder.py: reword the triggering sentence to the phrasing the
reporter verified returns 200. Meaning, the skill_manage reference, and the
## Skill Safety Rule block are all preserved. The reword is empirically
validated rather than understood, so a comment records the bisect and warns
that any rewrite must be re-verified against an OAuth token, not an API key.
- agent/conversation_loop.py: the Anthropic branch of the billing guidance no
longer asserts exhaustion as fact. It hedges the opening line, names the
content-filter alternative, and gives the operator a way to tell the two apart
(if the usage page still shows quota, suspect a content rejection). It also
points at `hermes auth reset anthropic`, because the credential exhaustion
latch replays the stored error for ~60 min without issuing a request — which
makes a real fix look like it did not work.
- hermes_cli/auth.py: document that CLAUDE_CODE_OAUTH_TOKEN is an OAuth token,
not an API key, despite auth_type="api_key". It stays in api_key_env_vars
because that tuple doubles as the credential-discovery list; removing it would
stop Hermes finding a `claude setup-token` credential at all.
Docs updated to match the reworded prompt.
Fixes#82154
A turn writing against a session already closed by compression died with
session_persistence_failed and a misleading "this is often a full disk"
dialog, even though the store was healthy and a live continuation existed
(#82001). Depth-1 recovery (find_live_compression_child) could not resolve
lineages with >=2 compression hops (root -> mid -> tip), reproduced
independently on two- and three-hop chains.
- run_agent.py flush chokepoint: on CompressionSessionClosedError, resolve
tip = db.get_compression_tip(old_id) (canonical bounded transitive walk),
adopt only when tip != old_id AND the tip row is live, retry the flush
exactly once (adoption budget); otherwise fail closed.
- gateway/session.py append_to_transcript: replace the depth-1 live-child
lookup with the same tip + liveness contract, so gateway transcript
reroutes follow full chains.
- agent/conversation_compression.py _adopt_live_compression_child: turn-start
recovery preflight now resolves via get_compression_tip with the same
liveness check, closing the last depth-1 consumer in this family.
- classify_persistence_error: new "compression_closed" bucket; the turn-end
explanation names compression rotation and tells the client to refresh the
session id instead of blaming a full disk.
Tests: depth-1 adoption, multi-hop chain adoption (agent + gateway), fail
closed with no continuation / stale-closed (ws_orphan_reap) tip, exactly-once
adoption budget, and error-wording guards (compression-closed never mentions
disk; real disk failures keep disk guidance).
Closes#82001
Co-authored-by: Al3xand3r1987 <125030427+Al3xand3r1987@users.noreply.github.com>
Co-authored-by: yuzilongleif-collab <235949691+yuzilongleif-collab@users.noreply.github.com>
`_moa_prepared_request` is a private handshake between the conversation
loop and MoAChatCompletions.create. It is attached whenever
agent.provider == "moa", on the assumption that agent.client is still the
in-process MoA facade.
Credential rotation, provider fallback and dead-connection cleanup all
rebuild agent.client from _client_kwargs between attempts, and
pending_moa_prepared_request deliberately carries a prepared request
across exactly that boundary. The rebuilt client is a native OpenAI
client while provider stays "moa", so the key reaches an SDK that has
never heard of it:
TypeError: Completions.create() got an unexpected keyword argument
'_moa_prepared_request'
That error is non-retryable, so every remaining turn on the session
fails. Both dispatch paths are affected: the non-streaming one calls
agent.client directly, and _create_request_openai_client returns
agent.client unchanged for provider "moa".
Re-check the live client at the point the key is attached, which covers
both paths at once. When the facade is gone, send the prepared prompt
without the handshake and log the downgrade.
A provider-confirmed rearm (#85846) resets the shared attempt budget, but
an earlier insufficient-progress verdict left _preflight_compression_blocked
armed, keeping the pre-API gate dark for the rest of the turn — a later
pressure spike could still grow unchecked until the provider overflow
handler fired. Clear the blocker and the stale pressure reading inside the
provider-confirmed rearm branch: the prompt is proven back below the
threshold, so the old verdict describes a request shape that no longer
exists.
Builds on @h-mascot's #84995, whose commit is preserved on this branch;
his rearm condition was superseded by #85846's latch-verified variant, but
the blocker-clear half was correct and is kept.
A marathon tool turn burned all compression_attempts on *successful*
pre-API compactions; the gate then went permanently dark and the context
grew unchecked until the provider rejected the request terminally
("Context length exceeded: max compression attempts (3) reached", session
f087963205f9, 2026-08-01). The budget now refunds at loop top when the
assembled request is back under threshold * 0.8 AND the compressor's own
should_compress() agrees the pressure is gone.
Anti-thrash intent of the cap is preserved (#11529): no-progress passes
never reach the refund margin, divergent-signal cases (should_compress
still True) keep the budget burnt, and the insufficient-progress blocker
is untouched.
Validation: new behavioral suite (7) + all 130 compression/context tests
via scripts/run_tests_hermetic.py.
Rückbau: Commit revertieren; kein Zustand, keine Migration.
(cherry picked from commit 041b489d566bfb1d6816d53d9acd5ac50b8d5af0)
The diff-apply salvage introduced stale-base revert hunks — the PR was 1246
commits behind main, and its diff for conversation_loop.py and moa_loop.py
silently dropped symbols added after the PR's base (e.g.
_CODEX_ACK_CONTINUATION_NUDGE, _INTERRUPT_SCAFFOLD_MARKER, cache_ttl plumbing,
finalize_turn import, _restore_user_after_reference_handoff).
Restored both files to origin/main and re-applied only the PR's additive
changes: _moa_reference_metrics_for_hook, _system_prompt_for_hooks, the
system_prompt= and moa_references= hook kwargs, _last_reference_metrics
attribute and accessors, and the slot_metrics population in the fan-out path.
Fixes CI ImportError: cannot import name '_CODEX_ACK_CONTINUATION_NUDGE' from
'agent.conversation_loop'.
Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).
Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.
Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.
Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.
New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.
Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.
Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.
Closes#29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
Two reliability gaps from #82777:
1. Rejection matcher required only a field-name mention, so a transient
5xx/timeout whose body echoed the request (which contains
context_management) permanently downgraded native compaction for the
session. Now requires rejection language (unknown/unsupported/invalid/...)
alongside the field name, and when a parsed HTTP status is available,
400 specifically — non-400 statuses never match. Message-only transports
(no status attribute) keep working unchanged.
2. compression.codex_responses_native was coerced with bool(), so the
strings "false"/"off" enabled the feature. Now uses the shared
utils.is_truthy_value helper.
Conversation-loop call site passes api_error.status_code through.
Sabotage-verified: reverting the matcher to field-name-only fails the new
echo and non-400 tests.
Simplify-pass follow-ups on the salvage stack (all guard tests re-run,
mutation-checked):
1. conversation_loop.py: moved `_preflight_compression_blocked = False`
from 9 per-site copies into the restart_with_rebuilt_messages handler
(its single consumer). Besides removing the 9 duplicated blocks, this
fixes a 10th pre-existing retry-loop site (content-filter stall
failover, #32421) that set the flag and broke WITHOUT clearing the
preflight block — a content-filter failover previously restarted with
preflight compression still blocked against the fallback's smaller
window, the same #84733 bug class. The outer-loop empty-response site
keeps its own clear (it never passes through the handler). New AST
guard test_restart_handler_clears_preflight_block pins the hoisted
clear (mutation-checked).
2. agent_runtime_helpers.py: extracted _raw_cache_ttl_from_config() —
prompt_caching_disabled_from_config and configured_cache_ttl were
verbatim copies of the same config read. Added VALID_CACHE_TTLS.
3. prompt_caching.py: added is_qwen_model() next to
ALIBABA_FAMILY_PROVIDERS; effective_cache_ttl and
anthropic_prompt_cache_policy now share both the family set and the
qwen predicate — neither can desync.
4. Guard-test hardening: assert every _try_activate_fallback reference
is a direct `if agent._try_activate_fallback(...):` site, so a future
`activated = ...` form can't silently escape the restart-discipline
guard.
Follow-ups on the salvaged #84782 (webtecnica):
1. conversation_loop.py: the empty-response fallback site sits directly
in the OUTER iteration loop, not the retry loop. The salvaged commit's
`break` there exited the conversation loop and ended the turn without
ever calling the just-activated fallback (caught by CI:
test_empty_response_triggers_fallback_provider). Restored `continue`
(which already re-runs the pre-API preflight at the top of the next
outer iteration) while keeping the `_preflight_compression_blocked`
reset. The other 9 sites are inside the retry loop, where `break` to
the restart_with_rebuilt_messages handler is correct.
2. test_prompt_cache_ttl_propagation.py: made the AST guard loop-aware —
retry-loop sites must break, outer-loop sites must continue (the old
assertion pinned the bug in (1)). Mutation-checked both directions.
3. test_failover_identity.py: added `model` to the SimpleNamespace agent
fixture — _redecorate_prompt_cache_for_provider now reads agent.model
for the per-destination TTL clamp (2 CI failures).
4. prompt_caching.py / agent_runtime_helpers.py: single source of truth
for the alibaba-family provider set — ALIBABA_FAMILY_PROVIDERS lives
in prompt_caching and anthropic_prompt_cache_policy imports it, so the
cache-policy opt-in and the TTL clamp can never desync.
5. auxiliary_client.py: threaded the configured tier into
_replan_synchronous_cache_sections via new configured_cache_ttl()
(no live agent on that path) — the aux half of #84733's report also
stopped regressing 1h to 5m. Guarded by
TestAuxFallbackReplanThreadsTtl (mutation-checked).
6. Dropped the redundant `or "5m"` at the two threaded call sites —
effective_cache_ttl already resolves None to "5m", and the `or`
masked the cache-disabled (None) semantics.
Adds a 'council' synthesis style to MoA (per preset via synthesis_style,
one-shot via the new /council command on CLI + gateway). Reference models
answer independently; the aggregator chairs the deliberation and produces
a user-facing report of consensus, per-model disagreements (with the
differing assumptions behind them), unique contributions, and a
recommendation with an explicit confidence level.
Inspired by Perplexity's Model Council rollout to Perplexity Computer
(changelog 08/04/26): pick a board of 2-8 models, run them independently,
synthesize where they agree/disagree and what each uniquely surfaces.
Salvage the plugin-owned static prompt idea from PR #51589 into the constrained #64167 contract: stable IDs, deterministic placement, bounded fail-open rendering, and full-prompt resume recovery without new session columns.
Co-authored-by: Topher Ross <biz@topherross.com>
Trim verbose comments in conversation_loop.py and run_agent.py to 2 lines
each. Fix the same bug class in the compression summary path at
chat_completion_helpers.py: remove _thinking_prefill from the explicit
pop tuple and move the generic underscore-key sweep to after
_drop_thinking_only_and_merge_users, so the drop pass can recognize
prefill stubs there too.
- warn (not debug) on final text-turn flush failure: a failure here
reopens the exact #81641 data-loss window with _persist_session as
the only remaining retry, unlike the verify siblings which retry
in-loop; include session id for triage
- trim the flush-site comment to sibling proportion, pointing to the
test module for the full incident narrative
- test: assert _persist_session presence before indexing, so a wiring
change fails with a clean assertion instead of ValueError from max()
A pure-text assistant turn (finish_reason=stop) had no durable write of
its own. Its answer reached the user through the streaming / interim
display path, which is display-only and never touches state.db, and the
first durable write was finalize_turn's _persist_session — after the
loop exits and behind post-turn work that can include micro-compaction's
aux-LLM call.
Anything that ended the process or tore the session down inside that
window lost a reply the user had already been shown. On a remote
(non-loopback) backend the window is easy to hit: WS 1006 closures drive
ws_orphan_reap teardown, and affected sessions ended up with user rows
and zero assistant rows in state.db.
The neighbouring exits of the same loop already close this gap:
* the tool-call exit flushes the assistant(tool_calls) block before
handing control to _execute_tool_calls (#49045)
* the verify-on-stop and pre_verify exits flush final_msg before
appending their nudge (#65919 §7)
Apply that same idiom to the ordinary text exit rather than adding a new
persistence mechanism. The intrinsic _DB_PERSISTED_MARKER dedup makes the
later _persist_session a no-op for this row, so no duplicate rows and no
extra write — the same write, just earlier.
Unlike the tool-call exit, a failed flush must not abort the turn: no
side effect runs after this point and the answer is already produced, so
the failure is logged and _persist_session remains the retry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Move the legacy ghost-row filter from inside the api_messages loop to
BEFORE repair_message_sequence_with_cursor. Dropping a ghost assistant
row between two user messages creates user→user which the repair can
now fix (previously the repair ran first and missed it).
Promote '[This response was interrupted by a user correction.]' to
module-level _INTERRUPT_SCAFFOLD_MARKER constant — used in both
_apply_active_turn_redirect (checkpoint_parts) and the ghost filter,
so they can never drift.
Update ghost-row test: the two consecutive user messages are now
merged by repair, so check for content as substring.
Sessions already poisoned by the incomplete #73146 else branch still replay
hidden assistant rows whose content is the raw interrupt scaffold. Skip those
rows when building provider messages so old state.db history cannot keep
seeding the echo loop.
The incomplete #73146 else branch still wrote the interrupt checkpoint into
the placeholder assistant row. Mid-tool steers then replayed that scaffold as
the model's own prior reply, which it echoed into a self-replicating ghost
loop. Carry the scaffold only on the user correction's api_content, matching
the assistant-tail branch.
Fix: _LENGTH_CONTINUATION_DROPPED_TOOLS_PREFIX ended with '(' but
_get_continuation_prompt still had f'({tool_list})', producing
'((write_file)' instead of '(write_file)'. Removed the '(' from
the prefix constant — the parenthesis belongs in the interpolation.
Widened: promoted the empty-response nudge (line 6993,
'You just executed tool calls but returned an empty response...')
to _EMPTY_TOOL_RESPONSE_NUDGE constant and added it to the
classifier's recognition set. Same bug class — its
_empty_recovery_synthetic metadata flag doesn't survive SessionDB
projection either.
Test: added parametrize case for the empty-response nudge (7→8 cases).
E2E: verified byte-for-byte string equivalence for all nudge constants.
aed114a69 taught _is_synthetic_compression_user_turn to recognize the
max-iteration nudge as ephemeral runtime scaffolding rather than a human
turn, since its role="user" metadata flag doesn't survive SessionDB
projection and a crash/interrupt mid-turn can persist it durably — becoming
the compaction anchor / auto-focus topic in place of the real task.
conversation_loop.py's retry loop appends several more role="user" rows
with the exact same "ephemeral, metadata-tag-only" shape, none of them
recognized by the classifier:
- The three _get_continuation_prompt variants (length-continuation nudge,
tagged _length_continuation_nudge) — two fixed strings plus a third that
interpolates the dropped-tool-call list.
- _CODEX_INCOMPLETE_NUDGE (codex/responses reasoning-only retry).
- The codex ack-continuation nudge (acknowledgment-only reply re-prompt).
- The dropped-tool-call nudge (tagged _dropped_toolcall_nudge) — persisted
across up to 3 consecutive retries before the finalization pop-loop
strips it; an interrupt/crash before that pop can persist it same as the
max-iteration case.
Promote the previously-inline nudge strings to named module-level constants
in conversation_loop.py (single source of truth for both construction and
recognition), then extend the classifier to recognize all of them — exact
match for the five fixed-content nudges, a stable-prefix check for the
dropped-tool-call continuation variant (its tool list is interpolated so it
can't be exact-matched, same treatment TODO_INJECTION_HEADER already gets).
Imported lazily inside the classifier to avoid a module-load-order cycle —
conversation_loop.py already imports FROM context_compressor.py at call
time for the same reason.
A background memory/skill review (agent/background_review.py) forks a
second, complete AIAgent in a daemon thread that deliberately shares the
live agent's own session_id for prompt-cache warmth. Nothing previously
stopped a user's next live turn from starting while that fork was still
mid-conversation, letting both stream against the same session_id and
credentials concurrently. That produced two observable failures:
- Doubled prompt-token accounting on the live turn's own calls (the two
concurrent request/response streams under one session_id confuse the
token-usage bookkeeping), triggering premature context compression.
- A lockup that a normal interrupt could not clear: the review fork is a
fully independent AIAgent with its own _interrupt_requested flag, and
was never added to the parent's _active_children list -- the only list
AIAgent.interrupt() actually walks for cross-agent cancellation -- so a
live-turn Ctrl+C had no propagation path to it at all.
Fix, three files:
1. agent/agent_init.py -- add _background_review_agent /
_background_review_lock tracking state to every AIAgent, mirroring the
existing _active_children pattern.
2. agent/background_review.py -- the review fork now registers itself on
the parent's _active_children right after construction (reusing the
same list/lock interrupt() already fans out to for real subagent
delegation), and unregisters on every exit path (success, the
tool-whitelist finally, and the outer exception safety-net). All
registration is defensive (getattr/try-except) so an AIAgent built
without going through agent_init.py's setup degrades to "no
cross-turn cancellation" instead of aborting the whole review.
3. agent/conversation_loop.py -- at the very start of every
run_conversation() turn, if a prior background review is still
in-flight, it is now proactively cancelled via interrupt() before the
live turn proceeds -- fire-and-forget, non-blocking, adds no latency.
Adds 3 regression tests to tests/run_agent/test_background_review.py,
confirmed to fail against the pre-fix code via a scripted revert.
Verified: ruff clean on all touched files; 66/66 background-review and
interrupt-propagation tests pass; 256/256 across turn_finalizer +
run_agent regression suites; no fork-only symbols in the diff.
Two corrections on top of the #71077 base (the whole bug class):
1. Turn boundary = last USER message, not last assistant message. A Codex
turn spans several assistant messages (assistant+tool_calls -> tool ->
... -> final assistant) whose reasoning items must replay together; the
last-assistant boundary would strip reasoning mid-chain from the active
turn (the gap flagged in PR #71077 review).
2. type="compaction" checkpoints (native server-side compaction, PR #81747)
are exempt: they carry already-pruned history, not per-turn reasoning.
Pruning filters items instead of popping the sidecar key.
Sibling site fixed in the same class: the Codex incomplete-continuation
dedup path blind-overwrote codex_reasoning_items on visually-duplicate
interim messages, which would drop the only copy of a checkpoint captured
on the earlier response. Extracted merge_interim_reasoning_items() into
agent/native_compaction.py; newer reasoning wins, prior checkpoints are
preserved unless the newer payload carries its own.
Own the surrogate-crash class at three chokepoints instead of leaf sites:
- finalize_turn scrubs final_response once where model text leaves the
conversation loop — covers oneshot stdout (#80366), NIM/any-provider
responses (#19819), and every delivery consumer of the turn result.
- _sanitize_gateway_final_response scrubs at the gateway chat-surface
boundary — Telegram utf16_len (#55309) and Signal formatting (#55143)
can no longer see a lone surrogate; raw-text surfaces keep passthrough.
- run_conversation walks the fully-built api_kwargs with
_sanitize_structure_surrogates so tool descriptions (session_search,
#50959) and every other request-body leaf are JSON-encodable before
any provider sees them.
Regression tests pin all three chokepoints plus helper semantics.
Cherry-picked alongside #79240 (TheophilusChinomona) and #80374
(rainbowgore) whose commits precede this one with authorship preserved.
Opt-in via compression.codex_responses_native (default: false). When enabled,
gpt-5.6-family models on the direct OpenAI API (api.openai.com) or a ChatGPT
Codex subscription send context_management=[{type: compaction,
compact_threshold: N}] on Responses requests. OpenAI compacts server-side and
returns an encrypted compaction output item; Hermes captures it into the
existing codex_reasoning_items sidecar and replays it on later turns in place
of the pruned history — inheriting persistence, session replay, the
cross-issuer guard, and the encrypted-replay kill switch with zero new state.
Scope is deliberately hard-gated (agent/native_compaction.py, re-checked per
request): gpt-5.6 family only — gpt-5.1/5.2 fail server-side on the field
(HTTP 500 / stream stall, no structured rejection; live-verified) — and
direct OpenAI/Codex routes only; xAI, GitHub/Copilot, OpenRouter, relays,
and local servers never see the field.
Hermes' local compression stays armed as the fallback owner: the native
threshold is clamped ~8K tokens below the local trigger so the server
compacts first, and a structured provider rejection of context_management
disables native compaction for the session and retries without it
(one-shot guard in TurnRetryState).
Live-verified E2E on api.openai.com/gpt-5.6: server compaction fired at a
4K threshold, checkpoints captured and replayed, recall preserved across
3 turns; gpt-5.1 with the flag enabled stays clean (field never sent).
Direction credit: PR #76950 by @laryhorb explored native Responses
compaction; this is a minimal reimplementation on current main.
Fixes 13 issues found in PR #20774 review:
1. Wiring: engine selection moved from run_agent.py to agent/agent_init.py
(where init_agent lives on current main). Transform hook moved from
run_agent.py to agent/conversation_loop.py (where run_conversation lives).
2. Prompt caching: replace copy.deepcopy with copy-on-write (shallow list
copy + clone only messages that are mutated). Use last_prompt_tokens
from update_from_response instead of re-estimating tokens every call.
System extension injection is idempotent (one-time cache break).
3. Signature mismatch: _message_signature renamed to _content_signature
and now excludes tool_calls/tool_call_id from the hash. This prevents
mismatches when _canonicalize_api_tool_calls re-serializes argument
JSON with sort_keys=True on the API copy.
4. update_model: accepts api_mode parameter (required by agent_init.py).
5. Reconciled with select_context: transform_api_messages is a separate
hook that runs AFTER select_context and sanitization, before
prompt-cache marker placement. Both hooks coexist with clear ordering.
6. Dedup/purge: kept as DCP-specific strategies (different semantics from
ContextCompressor._prune_old_tool_results — DCP deduplicates by
tool+args signature, not by content hash).
7. Removed copy.deepcopy: replaced with shallow list copy + copy-on-write
via _clone_if_needed. Only messages that are actually mutated get
cloned.
8. Removed redundant _ensure_refs call: _match_api_messages_to_refs no
longer calls _ensure_refs (the caller already called it).
9. _message_key still uses index (needed for positional ref assignment),
but _content_signature is cached per id(msg) to avoid re-hashing.
10. _inject_nudge: only injects into user messages, never falls back to
non-user messages (prevents role semantics violations).
11. Memory: _evict_inactive_blocks bounds blocks_by_id to
_MAX_INACTIVE_BLOCKS (50) deactivated blocks.
12. Merged _range_tool_schema and _message_tool_schema into a single
_compress_tool_schema. Merged _handle_range_compress and
_handle_message_compress into _handle_compress.
13. Dropped DCP_CONTEXT_ENGINE_PR_SPEC.md (temporary file, not for tree).
Config defaults kept minimal in hermes_cli/config_defaults.py (only
the keys the engine actually reads, not the full DCP-compatible surface).
Closes#20717
truncated_response_parts were joined with no separator at both the
ceiling exit and the success path, so a fragment ending mid-word ran
straight into the next one (#78577). insert a newline only when the
previous fragment ends non-whitespace and the next starts
non-whitespace, so existing separators are not doubled.
the scaffolding marks are hermes bookkeeping. only the chat-completions
transport strips underscore keys, so anthropic and bedrock requests on
continuation attempts 2+ would send the marks to strict providers. pop
them in the central api_messages sanitization next to _thinking_prefill.
also pin that a mark reloaded from a mid-crash persist on a prior turn's
message is never deleted by a later turn's ceiling cleanup.
a turn that exhausts all 4 length-continuation attempts used to persist
its interim fragments and '[System: ... continue ...]' user nudges into
the session transcript. every later user turn replayed the unanswered
nudges, so the model resumed the oversized response, truncated again,
and re-exhausted the ceiling - wedging the session regardless of input.
at the ceiling exit, drop the fragment/nudge scaffolding from the turn's
tail and store one settled assistant turn carrying the stitched partial
text. the marks are cleared on continuation success and on the
content-filter rollback so cleanup can never delete fragments whose text
was already consumed.
also stop labeling a finish_reason='length' stub a network error: report
it as a truncation (stream ended before completion) and say the partial
response is kept when the ceiling is exhausted.
The function-scoped import at the end of run_conversation loads
agent.turn_finalizer fresh from disk on the first turn that reaches it.
On a source/editable install whose checkout changed mid-session, that
pairs an old caller with a new callee at the exact seam where every turn's
work is persisted — the turn crashes on a signature mismatch after the
work is done. The lazy import was never cycle-forced: turn_finalizer
defers its own conversation_loop import.
Post-merge simplify finding on #81613: the wrapper's docstring claimed it
existed for 'existing callers', but every caller was introduced by the same
PR - there was never a pre-existing import path to preserve. All callers
(conversation_loop, tool_executor, run_agent's own flush handler, tests)
now import the canonical hermes_state.classify_persistence_error directly,
matching how is_disk_full_error is consumed. No behavior change; imports
stay lazy inside the exception handlers.
An enterprise deployment hit sustained SQLite write-lock contention on a
shared multi-gigabyte state.db (gateway + CLI processes writing
concurrently). Turns correctly failed closed with
session_persistence_failed, but the only user-facing wording claimed the
disk was full and the gateway rendered a generic failure.
The fast-fail semantics are deliberate and unchanged. This adds a pure
classifier (locked / disk / unknown) applied where the SQLite error is
still visible, threads the cause through the turn-completion explainer,
and stamps a machine-readable failure_reason
(session_persistence_failed:<cause>) plus a guaranteed non-empty error on
the result for downstream surfaces. The cron scheduler's explainer-text
suppression now matches every cause variant so refined wording cannot
leak into scheduled-job deliveries.
Review-pass follow-ups (three parallel reviewers, findings verified):
- hermes_state_search.py list_recent_user_messages now drops legacy
standalone compaction handoffs in the decode loop (SQL can't see them:
durable role=user, no display_kind). Closes the /undo N pairing skew
where the in-memory count (new predicate) and the DB soft-delete pick
(old predicate) targeted different turns on legacy sessions. Fetches
with headroom so the requested limit is still honored. 3 new tests,
mutation-checked (no-op'ing the skip fails 2/3).
- _should_skip_model_call_for_reference_handoff: single drive-check scan
(was two — once inside the restore helper, once after); the restore
helper no longer re-scans and its return value now decides the verdict.
- _final_response_from_messages replaced by the _HANDOFF_SKIP_FINAL_RESPONSE
constant it always returned (parameter was unused).
- _handoff_carries_live_user_content delegates to the canonical
_strip_context_summary_handoff_message — also fixes the edge where a
merged-shaped row with an EMPTY preserved prior tail was wrongly
treated as carrying live content.
- Site-level guard test for rollback.restore with a legacy handoff row
(predicate-in-context, complements the unit tests).
Follow-ups on top of the salvaged #80696 fix (review findings):
- Sibling sites: rollback.restore, gateway /retry, CLI /retry and /undo N,
and both CLI resume turn counters now use is_user_originated_turn so
legacy-persisted standalone handoffs (durable role=user, no display_kind)
can never be truncation targets or counted as user turns (#80622
suggested regression 4, dispatcher-wide).
- Site-1 guard: hoist the api_call_count decrement + iteration-budget
refund above the break so a skipped turn no longer leaks a budget unit
and finalize_turn logs the true call count (matches the ollama early-exit
and the site-2 sibling).
- Site-2 guard: run the handoff guard BEFORE reanchoring so a restored
user ask is what the anchor lands on, not a stale pre-restore index.
- SUMMARY_PREFIX: add the mid-tool-loop carve-out the code-side guard
already implements, so a literal-minded model doesn't halt an in-flight
exchange after in-place compaction.
- Skip path returns a short compaction status instead of replaying the
previous turn's answer (finalize_turn would append it as a fresh
assistant row — duplicate prose in transcript and delivery).
After a completed assistant stop, a standalone CONTEXT COMPACTION handoff
could occupy the sole user slot and resume stale Historical Task Snapshot
work with no new human ask. Guard post-compaction continues, hide
standalone handoffs from session dispatch, and harden SUMMARY_PREFIX for
the empty-after-handoff case (#80622).
The rough preflight estimate intentionally overestimates, but not by a
fixed margin: CJK text is counted at ~1.7x its o200k cost and
Responses-mode reasoning replay blobs at several times their billed
cost. Heavy sessions show rough estimates 2-3x real usage and compact
at 35-55% of the real window, stalling turns for minutes and discarding
detail (churn), because the defer guard only tolerated 5% rough growth
and sessions that never compressed had no baseline at all.
Pair every request's rough estimate (note_request_rough_estimate,
recorded in the conversation loop right after the pressure estimate)
with the provider's real prompt_tokens in update_from_response(), then
defer preflight while projected real usage — last real + rough growth
since that reading — stays under the threshold. Rough growth is itself
an overestimate of real growth, so the projection is an upper bound and
deferring below the threshold is safe; the provider's context-overflow
handler remains the backstop.
The baseline no longer ratchets on defer: it is refreshed by the
response pairing, and advancing it without a matching real reading
would shrink apparent growth and defer on stale data.
The api_messages build used a shallow msg.copy(), decoupling only
top-level fields. Every nested container (tool_calls entries and their
function dicts, multimodal content-part lists, reasoning_details) stayed
aliased to the persisted history, so ANY in-place transform on the send
copy silently rewrote the stored transcript.
Probed every send-path transform against that aliasing shape on main:
content strip loop safe (top-level reassign)
_canonicalize_api_tool_calls (repair) LEAKED <- #80616's fix
_sanitize_messages_surrogates LEAKED (multimodal parts,
tc ids/args, reasoning)
_sanitize_messages_non_ascii LEAKED (multimodal parts)
_sanitize_api_messages safe
_drop_thinking_only_and_merge_users safe
The retry loop already believed the copies were independent - it
sanitizes messages AND api_messages separately (~L3555) - so the
aliasing was accidental everywhere.
Fix at the chokepoint: _clone_message_for_send clones every container
(dict/list) recursively while sharing immutable leaves, so every
downstream in-place transform - current and future - is safe by
construction. Cost is container-count, not string-bytes: 100KB argument
strings and base64 payloads are shared (measured ~0.5ms vs ~0.1ms per
1500-message build; noise next to one json round-trip). Same clone
applied to the prefill-message insert (same class, same pipeline).
The class-wide invariant test runs the full send-path transform
pipeline over an adversarial fixture (malformed args, surrogates,
non-ASCII, multimodal parts, reasoning fields) and asserts the history
stays byte-identical; an AST contract pins the build-site wiring so the
shallow copy can't quietly return. Both mutation-verified: reverting
the clone to shallow fails 4 isolation tests, unwiring the build site
fails the AST contract.
0xGr1mm's branch fix (previous commit) remains as defense in depth at
the exact site the #80498 incident hit; his regression tests and the
class-wide invariant give layered coverage.
`_canonicalize_api_tool_calls` promises copy-on-write in its own docstring
— "the persisted history is untouched" — and the call site repeats it:
"Operates on api_messages (the API copy) so the original conversation
history in `messages` is untouched."
The canonicalize branch keeps that promise (`tc = {**tc, "function": {...}}`).
The repair branch does not:
except Exception:
tc["function"]["arguments"] = _repair_tool_call_arguments(...)
`api_messages` is built with `msg.copy()` — a SHALLOW per-message copy — so
every `tool_calls` entry is the same dict object the persisted history
holds. Assigning into `tc["function"]` therefore writes through to the
stored turn. The sibling loop two lines above only touches `am["content"]`,
one level deep, which is why the aliasing never showed up there.
On the unrepairable path `_repair_tool_call_arguments` returns "{}", so
that write replaces the model's real arguments with an empty object in the
transcript. A stream that dies mid `write_file` loses the file content it
had already streamed — the reported symptom in #80498, where a chapter
draft was silently reduced to `{}` and only a WARNING remained:
Unrepairable tool_call arguments for write_file — replaced with empty
object (was: {"content": "# 骨架-第25章\n> 承接...)
Mirror the canonicalize branch: build a new tool-call dict instead of
assigning into the shared one. The API copy still carries "{}" — the
repair's whole purpose is to never ship broken JSON — but the history keeps
what the model actually sent, so the transcript, session persistence and
any later retry still have it.
The in-place write was not an oversight in isolation: it predates the memo
refactor, which preserved it deliberately for byte-parity. The existing
`test_history_not_mutated` asserts exactly this invariant but restricts
itself to valid arguments, and its docstring records the gap — "(Malformed
args take the in-place repair path — pre-existing behavior)". That is why
a test file whose header already claims "the persisted history is never
mutated (copy-on-write preserved)" stayed green through the bug.
Four tests close it: history keeps the original bytes, the send copy is
still repaired, a broken call does not disturb its siblings, and repeated
sends stay lossless. On unpatched main three of them fail; the parity and
complexity tests are unaffected because the difference is only observable
when the history list is separate from the send copy — which is the shape
production uses.
Refs #80498
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Selecting an NVIDIA NIM model whose id reached config without the nvidia/
prefix produced a bare "HTTP 404: 404 page not found" — retried three times,
never naming the model. It reads exactly like an outage or an auth failure,
which is where the Discord thread spent its time before the id was spotted.
normalize_model_for_provider() had no branch for nvidia, so a bare id passed
straight through to the API. Repair it from the provider's curated catalogue:
a bare name that matches exactly one entry modulo the prefix gets it back.
That's a lookup, not a guess — build.nvidia.com also fronts local NIM
containers and third-party models, and anything absent from the catalogue is
left alone. Because the repair runs on every runtime setup, an already-broken
config self-heals on the next turn and prints what it changed.
If a bare id still reaches the wire, the 404 now explains itself. The
classifier consults the same catalogue: a prefix-less id the provider only
serves as vendor/model is a deterministic failure, so it classifies as
model_not_found instead of burning three retries on a retryable "unknown",
and the error trace names the id to use.
Fixes#78796
* fix(agent): adopt .env credential/base-url edits at the turn boundary
A Settings save (desktop PUT /api/env, hermes setup) updates .env and
the saving process's os.environ, but a live session worker keeps the
base_url/api_key captured at agent init until restart — an open chat
silently kept calling the old endpoint (e.g. a local-server key sent to
api.openai.com, failing with an opaque 401).
Add AIAgent._try_refresh_env_client_credentials(), called at the start
of each conversation turn: re-resolve the provider's env-sourced
credentials (load_env() is mtime-memoized, so an unchanged file costs
one stat()) and rebuild the client via the existing
_replace_primary_openai_client machinery when the user edited them.
The refresh reacts only to env edits — resolved values changed since
the last look — never to mere divergence from the agent's current
values: credential-pool rotation and failover legitimately move the
session off the env credential, and stomping those back would flap.
Config model.base_url / pool custom endpoints keep precedence: edits
are only adopted while the session still runs on the registry default
or the previously-seen env value.
Lift _get_env_prefer_dotenv out of _seed_from_env to module level
(get_env_prefer_dotenv) so both the pool seeder and the per-turn
refresh share the same .env-over-os.environ resolution, including the
op:// indirection handling.
Fixes#67821
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(agent): address sweeper review on env credential refresh
- Cover named custom providers (#67935): provider="custom" has no
PROVIDER_REGISTRY entry, so resolve the config block's key_env through
the same lookup the runtime resolver uses.
- Make the edit baseline transactional: a failed client rebuild rolls the
agent back and leaves _env_creds_seen un-advanced so the unchanged edit
is retried next turn.
- Recompute route-derived TLS material and default headers on a base-url
change, via a _reapply_route_client_config helper shared with
credential-pool rotation so the two paths cannot drift.
- Rebase onto main: get_env_prefer_dotenv keeps the scoped _get_secret
semantics from the profile-isolation fix (no raw os.environ reads).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: map jskang@lablup.com to rapsealk
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
