Route shaping dropped reasoning_details on routes that do not read the array
(Gemini, Vertex, Copilot, custom) before shape_wire_carriers could lift the
private native_assistant record out of it. Every reloaded turn (each gateway
turn, every --resume) lost the signature and flipped the cached prefix. The
private records now survive shaping; the transport still filters them per route.
The field-rejection recorder matched generic phrases ('not allowed', 'is
unsupported'), so a value complaint like 'reasoning_content is not allowed to be
empty' was recorded as a schema rejection and persisted: every later tool turn
then 400'd. Only unknown-field phrasing counts now, a must-echo route never
records reasoning_content, and the recorder runs on chat_completions only.
Also restores the stealth/missingno must-echo pad (f86276d2eb) as a table row.
A strict lane (the Portal's Fireworks :US models) rejects `reasoning` /
`reasoning_details` by name. The in-memory record stopped the 400 for the
rest of the process, but `hermes chat --resume` (or a gateway restart)
re-learned it with another 400 on its first request. The record now lives in
the session's model_config (same seam as the Anthropic rejected-thinking
fingerprints) and is loaded once per session_id. Still keyed by
(provider, host, model), so sibling models keep every carrier.
Every model reasons and intends that reasoning to be replayed, but the
chat-completions wire echoed reasoning_content only to DeepSeek/Kimi/MiMo/
Ollama (plus a hardcoded Portal stealth model) and reasoning_details only to
OpenRouter; turn_context popped `reasoning` everywhere, so Ollama's /v1
(reads only `reasoning`) never saw any.
- message_sanitization owns one route->carrier table
(_REASONING_CARRIER_ROUTES): strict schemas (Mistral, Groq, Cerebras,
SambaNova) get nothing, Fireworks reasoning_content only, OpenRouter-format
gateways (OpenRouter, Kilo, Vercel AI Gateway, Nous Portal) all three,
Novita/MiniMax /v1/TokenHub + reasoning_details, local/Ollama/Upstage +
reasoning, everyone else reasoning_content.
- The " " pad stays limited to the must-echo tier (DeepSeek, Kimi, MiMo);
Ollama leaves it (no echo requirement) and the stealth hardcode is gone.
- Reactive recovery: a 4xx unknown-field rejection that NAMES a reasoning
key this request carried is recorded per (provider, host, model) and the
retry re-shapes api_messages without it. Generic upstream errors never
count; a recorded key falls through, so no loop. History is untouched.
- Nous Portal reads reasoning_details again: the #118182 "replay budget"
does not exist in the Portal and does not reproduce live.
- Gemini thought_signature survives on Gemini hosts/providers even when the
model id is an alias.
- Vendor keep-reasoning flags (docs-only): GLM thinking.clear_thinking=false,
DashScope preserve_thinking / clear_thinking, Kimi K2.x thinking.keep=all.
- stale_thinking_reaches_wire follows the same table, so compaction trigger
and tail walks agree with what is sent.
Local Ollama exposes cloud models both as "<model>:cloud" and as
"<model>:<size>-cloud" (gpt-oss:120b-cloud). Treat the second form like
the first, requiring a ":" so provider ids such as "gmi-cloud" do not
match. Update the require-side comment and docstrings that still listed
only DeepSeek/Kimi/MiMo or claimed every member 400s without the field.
Ollama Cloud users reported reasoning being sent back wrong. Since #50480
the replay policy strips reasoning_content for every provider outside the
DeepSeek/Kimi/MiMo echo families, so on ollama-cloud an assistant tool-call
turn went back with no reasoning at all. Ollama's /v1 maps the field onto
the chat template's thinking input, so thinking models (kimi-k3, glm-5.3)
lost their own chain of thought between tool calls.
Add an "ollama" echo family (provider ollama-cloud, host ollama.com, or a
local server's ":cloud" model). Purely local Ollama models stay on the
strip side.
agent_runtime_helpers (two sites) and message_sanitization imported
_INTERRUPTED_PLACEHOLDER inside the function body. The placeholders
module is a leaf with no imports, so there is no cycle to dodge; import
it once at module level (each module still imports cleanly in a fresh
interpreter). conversation_loop keeps its local import.
Gate rP finding 7 (quality).
message_sanitization.close_interrupted_tool_sequence and the
turn_api_call repetition-loop exit each spelled the same hidden
assistant row literal (content "", display_kind hidden, api_content
_INTERRUPTED_PLACEHOLDER), and the conversation loop's redirect built it
field by field. Add hidden_interrupt_placeholder_row() to the leaf
placeholders module and use it at all three sites so the shape cannot
drift between them.
Gate rP finding 6 (reuse).
Code-health ratchet: agent/agent_runtime_helpers.py was 3787 > 3776 and
tests/agent/test_run_agent.py 7014 > 7012 (both over the 2,000 target, so
they may only shrink vs main).
- _INTERRUPTED_PLACEHOLDER / _LEGACY_INTERRUPTED_PLACEHOLDER (and their
rationale comment) move byte-identically into the leaf sibling
agent/agent_runtime_helpers_placeholders.py. Every importer (runtime
helpers via late import, conversation_loop, message_sanitization,
turn_api_call, turn_iteration_prep and five test files) now reads the
sibling; no shim is left in the facade. Nothing patches the old path.
- test_keyboard_interrupt_emits_cancelled_post_tool_hook moves unchanged
into tests/agent/test_run_agent_interrupt_hook.py, reusing the facade's
agent fixture.
Fold finding 4 (B02 gate, low): behaviour-neutral cleanups.
- close_interrupted_tool_sequence: the hidden branch only runs when the
stripped text is empty or already the placeholder, so `stripped or
_INTERRUPTED_PLACEHOLDER` is always the placeholder; say so.
- tool_executor: compute interrupt_skip_wording once for the cancelled
outcome instead of twice.
- move _neutralise_replay_echo_ghosts out of prepare_iteration to module
level (lazy imports kept inside to avoid the conversation_loop cycle).
- the finalizer test asserts the exact placeholder, not just non-empty.
close_interrupted_tool_sequence hid every synthetic closing turn behind
display_kind="hidden", but four of its five callers pass a real user-facing
notice — the truncation terminal ("Response truncated — …"), the context-
overflow partial text, tool-validation partials and abort_turn_on_interrupt's
"Operation interrupted: …" lines. Hiding those removed the turn's only
explanation from every transcript surface.
Narrow the hidden shape to what it was for: an empty final_response or the
bare interrupt placeholder (which previously rendered as the "Operation
interrupted." bubble users never asked for). Any other text is rendered
verbatim as before. Alternation safety is unchanged in both branches.
(cherry picked from commit a637af2ef44c1f0213c3138c458d6f6c48650988)
close_interrupted_tool_sequence() appends a synthetic assistant turn when an
interrupted tail is a raw tool result, so the next user message does not land as
tool -> user. The row carried a user-visible "Operation interrupted." string and
was persisted as an ordinary assistant message, so background preemption —
session reaping, watchdogs, batch guards — surfaced in the transcript as an
interruption the user was expected to act on.
Store it in the same hidden shape turn_api_call.py already uses for interrupt
placeholders: content="" plus display_kind="hidden" keeps it out of rendered
transcripts, while the api_content sidecar carries the LLM-visible text and is
substituted at API-build time. The alternation guarantee is unchanged.
(cherry picked from commit 22502b1c40c26a08eefc3d35cee067bd1d889bfc)
transports.types.ToolCall exposes call_id as a read-only view of provider_data,
so setattr raised AttributeError in validate_tool_calls for Responses-transport
batches. Write the backing value instead so id and call_id stay in agreement,
and cover both real transports end to end.
Refs #130363
Gate provider-id normalization on the effective id that serialization and
result pairing use (stripped; blank call_id falls back to id), and re-run it
after dedup/capping so a batch that becomes all-provider-prefixed only after
filtering is still normalized before it is staged. Idempotent; mint time only.
Refs #130363
Rewrite the call half of id and call_id independently so a composite id keeps
its response-item half when call_id is bare, and hash with surrogatepass so a
lone surrogate neither raises in validate_tool_calls nor collapses distinct ids.
Adds an end-to-end test through validate_tool_calls (real duplicate repair) and
assistant-row serialization.
Refs #130363
native_anthropic_accounting_projection rebuilt assistant rows from content only, so on
canonical input (preflight, tail walk) a promoted reply with content="" and an
api_content sidecar priced nothing for text that build_api_messages substitutes and
sends. The projection now lets a
non-empty string sidecar win over content, matching the generic wire shadow on main.
Post-build input is unchanged (the sidecar is already in content); ordered turns and
context-selection clones may overcount, never undercount.
The accounting invariant gains an assistant_api_content case that measures replayed
assistant text on the real default assembly: red before (preflight/tail_walk 0 vs
1999), green after.
native_anthropic_accounting_projection (added in #129620, bd3a75f, JoaoMarcos44,
2026-10-01) re-derived carrier precedence on its own: any ordered block whose type
was replayable made the ordered sidecar authoritative. The converter instead uses the
ordered sidecar only when a block survives _sanitize_replay_block, and otherwise
falls back to reasoning_details. With a dataless redacted_thinking sidecar plus valid
signed reasoning_details, the wire carried the details thinking while accounting
charged none of it (and dropped reasoning_content as if a carrier existed).
Extract assistant_replay_carrier() from _convert_assistant_message and use it for
the projection, so conversion and accounting share one precedence rule.
One Skip broke a tester's whole onboarding. The provider named every
tool call "call_0", and the desktop bound each setup card to its
request by call id alone. After the apps card was skipped, every later
card with "call_0" showed that old "Skipped" answer instead of its
picker. Composer text was also saved as a pick, so "hey" became a
plugin and a complaint became the layout.
- The agent renames a tool call whose id is already used earlier in
the session (call_0 -> call_0_d2), before the call is stored or run.
Ids already in history never change, so the prompt cache holds.
- The card's request binding is keyed by session, message and call id.
- Composer text on a card with fixed rows counts as a pick only when it
names a row by id or label. Otherwise setup_choose returns
outcome "typed" with the words in `said`, records no card facts, and
tells the model to reply and send the card again. The settled card
shows the typed words.
A legacy (no-digest) dict over a non-blank assistant row adopted the whole
decoded DB row: tool_calls / reasoning* / codex_* were overwritten with the
stored JSON (which still holds the escaped lone surrogate the sanitizer just
fixed, re-injecting it into the provider payload) and live-only fields were
popped. Resumed dicts (_rows_to_conversation stamps _row_id without a
digest) and compaction clones hit this path. Adopt content only, as before
this stack, via a content-only canonical handled like the metadata-only one.
_insert_message_rows dropped a clone's parent digest but only the flush
path restamped it, so clones made by archive_and_compact / replace /
rotation handoff / import reached the legacy path and the first live edit
after a clone was not persisted. Stamp the stored-row digest inside
_insert_message_rows (one batched SELECT, cold paths only; the flush path
statement count is unchanged) and drop the duplicate call in
append_messages_batch.
Define the _db_row_snapshot / _canonical_row keys once in
agent/message_metadata.py and import them everywhere instead of repeating
the literals.
_sanitize_surrogates ran the surrogate regex over every string leaf of the
outbound messages and kwargs on each request. Surrogates are never ASCII and
str.isascii() is an O(1) flag check, so gate the scan on it; same gate on
_strip_non_ascii. 50 KB ASCII leaf: 1017 us -> 0.1 us; output unchanged.
Partial salvage of #83839: kept the isascii fast-path idea as a single gate
at the top of _sanitize_surrogates (main refactored to a generic fix-callable
walk, so the PR's per-site hunks no longer apply); dropped the monkeypatch
"never invokes regex" tests because they are change-detectors.
(cherry picked from commit 36d2575a12e2ffae1a3286208f269fb3d92acbf5)
The image-rejection recovery no longer switches the session to
text-only or strips history; it records the (provider, model) and
build_api_request strips images from that model's requests only. Three
places still described the old behaviour:
- recover_before_classification docstring and the adjacent comment said
"switch session to text-only" / "mark session vision-unsupported".
- TestStripImagesDropsStaleApiContent's rationale claimed the strip runs
on persistent history and that leaving api_content would replay
rejected images every turn because the recovery gates on
_image_rejecting_models — false on both counts: current callers pass
per-call clones. Reworded as the generic helper contract (a rewritten
persisted row must drop its sidecar) with the no-op note.
- _strip_images_from_messages docstring, same fix.
Comments and docstrings only; no code change.
Two branches in turn_recovery.py carried the same isinstance +
_strip_images_from_messages guard and the same "Provider rejected a
corrupted image" notice: the new pre-classification corrupt branch and
the FailoverReason.image_corrupt branch. Both now call one module
helper, _strip_request_images_and_retry(agent, api_messages) -> bool,
so the strip-this-attempt-only policy lives in one place.
_IMAGE_REJECTION_PHRASES was rebound as unsupported + corrupt, which
made _looks_like_image_content_rejection silently cover corrupt payloads
and forced the recovery to test the corrupt list a second time to undo
that. The name stays (external references) but is now the
unsupported-only tuple; _IMAGE_CORRUPT_PHRASES is disjoint, and the
recovery asks the two questions explicitly:
`_corrupt or (model not yet rejected and unsupported)`.
Tests: the phrase-isolation matcher asks the same disjoint question the
recovery does; the image_corrupt source-contract check looks for the
helper call instead of the inlined strip.
init_agent seeds `_image_rejecting_models = set()` at the same site as
`_force_ascii_payload`, which the adjacent sanitize_outbound_kwargs already
reads as a plain attribute. The getattr/isinstance guards and the lazy
re-create in recover_before_classification implied the attribute could be
missing or mistyped on a real agent; it cannot, and defensive fallbacks for
impossible states hide wiring bugs instead of surfacing them. Every test
fixture that reaches these paths seeds the attribute already.
message_sanitization.image_model_key duplicated vision_message_prep's
_provider_model_key — the (provider, model) key the same mixin already uses
for its per-model vision bookkeeping. Two keying functions for the same
concept drift: one normalised the provider (.strip().lower()), the other did
not, so a provider spelled "OpenAI" in one place and "openai" in another
would have been tracked as two models. Key `_image_rejecting_models` on the
existing helper and delete the duplicate (and its __all__ entry).
No import cycle: vision_message_prep imports only lazy_forward,
tool_dispatch_helpers and utils, none of which import message_sanitization
or turn_recovery.
recover_before_classification matches _IMAGE_REJECTION_PHRASES, which
mixes two kinds of body: capability rejections ("does not support
images", "only text content type is supported", ...) and bad-payload
rejections. After the per-model tracking landed, BOTH added the
(provider, model) to agent._image_rejecting_models, so one truncated
screenshot rejected with "failed to decode image" made every later
request to that model text-only for the rest of the session, even
though the model can see fine.
Split the bad-payload phrases into _IMAGE_CORRUPT_PHRASES:
- "image data you provided does not represent a valid image"
(ChatGPT-account Codex backend)
- "failed to decode image" (Kimi / Moonshot and other
OpenAI-compatible providers)
_IMAGE_REJECTION_PHRASES stays the union so the turn still recovers on
them. For a corrupt match the recovery now strips the current attempt
only and retries iff something was stripped (mirroring the existing
image_corrupt branch), and leaves the model unmarked; only a capability
rejection records the model.
One test: a "failed to decode image" body strips the wire copy, keeps
history, and leaves _image_rejecting_models empty so the next request
to the same model carries its images.
The first head stored a single rejecting (provider, model) and kept the
turn-global `_vision_supported` as the recovery guard. In a fallback
chain that fails: model A rejects images and retries text-only, a later
error activates model B, the restart rebuilds api_messages from history
so B receives the images, and when B rejects them too the branch is
skipped because `_vision_supported` is already False — the request
falls through to generic error handling. Recording B also overwrote A,
so A was no longer treated as text-only on later turns.
`_image_rejecting_models` is now a set of every rejecting model, and it
is also the guard: each model's first rejection runs the recovery and a
repeat rejection from the same model still falls through, so the retry
cannot loop. image_model_key() names the key in one place.
Adds a test for the two-model sequence (fails on the previous head) and
one pinning that a repeat rejection from the same model does not retry.
Thanks to @ehz0ah for the review.
(cherry picked from commit 225fd76ccf90cd3ec509a9327f725ac02d996f24)
When a provider 4xx's on image content, recover_before_classification
ran _strip_images_from_messages on the canonical `messages` list and
reset _db_flush_scan_prefix. Since #117569 that function pops
_db_persisted on every rewritten dict, so the next flush rewrote those
rows: every image in the session — and every image-only message, which
the stripper deletes outright — was removed from state.db for good.
The rejection describes what the CURRENT model accepts, not what the
conversation holds. An automatic fallback to a text-only provider, or a
single /model switch, was enough to erase images the user had sent to a
vision model, and switching back found them gone. It is the same failure
as the ASCII strip in #117802, on the image path; neither open fix for
that issue touches this branch.
Keep the repair on the send path, where the per-call copy already lives
(_clone_message_for_send exists so send-path rewrites never reach the
persisted transcript, #80498):
- record the rejecting (provider, model) on the agent, a session-scoped
flag initialised beside _force_ascii_payload;
- strip the in-flight api_messages copy for the immediate retry;
- build_api_request calls strip_images_for_rejecting_model() on each
attempt's api_messages BEFORE provider conversion. The stripper knows
Hermes's own part types; a converted payload would slip past it
(Bedrock Converse image blocks carry no `type`). Keyed on the model,
so one that accepts images gets them again.
_strip_images_from_messages itself is unchanged, so its role-alternation
and sidecar guarantees still hold on the wire copy. The notice no longer
claims "text-only mode for this session" (_vision_supported resets every
turn) or that images were stripped from history.
(cherry picked from commit cbdb184c27f915ab138b2087f878aed7fcc7c76b)
sanitize_outbound_kwargs detached the agent.tools alias with copy.deepcopy
before the ASCII strip. The repo's send-path detach idiom is the structural
clone _clone_message_for_send (dicts/lists recursively, immutable leaves
shared), which is what every other outbound copy uses and is cheaper on
JSON-shaped, acyclic payloads. It is sufficient here because
_sanitize_structure only rebinds str leaves inside dict/list containers, so
the clone fully isolates the canonical tool schemas. Imported lazily inside
the function (conversation_loop imports this module) exactly as
turn_finalizer does. Drops the now-unused `import copy`.
api_kwargs["tools"] is rebuilt from agent.tools on every attempt
(_build_api_kwargs_for_mode: tools_for_api = agent.tools; transports set
api_kwargs["tools"] = tools without copying), so the list usually aliases
the canonical tool schemas. The ASCII retry path then runs
sanitize_outbound_kwargs with _force_ascii_payload set, and its in-place
strip rewrote agent.tools for the rest of the session.
Move the guard to the chokepoint: when the flag is set and tools IS
agent.tools, deepcopy before stripping. The deepcopy the contributor pick
added inside _recover_unicode_encode_error only protected the failed
request's kwargs, which are discarded before the retry; drop it and have
recovery skip an aliased tools list entirely (request-local lists are
still stripped for the diagnostic message).
The kept test now exercises the chokepoint directly: with the flag set on
kwargs whose tools aliases agent.tools, agent.tools must be byte-stable
afterwards. Verified red against the previous chokepoint.
The _db_persisted marker asserts that a message dict's persisted row is
durable as written; any in-place mutation must pop it or the flush scan
identity-skips the dict and state.db keeps the stale row forever. Six
mutation sites violated the contract:
- micro_compaction._merge_adjacent_user_turns rewrote content on a
carried-forward dict after superseding a stale micro marker. On the
archive-failure path nothing re-stamps, so the merged text never
reached state.db.
- repair_message_sequence passes mutated stamped survivors in place:
_merge_assistant_into (tool_calls union, content join,
reasoning_content carry), _prune_unanswered_tool_calls (tool_calls
rewrite), _merge_consecutive_users (content join).
- sanitize_tool_call_arguments rewrote corrupted/blank
function.arguments and prepended the corruption marker onto stamped
resumed rows, leaving the corrupt bytes durable and self-perpetuating
across resumes.
- _sanitize_messages (surrogate and non-ASCII recovery) and
_strip_images_from_messages (image-rejection recovery) rewrote live
dicts on the recovery path.
Each site now pops the marker when a persisted field actually changes,
and the agent-aware callers (repair_message_sequence_with_cursor,
turn_iteration_prep, turn_recovery) invalidate the bounded flush-scan
prefix so repaired rows are rewritten on the next flush.
Balanced-but-misnested argument JSON ({"a": [{"b": 1}, {"c": 2}}]}, the reporter's
deepseek-v4-flash shape where the "]" of an array of objects is dropped and the
neighbouring "}" closes in its place) has equal delimiter counts, so the closer
arithmetic had nothing to append and the call degraded to "{}" — silently losing
the tool call. The string-aware scan from #115096 now rewrites the text: a closer
that matches a deeper opener gets the missing inner closers inserted in front of
it, and the remaining stack is appended in order. Anything ending inside a string
still returns "{}" (unrecoverable content is never guessed).
Tests trimmed to invariants: string-aware balance and stack-order close (from
#115096), the five reporter shapes, and the streaming assembler seam
(_StreamingCall._assemble_tool_calls) repairing instead of flagging truncation,
with the mid-string-cut control still flagged.
Part of #115061 (JSON-repair half; the plugin button-callback half is a
separate design change the reporter agreed to file standalone).
Truncated tool-call arguments whose string values contain braces or brackets
("}", "if (x) {") were miscounted by the naive count-based repair: the
delimiter inside the value either cancelled a real unclosed brace or demanded
one closer too many, and the candidate never parsed. Closers were also
appended grouped (all } before all ]) instead of in stack order, so a
truncated array of objects ({"items": [{"n": 1}, {"n": 2) could not be
closed at all. Scan the prefix string-aware and append the closers the open
stack actually needs.
A tool call whose function.name violates the provider pattern
^[A-Za-z0-9_-]{1,64}$ — OpenAI's synthetic `multi_tool_use.parallel`, or a
whole shell command a weak fallback model put into `name` (372 chars) — is
persisted once and then 400s every later request on a strict endpoint, so
the session silently pins itself to the lenient fallback model (#51944).
`agent/message_sanitization.py::coerce_tool_name` is now the single owner of
the coercion (valid → identity, invalid runs → `_`, cut at 64, empty →
fallback); the Codex Responses adapter uses it in place of its private copy,
and the pre-call sanitizer's nameless-call repair becomes
`_repair_invalid_tool_call_names`, so both outbound builders that already
call `sanitize_api_messages` — the main loop (turn_request_assembly) and the
iteration-limit summary (chat_completion_helpers) — send valid names. Dict
tool calls are rewritten copy-on-write, so the summary path's shallow message
copy never edits persisted history; tool results follow through
`_realign_tool_result_names`. Deterministic, so identical stored bytes always
render identical wire bytes (prompt-cache prefix stays stable).
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
The summary path had grown a verbatim copy of turn_api_request's 3-line
surrogate/ASCII chokepoint — the same drift class this PR removes for the
hand-rolled kwargs builder. Move the two lines and the #50959 rationale into
`message_sanitization.sanitize_outbound_kwargs` and call it from both sites,
so the next sanitizer step added to the main loop cannot miss the summary.
Tighten two comments: "same kwargs builder" (cache_control redecoration is
not re-applied here) and a `_summary_text` note that is true for all three
summary branches, not just the chat one.
Some OpenAI-compatible gateways fronting Gemini backends emit the native
uppercase finish reasons (STOP, MAX_TOKENS) instead of the lowercase
OpenAI contract values. Every downstream comparison in Hermes uses
lowercase literals, so an uppercase reason silently fell through: a
clean STOP completion missed the stop handling and a MAX_TOKENS
truncation never entered the length-recovery path.
Adds normalize_finish_reason() as the single owner in
agent/message_sanitization.py (case fold + alias map: max_tokens->length,
end->stop, function_call->tool_calls) and wires it at both wire-intake
choke points: ChatCompletionsTransport.normalize_response and the
streaming chunk-capture loop in chat_completion_helpers. Non-string and
empty values pass through unchanged so existing 'or "stop"' defaults
and the Poolside int-reason path keep their behavior.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
