hermes-admin/hermes-agent
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

fix: replay prior reasoning to every chat-completions route that can read it

Browse Source 
Every model reasons and intends that reasoning to be replayed, but the
chat-completions wire echoed reasoning_content only to DeepSeek/Kimi/MiMo/
Ollama (plus a hardcoded Portal stealth model) and reasoning_details only to
OpenRouter; turn_context popped `reasoning` everywhere, so Ollama's /v1
(reads only `reasoning`) never saw any.

- message_sanitization owns one route->carrier table
  (_REASONING_CARRIER_ROUTES): strict schemas (Mistral, Groq, Cerebras,
  SambaNova) get nothing, Fireworks reasoning_content only, OpenRouter-format
  gateways (OpenRouter, Kilo, Vercel AI Gateway, Nous Portal) all three,
  Novita/MiniMax /v1/TokenHub + reasoning_details, local/Ollama/Upstage +
  reasoning, everyone else reasoning_content.
- The " " pad stays limited to the must-echo tier (DeepSeek, Kimi, MiMo);
  Ollama leaves it (no echo requirement) and the stealth hardcode is gone.
- Reactive recovery: a 4xx unknown-field rejection that NAMES a reasoning
  key this request carried is recorded per (provider, host, model) and the
  retry re-shapes api_messages without it. Generic upstream errors never
  count; a recorded key falls through, so no loop. History is untouched.
- Nous Portal reads reasoning_details again: the #118182 "replay budget"
  does not exist in the Portal and does not reproduce live.
- Gemini thought_signature survives on Gemini hosts/providers even when the
  model id is an alias.
- Vendor keep-reasoning flags (docs-only): GLM thinking.clear_thinking=false,
  DashScope preserve_thinking / clear_thinking, Kimi K2.x thinking.keep=all.
- stale_thinking_reaches_wire follows the same table, so compaction trigger
  and tail walks agree with what is sent.
This commit is contained in:
Teknium
2026-10-10 13:28:14 -07:00
parent c57331677d
commit 336b11584c
23 changed files with 562 additions and 213 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
agent/agent_init.py
View File

@@ -2449,9 +2449,9 @@ def init_agent(
agent.request_overrides = dict(request_overrides or {})
agent.prefill_messages = prefill_messages or [] # Prefilled conversation turns
agent._force_ascii_payload = False
# Every (provider, model) that rejected image content this session. build_api_request strips
# images from requests to those models only, so history keeps them for any model that can see.
# Routes that rejected image content / named reasoning keys; build_api_request strips those from wire copies only.
agent._image_rejecting_models = set()
agent._reasoning_rejecting_routes = {} # (provider, host, model) -> rejected reasoning keys
# Models whose Anthropic organization answered a fast request with a fast-mode limit of 0;
# agent.fast_mode stops sending ``speed`` to them for the rest of the session.
agent._fast_mode_unavailable_models = set()

10
agent/agent_runtime_helpers.py
View File

@@ -3497,14 +3497,14 @@ def intent_ack_continuation_mode(agent) -> str:
def copy_reasoning_content_for_api(agent, source_msg: dict, api_msg: dict) -> None:
"""Forward reasoning fields onto an API replay message; policy lives in ``agent.message_sanitization.apply_reasoning_content_policy``."""
from agent.message_sanitization import apply_reasoning_content_policy
apply_reasoning_content_policy(source_msg, api_msg, agent._needs_thinking_reasoning_pad())
apply_reasoning_content_policy(source_msg, api_msg, *agent._reasoning_replay_route())
def reapply_reasoning_echo_for_provider(agent, api_messages: list) -> int:
"""Re-pad or strip assistant turns' reasoning_content for the CURRENT provider after a
fallback switch: ``api_messages`` is shaped for the primary; require-side providers
"""Reconcile assistant turns' reasoning keys with the CURRENT route after a fallback switch or
a recorded field rejection: ``api_messages`` is shaped for the primary; must-echo providers
(DeepSeek/Kimi/MiMo) 400 without the pad, strict ones (Mistral, Cerebras, Groq) 400/422
with it. Idempotent; returns the number of assistant turns changed.
with any reasoning key. Idempotent; returns the number of assistant turns changed.
* Switching TO a strict provider that rejects the field (Mistral, Cerebras, Groq, SambaNova, …):
assistant turns built under a reasoning primary carry a ``reasoning_content`` pad (often a single space
@@ -3513,7 +3513,7 @@ def reapply_reasoning_echo_for_provider(agent, api_messages: list) -> int:
request falls back to Mistral, and Mistral 422s on the stale pad.
"""
from agent.message_sanitization import reapply_reasoning_echo
return reapply_reasoning_echo(api_messages, agent._needs_thinking_reasoning_pad())
return reapply_reasoning_echo(api_messages, *agent._reasoning_replay_route())
def _iter_httpx_pools_with_owner(http_client: Any):

310
agent/message_sanitization.py
View File

@@ -12,7 +12,7 @@ import json
import logging
import re
from functools import partial
from typing import Any, Callable, Iterable
from typing import Any, Callable, Iterable, NamedTuple
from agent.agent_runtime_helpers_placeholders import _INTERRUPTED_PLACEHOLDER, hidden_interrupt_placeholder_row
from agent.message_metadata import DB_ROW_SNAPSHOT
@@ -466,6 +466,7 @@ __all__ = [
"close_interrupted_tool_sequence", "coalesce_tool_call_id", "coerce_tool_name",
"deterministic_call_id", "matches_reasoning_echo_family", "needs_reasoning_echo",
"normalize_provider_tool_call_ids", "reapply_reasoning_echo", "reasoning_echo_family",
"reasoning_replay_route", "record_reasoning_field_rejection", "route_reasoning_carriers",
"sanitize_outbound_kwargs", "stale_thinking_reaches_wire", "strip_images_for_rejecting_model",
"tool_call_id_variants", "tool_result_id_variants", "uniquify_tool_call_ids",
]
@@ -624,36 +625,59 @@ def _set_provider_tool_id(tc: Any, key: str, value: str) -> None:
_tc_set(tc, key, value)
# -- reasoning_content policy: single owner of strip-vs-re-pad; adapters keep only SYNTAX --
# Require side (echo-back enforced or consumed by the template): the families below. Kimi
# is host-driven on purpose (aggregators re-exporting kimi reject it); DeepSeek V4 rejects
# empty-string pads → " ". Strict side (400/422 "Extra inputs are not permitted"): everyone
# else — Mistral, Cerebras, Groq, SambaNova, … Strip the key entirely, even a one-space pad.
# --------------------------------------------------------------------------- reasoning_content policy —
# single owner (audit F4) --------------------------------------------------------------------------- The
# strip-vs-repad decision was previously forked across the wire files in separate incident commits
# (2b3a4f0af8 strip for strict providers, b5495db701 re-pad for require-side, 94b3131be7/9a9f8a6d99 kimi
# pad). The POLICY — which provider direction gets which treatment — lives here as one rule table + apply
# functions; adapters keep only SYNTAX mapping (e.g. anthropic_adapter turning reasoning_content into a
# thinking block). Direction table: require-side (echo-back enforced, or consumed by the template): kimi
# — provider kimi-coding/kimi-coding-cn, or host api.kimi.com / moonshot.ai / moonshot.cn. Host-driven on
# purpose: aggregators re-exporting kimi models reject the echo. deepseek — provider "deepseek", model
# contains "deepseek", or host api.deepseek.com (#15250; V4 rejects empty-string pads, hence the " "
# single-space pad, #17341). mimo — provider "xiaomi", model contains "mimo", or host *.xiaomimimo.com.
# ollama — provider "ollama-cloud", host ollama.com, or a local ":cloud"/"-cloud" tag (Ollama's /v1 feeds the
# field back into the template's thinking slot; stripping it drops the model's own CoT mid tool loop).
# strict side (field rejected with 400/422 "Extra inputs are not permitted"): everyone else — Mistral,
# Cerebras, Groq, SambaNova, … (#45655). Strip the key entirely, even a single-space pad.
# -- reasoning replay policy: single owner; adapters keep only SYNTAX ----------------------
# Every model reasons and intends that reasoning to be replayed, so every chat-completions
# route gets its stored reasoning back BY DEFAULT, on the carrier(s) that route reads. Two
# tables own the decision:
# * ``_REASONING_ECHO_RULES`` — the MUST-ECHO tier: a tool-call turn without a non-empty
# ``reasoning_content`` is an HTTP 400 there, so a missing value is padded with " " (never
# "": DeepSeek V4 rejects it, #17341). Kimi is host-driven on purpose (aggregators
# re-exporting kimi do not need the pad).
# * ``_REASONING_CARRIER_ROUTES`` — which carriers a route reads (first match wins). Strict
# hosts come first: their schemas reject unknown message keys with 400/422 ("Extra inputs
# are not permitted" / "property X is unsupported" / "no such field", #45655, #70233).
# Every other route defaults to ``reasoning_content``, which is the de-facto standard.
# A route that still rejects a field is learned at runtime (``record_reasoning_field_rejection``)
# and the field is dropped for that (provider, host, model) only. Stored history keeps every
# carrier; only wire copies are shaped here.
_REASONING_ECHO_RULES: tuple = (
# (family, exact providers (raw), exact providers (lowered), model substrings (lowered), hosts)
("kimi", frozenset({"kimi-coding", "kimi-coding-cn"}), frozenset(), (), ("api.kimi.com", "moonshot.ai", "moonshot.cn")),
("deepseek", frozenset(), frozenset({"deepseek"}), ("deepseek",), ("api.deepseek.com",)),
("mimo", frozenset(), frozenset({"xiaomi"}), ("mimo",), ("api.xiaomimimo.com", "xiaomimimo.com")),
("ollama", frozenset(), frozenset({"ollama-cloud"}), (":cloud",), ("ollama.com",)),
)
_REASONING_ECHO_RULE_BY_FAMILY = {rule[0]: rule for rule in _REASONING_ECHO_RULES}
RC, R, RD = "reasoning_content", "reasoning", "reasoning_details"
REASONING_CARRIERS = (RC, R, RD)
_ALL_CARRIERS = frozenset(REASONING_CARRIERS)
_DEFAULT_CARRIERS = frozenset({RC})
_LOCAL_CARRIERS = frozenset({RC, R}) # llama.cpp/SGLang read reasoning_content, vLLM/Ollama read reasoning
_REASONING_CARRIER_ROUTES: tuple = (
# (carriers, providers (lowered), hosts) — evidence per row in the PR body / FINDINGS.
(frozenset(), frozenset({"mistral", "groq", "cerebras", "sambanova"}),
("mistral.ai", "groq.com", "cerebras.ai", "sambanova.ai", "hunyuan.cloud.tencent.com")),
# Fireworks' ChatMessage schema is additionalProperties:false with reasoning_content only.
(frozenset({RC}), frozenset({"fireworks"}), ("fireworks.ai",)),
# OpenRouter-format gateways read the unified array, the string and the alias.
(_ALL_CARRIERS, frozenset({"openrouter", "kilocode", "ai-gateway", "nous"}),
("openrouter.ai", "kilo.ai", "ai-gateway.vercel.sh", "nousresearch.com")),
# Vendors that document reasoning_details replay next to reasoning_content.
(frozenset({RC, RD}), frozenset({"novita", "tencent-tokenhub", "minimax", "minimax-cn"}),
("novita.ai", "tokenhub.tencentmaas.com", "tokenhub-intl.tencentcloudmaas.com", "minimax.io", "minimaxi.com")),
(_LOCAL_CARRIERS, frozenset({"ollama-cloud", "custom", "lmstudio", "upstage"}), ("ollama.com", "upstage.ai")),
)
class ReasoningReplayRoute(NamedTuple):
"""``pad``: must-echo tier (missing reasoning_content -> " "). ``carriers``: the reasoning
keys an assistant wire copy carries; ``None`` outside chat_completions, where the native
adapters own reasoning replay (only the must-echo pad is applied, details are untouched)."""
pad: bool
carriers: frozenset | None
def matches_reasoning_echo_family(family: str, provider: Any, model: Any, base_url: Any) -> bool:
"""True when (provider, model, base_url) matches one echo-back family (families can overlap;
@@ -662,8 +686,6 @@ def matches_reasoning_echo_family(family: str, provider: Any, model: Any, base_u
_, raw_providers, lowered_providers, model_subs, hosts = _REASONING_ECHO_RULE_BY_FAMILY[family]
model_lower = (model or "").lower()
if ":" in model_lower and model_lower.endswith("-cloud"):
model_lower += ":cloud" # Ollama's "<size>-cloud" tag (gpt-oss:120b-cloud) == ":cloud"
return (
provider in raw_providers or (provider or "").lower() in lowered_providers
or any(sub in model_lower for sub in model_subs) or any(base_url_host_matches(base_url, host) for host in hosts)
@@ -671,17 +693,59 @@ def matches_reasoning_echo_family(family: str, provider: Any, model: Any, base_u
def reasoning_echo_family(provider: Any, model: Any, base_url: Any) -> str | None:
"""``"kimi"`` / ``"deepseek"`` / ``"mimo"`` / ``"ollama"`` (first match in table order) when the
endpoint enforces reasoning_content echo-back, else ``None`` (strip side)."""
"""``"kimi"`` / ``"deepseek"`` / ``"mimo"`` (first match in table order) when the endpoint
enforces reasoning_content echo-back on tool-call turns, else ``None``."""
families = (rule[0] for rule in _REASONING_ECHO_RULES)
return next((f for f in families if matches_reasoning_echo_family(f, provider, model, base_url)), None)
def needs_reasoning_echo(provider: Any, model: Any, base_url: Any) -> bool:
"""True when the endpoint requires reasoning_content echo-back."""
"""True when the endpoint requires reasoning_content echo-back (the must-echo tier)."""
return reasoning_echo_family(provider, model, base_url) is not None
def route_reasoning_carriers(provider: Any, base_url: Any) -> frozenset:
"""Reasoning carriers the chat-completions route reads (before any runtime rejection)."""
from utils import base_url_host_matches
provider_lower = (provider or "").strip().lower()
carriers = next(
(c for c, providers, hosts in _REASONING_CARRIER_ROUTES
if provider_lower in providers or any(base_url_host_matches(base_url, h) for h in hosts)),
None,
)
if carriers is None:
from agent.model_metadata import is_local_endpoint
carriers = _LOCAL_CARRIERS if base_url and is_local_endpoint(str(base_url)) else _DEFAULT_CARRIERS
if RD not in carriers and carriers and _profile_declares_native_details(provider_lower):
carriers = carriers | {RD}
return carriers
def _profile_declares_native_details(provider_lower: str) -> bool:
"""A profile declaring ``native_reasoning_details_type`` consumes replayed details by contract."""
if not provider_lower:
return False
from providers import get_provider_profile
return bool(getattr(get_provider_profile(provider_lower), "native_reasoning_details_type", None))
def reasoning_replay_route(
api_mode: Any, provider: Any, model: Any, base_url: Any, *, echo_opt_in: bool = False,
rejected: Iterable[str] = (),
) -> ReasoningReplayRoute:
"""Replay decision for one route. Deterministic in its inputs, so the wire prefix is byte-stable
across turns and changes only on a route switch or a recorded rejection (``rejected``)."""
pad = bool(echo_opt_in) or needs_reasoning_echo(provider, model, base_url)
if (api_mode or "chat_completions") != "chat_completions":
return ReasoningReplayRoute(pad, None)
carriers = route_reasoning_carriers(provider, base_url) - frozenset(rejected)
# A strict route never gets a reasoning key, not even the must-echo pad.
return ReasoningReplayRoute(pad and RC in carriers, carriers)
def stale_thinking_reaches_wire(api_mode: Any, provider: Any, model: Any, base_url: Any) -> bool:
"""True when stale assistant reasoning text is actually replayed on the wire for the route.
@@ -689,12 +753,19 @@ def stale_thinking_reaches_wire(api_mode: Any, provider: Any, model: Any, base_u
walks must share: if they disagree, a reasoning-heavy session can look over-threshold
to preflight yet fully tail-protected to the walk — an infinite compaction loop.
``codex_responses`` never reads the text keys (continuity rides the encrypted sidecar).
A runtime rejection is not visible here; both sides then over-charge identically.
"""
if (api_mode or "") == "anthropic_messages":
from agent.anthropic_thinking_policy import native_anthropic_preserves_prior_thinking
if native_anthropic_preserves_prior_thinking(base_url, model):
return True
return (api_mode or "") != "codex_responses" and needs_reasoning_echo(provider, model, base_url)
if not api_mode:
# No route facts (a bare compressor): only the must-echo tier is known to replay.
return needs_reasoning_echo(provider, model, base_url)
route = reasoning_replay_route(api_mode, provider, model, base_url)
if route.carriers is None:
return (api_mode or "") != "codex_responses" and route.pad
return bool(route.carriers & {RC, R})
def native_anthropic_accounting_projection(messages: Any) -> tuple[Any, tuple[str, ...]]:
@@ -749,72 +820,147 @@ def native_anthropic_accounting_projection(messages: Any) -> tuple[Any, tuple[st
return projected, tuple(replayed_thinking)
def apply_reasoning_content_policy(source_msg: dict, api_msg: dict, needs_thinking_pad: bool) -> None:
"""Copy provider-facing reasoning fields onto an API replay message (mutates ``api_msg``).
``needs_thinking_pad`` is the require-side flag (``needs_reasoning_echo``)."""
def _replayable_reasoning_text(msg: dict) -> str | None:
"""The turn's reasoning text: a non-blank ``reasoning_content`` wins over ``reasoning``."""
for key in (RC, R):
value = msg.get(key)
if isinstance(value, str) and value.strip():
return value
return None
def _apply_legacy_must_echo_pad(source_msg: dict, api_msg: dict) -> None:
"""Native-adapter wires (``carriers=None``): the pre-replay contract, byte-for-byte. An explicit
value is kept verbatim (legacy "" upgraded to " "); a tool-call turn holding only another
provider's ``reasoning`` is padded instead of converted into a native thinking block."""
existing, reasoning = source_msg.get(RC), source_msg.get(R)
if isinstance(existing, str):
api_msg[RC] = existing or " "
elif isinstance(reasoning, str) and reasoning and not source_msg.get("tool_calls"):
api_msg[RC] = reasoning
else:
api_msg[RC] = " "
def apply_reasoning_content_policy(
source_msg: dict, api_msg: dict, needs_thinking_pad: bool, carriers: frozenset | None = None,
) -> None:
"""Shape an assistant replay message's reasoning keys for the active route (mutates ``api_msg``).
``needs_thinking_pad`` is the must-echo flag; ``carriers`` the keys the route reads
(``reasoning_replay_route``). ``carriers=None`` is a non-chat-completions wire: only the
must-echo ``reasoning_content`` rides there and ``reasoning_details`` is left to the adapter.
Keys the route does not read are removed, so a strict host never sees one (#45655, #70233).
"""
if source_msg.get("role") != "assistant":
return
if not needs_thinking_pad:
# Strict side: never carry the field — a reasoning primary pads history with " ",
# then a fallback to Mistral/Cerebras/Groq replays the pad and 422s. Also drops a
# non-string value (None after compaction): never pass null to the API.
api_msg.pop("reasoning_content", None)
return
existing, reasoning = source_msg.get("reasoning_content"), source_msg.get("reasoning")
# 1. Explicit reasoning_content already set. When the active provider enforces the thinking-mode
# echo-back (DeepSeek / Kimi / MiMo), preserve it verbatim — that includes their own space-placeholder
# written at creation time and any valid reasoning from the same provider. Sessions persisted BEFORE
# #17341 have empty-string placeholders pinned at creation time; DeepSeek V4 Pro rejects those with
# HTTP 400, so upgrade "" → " " on replay. When the active provider does NOT enforce echo-back, strip
# the field entirely. Strict OpenAI-compatible providers (Mistral, Cerebras, Groq, SambaNova, …)
# reject ANY reasoning_content key in input messages with HTTP 400/422 ("Extra inputs are not
# permitted"), even an empty string or a single-space pad. Stripping here covers the rebuild path;
# ``reapply_reasoning_echo`` covers the already-built api_messages path. Refs #45655.
if isinstance(existing, str):
# Explicit value: preserve verbatim, upgrading legacy "" to " " (DeepSeek V4 400s on "").
api_msg["reasoning_content"] = existing or " "
elif isinstance(reasoning, str) and reasoning and not source_msg.get("tool_calls"):
# Healthy session: promote internal 'reasoning' → 'reasoning_content'.
api_msg["reasoning_content"] = reasoning
text = _replayable_reasoning_text(source_msg)
effective = carriers if carriers is not None else (frozenset({RC}) if needs_thinking_pad else frozenset())
if carriers is None and needs_thinking_pad:
_apply_legacy_must_echo_pad(source_msg, api_msg)
elif RC in effective and (text is not None or needs_thinking_pad):
# Must-echo tier: every assistant turn carries the field; " " (not "") when there is no
# text, because DeepSeek V4 rejects empty string (#17341) and a legacy "" pad upgrades.
api_msg[RC] = text if text is not None else " "
else:
# tool_calls + 'reasoning' but no 'reasoning_content' means the reasoning came from
# ANOTHER provider (DeepSeek's own build pins reasoning_content for tool-call turns):
# pad without leaking foreign CoT. No reasoning at all: every assistant turn still needs
# the field; " " (not "") because DeepSeek V4 rejects empty string.
api_msg["reasoning_content"] = " "
# Also drops a non-string value (None after compaction) and a whitespace pad written
# for a must-echo provider: never pass null, never replay a pad as reasoning.
api_msg.pop(RC, None)
if R in effective and text is not None:
api_msg[R] = text
else:
api_msg.pop(R, None)
if carriers is not None and RD not in carriers:
api_msg.pop(RD, None)
def reapply_reasoning_echo(api_messages: list, needs_thinking_pad: bool) -> int:
"""Re-pad (or strip) assistant turns' reasoning_content for the ACTIVE provider.
def _reasoning_shape(msg: dict) -> tuple:
return tuple(msg.get(key) for key in REASONING_CARRIERS)
``api_messages`` is built once under the primary provider; a mid-conversation fallback
can switch providers, so baked-in fields must be reconciled: TO a require-side provider
re-applies the pad (else 400), TO a strict one strips it (else 422). Idempotent.
Returns the number of assistant turns changed.
def reapply_reasoning_echo(api_messages: list, needs_thinking_pad: bool, carriers: frozenset | None = None) -> int:
"""Reconcile already-built assistant turns with the ACTIVE route.
``api_messages`` is built once under the primary; a mid-conversation fallback or a recorded
field rejection changes the route, so baked-in keys are reconciled both ways: TO a must-echo
provider the pad is re-applied (else 400), TO a strict one the keys are stripped (else 422),
and a carrier the new route reads is restored from the text that is still present.
Idempotent. Returns the number of assistant turns changed.
"""
changed = 0
for api_msg in api_messages:
if api_msg.get("role") != "assistant":
continue
# 3. Healthy session: promote 'reasoning' field to 'reasoning_content' for providers that use the
# internal 'reasoning' key. This must happen before the unconditional empty-string fallback so
# genuine reasoning content is not overwritten (#15812 regression in PR #15478). Only promote for
# providers that enforce echo-back — strict providers reject the field (refs #45655).
# 4. DeepSeek / Kimi thinking mode: all assistant messages need reasoning_content. Inject a single
# space to satisfy the provider's requirement when no explicit reasoning content is present.
# Covers both tool-call turns (already-poisoned history with no reasoning at all) and plain text
# turns. Space (not "") because DeepSeek V4 Pro tightened validation and rejects empty string with
# HTTP 400 ("The reasoning content in the thinking mode must be passed back to the API"). Refs
# #17341.
if needs_thinking_pad:
if not api_msg.get("reasoning_content"):
apply_reasoning_content_policy(api_msg, api_msg, needs_thinking_pad)
changed += 1 if api_msg.get("reasoning_content") else 0
elif "reasoning_content" in api_msg:
api_msg.pop("reasoning_content", None)
changed += 1
before = _reasoning_shape(api_msg)
if carriers is None and needs_thinking_pad and api_msg.get(RC):
continue
apply_reasoning_content_policy(api_msg, api_msg, needs_thinking_pad, carriers)
changed += _reasoning_shape(api_msg) != before
return changed
# Provider error bodies (lowercased) for "this request carries a key my schema does not know".
_UNKNOWN_FIELD_PHRASES = (
"extra inputs are not permitted", "is unsupported", "no such field", "unknown field",
"unrecognized request argument", "unrecognized field", "unknown parameter", "unexpected field",
"additional properties", "unknown key", "not allowed",
)
_NAMED_FIELD_RES = {
RC: re.compile(r"(?<![\w])reasoning_content(?![\w])"),
RD: re.compile(r"(?<![\w])reasoning_details(?![\w])"),
# The bare word is also a top-level request param; only a message-path mention counts.
R: re.compile(r"messages[^\n]{0,80}?(?<![\w])reasoning(?![\w])"),
}
def rejected_reasoning_fields(error_body: Any, sent_messages: Any) -> frozenset:
"""Reasoning keys to drop after an unknown-field rejection that names one this request carried.
Strict schemas report one offending key per error, so the drop is widened to the keys the
same schema will reject next: a rejected ``reasoning_details`` / ``reasoning`` takes the
other non-standard carrier too (``reasoning_content`` still carries the text); a rejected
``reasoning_content`` (the most widely accepted alias) takes all three. One retry, not three.
"""
body = str(error_body or "").lower()
if not any(phrase in body for phrase in _UNKNOWN_FIELD_PHRASES):
return frozenset()
sent = {
key for msg in (sent_messages if isinstance(sent_messages, list) else ())
if isinstance(msg, dict) and msg.get("role") == "assistant" for key in REASONING_CARRIERS if key in msg
}
named = {key for key in sent if _NAMED_FIELD_RES[key].search(body)}
if not named:
return frozenset()
return frozenset(sent & (_ALL_CARRIERS if RC in named else {R, RD}))
def reasoning_route_key(agent: Any) -> tuple[str, str, str]:
"""``(provider, base_url host, model)``: one gateway serves models on different upstream lanes,
so a rejection learned for one model must not strip carriers from its siblings."""
from utils import base_url_hostname
return (
(getattr(agent, "provider", "") or "").strip().lower(),
base_url_hostname(getattr(agent, "base_url", "") or "") or "",
(getattr(agent, "model", "") or "").strip(),
)
def record_reasoning_field_rejection(agent: Any, error_body: Any, sent_messages: Any) -> frozenset:
"""Remember the reasoning keys this route rejected; returns the NEW ones (empty = no retry).
The next ``build_api_request`` re-shapes ``api_messages`` without them for this
(provider, host, model) for the rest of the session. A repeat rejection of an already
recorded key returns empty, so recovery can never loop. History is never touched.
"""
fields = rejected_reasoning_fields(error_body, sent_messages)
if not fields:
return frozenset()
known = agent._reasoning_rejecting_routes.setdefault(reasoning_route_key(agent), set())
new = fields - known
known.update(new)
return frozenset(new)
# Image / multimodal parts are deliberately NOT consolidated here: per-adapter handling is
# format-specific SYNTAX. The one shared image POLICY is ``_strip_images_from_messages``.

17
agent/reasoning_effort.py
View File

@@ -247,6 +247,23 @@ def thinking_toggle_extras(
return {"thinking": {"type": "enabled"}}, {}
# DashScope documents ``preserve_thinking`` (Qwen3.6+ and the Kimi K2.6/K2.7 it hosts) and a
# top-level ``clear_thinking`` (its GLM-5.x) as the switches that make the server KEEP replayed
# prior-turn ``reasoning_content``; without them the replay is accepted and dropped.
# https://www.alibabacloud.com/help/en/model-studio/deep-thinking
_DASHSCOPE_PRESERVE_MODELS = ("qwen3.6", "qwen3.7", "qwen3.8", "kimi-k2.6", "kimi-k2.7")
def dashscope_preserve_thinking_extras(model: Optional[str]) -> dict:
"""``extra_body`` keys that keep replayed reasoning on a DashScope-hosted model, else ``{}``."""
m = (model or "").strip().lower()
if any(token in m for token in _DASHSCOPE_PRESERVE_MODELS):
return {"preserve_thinking": True}
if m.startswith("glm-5"):
return {"clear_thinking": False}
return {}
def ox_alpha_reasoning_extras(reasoning_config: Optional[dict], model: Optional[str]) -> tuple[dict, dict]:
"""Ox Alpha (``x-preview-f-free``) ``reasoning_effort`` translation for the
opencode-zen profile (low/high/max only; anything else 400s)."""

55
agent/reasoning_params.py
View File

@@ -154,29 +154,35 @@ class ReasoningParamsMixin:
_build_assistant_message = _forward("agent.chat_completion_helpers", "build_assistant_message")
def _needs_thinking_reasoning_pad(self) -> bool:
"""True when the provider enforces ``reasoning_content`` echo-back on tool-call replays (DeepSeek, Kimi,
MiMo thinking all 400 without it; Ollama Cloud feeds it to the template). Cached per (provider, model, base_url), invalidated by
``switch_model()`` / ``_try_activate_fallback()`` — called ~16× per turn.
def _reasoning_replay_route(self):
"""Reasoning replay decision for the active route (``message_sanitization.reasoning_replay_route``).
DeepSeek v4 thinking and Kimi / Moonshot thinking both reject replays of assistant tool-call
messages that omit ``reasoning_content`` (refs 15250, #17400). Xiaomi MiMo thinking mode has the
same requirement.
Cached per (api_mode, provider, model, base_url, opt-in, rejected keys) — called many times
per turn — and recomputed after ``switch_model()`` / fallback activation or a recorded
field rejection, the only events allowed to change the replayed bytes mid-session.
"""
key = (self.provider, self.model, getattr(self, "_base_url_lower", self.base_url))
cached = getattr(self, "_thinking_pad_cache", None)
from agent.message_sanitization import reasoning_replay_route, reasoning_route_key
rejected = frozenset(getattr(self, "_reasoning_rejecting_routes", {}).get(reasoning_route_key(self), ()))
key = (getattr(self, "api_mode", None), self.provider, self.model, getattr(self, "_base_url_lower", self.base_url),
self._reasoning_echo_opt_in(), rejected)
cached = getattr(self, "_reasoning_route_cache", None)
if cached is not None and cached[0] == key:
return cached[1]
result = (self._needs_deepseek_tool_reasoning() or self._needs_kimi_tool_reasoning()
or self._needs_mimo_tool_reasoning() or self._needs_ollama_tool_reasoning()
or self._reasoning_echo_opt_in()
or ((self.provider or "").lower() == "nous"
and (self.model or "").lower() == "stealth/missingno"))
self._thinking_pad_cache = (key, result)
return result
route = reasoning_replay_route(key[0], self.provider, self.model, self.base_url,
echo_opt_in=key[4], rejected=rejected)
self._reasoning_route_cache = (key, route)
return route
def _needs_thinking_reasoning_pad(self) -> bool:
"""True for the must-echo tier: DeepSeek / Kimi / MiMo thinking 400 on a replayed tool-call turn
without a non-empty ``reasoning_content`` (#15250, #17400), plus a ``model.reasoning_echo`` opt-in
for gateways proxying them. Never true on a route that does not read the field."""
return self._reasoning_replay_route().pad
def _reasoning_echo_opt_in(self) -> bool:
"""``model.reasoning_echo`` opt-in for the *current* provider (covers gateways the host rules miss);
"""``model.reasoning_echo``: treat the *current* provider as must-echo (pad tool-call turns) — for
gateways proxying DeepSeek/Kimi/MiMo that the host rules miss. Replay itself is on by default;
fallback activation swaps the flag and ``restore_primary_runtime()`` restores it."""
return bool(getattr(self, "_reasoning_echo_flag", False))
@@ -208,24 +214,23 @@ class ReasoningParamsMixin:
"""True when the current provider is Xiaomi MiMo thinking mode."""
return matches_reasoning_echo_family("mimo", (self.provider or "").lower(), self.model, self.base_url)
def _needs_ollama_tool_reasoning(self) -> bool:
"""True for Ollama Cloud (provider, host, or a local ``:cloud``/``-cloud`` tag)."""
return matches_reasoning_echo_family("ollama", (self.provider or "").lower(), self.model, self.base_url)
_copy_reasoning_content_for_api = _forward("agent.agent_runtime_helpers", "copy_reasoning_content_for_api")
_reapply_reasoning_echo_for_provider = _forward("agent.agent_runtime_helpers", "reapply_reasoning_echo_for_provider")
@staticmethod
def _sanitize_tool_calls_for_strict_api(api_msg: dict, model: str | None = None) -> dict:
def _sanitize_tool_calls_for_strict_api(api_msg: dict, model: str | None = None, *, base_url: str | None = None,
provider: str | None = None) -> dict:
"""Strip Codex Responses fields from tool_calls for strict Chat Completions APIs (Mistral, Fireworks
400/422 on unknown fields). ``extra_content`` (Gemini thought_signature) is kept only for Gemini-family
models. Builds new dicts so the internal history keeps the Codex fields for a later fallback."""
400/422 on unknown fields). ``extra_content`` (Gemini thought_signature) is kept only on Gemini routes
(model family, Gemini host or provider). Builds new dicts so the internal history keeps the Codex
fields for a later fallback."""
tool_calls = api_msg.get("tool_calls")
if not isinstance(tool_calls, list):
return api_msg
from agent.transports.chat_completions import _model_consumes_thought_signature
strip = {"call_id", "response_item_id"} | (set() if _model_consumes_thought_signature(model) else {"extra_content"})
keep_signature = _model_consumes_thought_signature(model, base_url, provider)
strip = {"call_id", "response_item_id"} | (set() if keep_signature else {"extra_content"})
api_msg["tool_calls"] = [{k: v for k, v in tc.items() if k not in strip} if isinstance(tc, dict) else tc
for tc in tool_calls]
return api_msg

89
agent/transports/chat_completions.py
View File

@@ -246,35 +246,44 @@ def _is_openai_api_base_url(base_url: Any) -> bool:
return False
def _model_consumes_thought_signature(model: Any) -> bool:
"""True for Gemini-family targets, which require tool-call ``extra_content`` (thought_signature) replay.
Every other strict provider rejects it, so it is stripped for non-Gemini targets.
"""
m = str(model or "").lower()
return "gemini" in m or "gemma" in m
_GEMINI_SIGNATURE_HOSTS = ("generativelanguage.googleapis.com", "aiplatform.googleapis.com")
_GEMINI_SIGNATURE_PROVIDERS = frozenset({"gemini", "vertex", "copilot"})
def _route_replays_reasoning_details(base_url: Any) -> bool:
"""True when the target route reads replayed ``reasoning_details`` (OpenRouter's unified
reasoning array).
def _model_consumes_thought_signature(model: Any, base_url: Any = None, provider: Any = None) -> bool:
"""True for Gemini routes, which require tool-call ``extra_content`` (thought_signature) replay.
Every other chat-completions endpoint either ignores the field or, when its schema is
strict (Groq, Mistral, Cerebras, opencode relays: ``property 'reasoning_details' is
unsupported`` / ``Extra inputs are not permitted`` / ``no such field``), rejects the whole
request with HTTP 400/422 — so a reasoning turn produced earlier in the session wedges every
later turn once the model is switched (#70233). The stored history keeps the field; only the
wire copy drops it.
The Nous Portal read the field too (multi-turn reasoning continuity), but it enforces a
cumulative replayed-reasoning budget: replaying stored reasoning_details wedges long
sessions with a non-retryable 400 (#118182), so the Portal now strips like every other
route. Stored history keeps the field, so a route that genuinely replays it (OpenRouter,
#129037) still receives it.
A Gemini host or provider counts even when the model id is an alias without "gemini" in it;
every other strict provider (Fireworks, Cerebras) rejects the key, so it is stripped there.
"""
from utils import base_url_host_matches
return base_url_host_matches(base_url, "openrouter.ai")
m = str(model or "").lower()
return (
"gemini" in m or "gemma" in m
or (provider or "").strip().lower() in _GEMINI_SIGNATURE_PROVIDERS
or any(base_url_host_matches(base_url, host) for host in _GEMINI_SIGNATURE_HOSTS)
)
def _wire_reasoning_carriers(provider: Any, base_url: Any) -> frozenset:
"""Reasoning keys an assistant message may carry on this route; the rest are stripped.
``message_sanitization.route_reasoning_carriers`` owns the table; keys the route rejected
this session were already removed upstream by ``reapply_reasoning_echo``. The table replays
``reasoning_details`` to OpenRouter-format gateways (OpenRouter, Kilo, Vercel AI Gateway,
Nous Portal) and to vendors documenting it (Novita, MiniMax /v1, TokenHub), never to strict
schemas (Groq, Mistral, Cerebras, opencode relays: ``property 'reasoning_details' is
unsupported`` / ``Extra inputs are not permitted`` / ``no such field``, #70233).
The Portal is back on the list: the "cumulative replayed-reasoning budget" blamed for #118182
does not exist in the Portal code and ~440 live replay requests (up to 32 KB of reasoning)
returned no 400; the observed failures were an upstream content filter surfaced as
"Provider returned error". Portal lanes that reject a carrier are handled by the
per-(provider, host, model) reactive strip. Stored history always keeps every carrier.
"""
from agent.message_sanitization import route_reasoning_carriers
return route_reasoning_carriers(provider, base_url)
def _has_replayable_thought_signature(extra_content: Any) -> bool:
@@ -379,7 +388,7 @@ def _finish_kwargs(api_kwargs: dict[str, Any], sanitized: list, params: dict, *,
def _sanitize_message(
msg: Any, strip_extra_content: bool, strip_reasoning_details: bool = False,
msg: Any, strip_extra_content: bool, strip_reasoning: frozenset = frozenset(),
native_reasoning_details_type: str | None = None,
) -> dict | None:
"""Sanitized copy of ``msg``, or None when nothing needs stripping.
@@ -388,19 +397,19 @@ def _sanitize_message(
``response_item_id`` (and ``extra_content`` unless Gemini), an assistant
``tool_calls: []`` / ``null`` (strict providers reject both), ``name``
on tool results (schema-valid only on user/assistant messages; strict
providers reject it with ``contains item with unknown key name``), and
``reasoning_details`` unless the route replays it (``_route_replays_reasoning_details``).
On a replaying route, private ``<provider>.native_assistant`` carriers still go only to the
providers reject it with ``contains item with unknown key name``), and the reasoning
keys in ``strip_reasoning`` (carriers the route does not read, ``_wire_reasoning_carriers``).
On a route that reads ``reasoning_details``, private ``<provider>.native_assistant`` carriers still go only to the
profile that declared that exact type: another provider's signed replay is meaningless (or
rejected) elsewhere, and stored history keeps it for a return to the original provider.
"""
if not isinstance(msg, dict):
return None
strip_keys = [k for k in msg if k in _STRIP_MSG_KEYS or (isinstance(k, str) and k.startswith("_"))]
if msg.get("role") == "assistant":
strip_keys += [k for k in strip_reasoning if k in msg]
kept_details = None
if strip_reasoning_details and "reasoning_details" in msg:
strip_keys.append("reasoning_details")
elif isinstance(msg.get("reasoning_details"), list):
if "reasoning_details" not in strip_keys and isinstance(msg.get("reasoning_details"), list):
details = msg["reasoning_details"]
kept = [d for d in details if not (
isinstance(d, dict) and isinstance(d.get("type"), str)
@@ -455,11 +464,16 @@ class ChatCompletionsTransport(ProviderTransport):
Returns the input list unchanged when nothing needs sanitizing.
"""
strip_extra_content = not _model_consumes_thought_signature(kwargs.get("model"))
# A profile declaring a native carrier type consumes replayed details by contract.
native_type = getattr(kwargs.get("provider_profile"), "native_reasoning_details_type", None) or None
strip_reasoning_details = not (native_type or _route_replays_reasoning_details(kwargs.get("base_url")))
sanitized_pairs = [(m, _sanitize_message(m, strip_extra_content, strip_reasoning_details, native_type))
profile = kwargs.get("provider_profile")
provider = getattr(profile, "name", None) or kwargs.get("provider")
strip_extra_content = not _model_consumes_thought_signature(kwargs.get("model"), kwargs.get("base_url"), provider)
native_type = getattr(profile, "native_reasoning_details_type", None) or None
carriers = _wire_reasoning_carriers(provider, kwargs.get("base_url"))
if native_type and carriers:
# A profile declaring a native carrier type consumes replayed details by contract.
carriers = carriers | {"reasoning_details"}
strip_reasoning = frozenset({"reasoning_content", "reasoning", "reasoning_details"}) - carriers
sanitized_pairs = [(m, _sanitize_message(m, strip_extra_content, strip_reasoning, native_type))
for m in messages]
if all(s is None for _, s in sanitized_pairs):
return messages
@@ -478,7 +492,10 @@ class ChatCompletionsTransport(ProviderTransport):
path below (is_kimi, is_openrouter, ...) is only reached for unregistered providers.
"""
_profile = params.get("provider_profile")
sanitized = self.convert_messages(messages, model=model, base_url=params.get("base_url"), provider_profile=_profile)
sanitized = self.convert_messages(
messages, model=model, base_url=params.get("base_url"), provider_profile=_profile,
provider=params.get("provider_name"),
)
if _profile:
return self._build_kwargs_from_profile(_profile, model, sanitized, tools, params)

13
agent/turn_context.py
View File

@@ -1304,12 +1304,10 @@ def build_api_messages(
# and assistant rows may carry a sanitize-divergence sidecar.
api_msg["content"] = _api_content
# Pass reasoning back to the API for ALL assistant messages so multi-turn
# reasoning context is preserved.
# Replay stored reasoning on every carrier the active route reads (and strip the
# rest): message_sanitization.reasoning_replay_route owns the decision.
agent._copy_reasoning_content_for_api(msg, api_msg)
# 'reasoning' is trajectory-only (copied to 'reasoning_content' above);
# finish_reason is rejected by strict APIs (e.g. Mistral).
api_msg.pop("reasoning", None)
api_msg.pop("finish_reason", None)
# Fill empty non-final user/assistant wire copies so the pre-call sanitizer
# stops re-healing and flooding errors.log; durable history is untouched.
@@ -1323,10 +1321,11 @@ def build_api_messages(
# reject unknown fields. New dicts keep the internal list intact for Codex.
if agent._should_sanitize_tool_calls():
agent._sanitize_tool_calls_for_strict_api(
api_msg, model=_sanitize_model_for(agent, moa_config)
api_msg, model=_sanitize_model_for(agent, moa_config),
base_url=agent.base_url, provider=agent.provider,
)
# 'reasoning_details' is kept here; the chat-completions transport drops it on the
# wire for every route that does not replay it (OpenRouter/Nous do).
# 'reasoning_details' is shaped by the replay policy above on chat_completions and
# left intact for the native adapters (anthropic/bedrock rebuild signed blocks from it).
api_messages.append(api_msg)
# A provider-rejected Anthropic signature is suppressed outside canonical history and

15
agent/turn_recovery.py
View File

@@ -25,8 +25,7 @@ from agent.error_classifier import FailoverReason, classify_api_error
from agent.message_sanitization import (
_looks_like_corrupt_image_rejection, _looks_like_image_content_rejection, _sanitize_messages_non_ascii,
_sanitize_messages_surrogates, _sanitize_structure_non_ascii, _sanitize_structure_surrogates,
_strip_images_from_messages, _strip_non_ascii,
close_interrupted_tool_sequence,
_strip_images_from_messages, _strip_non_ascii, close_interrupted_tool_sequence, record_reasoning_field_rejection,
)
from agent.thinking_timeout_guidance import build_thinking_timeout_guidance, is_thinking_timeout
from agent.vision_message_prep import _provider_model_key
@@ -239,7 +238,9 @@ def recover_before_classification(
"""Recovery branches that run BEFORE ``classify_api_error``: UnicodeEncodeError
sanitization, Anthropic fast mode with no capacity (drop ``speed`` for that model),
provider image-content rejection (record the (provider, model);
build_api_request strips images from that model's requests only), and the Bedrock
build_api_request strips images from that model's requests only), an unknown-field
rejection naming a replayed reasoning key (recorded per (provider, host, model),
stripped on the retry), and the Bedrock
AnthropicBedrock SDK streaming fallback. Returns ``(retry_now, active_system_prompt)``;
the prompt may be ASCII-sanitized in place."""
if isinstance(api_error, UnicodeEncodeError) and getattr(agent, '_unicode_sanitization_passes', 0) < 2:
@@ -296,6 +297,14 @@ def recover_before_classification(
)
return True, active_system_prompt
# A strict schema rejected a replayed reasoning key BY NAME: record it per (provider, host, model)
# and retry once; build_api_request re-shapes api_messages without it. An unnamed upstream error
# never counts, and a key already recorded falls through to normal handling (no loop).
_sent = api_kwargs.get("messages") if isinstance(api_kwargs, dict) else None
if _status_ok and (_new := record_reasoning_field_rejection(agent, _err_body, _sent or api_messages)):
_vlines(agent, f"⚠️ {agent.model} rejected replayed {', '.join(sorted(_new))} — retrying without it for this model.")
return True, active_system_prompt
# AnthropicBedrock SDK raises "Unexpected event order" when Bedrock errors before
# message_start; fall back to native Converse for this session.
if (

10
hermes_cli/config_defaults.py
View File

@@ -280,11 +280,11 @@ DEFAULT_CONFIG = {
# Model name (any reasonable spelling) -> effort level; overrides agent.reasoning_effort
# when the current model matches. Edit in config.yaml (no CLI support: dots in keys).
"reasoning_overrides": {},
# Preserve assistant `reasoning_content` on history replay. Echo families (DeepSeek,
# Kimi/Moonshot, Xiaomi MiMo) are auto-detected by provider name/base-URL host; custom
# providers and OpenAI-compatible gateways proxying them are not. Set `reasoning_echo: true`
# on a `model:` entry or a `fallback_providers:` entry to opt in per provider. Default
# false: strict providers (Mistral, Groq, Cerebras) reject the field.
# Stored reasoning is replayed to every provider that can read it by default. This flag
# only adds the must-echo " " pad on reasoning-less tool-call turns, which DeepSeek,
# Kimi/Moonshot and Xiaomi MiMo require (auto-detected by provider name/host). Set
# `reasoning_echo: true` on a `model:` or `fallback_providers:` entry for a custom gateway
# proxying one of them. Strict providers (Mistral, Groq, Cerebras) never get the field.
"reasoning_echo": False,
# Turn liveness watchdog: a turn with no observable progress for `timeout_s` seconds is
# logged, force-interrupted so the UI can retry, and its lease stops renewing so stale-turn

15
plugins/model-providers/alibaba-coding-plan/__init__.py
View File

@@ -5,10 +5,21 @@ The CN profile checks its own key first and keeps the shared vars as ordered
fallbacks so existing CN users configured with the shared key keep working.
"""
from typing import Any
from agent.reasoning_effort import dashscope_preserve_thinking_extras
from providers import register_provider
from providers.base import ProviderProfile
alibaba_coding_plan = ProviderProfile(
class DashScopeProfile(ProviderProfile):
"""DashScope compatible-mode: ask the server to keep replayed prior-turn reasoning."""
def build_api_kwargs_extras(self, *, model: str | None = None, **context: Any) -> tuple[dict[str, Any], dict[str, Any]]:
return dashscope_preserve_thinking_extras(model), {}
alibaba_coding_plan = DashScopeProfile(
name="alibaba-coding-plan", aliases=("alibaba_coding", "alibaba-coding", "dashscope-coding"),
display_name="Alibaba Cloud (Coding Plan)",
description="Alibaba Cloud Coding Plan (Dedicated coding tier)",
@@ -17,7 +28,7 @@ alibaba_coding_plan = ProviderProfile(
base_url="https://coding-intl.dashscope.aliyuncs.com/v1", auth_type="api_key",
)
alibaba_coding_plan_cn = ProviderProfile(
alibaba_coding_plan_cn = DashScopeProfile(
name="alibaba-coding-plan-cn", aliases=("alibaba-coding-cn", "dashscope-coding-cn"),
display_name="Alibaba Cloud (Coding Plan, China)",
description="Alibaba Cloud Coding Plan, mainland-China endpoint",

19
plugins/model-providers/alibaba/__init__.py
View File

@@ -5,15 +5,26 @@ Profile names match models.dev catalog keys exactly so model metadata lines up
and ``model.provider: alibaba-cn`` resolves at runtime.
"""
from typing import Any
from agent.reasoning_effort import dashscope_preserve_thinking_extras
from providers import register_provider
from providers.base import ProviderProfile
alibaba = ProviderProfile(
class DashScopeProfile(ProviderProfile):
"""DashScope compatible-mode: ask the server to keep replayed prior-turn reasoning."""
def build_api_kwargs_extras(self, *, model: str | None = None, **context: Any) -> tuple[dict[str, Any], dict[str, Any]]:
return dashscope_preserve_thinking_extras(model), {}
alibaba = DashScopeProfile(
name="alibaba", aliases=("dashscope", "alibaba-cloud", "qwen-dashscope", "aliyun"), env_vars=("DASHSCOPE_API_KEY",),
base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
alibaba_cn = ProviderProfile(
alibaba_cn = DashScopeProfile(
name="alibaba-cn", aliases=("dashscope-cn", "alibaba-cloud-cn"),
display_name="Alibaba Cloud DashScope (China)",
description="Alibaba Cloud DashScope, mainland-China endpoint",
@@ -21,7 +32,7 @@ alibaba_cn = ProviderProfile(
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
alibaba_token_plan = ProviderProfile(
alibaba_token_plan = DashScopeProfile(
name="alibaba-token-plan", aliases=("dashscope-token-plan",), display_name="Alibaba Cloud (Token Plan)",
description="Alibaba Cloud Model Studio Token Plan (flat-token tier)",
signup_url="https://help.aliyun.com/zh/model-studio/",
@@ -29,7 +40,7 @@ alibaba_token_plan = ProviderProfile(
base_url="https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1", auth_type="api_key",
)
alibaba_token_plan_cn = ProviderProfile(
alibaba_token_plan_cn = DashScopeProfile(
name="alibaba-token-plan-cn", aliases=("dashscope-token-plan-cn",),
display_name="Alibaba Cloud (Token Plan, China)",
description="Alibaba Cloud Model Studio Token Plan, mainland-China endpoint",

12
plugins/model-providers/kimi-coding/__init__.py
View File

@@ -52,11 +52,17 @@ class KimiProfile(ProviderProfile):
return [model for model in models if model.strip().lower() != "k3"]
def build_api_kwargs_extras(
self, *, reasoning_config: dict | None = None, **context
self, *, reasoning_config: dict | None = None, model: str | None = None, **context
) -> tuple[dict[str, Any], dict[str, Any]]:
"""Moonshot treats extra_body.thinking and reasoning_effort as mutually
exclusive (400 on both): send effort when requested, else the toggle."""
return thinking_toggle_extras(reasoning_config, KIMI_K3_EFFORTS, KIMI_K3_OVERRIDES)
exclusive (400 on both): send effort when requested, else the toggle. K2.x keeps
replayed prior-turn reasoning_content only with ``thinking.keep: "all"`` (default null
ignores it; platform.kimi.ai/docs/guide/use-thinking-models); K3 always keeps it."""
extra_body, top_level = thinking_toggle_extras(reasoning_config, KIMI_K3_EFFORTS, KIMI_K3_OVERRIDES)
thinking = extra_body.get("thinking")
if isinstance(thinking, dict) and thinking.get("type") == "enabled" and "k2" in (model or "").lower():
extra_body["thinking"] = {**thinking, "keep": "all"}
return extra_body, top_level
def _kimi(name: str, aliases: tuple, env_vars: tuple, base_url: str) -> KimiProfile:

7
plugins/model-providers/zai/__init__.py
View File

@@ -58,10 +58,11 @@ class ZaiProfile(ProviderProfile):
enabled = reasoning_config.get("enabled") is not False
if not enabled and _has_token(model, _GLM_5_3_TOKENS):
# GLM-5.3 rejects disabled thinking; low is its lightest supported mode.
extra_body["thinking"] = {"type": "enabled"}
enabled = True
top_level["reasoning_effort"] = "low"
else:
extra_body["thinking"] = {"type": "enabled" if enabled else "disabled"}
# Preserved Thinking: with thinking on, the standard endpoint drops replayed prior-turn
# reasoning_content unless clear_thinking is false (docs.z.ai/guides/capabilities/thinking-mode).
extra_body["thinking"] = {"type": "enabled", "clear_thinking": False} if enabled else {"type": "disabled"}
if is_5_2:
effort = _glm_5_2_reasoning_effort(reasoning_config, model=model)
if effort is not None:

2
tests/agent/test_anthropic_preserved_thinking_replay.py
View File

@@ -87,6 +87,7 @@ def _session_db(tmp_path, *session_ids):
def _agent(db, session_id="s1", model="claude-opus-4-6", route=ANTHROPIC):
from agent.agent_runtime_helpers import copy_reasoning_content_for_api
from agent.message_sanitization import ReasoningReplayRoute
from agent.context_compressor import ContextCompressor
agent = SimpleNamespace(
@@ -97,6 +98,7 @@ def _agent(db, session_id="s1", model="claude-opus-4-6", route=ANTHROPIC):
reasoning_callback=None, stream_delta_callback=None, _stream_callback=None, log_prefix="",
)
agent._needs_thinking_reasoning_pad = lambda: False
agent._reasoning_replay_route = lambda: ReasoningReplayRoute(False, None)
agent._copy_reasoning_content_for_api = (
lambda source, target: copy_reasoning_content_for_api(agent, source, target)
)

1
tests/agent/test_deepseek_reasoning_content_echo.py
View File

@@ -319,6 +319,7 @@ class TestReapplyReasoningEchoForProviderSwitch:
model="gpt-5.5",
base_url="https://chatgpt.com/backend-api/codex",
)
agent.api_mode = "codex_responses" # the Responses wire never reads reasoning_content
msgs = self._codex_built_history()
changed = reapply_reasoning_echo_for_provider(agent, msgs)
# msgs[2] carried "summary from codex" — must be stripped for the

9
tests/agent/test_message_sanitization_policy.py
View File

@@ -163,10 +163,11 @@ class TestReasoningEchoFamily:
("xiaomi", None, "https://x", "mimo"),
("custom", "MiMo-7B", "https://x", "mimo"),
("custom", None, "https://api.xiaomimimo.com/v1", "mimo"),
("ollama-cloud", "kimi-k3", "https://x", "ollama"),
("custom", "kimi-k3", "https://ollama.com/v1", "ollama"),
("custom", "kimi-k3:cloud", "http://127.0.0.1:11434/v1", "ollama"),
("custom", "gpt-oss:120b-cloud", "http://127.0.0.1:11434/v1", "ollama"),
# Ollama has no echo-back requirement: replayed (as reasoning + reasoning_content) but never padded.
("ollama-cloud", "kimi-k3", "https://x", None),
("custom", "kimi-k3", "https://ollama.com/v1", None),
("custom", "kimi-k3:cloud", "http://127.0.0.1:11434/v1", None),
("custom", "gpt-oss:120b-cloud", "http://127.0.0.1:11434/v1", None),
("gmi", "gmi-cloud", "https://x", None),
("custom", "qwen3:8b", "http://127.0.0.1:11434/v1", None),
("openai", "gpt-5", "https://api.openai.com/v1", None),

12
tests/agent/test_provider_history_parity.py
View File

@@ -39,11 +39,10 @@ def test_native_carriers_follow_only_their_owner_on_each_request():
NOUS_PORTAL = "https://inference-api.nousresearch.com/v1"
def test_nous_portal_strips_replayed_reasoning_details_from_wire():
"""The Portal enforces a cumulative replayed-reasoning budget; replaying stored
reasoning_details wedges long sessions with a non-retryable 400 (#118182). Only the
wire copy is stripped — stored history keeps the field, so switching to a replaying
route (OpenRouter) still replays it."""
def test_nous_portal_replays_reasoning_details_on_wire():
"""The Portal forwards assistant carriers to its upstream lanes and the "cumulative replay
budget" blamed for #118182 does not exist (no 400 across ~440 live replay requests), so the
Portal reads reasoning_details again. Lookalike hosts never match and still strip."""
standard = {"type": "reasoning.encrypted", "data": "opaque-signature"}
history = [{"role": "assistant", "content": "answer", "reasoning_details": [standard]}]
original = deepcopy(history)
@@ -51,10 +50,9 @@ def test_nous_portal_strips_replayed_reasoning_details_from_wire():
for base_url in (NOUS_PORTAL, "https://stg-inference-api.nousresearch.com/v1"):
wire = transport.convert_messages(deepcopy(history), base_url=base_url)
assert "reasoning_details" not in wire[0]
assert wire[0]["reasoning_details"] == [standard]
assert history == original
# Substring lookalikes never matched the allowlist; they still strip (strict routes).
assert "reasoning_details" not in transport.convert_messages(
deepcopy(history), base_url="https://nousresearch.com.evil.io/v1")[0]

14
tests/agent/test_reasoning_echo_resolver_e2e.py
View File

@@ -94,15 +94,21 @@ class TestReasoningEchoResolverE2E:
copy_reasoning_content_for_api(agent, source, api_msg)
assert api_msg["reasoning_content"] == "the model's chain of thought"
def test_flag_absent_strips(self, tmp_path, monkeypatch):
"""No reasoning_echo -> flag False -> reasoning_content stripped on replay."""
def test_flag_absent_still_replays_but_never_pads(self, tmp_path, monkeypatch):
"""No reasoning_echo -> flag False: real reasoning is still replayed (replay is the
default on every route that can read it), but a reasoning-less turn is not padded."""
_write_home(tmp_path, monkeypatch, reasoning_echo=False)
agent = _agent_with_init_flag()
assert agent._reasoning_echo_flag is False
assert agent._reasoning_echo_opt_in() is False
source = {"role": "assistant", "content": "calling a tool",
"reasoning_content": "should be stripped"}
"reasoning_content": "the model's chain of thought"}
api_msg = dict(source)
copy_reasoning_content_for_api(agent, source, api_msg)
assert "reasoning_content" not in api_msg
assert api_msg["reasoning_content"] == "the model's chain of thought"
bare = {"role": "assistant", "content": "", "tool_calls": [{"id": "c1"}]}
api_bare = dict(bare)
copy_reasoning_content_for_api(agent, bare, api_bare)
assert "reasoning_content" not in api_bare

115
tests/agent/test_reasoning_replay_routes.py Normal file
View File

@@ -0,0 +1,115 @@
"""Stored reasoning is replayed to every chat-completions route that can read it, on the
carriers that route reads; strict schemas never receive a reasoning key; a route that still
rejects one is learned per (provider, host, model) and stripped on a single retry."""
from copy import deepcopy
from types import SimpleNamespace
import pytest
from agent.message_sanitization import (
apply_reasoning_content_policy, reapply_reasoning_echo, reasoning_replay_route,
)
from agent.transports.chat_completions import ChatCompletionsTransport
_DETAILS = [{"type": "reasoning.text", "text": "thought"}]
_TURN = {"role": "assistant", "content": "", "reasoning": "thought", "reasoning_content": "thought",
"reasoning_details": _DETAILS,
"tool_calls": [{"id": "c1", "type": "function", "function": {"name": "t", "arguments": "{}"},
"extra_content": {"google": {"thought_signature": "sig"}}}]}
_BARE = {"role": "assistant", "content": "", "tool_calls": [{"id": "c2", "type": "function",
"function": {"name": "t", "arguments": "{}"}}]}
def _wire(provider, base_url, model="m", rejected=()):
route = reasoning_replay_route("chat_completions", provider, model, base_url, rejected=rejected)
out = []
for source in (_TURN, _BARE):
api_msg = deepcopy(source)
apply_reasoning_content_policy(source, api_msg, route.pad, route.carriers)
out.append(api_msg)
return ChatCompletionsTransport().convert_messages(out, model=model, base_url=base_url, provider=provider)
@pytest.mark.parametrize("provider,base_url,model,expected", [
# strict schemas: nothing, not even the must-echo pad (#45655, #70233)
("mistral", "https://api.mistral.ai/v1", "deepseek-r1", set()),
("custom", "https://api.groq.com/openai/v1", "m", set()),
("custom", "https://api.cerebras.ai/v1", "m", set()),
("fireworks", "https://api.fireworks.ai/inference/v1", "m", {"reasoning_content"}),
# OpenRouter-format gateways, incl. the Portal (#118182 does not reproduce)
("nous", "https://inference-api.nousresearch.com/v1", "z-ai/glm-5.3",
{"reasoning_content", "reasoning", "reasoning_details"}),
("kilocode", "https://api.kilo.ai/api/gateway", "m", {"reasoning_content", "reasoning", "reasoning_details"}),
("novita", "https://api.novita.ai/openai/v1", "m", {"reasoning_content", "reasoning_details"}),
("ollama-cloud", "https://ollama.com/v1", "m", {"reasoning_content", "reasoning"}),
("custom", "http://127.0.0.1:8080/v1", "m", {"reasoning_content", "reasoning"}),
# everyone else: the de-facto standard alias
("zai", "https://api.z.ai/api/paas/v4", "glm-5.3", {"reasoning_content"}),
("openai-api", "https://api.openai.com/v1", "gpt-5", {"reasoning_content"}),
])
def test_each_route_gets_exactly_the_carriers_it_reads(provider, base_url, model, expected):
turn, bare = _wire(provider, base_url, model)
assert {k for k in ("reasoning_content", "reasoning", "reasoning_details") if k in turn} == expected
if "reasoning_content" in expected:
assert turn["reasoning_content"] == "thought"
assert "extra_content" not in turn["tool_calls"][0] # Gemini routes only
# A reasoning-less turn is never padded outside the must-echo tier.
assert "reasoning_content" not in bare
assert _TURN["reasoning_details"] is _DETAILS and "reasoning" in _TURN # history untouched
def test_must_echo_pads_and_aliased_gemini_keeps_its_signature():
_, bare = _wire("deepseek", "https://api.deepseek.com/v1", "deepseek-v4-flash")
assert bare["reasoning_content"] == " " # DeepSeek 400s without it; "" is rejected too
gemini_alias, _ = _wire("gemini", "https://generativelanguage.googleapis.com/v1beta/openai", "my-alias")
assert gemini_alias["tool_calls"][0]["extra_content"] == {"google": {"thought_signature": "sig"}}
class _Rejection(Exception):
status_code = 400
def __init__(self, body):
super().__init__(body)
self.body = body
def _agent(model):
return SimpleNamespace(
provider="nous", model=model, base_url="https://inference-api.nousresearch.com/v1",
api_mode="chat_completions", _force_ascii_payload=False, _image_rejecting_models=set(),
_reasoning_rejecting_routes={}, log_prefix="", _vprint=lambda *a, **k: None,
)
def _recover(agent, body, sent):
from agent.turn_recovery import recover_before_classification
return recover_before_classification(
agent, _Rejection(body), messages=[], api_messages=sent,
api_kwargs={"messages": sent}, active_system_prompt="",
)[0]
def test_named_field_rejection_strips_once_per_model_and_never_loops():
lane = _agent("z-ai/glm-5.3-flash:US")
sent = [{"role": "user", "content": "hi"}, deepcopy(_TURN)]
sent[1].pop("tool_calls")
body = "Extra inputs are not permitted, field: 'messages[1].reasoning_details'"
# A generic upstream error (content filter behind "Provider returned error") is not a rejection.
assert _recover(lane, "Provider returned error", sent) is False
assert lane._reasoning_rejecting_routes == {}
assert _recover(lane, body, sent) is True
rejected = lane._reasoning_rejecting_routes[("nous", "inference-api.nousresearch.com", "z-ai/glm-5.3-flash:US")]
assert rejected == {"reasoning", "reasoning_details"}
# The retry re-shapes the same api_messages for the narrowed route: reasoning_content survives.
route = reasoning_replay_route("chat_completions", "nous", lane.model, lane.base_url, rejected=rejected)
reapply_reasoning_echo(sent, route.pad, route.carriers)
assert sent[1]["reasoning_content"] == "thought"
assert "reasoning" not in sent[1] and "reasoning_details" not in sent[1]
# Same rejection again: nothing new to strip -> normal error path, no loop.
assert _recover(lane, body, [{"role": "assistant", "reasoning_details": _DETAILS}]) is False
# A sibling model on the same gateway keeps every carrier.
sibling = reasoning_replay_route("chat_completions", "nous", "moonshotai/kimi-k3", lane.base_url)
assert "reasoning_details" in sibling.carriers

22
tests/agent/transports/test_chat_completions_reasoning_details_replay.py
View File

@@ -1,7 +1,7 @@
"""``reasoning_details`` replay is route-scoped: OpenRouter reads it, every other
chat-completions route gets a wire copy without it (strict schemas 400/422 on the field,
wedging the session after an in-session model switch — hermes-agent#70233; the Nous Portal
additionally 400s on a cumulative replayed-reasoning budget — hermes-agent#118182)."""
"""``reasoning_details`` replay is route-scoped: OpenRouter-format gateways (OpenRouter, Kilo,
Vercel AI Gateway, Nous Portal) and vendors documenting it read it; strict schemas (Groq,
Mistral, Cerebras, Fireworks direct) 400/422 on the field and never receive it (#70233).
The Portal "replay budget" behind #118182 does not reproduce; it is back on the list."""
from openai import OpenAI
@@ -25,10 +25,14 @@ def test_auxiliary_wire_drops_reasoning_details_only_for_non_replaying_routes():
assert any("reasoning_details" in m for m in kwargs["messages"])
def test_openrouter_keeps_and_nous_strips_reasoning_details():
def test_openrouter_format_gateways_keep_and_strict_hosts_strip_reasoning_details():
transport = get_transport("chat_completions")
kwargs = transport.build_kwargs("m", _HISTORY, base_url="https://openrouter.ai/api/v1")
assert any("reasoning_details" in m for m in kwargs["messages"])
kwargs = transport.build_kwargs("m", _HISTORY, base_url="https://inference-api.nousresearch.com/v1")
assert all("reasoning_details" not in m for m in kwargs["messages"]), "Nous Portal strips (#118182)"
for url in ("https://openrouter.ai/api/v1", "https://inference-api.nousresearch.com/v1",
"https://api.kilo.ai/api/gateway"):
kwargs = transport.build_kwargs("m", _HISTORY, base_url=url)
assert any("reasoning_details" in m for m in kwargs["messages"]), url
for url in ("https://api.groq.com/openai/v1", "https://api.fireworks.ai/inference/v1",
"https://api.mistral.ai/v1"):
kwargs = transport.build_kwargs("m", _HISTORY, base_url=url)
assert all("reasoning_details" not in m for m in kwargs["messages"]), url
assert "reasoning_details" in _HISTORY[1] # durable history is untouched

5
tests/fixtures/provider_wire_snapshot.json vendored
View File

@@ -7059,6 +7059,7 @@
"unset": {
"extra_body": {
"thinking": {
"keep": "all",
"type": "enabled"
}
},
@@ -8880,7 +8881,6 @@
},
{
"content": "",
"reasoning_content": " ",
"role": "assistant",
"tool_calls": [
{
@@ -8939,7 +8939,6 @@
},
{
"content": "",
"reasoning_content": " ",
"role": "assistant",
"tool_calls": [
{
@@ -8997,7 +8996,6 @@
},
{
"content": "",
"reasoning_content": " ",
"role": "assistant",
"tool_calls": [
{
@@ -12187,6 +12185,7 @@
"high": {
"extra_body": {
"thinking": {
"clear_thinking": false,
"type": "enabled"
}
},

17
tests/plugins/model_providers/test_zai_profile.py
View File

@@ -48,7 +48,7 @@ class TestZaiThinkingWireShape:
extra_body, top_level = zai_profile.build_api_kwargs_extras(
reasoning_config={"enabled": True, "effort": "medium"}, model="glm-5"
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {}
def test_explicitly_disabled_sends_disabled_marker(self, zai_profile):
@@ -73,7 +73,7 @@ class TestZaiGLM52ReasoningEffort:
reasoning_config={"enabled": True, "effort": "high"},
model="glm-5.2",
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {"reasoning_effort": "high"}
@pytest.mark.parametrize("effort", ["low", "medium", "minimal"])
@@ -84,7 +84,7 @@ class TestZaiGLM52ReasoningEffort:
reasoning_config={"enabled": True, "effort": effort},
model="glm-5.2",
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {"reasoning_effort": "high"}
@pytest.mark.parametrize("effort", ["xhigh", "max"])
@@ -93,7 +93,7 @@ class TestZaiGLM52ReasoningEffort:
reasoning_config={"enabled": True, "effort": effort},
model="glm-5.2",
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {"reasoning_effort": "max"}
def test_disabled_sends_no_effort(self, zai_profile):
@@ -159,7 +159,7 @@ class TestZaiGLM53ReasoningEffort:
reasoning_config={"enabled": True, "effort": effort},
model="glm-5.3",
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {"reasoning_effort": expected}
@pytest.mark.parametrize(
@@ -199,7 +199,7 @@ class TestZaiGLM53ReasoningEffort:
reasoning_config={"enabled": False},
model=model,
)
assert extra_body == {"thinking": {"type": "enabled"}}
assert extra_body == {"thinking": {"type": "enabled", "clear_thinking": False}}
assert top_level == {"reasoning_effort": "low"}
@@ -243,7 +243,7 @@ class TestZaiFullKwargsIntegration:
provider_name="zai",
)
assert kwargs["reasoning_effort"] == "max"
assert kwargs["extra_body"]["thinking"] == {"type": "enabled"}
assert kwargs["extra_body"]["thinking"] == {"type": "enabled", "clear_thinking": False}
@pytest.mark.parametrize("model,reasoning,thinking,effort", [
("glm-5.3-flash", {"enabled": False}, "enabled", "low"),
@@ -263,5 +263,6 @@ class TestZaiFullKwargsIntegration:
provider_name="zai",
)
assert kwargs.get("reasoning_effort") == effort
expected = {"type": thinking} if thinking is not None else None
expected = None if thinking is None else (
{"type": thinking, "clear_thinking": False} if thinking == "enabled" else {"type": thinking})
assert kwargs.get("extra_body", {}).get("thinking") == expected

2
website/docs/developer-guide/session-storage.md
View File

@@ -239,7 +239,7 @@ CREATE INDEX IF NOT EXISTS idx_messages_session_id ON messages(session_id, id);
Notes:
- `tool_calls` is stored as a JSON string (serialized list of tool call objects)
- `reasoning_details`, `codex_reasoning_items`, and `codex_message_items` are stored as JSON strings
- `reasoning_details` is always kept in history; on the chat-completions wire it is replayed only to OpenRouter and the Nous Portal (every other chat-completions route gets a copy without it, since strict schemas reject the field)
- `reasoning`, `reasoning_content` and `reasoning_details` are always kept in history. On the chat-completions wire every assistant turn replays its stored reasoning on the carriers the route reads (`agent/message_sanitization.py::route_reasoning_carriers`): `reasoning_content` everywhere by default; plus `reasoning` and `reasoning_details` on OpenRouter-format gateways (OpenRouter, Kilo, Vercel AI Gateway, Nous Portal); `reasoning_details` on Novita, MiniMax `/v1` and TokenHub; `reasoning` on local servers, Ollama and Upstage. Strict schemas (Mistral, Groq, Cerebras, SambaNova) get no reasoning key, and Fireworks gets `reasoning_content` only. Only DeepSeek, Kimi and MiMo pad a reasoning-less tool-call turn with `" "`. When a route rejects a field by name ("Extra inputs are not permitted, field: messages[N].reasoning_details"), Hermes records it for that provider/host/model, strips it and retries once. The record lasts for the session and never touches history.
- Desktop history hydration retains assistant sidecars in both REST and JSON-RPC (`session.resume`, `session.activate`, `session.history`) projections, including rows with reasoning and tool calls. REST may return the SQLite JSON string while RPC returns decoded items; Desktop accepts both. A final Responses reply may live only in `codex_message_items` while `content` is empty. Canonical content still takes precedence, and analysis/commentary items are not promoted to reply text.
- `reasoning` stores the raw reasoning text for providers that expose it
- A reasoning-only clean stop (empty `content`, `finish_reason=stop`, reasoning present) is answered with the reasoning text, but the assistant row is never written with that text as `content`: `content` stays empty, the text lives in `reasoning`/`reasoning_content`, and `api_content` carries it so the next request replays the answer byte-identically. History surfaces therefore show it as reasoning, not as a reply.