e9a77f4188's `_RUNTIME_MAIN_API_KEY == old` guard had no test: reverting it left every kept test
green, though rotating under scoped_runtime_main then raises KeyError 'base_url' (or, with the
.get() default, overwrites the turn's mirrors with the scoped copy). Extend the existing aux-route
test with the scoped rotate step; red with the guard removed (mirror model becomes 'm') and red on
the pre-guard code (KeyError), green on HEAD.
Fold finding C15 #2 (gate 2ab toothless guard).
The credential refresh moved _primary_runtime["anthropic_api_key"] to the new token but left
_primary_runtime["is_anthropic_oauth"] at the old token's value, while self._is_anthropic_oauth is
recomputed from the new token. _build_anthropic_client_from_runtime restores the key and flag
together after a fallback, so a refresh that changed token type (Console key -> OAuth) would
restore the new key with the stale flag. Update the flag under the same identity guard.
Fold finding C15 #1 (gate quality review).
Manual /compress and turn-start compaction run before the next main request, so the
holder sync inside the main-request refresh never fired and the summary call carried the
revoked token. Replaces the bare holder-move test (its assertions are now checked through
compress_context), keeping the stack at two tests. Red without the pre-compression refresh.
Co-authored-by: Atul <atul@trycoral.ai>
Gate finding B15-1 (Medium): the pre-dispatch refresh runs inside the request
worker's copied Context (_context_thread_target -> contextvars.copy_context().run),
so re-calling _publish_runtime_main rebound the ContextVar only in that copy; the
turn thread's later `auto` aux calls (approvals, goal judge, plugin llm) still read
the revoked token. rotate_runtime_main_api_key mutates the already-published dict
(shared by both contexts) when it still holds the old key, then refreshes the
legacy mirrors. No cross-session leak: set_runtime_main builds a fresh dict per
turn, so the shared object is this turn's only.
The kept e2e test now runs the refresh on a thread under _context_thread_target;
red on the previous head, green here.
turn_context publishes the auxiliary main runtime at turn start, before the
first request can trigger the silent Anthropic refresh. Syncing the agent-side
holders alone left that already-published runtime on the revoked token, so
same-turn `auto` aux calls (approvals, goal judge, plugin llm) still 401d.
Re-call _publish_runtime_main when the published key is still the old token;
identity-gated so a fallback or other-route publish is never overwritten.
Also keep #121757's end-to-end aux-route test, rewritten to production
order (publish, then refresh, then read).
Co-authored-by: SoyMarketing <yo@soymarketing.org>
Co-authored-by: mahdi <mahdi.nikfallah@yarnhub.org>
Claude Code revokes the old OAuth token when it refreshes, but
_try_refresh_anthropic_client_credentials only updated _anthropic_api_key.
agent.api_key, context_compressor.api_key (sent as the aux main_runtime for
compression) and the _primary_runtime restore snapshot kept the revoked
token, so every compression summary 401d for the rest of the session.
Sync each holder that still carries the old token (identity-matched, so a
compressor or snapshot on another route keeps its own key).
Rebuilt by hand on main from #121859 (conflicted in the test file).
Fixes#121756
Co-authored-by: SoyMarketing <yo@soymarketing.org>
- Plugin hooks written against the older build_api_kwargs_extras keywords (no **context) no
longer raise TypeError on every request: lmstudio_reasoning_options is forwarded only when the
agent probed it (LM Studio). Pinned by TestProfileHookKeywordCompat, which fails if the field is
passed unconditionally.
- tencent-tokenhub: drop base_url and supports_model_listing=False. With them, first-time
`hermes model` setup reached the profile path and offered only the 3 curated Hy models instead
of probing {base}/models as main does; without a profile base_url, setup and the /model picker
take the same paths they took with no profile.
- One name for LM Studio's probed options end to end (lmstudio_reasoning_options), matching the
other provider-prefixed hook keys (ollama_num_ctx, qwen_session_metadata).
- New base-url:github-models-chat snapshot cell (openai/o4-mini, asserted chat_completions): the
gpt-5.5 cell goes to Responses and never reached the legacy GitHub reasoning branch.
- Comment/docstring truth: the host branches serve agents with no registered profile (not only
base_url-only ones); the LM Studio effort omission is done by the profile.
lmstudio and tencent-tokenhub were the last chat-completions providers with no ProviderProfile,
so their requests were built by the legacy flag branch in ChatCompletionsTransport.build_kwargs
(name check for LM Studio, host check for TokenHub). They now have profiles that reproduce the
same wire through build_api_kwargs_extras:
- lmstudio: top-level reasoning_effort from the model's published allowed_options. The agent's
cached probe is passed to the hook as reasoning_options; auxiliary calls (no probe) keep the
generic nested reasoning they already sent. base_url is left empty on purpose: a loopback
base_url would map every 127.0.0.1 / localhost endpoint to LM Studio in URL-to-provider
inference and turn off local-server detection for Ollama, llama.cpp and vLLM. Model listing
and the doctor probe stay off so the picker keeps LM Studio's own chat-model probe.
- tencent-tokenhub: top-level reasoning_effort clamped to low/medium/high, omitted when thinking
is off, high when no level is chosen (default_reasoning_config on main-loop turns). Model
listing and the doctor probe stay off so the picker keeps the curated Hy list.
The name-based LM Studio branch is removed from the legacy path. The host-based branches stay:
an agent built from a base_url alone (run_agent.py main, the batch runner, library callers)
carries provider="" and still reaches the legacy path. 11 base_url-only snapshot cells now pin
those host-detected requests (OpenRouter, Nous, Kimi, Kimi coding, TokenHub, GitHub Models,
NVIDIA NIM, Qwen portal, a localhost LM Studio port, Ollama, a generic endpoint) so the change
that retires the legacy path can prove it sends the same requests.
Deliberate deltas: model ids prefixed lmstudio: / tencent-tokenhub: are now stripped like every
other profiled provider's; TokenHub auxiliary calls with a reasoning config send top-level
reasoning_effort (the main-loop shape) instead of nested extra_body.reasoning.
Review threads left open on three merged PRs, each a gap in what the tests actually guard:
- #133740 (relay usage merge): no checked-in test failed on the base. The merged fix keeps
message_start's input/cache counts when the SDK's MessageDeltaUsage serializes them as null,
but the existing message_delta test only sent output_tokens, so it passed with the old merge
too. New test_anthropic_stream_accumulator_null_delta_usage_keeps_message_start_counts sends
nulls for input and cache fields; it fails with the old `{**current_usage, **usage}` merge.
- #133767 (error-body table): the #49769 cell asserted the 402 would be reclassified as
retryable. The fix that landed (#134157) keeps it `billing` and retries in the main loop's
recovery chain, so the cell still XFAILed on main and would pass even if the retry broke.
It now pins what the recovery depends on (billing + a parseable budget); the retry itself is
driven end to end by test_affordable_402_main_loop.py.
- #133767 (prompt-cache contract): the marker check needed cache_control in only one request,
so a later request that lost its markers passed. It now requires markers on every request.
The module docstring claimed native Anthropic layout coverage the cells don't have; it now
says envelope layout.
Greptile on a6b51f6ed4e (5/5, non-blocking): `_detect_upstream_elision`
only scans strings, so once an MCP result also carries an image it reaches
`make_tool_result_message` as a part list and an elided payload
("... 13 more items", has_more:true) lost the "data shown is INCOMPLETE"
notice. A part list is now judged by its first text part, and the notice
is appended there, inside the untrusted wrapper; the image is untouched.
Port of #46542 by @MarkVLK onto current main:
- hermes_cli/runtime_provider.py: _copilot_runtime_api_mode must not veto
a stale configured codex_responses mode for chat-only Copilot models;
recompute from the live model capabilities.
- agent/transports/codex.py: request summary:"auto" on the GitHub
Responses API so reasoning summaries surface.
- agent/agent_init.py: recompute api_mode at init-time fallback so the
fallback provider/model gets its own mode (and the primary-runtime
snapshot reflects it).
- tests: regression tests for the stale-mode veto, the GitHub Responses
summary, and the init-time fallback recompute.
Conflicts resolved against main's refactored fallback ladder
(agent/chat_completion_helpers.py) and its expanded transport tests.
Co-authored-by: Mark Vlcek <markvlcek@gmail.com>
Fixes#46527
OpenRouter answers a request whose max_tokens exceeds the remaining balance with
"402 ... You requested up to 65536 tokens, but can only afford 56272". The account has
credit; only the cap was too large. The main loop classified it as terminal billing and
failed the turn (or rotated off a credential that still had credit), so a request that
needed a few hundred tokens was dropped (#49769).
recover_after_classification now retries once with the output cap lowered to the affordable
budget before pool rotation, reusing the auxiliary client's _affordable_max_tokens_from_error
parse (same 64-token margin and 512-token floor) and the existing one-shot
_ephemeral_max_output_tokens seam. A second credit-limited 402 on the same call, or a body
with no parseable budget, keeps today's billing handling. Classification is unchanged.
Approach from #49785 by @tcconnally (closed unmerged), applied at the recovery layer.
Co-authored-by: Thomas Connally <51974392+tcconnally@users.noreply.github.com>
tirith ran `tirith check` on every terminal command, default-on with fail-open, as a
pinned PM binary auto-downloaded at startup. Measured against the shipped 0.4.2 binary
it caught 14/16 attack commands but flagged 12/24 everyday agent commands
(`echo $((6*7))`, `"$VAR/x.sh"`, `gh api | python3`, LAN curls, `.dev` hosts,
`pip install` via analysis_incomplete), and in cron/-q deny mode each of those is a
hard block. It only ever saw terminal strings (execute_code, writes and the browser
bypass it), fail-open meant no protection wherever the binary could not install
(Windows, Termux, x64 musl), and it produced a steady issue/PR stream (fail_open
semantics four times, install loops filling disks, a SIGSEGV hang, session-approval
scoping, third-party promo text on approval cards).
Removed: tools/tirith_security.py, the tirith branches of the command guard (the
is_tirith tuple, session-only "always" downgrade, import-error/fail-open paths,
unattended tirith pass), the CLI and gateway startup installers, the PM package and
lock pins, the nix runtime dep, security.tirith_* defaults, the CLI warning string
and three tirith tips (gaps refilled from the tail so translations survive), plus
the gateway's "no automated assessor" warning that only existed to point at tirith.
Config v50 drops security.tirith_* and enables nothing in its place. Hermes ships
no tirith plugin; if tirith upstream publishes one, it goes through normal catalog
admission under their name.
* fix(agent): free-tier rate limits end an attended turn at once
A free-tier (no sign-in) session that hit the service-wide rate limit slept
up to 600 s, retried into the same refusal, slept again, and then sent its
third attempt on a guest JWT that had expired during the wait. The welcome
host answered 403 "invalid jwt", which read as the tier being switched off.
The user watched a spinner for about 18 minutes and then saw the wrong card.
- An attended session (desktop, TUI, the interactive CLI, ACP) on the free
model ends the attempt cycle at once when the cooldown is longer than
60 s: fallback first, else the reset time and the sign-in / model doors.
"Attended" is keyed on the session source, so `hermes chat -q`, cron and
messaging sessions keep the long wait. The source vocabulary moves from
run_agent.py into agent/session_source.py, which adds is_attended().
- on_free_model() (welcome host + anonymous credential) is the one free-tier
check, used by the cutoff, the "free model is busy" wait copy and the
outage copy.
- A welcome-host 403 "invalid jwt" is its own route kind, session_expired:
the existing one-shot route heal re-exchanges the credential and retries;
if that fails the card is the retryable outage card.
- interruptible_backoff_sleep adopts a fresh Nous key after each wait.
_adopt_nous_key_before_expiry owns its refresh failure, so both call
sites drop their try/except.
- A retry wait over 60 s is announced in the transcript when it starts, for
every provider; shorter waits stay buffered. Every wait line names the
attempt that runs after it.
- max_retries_exhausted_result's max_retries becomes attempts (what it
carries); a one-attempt rate-limit end gets its own lead.
- One Retry-After reader, retry_utils.provider_retry_after_seconds, shared
by compute_error_backoff and the recovery ladder.
- _session_default_route replaces _session_default_model everywhere in
tui_gateway, so a not-yet-built free-tier session reports nous/welcome on
create, reuse, resume, branch and row persist, not the silent default.
* fix(agent): free-tier cutoff no longer ends subagent or library turns, paid waits look like main (review)
A delegated child of a TUI/Desktop session and a library caller (no platform) are unattended, so they keep waiting.
Rate-limit wait lines keep main's attempt numbers and stay buffered for every provider; only a non-rate-limit Retry-After over 60 s is announced, as on main.
The unattended free-tier wait line keeps the reset time beside "The free model is busy".
The free-tier cutoff names the reset time for any cooldown it fires on (over 60 s), not only from 120 s.
session.info for an env-seeded model off the free tier carries the model alone again, as on main.
Behaviour tests for the cutoff, the subagent and unattended waits, and a paid long wait.
The fix merged as 90194c64ff, so the run-time xfail for #81867 would only ever
pass. Drop the known_failure wrapper and reword the module docstring from
"gap" to "regression". The cell is red with the pre-fix agent/prompt_caching.py
(messages[1] goes from parts to a plain string) and green on main.
Three regression nets, each built from production evidence:
- tests/agent/transports/test_provider_wire_snapshot.py + tests/fixtures/provider_wire_snapshot.json:
the request every registered provider is sent (51 providers x reasoning unset/high/off), built
through the production assembly path (build_api_request: cache decoration, sanitization,
transport kwargs, Codex preflight, middleware). A shared-code edit that changes what any
provider receives fails naming the provider and the first diverging key; a new provider fails
until its wire is recorded. Network and DNS are blocked so capability probes (Ollama
/api/show, LM Studio, catalogs) record their offline answer, not a live server's.
Mutation check: +1 on the shared timeout fails 38 providers.
- tests/agent/test_error_classifier_observed.py: error bodies Hermes actually received (relay
traces and agent.log, scrubbed), raised as the real openai/httpx exception types. Two known
gaps are run-time xfails that flip to passing when their fix lands: #49769 (OpenRouter
"can only afford N" 402 classified as terminal billing) and #85005 ("not a multimodal model"
not recognised as an image rejection).
- tests/agent/test_prompt_cache_prefix_contract.py: a real AIAgent on a caching route against
the recording fake provider; every request must re-send the previous request's content and
only append. Pins a live cache bust as a run-time xfail: the #81867 skill-scaffold split is
applied only while the skill message is inside the last-N marker window, so request 3 of a
skill session re-sends the scaffold as a plain string and re-writes it (~20k tokens on call 2
of skill-invoked sessions in agent.log).
Inline `!`cmd`` expansion in SKILL.md bodies ran for every skill whenever the
global skills.inline_shell flag was on — a community hub skill and a builtin
were expanded identically. The hub's install gate (INSTALL_POLICY) blocks a
community install on the caution/dangerous verdicts the inline-shell DSL scans
as, but --force or a pre-scanner install puts that skill on disk anyway, and
the next view/slash/preload would auto-execute it on the host.
Gate expansion at the single preprocess_skill_content choke point (all three
surfaces route through it): community-trust hub lock entries never expand;
trusted/builtin hub entries and anything without a hub lock entry (bundled-
synced, user-created, project/external) keep the flag's contract. Lock read
failures skip the gate, like every other provenance consumer.
Fixes#63307
plugin-catalog/mem0.yaml (#131267) points at mem0ai/mem0's own
integrations/hermes-plugin-mem0 at a reviewed pin. The entry stays dormant
while a bundled copy exists (bundled providers win in find_provider_dir), so
deleting the in-tree copy is what lets hermes_cli/memory_provider_migration.py
install it for every home whose memory.provider is "mem0". The vendor copy
keeps the provider name, mem0.json, the MEM0_* keys and the setup wizard.
Also dropped: the hermes-agent[mem0] extra, its win-arm64 platform row and
exclude-newer row, the pm/extras.py anchor (uv lock removes only mem0ai,
posthog, qdrant-client and sqlalchemy), the mem0-specific doctor check (the
generic check now names `hermes plugins install mem0`), and the tests that
imported plugins.memory.mem0. Prose that lists the bundled set now names mem0
as a catalog provider. MEM0_* env hints and redaction stay: the catalog copy
still reads them.
* fix(bedrock): route GPT-6 family and prefixed OpenAI inference profiles, size their 1M window
An enterprise deployment on provider bedrock with model us.openai.gpt-6.1-sol
compressed at 96,000 tokens with a 400K compression cap configured. Three gaps,
all on origin/main c53536929e:
1. agent/bedrock_adapter.py:77-79 BEDROCK_OPENAI_RESPONSES_MODEL_IDS held the
GPT-5.5 and GPT-5.6 IDs only (unchanged since 16476fad10). AWS now publishes
cards for GPT-6 Astra, GPT-6 Sol, GPT-6 Luna and GPT-6.1 Sol.
2. agent/bedrock_adapter.py:165-167 is_openai_bedrock_model was an exact set
match, so us.openai.gpt-5.6-sol returned False while openai.gpt-5.6-sol
returned True. Every cross-Region inference profile of a Bedrock OpenAI model
was misclassified.
3. agent/bedrock_adapter.py:1169-1195 BEDROCK_CONTEXT_LENGTHS had no GPT-6
rows, so get_bedrock_context_length fell to the 128,000 default silently
(the fallback WARNING fired only for application-inference-profile ARNs).
agent/context_compressor.py:1181 applies the 75% small-window floor, which
is exactly the observed 96,000.
Per the AWS model cards (model-card-openai-gpt-6-1-sol.html and siblings):
the bare openai.* ID is served in-Region by bedrock-mantle (Responses, Chat
Completions, no geo or global IDs), and the us./global. profile IDs are served
by bedrock-runtime where Converse is listed as supported. GPT-6.1 Sol states a
1M context window with 131,072 max output tokens, the other cards state
1,050,000. 272K on these cards is the long-context pricing tier boundary, not
the window.
Changes
- parse_bedrock_openai_model_id(model_id) -> (bare_id, profile_prefix) | None.
is_openai_bedrock_model is True for every endpoint form of an allowlisted ID,
GPT-OSS stays False. New bedrock_openai_uses_mantle is True only for the bare
in-Region ID. Both callers (hermes_cli/runtime_provider_backends.py primary,
agent/auxiliary_client.py auxiliary) route Mantle on that predicate and keep
prefixed IDs on Converse via bedrock-runtime, the path AWS documents for them.
The Claude profile-prefix regex now derives from the same prefix tuple.
- GPT-6 generation added to the allowlist, the static picker catalog and the
context table. OpenAI rows sized from the cards (6.1 Sol 1,000,000, others
1,050,000). The table/metadata pairing test covers the new rows.
- get_bedrock_context_length logs one WARNING per model ID per process when
the static default is used for any ID, naming the model, the value, that it is
a fallback, and the model.context_length override.
- website/docs/guides/aws-bedrock.md: route table, model table and the
cross-Region note describe both endpoint forms.
The empty compression summary the deployment also reported is not addressed
here. Converse is a valid path for the prefixed ID, so routing does not explain
it, and no request trace exists yet.
* chore(bedrock): keep auxiliary_client.py within its code-health line cap
Rewrap the _build_bedrock_client docstring back to two lines so the file does not grow past its value on main (code health ratchet, FILE_LINES).
---------
Co-authored-by: unsupportedpastels <theoldwizard123@pm.me>
PR #133714 ("test: pin each provider's wire, real error bodies, and the
prompt-cache prefix contract") was merged before review and discussion
finished. This reverts both of its commits so main is back to the state
before it landed:
- 74306c1d1b test: the skill scaffold prefix cell is a plain regression test now that #133715 landed
- 560e53638c test: pin the wire every provider receives, real error bodies, and the prompt-cache prefix contract
Test-only revert: removes the three test files and the wire snapshot
fixture; no production code is touched. Discussion continues on #133714.
The fix merged as 90194c64ff, so the run-time xfail for #81867 would only ever
pass. Drop the known_failure wrapper and reword the module docstring from
"gap" to "regression". The cell is red with the pre-fix agent/prompt_caching.py
(messages[1] goes from parts to a plain string) and green on main.
Three regression nets, each built from production evidence:
- tests/agent/transports/test_provider_wire_snapshot.py + tests/fixtures/provider_wire_snapshot.json:
the request every registered provider is sent (51 providers x reasoning unset/high/off), built
through the production assembly path (build_api_request: cache decoration, sanitization,
transport kwargs, Codex preflight, middleware). A shared-code edit that changes what any
provider receives fails naming the provider and the first diverging key; a new provider fails
until its wire is recorded. Network and DNS are blocked so capability probes (Ollama
/api/show, LM Studio, catalogs) record their offline answer, not a live server's.
Mutation check: +1 on the shared timeout fails 38 providers.
- tests/agent/test_error_classifier_observed.py: error bodies Hermes actually received (relay
traces and agent.log, scrubbed), raised as the real openai/httpx exception types. Two known
gaps are run-time xfails that flip to passing when their fix lands: #49769 (OpenRouter
"can only afford N" 402 classified as terminal billing) and #85005 ("not a multimodal model"
not recognised as an image rejection).
- tests/agent/test_prompt_cache_prefix_contract.py: a real AIAgent on a caching route against
the recording fake provider; every request must re-send the previous request's content and
only append. Pins a live cache bust as a run-time xfail: the #81867 skill-scaffold split is
applied only while the skill message is inside the last-N marker window, so request 3 of a
skill session re-sends the scaffold as a plain string and re-writes it (~20k tokens on call 2
of skill-invoked sessions in agent.log).
Simplify pass over the skill-scaffold split fix:
- _scaffold_parts builds the [scaffold, tail] parts for both the marked split
(_apply_cache_marker) and the unmarked one (_split_unmarked_scaffolds). The
stripper only recognises the unmarked split while both stay byte-identical,
so they now come from one place.
- _strip_prior_decoration owns the "strip every list-content message before
re-planning" gate. build_prompt_cache_plan no longer strips and then hands
the result to apply_anthropic_cache_control, which stripped it again.
- strip_anthropic_cache_control checks the registry only for plain-text
two-part user messages, on the text it already joins for the flatten. The
old check also ran for non-text parts, where a lookup miss compared
None == None.
- apply_anthropic_cache_control docstring: drop the duplicated "Idempotent"
paragraph and the claim that only marked messages pay the copy cost (false
since the gate widened).
- Tests: cover the direct native tool-cache layout, which plans without
apply_anthropic_cache_control (red with its _split_unmarked_scaffolds call
removed). The re-plan test no longer depends on the marker window size.
apply_anthropic_cache_control only flattened rows that still carried a marker. After
#133715 a skill message outside the marker window goes out as an UNMARKED
[scaffold, tail] split, so a second pass over the planner's own output (MoA reference
trim, failover re-decoration) saw a plain two-part list and put the new breakpoint on
the volatile tail. Use the same list-shape gate build_prompt_cache_plan already has,
so the re-plan equals a fresh plan from the canonical string.
Test is red on the PR head, green with the fix.
A skill-invoked turn's user message is sent as [scaffold, tail] text parts so the scaffold can
carry a cache breakpoint (#81867). The split only happened inside _apply_cache_marker, i.e. only
while the message held one of the last-N markers. After the first tool round pushes it out of
the window, the same message went out as one plain string — different content to the provider,
so request 3 re-wrote the entire skill body. agent.log shows it on call 2 of skill-invoked
sessions: 6 of 8 sessions re-wrote ~20k tokens there, versus 1 of 147 plain sessions.
Every registered scaffold message is now sent split, marked or not, so its content is stable for
the whole session. The failover stripper flattens the unmarked split too (registry lookup, since
an unmarked two-part user message is otherwise indistinguishable from an organic one).
Introduced by 214f2b82db (perf(cache): split skill turns at a builder-declared stable/volatile
boundary, joaomarcos, 2026-08-08, landed via #82423).
Tests: one round-trip cell outside the marker window pins identical parts (red on
origin/main) and the stripper flattening the unmarked split (red with the
stripper clause removed).
Gate C11 r4 quality: the `flags &= {True, False}` set filter let YAML
`1` count as an opt-in because 1 == True in a set. Keep only bool values
so a stray 1/0 never promotes private reasoning. New parametrize row
(acme-int, value 1) is red without this.
Gate C11 r4 2c: startup, gateway and TUI build the agent with
provider="custom" and the named identity in requested_provider; only a
/model switch sets provider to the custom:<slug>. Matching entries against
agent.provider alone left the live-entry set empty at startup, so an
unflagged sibling on a shared endpoint still inherited the opt-in and a
sibling's false cancelled the provider that opted in. Collect the
custom:<slug> identity from both fields. The acme-plain parametrize row now
uses the startup shape (red before this change) and one opted-in row pins
the positive startup path.
Gate finding (C11 r3 quality, Low): 5769ff4aa9 removed the only writer of
runtime_capabilities["answer_in_reasoning"] (the agent_init seed); every
remaining writer stores resolve_native_compaction_capabilities(), which
yields only native_compaction. The reader was an unconfigurable second
trust source that could override the live config read, reachable only
from a test seam. The test now opts in the way users do, through a
custom_providers capabilities block on its base_url.
Also import _entries_for_route from hermes_cli.config like its neighbour.
Gate finding (C11 r3, Low): the provider-level loop returned the first
entry on the base_url carrying an answer_in_reasoning bool, whatever
provider was live. Two providers: entries on one endpoint (LiteLLM/vLLM
router) leaked one's opt-in to the other, and an earlier sibling's true
overrode the live entry's explicit false (probe: vllm-nemo true + vllm-ds
false on one URL, live custom:vllm-ds -> True).
Match the live custom:<slug> via custom_provider_slug, the helper that
derives that identity from an entry. With no slug match (bare 'custom'),
trust only when every flagged sibling on the endpoint opts in.
52e8ecd495 merged the constructor `capabilities=` answer_in_reasoning bool
into runtime_capabilities at startup so a provider-level opt-in reached
the promotion gate. The gate now re-reads the provider-level
`capabilities:` block on the live route (previous commit), which covers
CLI/TUI and /model switches the seed never did, so the seed is redundant.
Forward removal, not a revert: 52e8ecd495's test coverage stays, but the
fixture now opts in through the custom_providers entry-level
`capabilities` block instead of the constructor (fold2 finding 1).
The provider-level `capabilities: {answer_in_reasoning: true}` block only
reached the promotion gate through the constructor `capabilities=` arg,
which only the gateway passes. CLI and TUI never pass it, and every
/model switch rebuilds runtime_capabilities without it, so the documented
provider-level opt-in was silently ignored on CLI/TUI and lost after
/model in the gateway (fold2 finding 1, 2c review).
Read the entry-level block live, like the per-model key: when no
per-model capability is set, scan the custom_providers entries serving
the live base_url. It now holds on any model of that provider, is never
inherited by a fallback on another base_url, and OpenRouter stays
untrusted. The trusted-route test opts in through the entry-level block
with no constructor capabilities and gains /model-switch and fallback
rows (the first two are red without this change).
The exhaustion test pins an OpenRouter route, where the promotion gate
returns False before it reads runtime_capabilities, so setting
answer_in_reasoning=False there changed nothing (quality review). Also
collapse the leftover 3-5 blank-line runs to two.
The live-trust refactor dropped the writer of
runtime_capabilities["answer_in_reasoning"] but kept its reader, so an
explicit opt-in passed as AIAgent(capabilities={"answer_in_reasoning": True})
(what _lift_model_capabilities builds from a provider-level `capabilities:`
block and gateway/run.py passes through) never reached the gate and private
reasoning-only stops on that route were retried instead of promoted.
Seed it into runtime_capabilities at startup, merged into the existing
expression so agent_init.py does not grow past its size cap. The opt-in is
scoped to the primary route: a fallback rebuild drops it and the primary
restore brings it back from the snapshot. The fixture now opts in through
the constructor and the trusted-route test gains a constructor-opt-in row
(red without this change).
The code-health ratchet blocks growth of agent_runtime_helpers.py
(3909), agent_init.py (2527), chat_completion_helpers.py (4122) and
finish_text_response (309 lines); the gate added 60+ lines to those
facades.
Move answer_in_reasoning_capability into a topical sibling and evaluate
it from the agent's live route at the promotion site instead of caching
the decision in runtime_capabilities. That removes the plumbing at
startup, primary snapshot/restore, switch_model and fallback activation:
a live check cannot go stale across a route change, so the whole
sibling-site class is covered by construction. The unused constructor
`capabilities={"answer_in_reasoning": ...}` hook goes too (no caller
sets it); an explicit runtime_capabilities bool and the
custom_providers per-model opt-in still work.
Keep two behavioural invariants for the route gate: an untrusted
OpenRouter/DeepSeek route retries past private reasoning to the visible
answer while the local Nemotron parser route still promotes in one call
(one parametrized test), and exhausted recovery never echoes private
reasoning. The capability-helper, constructor-plumbing and fallback-
restore unit tests duplicate what those end-to-end paths already cover.
A clean-stop response with empty content and a reasoning payload was
promoted to the visible answer on every route. A field name is not
provenance: OpenRouter, DeepSeek and other adapters put private
chain-of-thought in `reasoning`, so users saw the model's private
thoughts as the reply (#111761, #114082). Exhausted recovery also echoed
a 500-char reasoning preview.
Promotion now requires a trusted route: an explicit
`answer_in_reasoning` runtime capability, a custom_providers per-model
opt-in, or the local Nemotron-3.5-Lightning parser route from #109205.
The decision lives in runtime_capabilities and is re-derived at startup,
fallback activation, primary restore and model switch. Exhausted
recovery returns the empty sentinel without a reasoning preview.
Rebuilt by hand on current main (keeps the Anthropic signed-thinking
skip and the _refund_api_call import added since the PR's base).
Salvaged from #112833 (a339569d77, 15a19b2fc0, 1d240e7b24).
The getattr(bundle_ctx, '_ctx', bundle_ctx).cert_store_stats() hedge
asserted nothing a non-empty certifi bundle would not satisfy either way.
Assert instead that platform_ssl_context() returned the (bundle, union=True)
cached context, which goes red if the platform path ever builds a
replacing (union=False) bundle context.
Fold3 finding 2 (quality Low).
Gate finding (C02 fold2 #2, quality): the SSL_CERT_FILE contract of
platform_ssl_context (bundle honoured on top of the platform store,
cached once, distinct from the plain platform context, CAs loaded) is an
agent/ssl_verify.py behaviour, but it was asserted inside the Telegram
close-wait test. Move it into the existing SSL_CERT_FILE test in
tests/agent/test_ssl_verify.py (no new test function); the Telegram test
keeps only its every-client-shares-the-context checks.
The rotation-state gate test now pins only that a durable fallback
streak never blocks should_compress (renamed to say so); its
clear-and-reload tail re-tested the loader, not the gate.
test_fallback_streak_alone_does_not_block_compaction duplicated
test_fallback_boundaries_wait_for_real_effectiveness_verdict, which
already asserts should_compress stays open with a streak of 2.
Rotating compaction (compression.in_place: false) calls on_session_start
with boundary_reason="compression"; bind_session_state zeroes
_fallback_probe_at and reloads it from the fresh child row, which has no
model-config key. The fallback streak was carried but the deadline was
not, so the benched summary model was re-benched with a new window on
every rotation and never probed again. Capture the deadline before the
rebind and persist it onto the child row alongside the streak.
Keep one test per invariant: effective fallback compactions no longer block
auto-compaction (test_fallback_boundaries_wait_for_real_effectiveness_verdict)
and the bench probe deadline survives a compressor rebuild (which also pins
that a streak >= 2 benches the summary model). Drop the window-arithmetic
tests that pinned the old in-memory clock and the unrelated session-reset
counter test.
The bench probe clock was in-memory monotonic time, so every rebuilt
compressor (the gateway rebuilds the AIAgent on cache eviction) came up
with a durable fallback streak but no clock, armed a fresh 300s window and
benched again. If rebuilds arrive faster than the window, the summary model
is never re-probed and the session stays on deterministic summaries until
a manual /compress.
Store the deadline as wall-clock time on the session row (model_config
key, the same durable path the proactive-prune runway uses) and load it in
bind_session_state, mirroring the anti-thrash recovery deadline (#100185).
A deadline more than one window in the future (clock jumped back) is
re-armed from now.
Two fallback summaries in a row tripped the anti-thrash breaker even when
each fallback compaction was effective, so automatic compaction was blocked
and the session rode into the provider's hard context limit. Only
ineffective compactions now trip the breaker; a fallback streak >= 2 instead
benches the summary model (deterministic drop, one probe per recovery
window), which still stops the paid fallback loop from #63008. force=True
(/compress) always calls the model.
Salvaged from #129431.
Co-authored-by: Pascal Bérard <pberard@nvidia.com>
The whole provider now lives in plugins/model-providers/solstice/ on the
generic seams:
- auth.py: NAS config discovery plus the brokered exchange/refresh, given
to the generic PKCE flow as token_request. A refresh that NAS rejects
(it folds every Google failure into 502) is treated as a dead grant
needing re-login.
- transport.py: a GeminiNativeClient subclass on the
generateContentPerUserQuota methods with Bearer auth (~30 lines, where
the PR duplicated a 257-line adapter).
- __init__.py: a hidden profile (create_client, fallback models,
default_aux_model).
Removed from core: hermes_cli/solstice.py, agent/solstice_adapter.py, and
the provider's branches in main.py, auth_commands.py, models.py,
runtime_provider.py, model_setup_flows.py and agent_runtime_helpers.py.
The unused SOLSTICE_CLIENT_ID env read is gone too.
Per-user-quota OAuth inference for Hermes Agent, hidden by default until
public launch is cleared. Auth (OAuth2 + PKCE with the user's account) is
brokered by nous-account-service, which holds the confidential client secret;
inference goes direct to the per-user-quota endpoint.
Built fresh on current main; commit history carries only the codename.
Provider surface:
- plugins/model-providers/solstice/ — profile + hidden enable gate
- hermes_cli/solstice.py — PKCE loopback login, NAS-brokered config/exchange/
refresh, token store, quarantine; callback path unchanged
- agent/solstice_adapter.py — per-user-quota transport over the native
adapter's OpenAI-compatible facade
- providers/: _HIDDEN_GATES + register_hidden_provider_gate + a hidden filter
in list_providers (kept out of the picker/registry/doctor until enabled)
- wired into main.py (_PROVIDER_MODEL_FLOWS), auth_commands.py (_PROVIDER_ALIASES,
_OAUTH_ADD_SPECS, _is_known_provider allowlist), models.py (catalog fetcher),
runtime_provider.py (resolution rung), model_setup_flows.py, agent_runtime_helpers.py
Kept verbatim (the live wire contract, until the broker is renamed): the
inference host, the OAuth scope, the deployed broker routes, and the
registered redirect callback path.
The enable gate is unchanged (open item: gate on stored credentials).
Verified: 226 focused tests pass; runtime resolution, client construction
and auth-add all exercise the real paths.
Fleet telemetry shows built-in memory writes failing 34-47% and automatic
compression failing ~15%, and several merged fixes each target one cause
(#133038 for memory, #133054/#133040/#133051/#133032 for compression). Both
metrics only carried outcome, so the fleet could not tell which cause a fix
moved.
- hermes.memory.op.count and hermes.compression.count gain one closed-enum
dimension, failure_class; `none` on success.
- Memory: every refusal/failure return in tools/memory_tool.py and
tools/memory_tool_store.py names its class (no_match, ambiguous,
over_budget, would_empty, drift, read_failed, retry_cap, scan_blocked,
missing_content, missing_old_text, invalid_args, stale_entry, disabled,
staged, gate_refused) through a context variable; the result the model
sees is unchanged. Plugin providers report provider_error / exception.
- Compression: the attempt log's own failure_class, collapsed onto the
compressor's closed names (exception:<Type> / rollback:<Type> keep only the
prefix, anything else reads other, a classless failure unknown). Skipped
rows carry the skip reason, and the skip set is now defined once in the
contract.
- Package schema v4 / event schema v4; v3 field sets of both metrics stay
valid so pending rows drain. Doc table rows and the consent copy (CLI
setup and every Desktop locale's collectedActivity) name the new reason.
