* fix(bedrock): route GPT-6 family and prefixed OpenAI inference profiles, size their 1M window
An enterprise deployment on provider bedrock with model us.openai.gpt-6.1-sol
compressed at 96,000 tokens with a 400K compression cap configured. Three gaps,
all on origin/main c53536929e:
1. agent/bedrock_adapter.py:77-79 BEDROCK_OPENAI_RESPONSES_MODEL_IDS held the
GPT-5.5 and GPT-5.6 IDs only (unchanged since 16476fad10). AWS now publishes
cards for GPT-6 Astra, GPT-6 Sol, GPT-6 Luna and GPT-6.1 Sol.
2. agent/bedrock_adapter.py:165-167 is_openai_bedrock_model was an exact set
match, so us.openai.gpt-5.6-sol returned False while openai.gpt-5.6-sol
returned True. Every cross-Region inference profile of a Bedrock OpenAI model
was misclassified.
3. agent/bedrock_adapter.py:1169-1195 BEDROCK_CONTEXT_LENGTHS had no GPT-6
rows, so get_bedrock_context_length fell to the 128,000 default silently
(the fallback WARNING fired only for application-inference-profile ARNs).
agent/context_compressor.py:1181 applies the 75% small-window floor, which
is exactly the observed 96,000.
Per the AWS model cards (model-card-openai-gpt-6-1-sol.html and siblings):
the bare openai.* ID is served in-Region by bedrock-mantle (Responses, Chat
Completions, no geo or global IDs), and the us./global. profile IDs are served
by bedrock-runtime where Converse is listed as supported. GPT-6.1 Sol states a
1M context window with 131,072 max output tokens, the other cards state
1,050,000. 272K on these cards is the long-context pricing tier boundary, not
the window.
Changes
- parse_bedrock_openai_model_id(model_id) -> (bare_id, profile_prefix) | None.
is_openai_bedrock_model is True for every endpoint form of an allowlisted ID,
GPT-OSS stays False. New bedrock_openai_uses_mantle is True only for the bare
in-Region ID. Both callers (hermes_cli/runtime_provider_backends.py primary,
agent/auxiliary_client.py auxiliary) route Mantle on that predicate and keep
prefixed IDs on Converse via bedrock-runtime, the path AWS documents for them.
The Claude profile-prefix regex now derives from the same prefix tuple.
- GPT-6 generation added to the allowlist, the static picker catalog and the
context table. OpenAI rows sized from the cards (6.1 Sol 1,000,000, others
1,050,000). The table/metadata pairing test covers the new rows.
- get_bedrock_context_length logs one WARNING per model ID per process when
the static default is used for any ID, naming the model, the value, that it is
a fallback, and the model.context_length override.
- website/docs/guides/aws-bedrock.md: route table, model table and the
cross-Region note describe both endpoint forms.
The empty compression summary the deployment also reported is not addressed
here. Converse is a valid path for the prefixed ID, so routing does not explain
it, and no request trace exists yet.
* chore(bedrock): keep auxiliary_client.py within its code-health line cap
Rewrap the _build_bedrock_client docstring back to two lines so the file does not grow past its value on main (code health ratchet, FILE_LINES).
---------
Co-authored-by: unsupportedpastels <theoldwizard123@pm.me>
hermes_cli/AGENTS.md and cli-internals.md stated "after the commit point
nothing fails" without the one documented exception (an autostash whose
restore conflicted stays parked: `partial`, exit 1, #122557) and without
the lost-completion outcome P2 introduces (a completion that could not
start or died without a result is an owed `completion` follow-up, tail
re-armed, exit 0). update_receipt's module docstring still said receipts
live under <HERMES_HOME>; they live under the ROOT home
(get_default_hermes_root()), never a sticky profile's.
Review P6.
main moved the update-pipeline long form from hermes_cli/AGENTS.md into
developer-guide/cli-internals.md (f3af936694). The C3 contract (nothing fails
after the commit point; root-home receipts that are on disk from the start)
now lives there, and the AGENTS.md rule bullet states it in the 30k chain budget.
Desktop's web picker had no way to send one capability through the Nous
Tool Gateway while the other used your own key. "Use for Search/Extract"
on the Nous Subscription row wrote the row's vendor string (firecrawl)
into web.<cap>_backend, which the runtime reads as BYOK Firecrawl: the
gateway was never used, and the managed and BYOK rows lit up together.
- The managed per-capability pick now stores `nous`; search/extract
dispatch, the registry and the Firecrawl client resolve the route per
capability (search: managed Perplexity, extract: managed Firecrawl).
- A whole-toolset Nous pick clears stale per-capability pins (they
outranked the new selection).
- /api/tools/toolsets/web/config reports search_via_nous /
extract_via_nous and each row's managed_nous_feature; the panel lights
the right row, labels the route, and refetches after a pick instead of
mirroring the vendor string.
Fixes#100513. Builds on the approach in #100539 by @liuhao1024 and the
self-healing pin clear from #122676 by @praggybuilds.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: Murphy (praggybuilds) <praggybuilds@users.noreply.github.com>
Independent-review findings on the plugin PCM streaming path:
- stream_sample_rate was checked, then re-read and int()-ed in the
adapter: NaN/inf raised out of resolve_streaming_provider (the speaker
then played nothing) and 0.5 became sample_rate=0. Validate once
(finite, >= 1 Hz) and hand that int to _PluginPCMStreamer.
- `nous` is an alias of the openai path in _get_provider but not a
reserved name, so a pinned `tts.streaming.provider: nous` could stream
a plugin registered as `nous` while the sync path speaks OpenAI.
- Document that stream() must be thread-safe: the speaker prefetches up
to 3 sentences, unlike the serialized synthesize() path.
Follow-up to the salvaged plugin-streaming commit. The adapter read
voice/model from the plugin's own tts.<name> section, so a streamed reply
spoke the plugin default while synthesize() got tts.voice: the voice
changed with the surface. It also skipped the sync dispatcher's routing:
a same-named `type: command` provider (which wins for synthesize) lost
to the plugin on the streaming path, and an is_available() == False
plugin was streamed anyway.
- tools/tts_tool_plugins: one routing guard (_plugin_route_key) and one
voice/model/speed builder (_plugin_voice_kwargs), shared by
synthesize() dispatch and the new _plugin_pcm_streaming_provider
(streams_pcm + positive stream_sample_rate + is_available(), read at
resolve time)
- tools/tts_streaming: _PluginPCMStreamer stays a thin adapter;
stream_channels/stream_sample_width dropped (the contract is int16
mono, which every sink assumes)
- agent/tts_provider: streams_pcm / stream_sample_rate declared on the
ABC with the contract
- tests: two invariant tests next to the plugin dispatch tests (the
streaming test module skips without numpy); docs for plugin authors
Before the type-ahead fix, an Enter pressed while `hermes chat` started
could save "No thanks" before the offer was on screen, and every surface
then treated the profile as answered. Every answer now also writes
`telemetry.shared_metrics.offer_version`. An "off" answer without it reads
as `reask`: undecided, so the CLI, Desktop strip and dashboard banner offer
it once more with a one-line reason. Any response settles it, including
Esc in the terminal and the banner close, which keep "No thanks", so
nobody is asked a third time. Opt-in answers were always deliberate picks
and are never re-asked.
The provider-level `capabilities:` block is no longer handed to the agent
by the gateway; the promotion gate re-reads it on the live route in CLI,
TUI and gateway alike. Say so, and that it survives /model on the same
provider but is not inherited by fallback or other providers (OpenRouter
never trusted) (fold2 finding 1).
Reasoning-only clean stops are now promoted to the reply only on trusted
routes, so a local parser that puts the answer in `reasoning` needs an
explicit opt-in. Document the per-model key and the provider-level
`capabilities:` block next to prompt_caching, plus the route scoping
(no fallback inheritance, OpenRouter never trusted). Finding from the 2c review.
Generic plugin-OAuth support that the Solstice provider needs. None of it
names a provider:
- OAuthPKCEConfig.token_request: a confidential client whose secret lives
behind a broker hands the grant to the plugin in place of a token_url
POST. The old refresh_token is kept when the response omits one, which
Google does on refresh.
- Plugin OAuth rows rotate 2 minutes ahead of their stored expires_at_ms
(shared with the anthropic rule). Opaque bearers such as Google's ya29.*
carry no JWT exp, so this stored expiry is the only clock they have.
- A row whose refresh grant just went DEAD stays DEAD on the inference 401
that follows; before this it was re-benched EXHAUSTED and the dead token
was replayed every cooldown.
- The auxiliary client serves an OAuth plugin that ships create_client from
its pooled row on the main runtime's endpoint (compression, titles). It
used to return "not directly supported".
- A 401 on a plugin OAuth row prints a re-sign-in hint, not the API-key
advice.
- The Gemini native adapter exposes its method names, auth headers and
error builder so a Bearer-auth subclass can reuse it without the API-key
remedies.
Review follow-ups for the failure_class split: finish_compression_attempt now
decides skipped/failed from the normalized class it records (a padded or
mixed-case skip name no longer reads failed with a skip reason, and a
non-string class no longer raises); the pre-chat consent offer names the new
reason like the setup copy; the setup copy keeps 80 columns; the doc says a
refused batch's op rows carry the class of the op that stopped it.
Fleet telemetry shows built-in memory writes failing 34-47% and automatic
compression failing ~15%, and several merged fixes each target one cause
(#133038 for memory, #133054/#133040/#133051/#133032 for compression). Both
metrics only carried outcome, so the fleet could not tell which cause a fix
moved.
- hermes.memory.op.count and hermes.compression.count gain one closed-enum
dimension, failure_class; `none` on success.
- Memory: every refusal/failure return in tools/memory_tool.py and
tools/memory_tool_store.py names its class (no_match, ambiguous,
over_budget, would_empty, drift, read_failed, retry_cap, scan_blocked,
missing_content, missing_old_text, invalid_args, stale_entry, disabled,
staged, gate_refused) through a context variable; the result the model
sees is unchanged. Plugin providers report provider_error / exception.
- Compression: the attempt log's own failure_class, collapsed onto the
compressor's closed names (exception:<Type> / rollback:<Type> keep only the
prefix, anything else reads other, a classless failure unknown). Skipped
rows carry the skip reason, and the skip set is now defined once in the
contract.
- Package schema v4 / event schema v4; v3 field sets of both metrics stay
valid so pending rows drain. Doc table rows and the consent copy (CLI
setup and every Desktop locale's collectedActivity) name the new reason.
#133567 taught git_pack_tidy to delete tmp_* transfer files, but
gitlock.clear_stale_tmp_packs already does that a few lines earlier in the
same update step (since Sep 6), with a stricter guard: it skips while any
git process runs. Remove the copy, keep the shared _unlink helper, and
correct the docs line about when the files go away.
Plugins can now call ctx.register_automation_blueprint(key, *, title,
description, schedule_template, prompt_template, category, slots,
deliver_default, skills, tags). The fields mirror the built-in
AutomationBlueprint, and slots are BlueprintSlot field dicts, so the call
also crosses the plugin-host boundary.
- One lookup, cron.blueprint_catalog.list_blueprints() / get_blueprint(),
merges the built-ins with the active profile's plugin blueprints at call
time. /blueprint (CLI/TUI/messengers), /api/cron/blueprints and the
instantiate route all use it. Each dashboard route resolves inside the
?profile= scope, off the event loop.
- Keys are namespaced <plugin>:<key>, so a plugin cannot shadow a built-in.
A ':' in the key, a duplicate key, an unknown placeholder, a bad slot type
or a schedule that does not parse is warned and ignored at registration.
- Catalog entries carry source/plugin. Desktop fetches the catalog for the
profile it creates jobs in, keyed per profile, and shows the plugin name
on the blueprint row and in the "Start from" select. The dashboard shows
a plugin badge.
- A created job is an ordinary cron job with the prompt and schedule already
filled in, so it keeps running if the plugin is later disabled.
- The ctx content registrars (register_skill, now joined by
register_automation_blueprint) move to hermes_cli/plugins_content.py.
The dashboard's blueprint types move to web/src/lib/automation-blueprints.ts.
Closes#132373. Requested by @enckequity.
A Desktop built before the blobless fix keeps walking history during
`hermes update`. On a treeless (tree:0) checkout every walk downloads trees
again, and #129514 filled a 434 GB disk during the minutes of dependency work.
The one-time conversion (#133016) ran at the very end of the update, after that
window. Both update hand-offs (update_completion._prepare and an older
updater's _update_takeover.prepare) now run it first, in the bootstrap
interpreter, and the late call is gone.
Re-running install.sh / install.ps1 over a treeless checkout now converts it
too. The filter is recorded only after the refetch succeeds, so a failure is
retried by the next update instead of leaving a blob:none filter over a
history that has no trees.
The pack cleanup also sweeps tmp_* transfer files that killed fetches left
behind. They are more than an hour old, a storm leaves gigabytes of them,
and git itself only prunes them after two weeks. The docs gain the recovery
steps for a disk that the storm already filled.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: Yuan Li <dskwelmcy@163.com>
Salvage follow-up for #129189 on top of #129209:
- An unknown or self-referential `inherit_from` now logs a warning and
registers the task without inheritance instead of raising, so one bad
base no longer fails the whole plugin load (the #129191 approach).
- `provider: auto` on an inheriting task means "no preference", like the
empty strings: "Reset all to auto" (CLI + dashboard) writes exactly that,
and before this it pinned the slot to the main provider while the model
was still inherited from the base (mismatched route).
- Once the user pins a route on the task (non-auto provider, model or
base_url), provider/model/base_url/api_key/api_mode/key_env/effort all
come from the task's own block. Mixing per key let a pinned provider pick
up the base's base_url and api_key.
- The CLI picker resolves the inherited view only for inheriting tasks.
Tests collapse to behavior contracts: base followed and changes
propagate, picker placeholders keep following, a pin wins whole, bad
bases warn and are ignored, re-registration cycles are cut.
Co-authored-by: Yuan Li <dskwelmcy@163.com>
`PluginContext.register_auxiliary_task` already lists a plugin task in the `hermes model`
picker and layers its defaults into `auxiliary.<key>`, but every REST/UI surface behind the
Models page enumerated only the built-in `_AUX_TASK_SLOTS`:
- `GET /api/model/auxiliary` never listed the task
- `POST /api/model/set` (scope=auxiliary) rejected it: `400 unknown auxiliary task`
- `__reset__` and the empty-task broadcast skipped it; `_stale_aux_pins` ignored it
- Desktop Settings → Models and the dashboard ModelsPage hardcoded the built-in list
So a plugin's side model could only be pinned by hand-editing config.yaml per profile.
Fix, at the existing seams:
- `web_server_config._aux_task_slots()` = built-ins + plugin keys from
`get_plugin_auxiliary_tasks()`, fail-soft; every consumer in that module uses it.
- `GET /api/model/auxiliary` reads plugin tasks INSIDE `_profile_scope`, so a request for
profile B enumerates B's plugins (the plugin manager keys on the same context-local home) —
the profile-scope gap flagged on #40922. Built-in rows are byte-identical; plugin rows are
appended with `label`/`hint`/`plugin` from the registration.
- Desktop `model-settings.tsx` and dashboard `ModelsPage.tsx` render served plugin rows after
the built-ins using the server label/hint (pure `auxTaskRows`; no new i18n strings, no-op on
older backends).
Tests: real plugin loaded from a temp HERMES_HOME through PluginManager discovery, two
profiles in one process (listing scoped per profile; foreign task still 400), assignment /
broadcast / reset / stale-pin over a plugin slot, discovery failure leaves built-ins intact —
all red on main. Vitest covers the pure row merge and the rendered plugin row.
Fixes#40880. Salvages #40922.
Co-authored-by: Tranquil-Flow <tranquil_flow@protonmail.com>
register_auxiliary_task's defaults start from a fixed shape and every key
is a full replace, so a plugin whose side model is a near-copy of a
built-in has to restate the whole block and silently drifts when that
built-in's shape changes.
inherit_from names a built-in or already-registered task whose effective
config becomes the base, resolved at read time so the plugin tracks the
base's current values instead of snapshotting them. Precedence is base,
then plugin defaults, then user config.
Self-referential inherit_from is rejected at registration rather than
left to the read-time guard: same-owner re-registration is allowed, so
the bad value would otherwise surface as unbounded recursion in a config
read. The guard stays for a hand-edited registry.
Closes#129189
Third-party drivers (e.g. a Codex-style computer-use reimplementation) could
only replace cua-driver by patching core. computer_use now follows the
single-select provider model Hermes uses for memory providers, context
engines and cron schedulers:
- plugins/computer_use/__init__.py discovers bundled
plugins/computer_use/<name>/ then user $HERMES_HOME/plugins/<name>/
(marker: register_computer_use_provider / ComputerUseProvider) and loads
ONLY the provider computer_use.backend names (default "cua"), read per
call so each profile gets its own. Unselected providers are never
imported; listing reads plugin.yaml only. An unknown/unloadable name
returns an error naming it and the installed providers; no fallback.
- The built-in cua-driver is an ordinary bundled provider at
plugins/computer_use/cua/ (driver implementation stays in
tools/computer_use). tools/computer_use/backend.py gains
ComputerUseProvider (create_backend(permission_mode=...), is_available(),
optional doctor()).
- The tool's check_fn and backend construction go through the selected
provider; the model-facing schema is unchanged for every provider
(prompt-cache invariant). The test-only HERMES_COMPUTER_USE_BACKEND env
switch is gone (tests patch _new_backend).
- General plugin discovery treats computer-use providers as exclusive
(not eagerly imported, not listed as toggleable) and skips the bundled
computer_use/ category, same as memory/, context_engine/, cron_providers/.
- hermes tools and the Desktop toolset panel list installed providers next
to cua; computer-use doctor/status/permissions and the Desktop status
payload skip cua-driver/TCC probes when another provider is selected.
`ctx.inject_message` could only reach an existing gateway session (`session_key`), so an
orchestrator plugin that opens a chat/thread itself (a bot-authored Discord forum post never
comes back as an inbound message) had no way to get an agent turn there.
`origin` takes the SessionSource.to_dict() shape. The gateway creates (or continues) that chat's
session through the adapter's normal message path, in the profile whose plugin manager loaded the
plugin, and the reply goes back to the origin chat:
- Profile: resolved from the manager's immutable home, never an ambient one. The origin may not
name another profile, the gateway's own routing for that chat through that profile's adapter
must land in the plugin's profile, and the profile needs its own connected adapter for the
platform (another profile's bot is never borrowed). Refusals are synchronous False + one log line.
- Authorization: same current-allowlist check as the session_key path (no bypass for host code).
- Multiplex: handle_message runs inside the profile's runtime scope so the spawned turn AND the
reply send after it inherit that profile's home and secrets (applies to the session_key path too,
which used to send from the default loop context).
- allow_gateway_injection is read from the plugin's own profile config.
- Plugin/system turns (event.internal) skip the human first-contact notes (onboarding intro and
the "no home channel" nudge), which were posted into the plugin's fresh thread.
Requested by canonicle-desk in #132609.
Closes#132609
Plugins that drive the agent's browser below the browser tools (Secure ENV,
catalog review on #132272) had no public way to send a raw CDP command over a
task's supervisor connection, so they bumped CDPSupervisor._next_call_id,
inserted futures into _pending_calls and sent on _ws. That breaks silently on
any internal change and collides between plugins.
SUPERVISOR_REGISTRY.capture(task_id) now returns a CapturedCDP pinned to the
supervisor's current socket and default page session:
call(method, params=None, *, session_id=None, timeout=10.0) -> raw reply
is_valid() / page_session_id / task_id / cdp_url
The validity check and the send run in one supervisor-loop step, so a call
never reaches a newer connection: after a reconnect, stop or registry
replacement every call raises CapturedCDPInvalid. Calls in flight when the
socket drops now fail at once (the supervisor fails pending replies on
teardown) instead of sitting out their timeout.
No model tool, config default or new capability: in-process Python could
already reach this. The handle enforces no origin/consent/ownership policy.
Reduced-scope salvage of #132752 (API shape and names from that PR); its
attach-reply handoff, old-wire cleanup and dispatch fences are not needed
because the supervisor closes the old socket on every reconnect, which drops
its sessions with it.
Co-authored-by: web3blind <264741654+web3blind@users.noreply.github.com>
The receiver buffers PCM per speaker with no conversation identity, and a same-voice-channel /voice join keeps that receiver. The poll and the leave flush then read the binding after draining, so audio captured for channel A was stamped with B and passed the post-STT check. The gateway join handler now discards the receiver's unemitted audio, under its lock, before moving the binding to a different text channel; a first join or same-channel rejoin keeps it. Nothing awaits between the discard and the binding write.
Reported by andrexibiza (F1) on #133471 and #130312.
The listen loop transcribes a check_silence() batch serially, so a later
utterance read its binding after a /voice join landed during an earlier
utterance's STT and was posted to the new channel. Capture the binding
once per batch (and per leave flush) and pass it in.
(cherry picked from commit e4b020a0fd21833a89fc1fa7f904db489d797d9c)
_process_voice_input awaits WAV conversion and STT, then calls the
gateway callback with only guild, user and transcript, and the callback
routes by the guild's binding as it is at that moment. A `/voice join`
from another text channel while the bot is already connected rewrites
_voice_text_channels and _voice_sources, so an utterance captured for
the old conversation became a turn in the new one, with its channel
prompt, bound skills, transcript echo and replies.
The bound text channel is captured when the utterance is accepted and
compared after STT; if the binding moved, the stale work is dropped and
logged. Rejoining from the same channel keeps the conversation and is
dispatched as before. Nothing awaits between that check and the
callback's own read of the binding.
Reported by andrexibiza (F6) on #128788.
(cherry picked from commit 47d515eb4e1633945212e42d6cf77aba41906ea8)
hermes.compression.count recorded every non-committed attempt as outcome=failed
except lock contention. Structural no-ops (insufficient_messages,
no_compressible_window, empty_post_handoff_window: the transcript is all
protected tail, no model call is made), a user stop (explicit_interrupt) and
superseded/stale attempts all landed in `failed`, so the fleet dashboard read a
session that simply had nothing summarizable as a broken compactor.
The outcome now comes from the resolved failure class (the previous commit keeps
the compressor's class instead of the caller's generic `no_progress`), and those
classes count `skipped`. Real failures (summary errors, would_grow, no_progress
after a real summary, stalls, commit errors) stay `failed`.
Two install routes refused a catalog memory provider that config already
names (memory.provider), so the install could never succeed and was
retried on every agent start / every click — the hindsight (91%) and
mnemosyne (87%) failed rows in hermes.extension.install.count.
1. Agent init (`recover_at_startup`) installs the configured provider
through the consent gate. Under the classic CLI that gate's input()
runs on the agent-build thread behind prompt_toolkit, so the turn
hangs on "Initializing agent..." forever; with no terminal (Desktop,
gateway, `chat -q`) it is refused — "Install declined: dependency
install skipped (non-interactive)" — and recorded failed on every
process start. Agent init now never asks: a provider that shipped in
core installs with its built-in's consent (security.allow_lazy_installs
still gates it), any other catalog provider is not attempted and the
user is told the one command that installs it.
2. The dashboard/Desktop Install click (`dashboard_install_plugin`) on a
plugin memory.provider already selects hits the publication veto that
is meant for replacing an active plugin, so it is refused every time
while the same plugin unselected installs and enables. The click now
carries the consent the fresh-install enable already acts on; a forced
reinstall keeps the veto.
Review fixes for the once-per-episode connect_failed dedupe:
- The episode was latched before the collection-consent gate, so a failure
seen while shared metrics were off suppressed the first consented day's
connect_failed. _record_connect now returns without latching when
collection is off: only a recorded failure opens an episode.
- Local episode state was keyed by the privacy-projected platform name, so
two custom adapters (both emitted as "plugin") shared one episode and one
adapter's recovery closed the other's outage. The key is now the adapter's
real platform value; projection happens only at emission.
- Docs: the hermes.platform.health row describes the episode semantics.
The pre-first-token window used the env stale base, so an explicit per-provider
stale timeout was ignored (900 configured, window capped at 180). It now goes
through the main loop's _cloud_stale_timeout_for with the attempt's routed
provider, so the explicit setting wins exactly as it does for the main loop.
Review fixes for the #100501 chat-stream no-progress watchdog:
- The no-progress window is inter-chunk. Before the first token a remote
endpoint gets the main loop's cloud stale patience (_cloud_stale_timeout:
context-scaled, reasoning-model floor), capped at the request timeout, so
o3 / deepseek-r1 / a MoA reference slot thinking silently is not cut at 60s.
- The watchdog pins its attempt's socket at construction and fences the
response body's release under its lock; the timer decides and shuts down
under the same lock with a generation token. A timer that already woke can
no longer reset a later request that reused the keepalive connection.
- The default relay callback (recovery rungs, credential retries) passes the
call's task, so auxiliary.<task>.no_progress_timeout survives the retry.
- A stall after the terminal chunk (finish_reason / usage received) keeps the
completed response and its billed usage instead of raising into fallback.
A streamed auxiliary call over chat-completions (every OpenAI-compatible
compression route: custom/vLLM/proxy, OpenRouter, ...) blocked inside the SDK
chunk iterator until the httpx read timeout (the full auxiliary budget, >= 300s
for compression). The host compression wait gave up first, so the attempt
ended as stall_interrupted / commit_fence_cancelled with no fallback. Only the
Codex Responses wire had a no-progress watchdog.
Arm the same 60s substantive-progress window (auxiliary.<task>.no_progress_timeout)
on the chat stream: a Timer shutdown()s only this attempt's socket (FD-safe
from a stranger thread, #70773), the owner closes the stream in its finally and
raises a "stalled ... timed out" TimeoutError, so the existing retry/fallback
ladder runs. Local endpoints keep silent prefill on the request timeout. The
fallback warning names no_progress_timeout instead of the request timeout.
Fixes#100501
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Catalog category "memory" also holds hook plugins, context engines and
skill packages. Deciding from the category made `--enable` overwrite the
user's memory.provider with an unloadable name and never enable the plugin.
Decide from the installed tree (_is_memory_provider_dir), and save
memory.provider only when find_provider_dir resolves the name.
The install prompt for a memory provider now sets memory.provider; say so on
the Hindsight setup and plugin install pages and drop the claim that the
plugin is enabled under plugins.enabled.
Fleet telemetry (14d, v0.21.5) shows the builtin store returning
success=False on 34-47% of add/replace/remove calls. Live `hermes chat -q`
runs against origin/main reproduced the dominant causes:
- Drift guard refused every replace/remove/batch on a file whose only
difference from the tool's own layout was whitespace (blank lines
around the delimiter, an empty entry). Parsing only strips whitespace,
so flushing such a file loses nothing; the round-trip signal could not
protect any content and turned the store read-only until hand-fixed
(#107270). The guard now fires only on the signal that can actually
lose foreign text: one entry larger than the whole store's limit.
- An over-budget add/replace/batch said "would exceed the limit" but not
by how much, so models retried with consolidations that were still too
big. Every over-budget message now states the exact chars to free and
points at the one-batch fix.
- An old_text that matched nothing (paraphrased, stale after a concurrent
write, or several entries glued with the delimiter) returned no hint on
the batch path. Zero-match errors now carry `closest_entries` (top word
overlap) on both paths, and flag a delimiter-spanning old_text.
- A batch replace missing its own `content` now says to set it on the
operation object.
Fixes#107270. Same diagnosis as #107278 (@KoNit-K) and #61612 (@webtecnica),
which normalized the round-trip comparison instead of dropping it.
hermes.provider_setup.count recorded every user cancel as event=failed
failure_class=cancelled, on every surface: Esc/Ctrl-C/Back in the CLI pickers,
Cancel on a Desktop/dashboard OAuth sign-in (DELETE /sessions), a cancel that
landed before the start response, and consent declined on the provider page.
A device code left to run out was also failed/other on the Nous, xAI and
Codex dashboard pollers (only the free-tier `timeout` reason and Codex
`expired` status were abandoned). Every provider's failure rate was inflated by
walk-aways (openai-codex desktop: 64 of 75 failed rows were `cancelled`).
- provider_setup_fields: failed + cancelled/expired is recorded abandoned, so
every emitter (CLI, web routers, pollers) gets the same answer.
- setup_failure_class: a lapsed device code (DeviceCodeExpired on the Nous poll,
AuthError code device_code_timeout/timeout, RFC 8628 expired_token) is
`expired`; access_denied is `cancelled`; an error whose __cause__ is a network
error is `network`.
- Dashboard/Desktop pollers keep the exception's closed class on the session
(note_oauth_failure) so `status=error` no longer collapses to `other`;
free-tier anon_unreachable is network, a start-route 504 is network.
- DELETE /sessions settles the metric immediately: a Nous/xAI poll blocks for
the code's lifetime, so the cancel row used to land minutes later or never.
Live (real dashboard server, Desktop-owned, isolated HOME, real providers):
base: nous/openai-codex cancel -> failed/cancelled (xai cancel never
settled); nous code left to lapse -> failed/other
fix: nous/xai-oauth/openai-codex cancel -> abandoned/none, each at cancel
Live (real `hermes model` in tmux, Ctrl-C at the OpenRouter key prompt):
base failed/cancelled, fix abandoned/none.
Command STT providers received the raw audio file with no format
normalization: _transcribe_command_stt only checked .exists() before
substituting {input_path}. Desktop voice recordings arrive as WebM/Opus
48 kHz and providers with container/sample-rate contracts (e.g. Tencent
16k_zh) reject the raw webm, so desktop voice chat failed on every
message. _transcode_audio_for_stt (ffmpeg -> 16 kHz mono m4a) existed
but was wired only into the OpenAI path's rejection retry.
Add stt.providers.<name>.normalize (default off, so whisper-style CLIs
that accept any container keep the original file): when truthy, the
input is transcoded into the per-call temp dir and {input_path} points
at the normalized file. Transcode failure returns an error envelope
instead of silently feeding the raw container to a provider the user
declared needs normalization.
Fixes#81811
Co-authored-by: embwl0x <embwl0x@users.noreply.github.com>
Review fixes for Settings ▸ Plugins:
- schema form is bound to one profile: keyed by (profile, plugin), renders
only rows loaded for that profile ($agentPluginsProfile), and its save
writes that profile or refuses; a scope switch discards the draft
- Capabilities gear carries its selected profile (&profile=), which
Settings adopts into its scope before resolving the page
- overview / missing / empty states keep the Applies-to selector
- desktop pages and automatic agent pages use disjoint identities
(kind-tagged uids; ?plugin= vs ?agent= routes)
- the folded schema form is addressed by agent key, so no plugin-chosen
sub-page id (e.g. config) is reserved
config_schema pages rendered raw keys as labels, ran the description into
the secret note without punctuation, floated Save mid-canvas and laid
fields on a two-column grid with an orphan cell.
- one ListRow per field (label + helper left, control right; JSON full width)
- label: manifest label/title, else the key in sentence case with acronyms
kept (maps_api_key -> Maps API key)
- helper sentences joined with punctuation (joinSentences)
- Save on the page heading row (Passwords & Logins' pattern), disabled
until dirty, spinner while saving
- only required fields are marked (settingsForm.optional -> .required)
- overview: Manage plugins as a heading-row button
- rail: a plugin row whose sub-page is open is named, not filled
- hello-runtime example: readable Chip style / gateway status copy
The hello-runtime runtime fixture registers a page with two sub-pages backed
by ctx.storage. The SDK guide gets a 'Plugin settings pages' section, plus
contract and export-table rows.
Gate finding B01 #1 (2c + quality): since api_server (/v1) runs now ride
the cron drain floor, the docstrings and comments for
parse_cron_drain_timeout, _load_cron_drain_timeout,
_get_cron_drain_timeout, the DEFAULT_GATEWAY_CRON_DRAIN_TIMEOUT comment,
cli-config.yaml.example and the messaging guide still called it a
cron-only floor. Say it covers /v1 runs too, that 0 opts them out as
well, and why: an interrupted /v1 run fails its waiting caller.
An independent live review of the tidy found it could lose data: a merge
under pack.packSizeLimit published only the first of several output packs
(45 of 60 commits gone), erasing "commit-free" on-demand packs dropped trees
and blobs a local branch still needed once upstream had pruned them, a
process killed mid-erase left a multi-pack-index naming a deleted pack, and
a classification timeout was cached as "holds commits" forever.
The erase rule is now proof against local packs alone: a promisor pack goes
only when every object in it is also in another promisor pack (checked by
binary search over the .idx files, no git process per pack). Nothing stored
locally can be lost and nothing depends on the remote still serving it, so
the marker heuristics and commit classification are gone. Merges run with
pack.packSizeLimit=0 and refuse anything but exactly one output; the
multi-pack-index is dropped before any pack is removed, so a kill at any
point leaves git reading; a lock file keeps two tidies from each erasing a
pack because the other still holds its copies.
.keep files written by a fetch-pack/index-pack that died (older than an
hour) no longer pin their packs. git's own repack never touches .keep'd
packs, and on a real dev checkout 53 of them held 4.5 GB of duplicated
history downloads.
Each `hermes update` now spends at most 60 s on the on-demand packs a
partial clone piles up (#129712), and every finished unit is kept, so a
large checkout converges over a few updates instead of blocking one
update for 20+ minutes:
1. Erase on-demand packs (empty .promisor marker, older than an hour)
that hold no commits. Their objects are copies of what the promisor
remote serves; git fetches what it needs again. This frees disk,
which the old gc fold never did. A pack holding commits is kept
because a ref may point into it; a pack git cannot classify in time
counts as holding one.
2. While more than 50 promisor packs remain, merge the smallest few
with `git pack-objects --stdin-packs` (cost proportional to those
packs, no reachability walk), marker first and index last so the
merged pack is a promisor pack from the moment git sees it.
Verdicts and a merge batch size that halves after a timed-out merge are
remembered in .git/hermes-pack-tidy.json, so no update repeats work an
earlier one could not finish. bounded_probe_run gains `input=` for
stdin-fed git commands.
Installers have cloned with --filter=tree:0 since c3e9ac87a7. A treeless
clone keeps commits only, and git requests a missing tree without telling
the server which trees it already has, so every checkout and every
path-filtered history walk re-downloads complete directory snapshots
(~0.58 MB per commit for this repo). GitHub's own guidance is that
treeless clones are for throwaway CI builds, not long-lived checkouts;
the blobless layout keeps every commit and tree local and fetches only
file contents on demand.
- install.sh / install.ps1 clone --filter=blob:none (same release tags,
--commit pins and ancestor checks the treeless switch was made for).
- hermes update (post-update step) converts an existing tree:0 checkout once with
`git fetch --refetch --filter=blob:none origin` (~120 MB, ~8 s
from GitHub) and records the new filter only after it succeeds, so a
failed refetch retries next update.
- The shallow-heal conversion unshallows as blob:none instead of tree:0.
- checkout_contains asks git with GIT_NO_LAZY_FETCH: a recorded SHA the
checkout lacks is "not contained", never a download of that commit's
whole history.
Teknium ruled the post-admission consumer hook (#129958) must fail OPEN, so
reshape the salvaged implementation into one small seam:
- `{"action": "handled", "reply": str|None}` is the only decisive result;
None, any other value, a raising callback or a timeout runs the ordinary
agent turn. One buggy plugin can no longer blackhole every inbound
message on a profile (the cherry-picked version failed closed on any
exception or malformed return from ANY registered callback).
- Fire under the message's ROUTED profile scope. The receiving bot's
handler binds the transport home so auth can read its .env; when that
bot serves a chat for another profile, the ambient scope at the hook was
the wrong profile and that profile's consumers never ran.
- Snapshot-only payload (session_key, platform, source dict, message_id,
text): no live gateway runner / session store handles, so the plugin
isolation host needs no HOST_DEGRADED_HOOKS entry.
- Bounded by plugins.hook_callback_timeout (fail-open) instead of the
fail-closed policy-hook set; the required-but-discarded receipt, alias
table, destination snapshot/restore machinery and hard-coded English
failure notice are gone.
- Tests trimmed to three invariants through the real _handle_message and
real plugin dispatch: consume path, raise path proceeds, routed profile
scope bound.
