Commit Graph

3098 Commits

Author SHA1 Message Date
Hermes Agent
1bb261251b fix(gateway): tolerate invalid UTF-8 update output
(cherry picked from commit 1dee620462c43daacd88783f446c32c6354f5b02)
(cherry picked from commit 295f32dad9b6ad9c3cc61bc0f0e4941ee0ba7617)
2026-08-08 12:30:19 -07:00
Brooklyn Nicholson
a0d406dcd8 fix(personality): stop writing personality into agent.system_prompt
Persist display.personality only; apply rendered text as an in-session
overlay across CLI, TUI config.set, and gateway /personality.

Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: EMT5320 <1908937833@qq.com>
2026-08-08 14:01:56 -05:00
Drexuxux
93964fda3d fix(api-server): resolve reasoning for the request's model, not model.default
e81d18dfb collapsed six per-surface copies of reasoning resolution onto
resolve_reasoning_config() and, in its own words, "fixes the gateway
resolving reasoning against config model.default instead of the session's
effective model". It did not touch gateway/platforms/api_server.py, which
kept that defect.

_create_agent() called GatewayRunner._load_reasoning_config() with no
model on its first line — before the model precedence chain (browser lock
-> session /model -> session row -> route -> per-request -> defaults) has
run. Per-model agent.reasoning_overrides therefore keyed off model.default
on the one surface where every request names its own model: a request for
a model with an override silently got the global effort instead.

Resolve after the chain settles, so the override follows the model the
request actually runs. An explicit per-request reasoning parameter still
takes precedence over config.

The existing test stub for _load_reasoning_config took no arguments (it
mirrored the old call); it now matches the real signature, as the sibling
stub in the same file already did.
2026-08-08 11:13:27 -07:00
Teknium
238351a60c fix(gateway): widen container->host media translation to home, cache, and in-process gateways
Follow-ups on the salvaged commit (#37207 by @charzhou):

- Persistent /root home mount translates too: an agent writing
  /root/out.png produced a real host file under
  <sandbox>/docker/default/home the gateway could not find.
- /root/.hermes cache mounts translate to the HOST cache (longest-prefix
  beats the home mount), so MEDIA:<agent_visible_image> paths deliver.
- /root/.hermes/* OUTSIDE a cache mount never translates through the home
  mount: those are the sandbox's credential copies (.env, auth.json) that
  sit outside the host-side denylist prefixes — fail closed.
- Run the idempotent terminal-config->env bridge before mount parsing so
  in-process gateways (Desktop backend, hermes serve) see the active
  backend and docker_volumes (covers #42299's /output case there too).
2026-08-08 10:32:54 -07:00
CharZhou
a7dd885439 fix(gateway): support Docker /workspace media paths in gateway delivery
Translate MEDIA paths under configured Docker volume mounts (and the
default persistent /workspace) to host paths before media delivery
validation, using longest container-prefix match so host:/workspace and
/output export mounts work.
2026-08-08 10:32:54 -07:00
kshitij
73997c41bb fix(tts): split long speech by provider and platform limits
Salvage of PR #17973 by @TKCen (Sebastian Hänisch), re-implemented on
current main to preserve speed/instructions/provider params,
prepare_spoken_text normalization, OPUS_VOICE_PLATFORMS, is_write_denied
path security, microsecond timestamps, and the streaming-TTS gate.

- Split long TTS text into provider-safe chunks instead of truncating
- Pack generated audio against platform upload limits (Discord 10MB,
  Telegram 50MB, configurable via tts.delivery_profiles)
- Combine chunks with ffmpeg (OGG/Opus re-encoded, MP3 stream-copied)
- Multi-file delivery when combination fails or would exceed limits
- Remove hard [:4000] truncation from all callers (cli.py, voice.py,
  gateway/run.py, gateway/platforms/base.py)
- Gemini TTS raises ValueError instead of silently truncating when
  composed prompt exceeds the provider limit

Simplify-code fixes: removed dead all_touched_paths set, added
try/finally for scratch file cleanup on exception, clean error response
on chunk failure instead of leaking stale file_path.
2026-08-08 22:54:20 +05:30
Teknium
a51a4cb096 fix(api-server): mark replayed tool calls completed in Responses output items
The non-streaming /v1/responses path built function_call and
function_call_output output items with no status field (and no item id),
while the SSE streaming path correctly emits status in_progress ->
completed. Spec-strict OpenAI clients reading the non-streaming output
array could interpret the status-less function_call items as pending
calls the CLIENT must execute — but these tools were already executed
server-side by the Hermes agent and are replayed for structured tool UI
only. Reported by a community user whose GPT-5.6 client concluded 'a
server should not tell an OpenAI client to execute a tool the server
already executed itself'.

- _extract_output_items now stamps status: completed and spec-shaped
  item ids (fc_/fco_) on replayed items, matching the streaming path
- test updated to pin status + id shape
- docs example updated + explicit note that output tool calls are
  replayed, never pending
2026-08-08 05:22:02 -07:00
Teknium
5dc0fa3889 fix: post-merge audit follow-ups for #81138/#81139/#81141/#81148
Four fix-forwards from the adversarial post-merge audit of the Aug 7
unreviewed merge batch:

- estop (#81148): is_engaged() now fails SAFE (engaged) on stat errors;
  the gateway estop gate lets recognized slash commands and replies owned
  by in-flight work (update prompts, clarify, slash-confirm, tool
  approvals, running sessions) through instead of consuming them; new
  gateway /pause [reason|off] command gives messaging-only operators an
  in-band engage/resume path (busy_policy=dispatch so it works mid-run).
- cron monitor mode (#81138): execution-mode invariants (monitor x
  no_agent, monitor_script x monitor_url, no_agent-requires-script) now
  have ONE owner (_validate_job_mode_invariants) called from BOTH
  create_job and update_job, so the create-time invariant can no longer
  be silently violated through the update door.
- cron notepad (#81139): remove_job now clears the job's notepad rows
  (clear_notepad was dead code -> orphaned KV state forever); clear is
  best-effort and no-ops without creating notepad.db.
- delegation batch gate (#81141): template-marker regex narrowed to
  multi-word placeholder shapes only (<feature name>, {file_path}) so
  generics (Vec<T>), HTML tags, JSON snippets, glob braces and f-string
  style no longer reject legitimate batches; duplicate-goal rejection
  removed (best-of-N fan-outs are legitimate).
2026-08-08 05:21:09 -07:00
kshitij
1005a057f0 review follow-ups: canonical classifier in hermes_state, compression-busy=locked, hedged gateway wording, drop dead constant
- Move classify_persistence_error into hermes_state beside is_disk_full_error
  and delegate the disk bucket to it (fixes 'ENOSPC writing state.db' and
  'not enough space' classifying as unknown). run_agent keeps a thin lazy
  delegating wrapper so the documented import path and fast import survive.
- Classify CompressionSessionBusyError (and its RPC-wrapped message forms)
  as 'locked': the motivating #81227 failure mode stringifies to 'is being
  compressed by another writer', which the substring heuristic missed.
- Export PERSISTENCE_ERROR_CAUSES and iterate it in the cron explainer
  suppression instead of a hardcoded tuple, so a future cause bucket cannot
  silently desynchronize cron delivery.
- Hedge the gateway locked/unknown recovery wording ('should already be
  saved' instead of 'was recorded') to match the explainer - the early
  turn-start persist may also have failed.
- Drop STATE_DB_WAL_WARN_BYTES (speculative dead constant with no consumer;
  the pre-existing 50 MB doctor WAL check covers the warning).
- Tests: compression-busy classification, is_disk_full_error delegation,
  causes-tuple coverage; mutation-checked red-green.
2026-08-08 14:18:26 +05:30
Victor Kyriazakos
01bc8a8752 fix(gateway): honest recovery message for session-persistence failures instead of 'unknown error'
Two defects in _normalize_empty_agent_response surfaced together during a
state.db lock-contention incident on an enterprise Slack deployment:

- the error lookup used dict.get's default, which an explicit
  'error': None value bypasses, rendering 'The request failed: None' /
  'unknown error';
- persistence failures fell through to the generic branch, whose 'use
  /reset' advice is harmful for this failure mode (destroys conversation
  context, fixes nothing).

Persistence-failed turns (failure_reason session_persistence_failed:*,
with a legacy fallback on the error text) now get a dedicated message:
storage was temporarily unavailable, the message was recorded, send it
again — with a disk-specific variant. No /reset suggestion. All other
branches unchanged.
2026-08-08 14:18:26 +05:30
kshitij
063f6941e5 fix: drop redundant None-guard on agent_result in agent:end payload
agent_result is guaranteed non-None at this point — line 17926 calls
.get() on it unconditionally, and line 18099 in the same block does
the same without the guard. The if/else was dead defensive code
inconsistent with surrounding access patterns.
2026-08-08 14:06:30 +05:30
Ken Weiner
b68a532295 fix: handle replacement transforms after CLI streaming 2026-08-08 14:06:30 +05:30
Ken Weiner
1f5a22264c fix: add model and provider to agent:end hook payload
Gateway hook plugins have no way to know which LLM model or provider was used
for a turn. The agent result dict already contains model and provider from
finalize_turn(), but neither field was forwarded into the agent:end hook
context.

Add agent_result.get("model") and agent_result.get("provider") to the
agent:end emit payload so gateway hooks can read them via context.get().
2026-08-08 14:06:30 +05:30
HexLab98
2e18e29723 fix(gateway): self-reacquire scoped lock by PID alone
After Discord reconnect, on-disk start_time can be null while the live
record has a fingerprint. Requiring equality made the gateway treat its
own PID as a foreign token holder (#81468).
2026-08-08 13:50:08 +05:30
kinsolee
c750d5354a fix(sessions): prevent oversized transcripts from exhausting memory 2026-08-08 13:36:08 +05:30
kshitij
643910afe3 refactor(gateway): narrow worker-start guard to Exception
Thread.start() failure is RuntimeError; catching BaseException here
swallowed KeyboardInterrupt/SystemExit without re-raise (unlike _worker,
which forwards them into the future).
2026-08-08 13:31:04 +05:30
kshitij
38cd1999cb fix(gateway): fail open + release admission slot when history-lookup worker cannot start 2026-08-08 13:31:04 +05:30
HenryG
271867f6fa fix(gateway): bound media history workers 2026-08-08 13:31:04 +05:30
HenryG
e52acf76a1 fix(gateway): keep media history reads off event loop 2026-08-08 13:31:04 +05:30
gnanirahulnutakki
e6b168855b fix(gateway): keep auto vision preprocess concise
Replace the 'describe everything in thorough detail' auto image-preprocess
prompt with a concise 2-4 sentence summary prompt so image-bearing gateway
messages stop generating ~2000-char descriptions (35s+ on local models).

Prompt-only variant of #10852: the max_tokens=500 cap and the
preserve_max_tokens aux-client plumbing from the original PR are
intentionally dropped to stay compatible with the max-tokens-knob policy
direction (#75253 removes hardcoded vision caps).

Fixes #10809
2026-08-08 13:28:19 +05:30
Gille
9c69d98864 fix(terminal): preserve SSH remote home cwd 2026-08-07 18:41:57 -06:00
kshitij
b5c2116783 fix: defer O(n) fallback_data construction to failure path in _save_entry
The entry_data/lock_held path (used by mark_turn_active/clear_turn_active)
was eagerly constructing a full O(n) snapshot of all routing entries under
the lock on every turn, even when the DB upsert would succeed.  Defer the
fallback_data construction to the except branch where it's actually needed,
and respect lock_held to avoid re-acquiring a non-reentrant _lock.
2026-08-08 01:12:16 +05:30
Dominic Bejar
c5e032c804 fix(gateway): close ambiguous recovery cleanup gaps 2026-08-08 01:12:16 +05:30
Dominic Bejar
5ff328cc76 fix(gateway): make active turn markers failure-atomic 2026-08-08 01:12:16 +05:30
Dominic Bejar
59a128c6fb fix(gateway): harden active turn marker lifecycle 2026-08-08 01:12:16 +05:30
Dominic Bejar
6774760b6f fix(gateway): recover exact turns after unclean exits 2026-08-08 01:12:16 +05:30
Teknium
5c29566e8d feat(terminal): graceful degradation for remote backend connection failures
Connection-class infrastructure failures on remote terminal backends (SSH
host unreachable/timed out, Docker daemon down or missing, remote file
sync failing on a dead link) previously surfaced to the model as raised
RuntimeError tracebacks. The model got a stack blob with no guidance and
the failure was indistinguishable from a tool bug.

Now:

- New EnvironmentConnectionError(RuntimeError) in tools/environments/base.py
  carrying a reason + retry_hint. Subclassing RuntimeError keeps every
  existing catcher working.
- ssh.py classifies connect-refused, connect-timeout, scp, remote mkdir,
  bulk upload/download, and remote rm failures as connection errors.
- docker.py classifies all four _ensure_docker_available() failure paths
  (missing exe, non-executable exe, daemon timeout, `docker version`
  failure).
- terminal_tool catches EnvironmentConnectionError and returns a
  structured tool result the model can act on:
    {"status": "degraded", "reason": ..., "retry_hint": ..., "exit_code": -1}
  The failed backend is evicted from the environment cache so a later
  call retries from scratch — recovery is automatic once the backend is
  reachable again.
- Config gate terminal.degraded_mode: warn|fail (default warn) in
  config.yaml, bridged as TERMINAL_DEGRADED_MODE across all four bridge
  sites (cli.py env_mappings, gateway/run.py _terminal_env_map,
  TERMINAL_CONFIG_ENV_MAP, DEFAULT_CONFIG). "fail" preserves the
  historical error+traceback tool result.
- Command failures (nonzero exit, command-not-found) are NOT touched —
  only infrastructure failures classify as degraded.

Tests: tests/tools/test_terminal_degraded_mode.py (15 tests) covering
exception classification for ssh+docker, structured degraded results,
no-caching of degraded envs, recovery after the backend returns,
nonzero-exit results unaffected, fail-mode preservation, invalid-mode
fallback to warn, and the four-site config bridge invariant.

Inspired by: Claude Cowork degraded-backend behavior (idea-level,
docs-only evidence).
2026-08-07 09:07:55 -07:00
Teknium
9fad45fcda feat(kanban,mcp): orphaned-card reconciliation + per-server MCP identity header
Two small config-gated features:

1. Kanban orphaned-card reconciliation (kanban.reconcile_orphans, default
   true, config.yaml): a running card with broken claim bookkeeping
   (claim_lock or claim_expires NULL — crash mid-claim, manual SQL, DB
   restore) is invisible to all existing recovery paths
   (release_stale_claims requires claim_expires NOT NULL,
   detect_crashed_workers requires host-local lock + pid,
   detect_stale_running is config-disabled by default) and shows Running
   forever. New reconcile_orphaned_running() pass in kanban_db.py runs
   each dispatch_once tick: requeues orphans to ready with an explanatory
   comment, closes any leaked run, emits a 'reconciled' event, and defers
   when the recorded PID is still alive on this host (never requeue
   beside a live worker). Surfaced via DispatchResult.reconciled_orphans.

2. Per-server MCP identity header (mcp_servers.<name>.identity_header,
   config.yaml): optional {name, value_from: static|profile, value}
   mapping; the header is attached to that server's HTTP/SSE transport
   requests. 'static' sends the config value; 'profile' resolves the
   active Hermes profile name once at connect time (no per-call
   mutation). Explicit per-server headers of the same name (any casing)
   win. Invalid blocks warn-and-ignore; stdio servers warn-and-ignore.

Tests: tests/gateway/test_kanban_reconcile_orphans.py (9),
tests/tools/test_mcp_identity_header.py (13), all written first (RED)
then implemented (GREEN). No new HERMES_* env vars.

Inspired by: openai/symphony tracker reconciliation (Apache-2.0) +
Poke per-user MCP identity (idea-level).
2026-08-07 08:58:20 -07:00
Teknium
5db1b72b1f feat(cli): global emergency stop — hermes pause / hermes resume
Resumable ESTOP sentinel at $HERMES_HOME/ESTOP that halts NEW work only:

- agent/estop.py: sentinel engage/disengage/is_engaged (single stat, no
  caching), optional reason + timestamp stored as JSON, paused_reply()
  notice, check_paused() log-once-per-engagement helper. Corrupt/empty
  sentinel still pauses (fail safe); a `touch ~/.hermes/ESTOP` works.
- cron/scheduler.py: tick() skips dispatch while engaged (logged once per
  engagement, not per tick). Due jobs simply wait for the next tick after
  resume — in-flight runs are never touched.
- gateway/kanban_watchers.py: dispatcher skips auto-decompose and worker
  spawning while engaged; zombie reaping still runs and running workers
  finish naturally.
- gateway/run.py: new gateway turns (post-auth, non-internal) get a brief
  "Hermes is paused" reply instead of an agent run. Internal events
  (in-flight background completions) bypass the gate.
- hermes_cli/subcommands/pause.py: `hermes pause [--reason]` and
  `hermes resume`, wired into main() and _BUILTIN_SUBCOMMANDS.
- hermes_cli/status.py: `hermes status` shows a PAUSED banner (one stat).
- tests/test_estop.py: 20 tests — sentinel lifecycle, reason surfacing,
  log-once, cron skip + resume, kanban gate, gateway paused reply +
  internal bypass, CLI idempotence, builtin-set parity, status line.

Never kills in-flight work; resumable with no restart. Footprint ladder:
CLI command only, no new model tool, no new env vars.

Ported from: gastownhall/gastown estop.go (MIT); related prior art:
#26778 (/panic — kill/exit semantics, deliberately different: ours is a
resumable pause), #44617 (interrupt in-flight cron — out of scope here).
2026-08-07 08:58:14 -07:00
kshitij
83bad5cdda fix: follow-ups for salvaged PR #80795
- Drain the eviction plan (pop + del) before trim_memory: the batch
  thread previously held every evicted agent in its local list while
  gc.collect + malloc_trim ran, so the in-pass trim freed almost
  nothing, the next tick re-read a still-high RSS, and the valve
  over-evicted an extra batch of warm prompt caches per cycle.
- Clear _db_flush_scan_prefix in _release_evicted_agent_soft: it is a
  shallow copy of the flushed transcript (stamped on every successful
  flush) sharing every message dict — and pressure-evictable agents
  have flushed by definition, so it pinned the multi-MB content strings
  on exactly the agents the valve targets.
- Config-read failure now falls back to resolve_agent_cache_bounds({})
  instead of bare AgentCacheBounds(): the dataclass default disables
  the pressure pass, but an absent config section means 'auto' — a
  transient read failure must not permanently switch off the OOM valve.
- protect_recent: false (YAML bool; False == 0) keeps the default MRU
  protection instead of silently disabling it.
- 'No evictable session' warning now distinguishes sessions blocked on
  un-flushed persistence (e.g. session DB never initialized — NFS
  HERMES_HOME) from mid-turn agents, so operators can diagnose why the
  valve isn't shedding instead of being pointed at running turns.
- _cgroup_limit_bytes checks the process's own cgroup (via the existing
  gateway.cgroup_cleanup._own_cgroup_path) before the root files, so a
  systemd unit's MemoryHigh=/MemoryMax= is detected — the root
  memory.high/max read 'max' on those deployments.
- Tests: 5 new guards; drain-before-trim and scan-prefix-clear
  mutation-checked (revert each fix -> its guard fails).
2026-08-07 21:02:33 +05:30
HexLab98
6bbe55dd09 fix(gateway): bound the agent cache by memory, not just count and age
The per-session agent cache is capped at 128 entries with a 1h idle TTL, and
neither bound knows how many bytes it holds. Each cached agent pins
_session_messages -- the full transcript including tool output, tens of MB on
a session with 100+ tool calls -- so a gateway serving many chats keeps every
warm transcript resident: agents that took a turn inside the TTL are never
idle-swept, and the idle sweep additionally defers finalizable sessions until
they expire. RSS climbs until the cgroup throttles and SIGTERM can no longer
flush inside systemd's stop timeout.

Add the missing bound. Each session-expiry watcher tick compares the process's
anonymous RSS against a budget and, when over, sheds LRU agents through the
same soft-eviction path the cap enforcer uses, then runs malloc_trim so the
freed arenas actually return to the OS. Evicted sessions rebuild their
transcript from the persisted session on the next turn.

Three classes of session are never shed: agents mid-turn, the most recently
used ones, and any session whose transcript has not finished reaching disk
(_last_flushed_db_idx vs len(_session_messages) -- the same divergence the FTS
write-corruption guard reacts to when it preserves live history).

memory_high_mb defaults to "auto", deriving the budget from the cgroup limit
the gateway runs under, so a MemoryHigh/MemoryMax on the unit is respected
without a second number to keep in sync. The two existing bounds become
configurable alongside it under agent.agent_cache.

protect_recent is clamped to half the cache: a couple of sessions can exhaust
the budget on their own, and a fixed MRU guard would then protect everything
and leave the gateway climbing with nothing it would shed.

Fixes #80764
2026-08-07 21:02:33 +05:30
kshitij
4eabb595f0 fix(agent): finish the #80622 bug class — sibling predicates, refund ordering, prompt carve-out, honest skip response
Follow-ups on top of the salvaged #80696 fix (review findings):

- Sibling sites: rollback.restore, gateway /retry, CLI /retry and /undo N,
  and both CLI resume turn counters now use is_user_originated_turn so
  legacy-persisted standalone handoffs (durable role=user, no display_kind)
  can never be truncation targets or counted as user turns (#80622
  suggested regression 4, dispatcher-wide).
- Site-1 guard: hoist the api_call_count decrement + iteration-budget
  refund above the break so a skipped turn no longer leaks a budget unit
  and finalize_turn logs the true call count (matches the ollama early-exit
  and the site-2 sibling).
- Site-2 guard: run the handoff guard BEFORE reanchoring so a restored
  user ask is what the anchor lands on, not a stale pre-restore index.
- SUMMARY_PREFIX: add the mid-tool-loop carve-out the code-side guard
  already implements, so a literal-minded model doesn't halt an in-flight
  exchange after in-place compaction.
- Skip path returns a short compaction status instead of replaying the
  previous turn's answer (finalize_turn would append it as a fresh
  assistant row — duplicate prose in transcript and delivery).
2026-08-07 19:44:35 +05:30
kshitij
f3ec2f36f8 perf(slack): parallelize conversations.info lookups with asyncio.gather
The dedup fix in #80679 reduced ~1.6k sequential API calls to ~30,
but those ~30 calls were still sequential. Extract the per-base-ID
resolution into an async helper and run all lookups concurrently
via asyncio.gather. Turns a 6-15s serial block into <1s.
2026-08-07 18:40:33 +05:30
kyssta-exe
025fc7e74b fix(slack): dedupe thread-qualified channel lookups (#80668) 2026-08-07 18:40:33 +05:30
kshitij
e5e96e8bb5 fix: harden _await_disconnect_step against outer cancellation + add claim keys
Follow-up to #80700:

1. _await_disconnect_step was missing the try/except CancelledError around
   asyncio.wait() that _await_adapter_cleanup_with_timeout already has.
   When the outer fatal-handler timeout cancels disconnect() mid-step,
   asyncio.wait does NOT cancel its inner task — the task was orphaned
   with no observer. Add the same cancel+detach+re-raise pattern.

2. _queue_retryable_fatal_platform omitted credential_claim/listener_claim
   keys that all 3 startup-path queue sites include. These are consumed by
   the multiplex reservation logic to prevent secondary profiles from
   taking the endpoint while a primary is queued. Pre-existing latent bug
   — now fixed since the extraction makes it trivial.
2026-08-07 18:32:45 +05:30
HexLab98
7141a6dc3a fix(gateway): queue reconnect before fatal disconnect wedges (#80598)
After a network outage the Telegram fatal handler could hang inside
disconnect() and never populate _failed_platforms, so the reconnect
watcher had nothing to retry and the process stayed permanently deaf.
Queue retryable platforms before any disconnect await, bound the fatal
handler with an outer detach deadline, and release the Telegram token
lock / PTB close steps with detach-on-timeout so recovery cannot stall.
2026-08-07 18:32:45 +05:30
kshitij
2a0d0bc698 refactor(gateway): drop dead degraded token field; de-churn salvage diff
Follow-up to the #80376 salvage:

- TurnLeaseToken.degraded is dead code since acquire() started raising
  TurnLeaseTimeoutError: the only constructor site passes the default and
  repo-wide there are zero external readers. Remove the field, its ctor
  param, the repr segment, and the always-False guards in rebind()/release();
  the class docstring's 'retained for compatibility' claim described
  consumers that do not exist.
- Revert pure black-rewrap churn in test_config_env_bridge_authority.py and
  test_turn_lease.py (hunks on functions this change does not touch), keeping
  the functional Windows-env/encoding additions.
- Turn the default-value test into an invariant: config default must equal
  gateway.turn_lease.DEFAULT_LEASE_WAIT instead of pinning the 1800 literal.
- Rewrap the dangling 'Released' comment line in gateway/run.py.

Verified: 20/20 targeted gateway tests, ruff clean; mutation check — with
gateway/turn_lease.py reverted to pre-fix main the module fails, restored it
passes.
2026-08-07 18:23:57 +05:30
poisdahl
3a3aed3c1f fix(gateway): keep pending turn lease acquires registered 2026-08-07 18:23:57 +05:30
texasich
b3e9e91709 fix(gateway): configure turn lease timeout via yaml 2026-08-07 18:23:57 +05:30
texasich
b2b681fefd fix(gateway): harden turn-lease timeout rejection 2026-08-07 18:23:57 +05:30
texasich
29af112cd4 fix(gateway): fail closed when session turn lease times out 2026-08-07 18:23:57 +05:30
Gille
f346458f29 fix(cron): surface initial scheduler registration failures 2026-08-07 17:45:06 +05:30
WintleChoung
e245a98781 fix(matrix): propagate sender MXID + reply context to MessageEvent
The Matrix adapter built MessageEvent from inbound room events but dropped
the sender's MXID and display name on the event itself -- only 'source'
carried them. Other adapters (signal/slack/telegram/discord/mattermost/irc)
have the same gap; this PR fixes matrix and adds the supporting
top-level MessageEvent fields so the rest can follow.

Downstream effects for matrix specifically:
  - gateway prompt assembly can now read event.user_name (or source)
    without having to dig into source per platform
  - reply context (reply_to_text / reply_to_author_id /
    reply_to_author_name) is parsed from the inline > <@user:server> ...
    Matrix fallback format before stripping, instead of discarded
  - the gateway's existing [Replying to: "..."] renderer can now show
    who the user was replying to (was always anonymous for matrix)

MessageEvent gains two optional top-level fields (user_id, user_name,
both default None) so non-IM producers (cron/webhook/autonomous) remain
unaffected. Source still carries the same values for callers that
already read from there.

Tests cover:
  - non-reply message carries sender user_id/user_name on MessageEvent
  - different senders (alice, bob) both propagate
  - reply message carries reply_to_message_id + reply_to_text +
    reply_to_author_id + reply_to_author_name, parsed from the
    > <@carol:example.org> original question\n\nactual reply shape
  - non-reply message does NOT spuriously set reply_to_* fields

Sibling matrix tests (148 across test_matrix*.py) remain green.

Authored by WintleChoung <cwt@users.noreply.github.com>
Salvaged from PR #80293.
2026-08-07 14:34:55 +05:30
kshitij
416d2a0157 fix(gateway): re-signal interrupts when work is still live at settle-window exit
Review follow-up for the salvaged #79881/#63963 stack: the shutdown
interrupt fires exactly once, but work can materialize AFTER that one
shot on BOTH sibling paths:

- a /v1/runs task admitted before the drain populates
  _active_run_agents only once _create_agent returns
  (queued-before-agent window);
- a _running_agents entry claimed as _AGENT_PENDING_SENTINEL is
  promoted to the real agent by track_agent() on its own schedule,
  after the one-shot walk skipped the sentinel.

Either way the settle loop waited on work nothing signaled, and the
turn went straight to the post-interrupt tool-subprocess kill — the
exact amputation the fix exists to avoid, in a rarer window.

If any work is still live when the settle loop exits, re-invoke
_interrupt_running_agents (which already skips sentinels and folds in
the API-server helper) so late-materializing agents on either path get
the cooperative interrupt. Regression test drives the real stop() path
with an accelerated loop clock and asserts exactly two interrupt
signals.
2026-08-07 14:11:18 +05:30
briandevans
d9ddfb23d5 fix(gateway): interrupt every in-flight API turn on shutdown, not just /v1/runs
The shutdown drain ACCOUNTS for API-server work but never INTERRUPTS it.
`_drain_active_agents()` folds `_active_api_run_count()` into both its wait
loop and its `timed_out` verdict, while `_interrupt_running_agents()` iterates
`self._running_agents` only -- a dict no API turn ever enters, because the
API server owns its own agent lifecycle. `gateway/run.py` states the gap
against itself: "API-server / desk sessions have the same structural gap
(#63529)."

The user-visible result is that every gateway restart with a live API or
desktop turn burns the full drain timeout and then runs
`_kill_tool_subprocesses("post-interrupt")`, which amputates the turn's tool
subprocesses with no cooperative interrupt and no resume marker.

There are seven API agent-entry points. Six funnel through `_run_agent()`
(both session-chat routes, and `/v1/chat/completions` + `/v1/responses` in
streaming and non-streaming form) and are counted by `_inflight_agent_runs`;
the seventh, `/v1/runs`, runs its own lifecycle and is counted through
`_active_run_tasks`. None of the six has a run_id, so the run_id-keyed
`_active_run_agents` cannot reach them, and only two pass `agent_ref` -- which
lands in a caller-local list, not a registry.

So register once at the single unconditional creation site inside
`_run_agent`, beside the existing `_publish_turn_process_ownership()` call,
and unregister in the same `finally` that already clears it. That one
symmetric pair covers all six callers. The registry is adapter-owned and
keyed by object identity, kept separate from `_active_run_agents` because
that dict is run_id-keyed and scoped to the public `/v1/runs` stop API.

`interrupt_active_runs()` then walks both registries, deduped by identity, so
the interrupt set matches the set the drain waits on. The settle window after
the interrupt now polls API work as well: the interrupt is cooperative, and
without this the window closes the instant `_running_agents` is empty -- which
it always is for API turns -- and the tool kill lands on a turn that was asked
to stop microseconds earlier.
2026-08-07 14:11:18 +05:30
dsad
51fa7db469 fix(gateway): interrupt api server runs on shutdown timeout 2026-08-07 14:11:18 +05:30
kshitij
2d9b809ff0 fix(yuanbao): preserve archived history on recall redaction
Sibling-site fix for #80216: yuanbao recall redaction also calls
rewrite_transcript() and was subject to the same archived-history
data loss when active_only defaulted to False. Pass active_only=True
at both yuanbao call sites — load_transcript only returns active
rows, so the redacted content is in the active set and the archived
pre-compaction history should survive the rewrite.

Also drops the stale 'callers that mean to purge (e.g. yuanbao
recall redaction) keep the default' note from the rewrite_transcript
docstring — no caller intentionally purges archived rows.
2026-08-07 13:56:18 +05:30
poisdahl
30c1421acf fix(gateway): make retry archive preservation fail-safe 2026-08-07 13:56:18 +05:30
Adolanium
56fbac6b38 fix(gateway): preserve archived compaction history on /retry
/retry truncates the live transcript to before the last user message
and persists it via SessionStore.rewrite_transcript, which calls
replace_messages() with the default active_only=False. That DELETEs
every row for the session, including the soft-archived
active=0/compacted=1 rows that in-place compaction keeps on disk
(#38763), so any /retry after a compaction permanently wiped the
archived history. #57803 named this call site as a residual gap after
its global-default approach was rejected; the TUI sibling was fixed
in #80195.

The handler now probes has_archived_messages() (new SessionStore
wrapper, auto-exposed through AsyncSessionStore) and passes
active_only=True when archives exist, so only the live rows are
replaced. rewrite_transcript gains an active_only parameter that
defaults to False, keeping the destructive semantics yuanbao recall
redaction depends on. Also corrects the rewrite_transcript docstring,
which still listed /undo as a caller even though /undo soft-archives
via rewind_session.

The regression test drives _handle_retry_command against a real
SessionStore and SessionDB seeded with archived compaction rows and
asserts the archives survive.
2026-08-07 13:56:18 +05:30
kshitij
a658dfe509 fix: address self-review findings on the check_fn/ensure_deps_fn split
- gateway/config.py: rewrite the stale enablement-pass header comment that
  still described check_fn as 'the single source of truth for are-my-env-
  vars-set' / 'lazy-installs it' — both false under the new contract.
- teams: check_requirements docstring wrongly claimed credential checks
  (body checks only SDK/aiohttp presence); derive install_hint from the
  canonical LAZY_DEPS pins + sys.executable instead of hardcoding
  '~/.hermes/hermes-agent/venv/bin/pip' and version pins (wrong under
  HERMES_HOME overrides / profile installs; pins go stale on CVE bumps);
  connect() fatal-error hints now point at the venv pip instead of bare
  system pip (the PEP 668 trap the docs warn about).
- teams docs: drop exact version pins from the two manual-install commands
  (LAZY_DEPS is the source of truth; unpinned installs still work and the
  text can't go stale).
- hermes_cli/status.py: per-entry exception guard around check_fn so one
  raising probe can't abort the listing of all remaining plugin platforms
  (aligns with the other three call sites).
- tests: rename test_register_check_fn_is_active_lazy_installer ->
  test_register_splits_passive_probe_from_active_installer (name said the
  opposite of what it verifies).
2026-08-07 13:28:43 +05:30