Follow-up to the salvaged mirror commit.
- Opt-in (`mirror_to_session: true`, or `hermes webhook subscribe --mirror-to-session`),
default off like cron's `mirror_delivery`. The mirrored text lands with user authority in
the target chat, and on `deliver_only` routes it is the raw rendered payload, so a route
author has to ask for it. Only a real boolean `true` opts in (a YAML string "false" no
longer does).
- The mirror runs inside the routed profile's scope, so a `/p/<profile>/` delivery writes
into THAT profile's state.db. A Telegram DM chat_id is the user's id on every bot; an
unscoped mirror landed in the default profile's DM with the same person (live-probed).
- Tests trimmed to two invariants against a real state.db: an opted-in delivery lands in the
routed profile's chat session and not the default profile's; a route without the opt-in
(including `mirror_to_session: "false"`) never touches the target transcript.
- Docs: webhooks guide section "Replying to a delivery" (default, profile scope, trust note),
CLI reference row, bundled hermes-agent skill reference.
A webhook route that delivers to telegram/discord/... runs in an ephemeral
webhook:<route>:<delivery_id> session. The delivered text never reaches
the target chat's own session, so when the user replies there ("so he's
out?") the agent has no idea it just sent them anything and asks what
they mean.
After a successful cross-platform send, mirror the delivered text into the
target chat's transcript as a labelled user turn ("[Webhook delivery:
<route>]\n..."), the same path and role convention cron briefs use
(cron.scheduler_delivery._maybe_mirror_cron_delivery, #2221). Best-effort:
never fails the delivery, skipped when the chat has no session yet.
Route opt-out via mirror_to_session: false. delivery_info now carries the
route name and the flag for both agent-run and deliver_only routes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZ15jNgPUHdPsMw4wWon6v
#120307 and #120295 merged, so their cells are plain tests and the probe
machinery in core/_pending_fixes.py has no consumer left (known_failure stays).
torn-tree also excuses 'No module named' (a later release pair can die on a new
module instead of a moved name); the gemini listing gate now requires the 401
from the models endpoint, so an outage or 5xx fails the cell.
Turn.frame_idx was the submitter's frame count, but op_shared_owner_drop
reads the watcher's connection, whose history is shorter: the scan
started past the watcher's early deltas and timed out (seed=11,
9:shared_owner_drop:A, ~1 in 4 runs). Record every attached connection's
frame count at submit and scan from the one being read.
CI (the PR merged with main) went red with 26 XPASS(strict): main fixed
the long-scalar re-fold (#119844, 7f595056), the explicit-null strip
(93c5856b), the phantom `agent: {}` (0698be87), the transient-read clobber
on the TUI RPC and dashboard PUT (db2f07c5/061da070) and the
`hermes-agent --help` argv bug (#54648). Their marks and the workarounds
that kept them out of other cells are gone.
Still-open gaps XFAIL only on their own failure message
(_pending_fixes.known_failure): #119928 env-lock refusal reported as
success (fix#119929) and the torn-tree update dying at import (fix
#120339). The dashboard explicit-null cell was only xfailing because seed
605 no longer generates a null leaf; it uses seed 607 and passes.
_base_config_version imports N-1's DEFAULT_CONFIG in the N-1 venv instead
of string-matching `git show` source.
#120319, #120374 and #120299 are on main, so their probes and every Gap
naming them go (the module's own rule); those cells are plain tests now.
The two static strict xfails with an open fix PR (#95375 cli
resize_scrollback, fix#120321; #62259 Gemini listing, fix #62267/#116509)
turned main red the moment the fix merged (XPASS). They now go through
_pending_fixes.known_failure: a run-time xfail only while the cell fails
with that gap's own message (a turn rendered more than once; an empty live
listing), any other failure stays red, and the fix just makes it pass.
The terminal job only needs the Ink TUI, not every workspace (desktop/electron).
The e2e-upgrade bwrap check used different flags from _bwrap_usable (no --proc,
no --die-with-parent), so it could pass while the suite fell back to running the
real updater unsandboxed; it now asserts _helpers.BWRAP_OK itself.
It scanned the connection's frames from 0, so any earlier turn's delta on the same
session matched at once and the interrupt/drop op fired before the new turn
streamed.
Collection no longer runs git (every CI shard imported the module and paid for
three git calls, and a git failure broke collection instead of skipping). The N-1
venv now comes from N-1's uv.lock via `uv sync --locked --extra all` into
install/venv with the user's uv config hidden, the installer's tier 0, instead of
a fresh resolve of .[all] against today's PyPI.
rc != 0 also counted the harness's SIGKILL (-9) after a 240 s hang as a fail-fast.
The cell now needs a real non-zero exit AND the resolver's
"provider 'custom' resolved without credentials" message, under a 90 s leg
timeout (the fixed tree exits in ~5 s).
A strict xfail(raises=AssertionError) also swallowed boot failures and timeouts, and
flips main red (XPASS) the moment its fix merges. Each gap now has a probe that
reproduces the defect's mechanism on the tree under test; the xfail applies only
while the probe reports the defect open, and only for the bug-specific exception
class the cell raises when it observes that leak (TenantLeak, LaunchProfileBleed,
RoutingLeak, PromptLeftPending). Same pattern as #120344's delivery suite.
The six provider keys sat in job-level env, so every step saw them: uv sync
(and any sdist build backend it runs), setup-uv, checkout and the retry
action. The job now carries only secrets.X != '' booleans for the gate, and
the values are set on 'Run live canaries' alone. actionlint clean.
run_tests.sh starts every run from env -i with an allowlist, so the e2e
job's HERMES_E2E_REQUIRE_TUI=1 never reached the terminal suite (a missing
ui-tui build skipped instead of failing) and the upgrade suite's CI branch
in sandbox_required_reason() was dead. With ui-tui/dist removed and
HERMES_E2E_REQUIRE_TUI=1: before, 2 skipped; after, 2 failed.
An approval.respond RPC can end the wait before the gateway attaches the hook
that withdraws the sent request, so it stays in open_requests. Red in 1 of 4
loaded union runs, deterministic with the gap widened, green with #120374.
A race cannot be strict; drop the mark when #120374 lands.
tests/e2e/core/terminal drives the real `hermes --tui` over a PTY, so the e2e
job now installs the Node workspaces and builds ui-tui, and
HERMES_E2E_REQUIRE_TUI=1 makes a missing build fail instead of skip.
tests/e2e/core/upgrade runs a real N-1 -> HEAD `hermes update`: it needs full
history + tags, bubblewrap (every updater runs sandboxed so it can never reach
a real gateway or systemd), the warm uv cache, and up to ~15 min for one file.
It gets its own 60-minute job instead of stretching the e2e job.
The C7/C11 suites were built on test/core-tenancy next to seven production
fixes that land as their own PRs. Without them, exactly four cells fail on
base: the bare-custom OpenRouter key leak, the TUI /model switch legs
(explicit --provider vs alias, Anthropic refresh onto a foreign host), the
gateway's shared 'default' terminal env for routed cron work, and the Desktop
backend's launch-profile model/.env. Each is strict, so it flips red once its
fix lands and the mark comes off in the same PR.
Runs tests/e2e/core/live (`-m live`) nightly, on `v*` tags and on
workflow_dispatch (optional -k filter). Secrets-gated on LIVE_*_API_KEY
repo secrets (each case skips without its key; the job no-ops when none
are configured), main-repo only, one run per ref (never cancels a release
gate), 25-minute timeout. Uses direct pytest because scripts/run_tests.sh
starts from `env -i` so no credential can reach a test. Publishes a
usage/cost table to the step summary and uploads junit + usage JSONL.
(cherry picked from commit 75c5656ff3905512bf93c1fd887bbd5e85396f29)
Classes: C9 provider wire-format drift, C17 prompt-cache hits, C11 real
auth / credential routing / `/models` parse.
Mocks encode our own belief about each vendor's wire format; vendor-side
schema changes, reasoning-replay rules, streaming shape changes and cache
behaviour are only observable against the real APIs. This drives the REAL
AIAgent + real adapters (chat_completions, anthropic_messages via OpenRouter,
codex_responses for xAI/OpenAI, GeminiNativeClient) in a temp HERMES_HOME
through a scripted 3-turn conversation with a deterministic registered tool:
turn 1 one forced tool call; valid JSON args; result round-trips
turn 2 two tool calls in ONE assistant message; both results round-trip
turn 3 no tools; answer from replayed history (tool results + reasoning
replay) under a cache breakpoint
Invariants per turn: no 4xx on any agent-loop request (429 excepted), the
turn completes, no <think>/DSML/control-token/raw tool JSON in user text,
the credential resolved from env is the one sent and ONLY to the provider's
host, tool results are persisted to state.db. Anthropic-family routes also
assert 1..4 well-formed breakpoints (none on role:tool / inside
tool_result.content[]) and cache-read tokens > 0 on turn 3 (warm once).
A `/models` listing check per provider uses the live fetchers without the
curated fallback (Gemini is strict-xfail on open #62259).
Spend: cheap models, <= 3 turns, max_tokens 2048, max_iterations 6, a hard
per-test token and $ guard; usage + estimated $ printed per test and
appended to $HERMES_LIVE_USAGE_FILE. Every case skips cleanly without its
key. `live` is excluded by default addopts; select with `-m live`.
(cherry picked from commit 3bd7de6991bcb03bfe5d11945de6f2317c534b7d)
Class C6 (bricked installs, stale code, lost state after hermes update). A local bare
origin (--shared, no network) parks main at release N-1; a git-mode install with its
own uv venv gets user state written by the N-1 CLI itself (sessions, a named profile,
a cron job, a hand-edited config with comments / long quoted values / a legacy MCP
disabled flag), then the real `hermes update --yes` runs to HEAD.
Legs: clean, local edits + orphan autostash, SIGKILL mid-fast-forward (lock-only must
heal; torn tree pinned as a strict xfail: it bricks every entry point), SIGKILL when the
dependency sync starts (new code on the old venv must heal), origin unreachable (must
fail non-zero and change nothing). After each: exit code matches reality, the venv
serves HEAD's tree (every top-level package + HEAD-only modules import from outside
the checkout) and satisfies HEAD's dependency set, --version / doctor / one-shot turn
succeed in fresh processes, both profiles' state.db pass integrity_check with unchanged
rows and byte-identical messages, the cron job survives, and config.yaml equals HEAD's
own non-interactive migration of the pre-update bytes (plus independent value/comment
checks). Everything runs in the bubblewrap sandbox so the updater's all-profile gateway
scan and systemctl calls can never reach a live install.
(cherry picked from commit 9b275a1dfc644bdddf6ee77091a7126aeb536722)
Class C18 (config/settings persistence): seeded-random property tests on the real
load/save/set path. Round-trip byte stability of untouched keys, config set changes
exactly one key (in-process, hermes config set subprocess, tui_gateway stdio JSON-RPC,
dashboard PUT /api/config), a failed/empty read never clobbers config.yaml (#113301
class), .env loading idempotent incl. self-references (#109902), every migration from
all historical _config_version values idempotent, and every top-level section tolerates
null. Open bugs found are pinned as strict xfail (TUI/dashboard empty-read collapse,
save_config dropping explicit nulls, #119844 refold, #119928).
(cherry picked from commit 66f808b2ec64a80a3c270e3e8d8e99c9d33ade9a)
Class C6 (bricked installs / stale modules after hermes update): every module the
package ships is imported from a clean first-party sys.modules in a fresh sandboxed
interpreter; a stale-graph leg replays the pre-handoff v2026.9.14 updater graph and
imports the post-purge restart modules (file_signature burst #111942 class); and a
matrix of real entrypoints (--version, doctor, -z one-shot against the fake provider
with state.db integrity_check, serve READY on stdout + clean SIGTERM with no orphan,
console scripts) runs in fresh processes. Every process is bubblewrap-sandboxed (own
PID namespace, no user systemd bus, real ~/.hermes read-only) so no probe can reach a
live install.
(cherry picked from commit 6635ace39ae6c04d9391da7e1321277a28f8b219)
Class C2 (duplicate / vanishing / garbled transcript rendering) on the terminal
surfaces. Drives a real `hermes chat --cli` and a real `hermes --tui` (Node
frontend + tui_gateway child) as session leader of a real PTY against the
recording fake provider (long multi-chunk stream, tool-call turn, reasoning
turn, width resizes mid-stream via SIGWINCH), renders the byte stream through a
dependency-free VT emulator (_vt.py, cross-checked identical to pyte on
captured CLI and TUI streams) and asserts on the settled grid + scrollback:
every reply exactly once and verbatim (== scripted stream), every prompt echoed
once, conversation order, tool call once, reasoning at most once, rendered ==
persisted state.db rows, /exit -> 0 with no process left in the PTY session or
among tracked descendants (setsid'ed helpers included).
cli-resize_scrollback is a strict xfail (raises=DuplicateRender) for the open
live bug #95375: a width-changing resize on a 24-row terminal re-prints turns
that already sit in scrollback (warmup reply rendered 3x).
Red-proof: re-render final panel after stream, double-appended deltas,
tui_gateway double event emit, resize replay without clear, gateway outliving
the TUI (EOF+SIGTERM ignored), detached helper leak -> all red.
(cherry picked from commit 9e1ab342650fcbee791f0cc35a3ec679052e9492)
Class C20 (interactive prompts): a clarify question or dangerous-command approval
never shows, is lost on a session switch / reconnect / the other window, the answer
never reaches the tool, the approval outcome is ignored (denied command runs), or a
prompt stays pending after the turn.
test_interactive_roundtrip.py reuses the lane's real `hermes serve` harness with
approvals.mode=manual (not yolo). The fake provider issues clarify and
terminal(`rm -rf <victim>`) tool calls and records the tool result the agent sends
back. Matrix {clarify, approval-deny, approval-approve} x {answer at once, after
switching away and back (re-delivered via session.resume open_requests), from a
second connection, after an abrupt ws drop + reconnect, via the approval.respond
RPC}. Asserts the server->client request arrives in bounded time, the answer
reaches the tool (clarify user_response / "denied by user" + victim intact /
exit_code 0 + victim gone), the other window gets a card-settling signal,
approval.pending and open_requests are empty afterwards, and the turn completes.
Red-proof (each fails, restored after): clarify callback drops the answer;
approval response choice ignored (deny runs the command); open_requests not
re-delivered on resume; approval.respond RPC routing dropped.
(cherry picked from commit 372e0c04e56c54ceb237937bae7a990212fcfbe6)
Class C14 (multi-client session ownership / routing / switching): prompts persisted
to a stale session after a switch, events rendered twice or leaking into another
chat, zombie leases ("already has a live owner"), a ws drop or client absence
killing the in-flight turn, a live session hard-deleted.
tests/e2e/core/terminal/_gateway_client.py spawns ONE real
`python -m hermes_cli.main serve --port 0` per module (Desktop's argv) in an
isolated HOME with only the recording fake provider, and drives it with
Desktop-shaped WebSocket JSON-RPC clients (gateway.ready, client.capabilities,
server->client request answers, abrupt TCP drop, reader pause for absence).
test_multiclient_session_model.py runs a seeded fuzzer (seeds 11/23/37) over 3
clients: create, switch(session.resume), fast and slow prompts, interrupt
mid-stream, delete of live (must be refused) and closed sessions, ws drop +
reconnect + resume mid-turn, private-chat owner death (idle and running) +
takeover, shared-chat owner death, client absence. A reference model checks after
every step and at the end: prompts persisted exactly once, in order, to the
session selected at send time; per-connection event seq strictly increasing (no
duplicate delivery); no events for sessions a connection never attached to;
canaries only in their own session; one live runtime per chat; orphaned runtimes
released in bounded time and resumable/promptable by another client; drop/absence
never cut a turn short (full reply persisted, message.complete status complete).
Red-proof (each fails, restored after): duplicate event write in write_json;
semantic revert of 67de93862c (#98028/#100325, orphan reaper ignores turn
activity); semantic revert of de25545dce (rebind instead of fan-out); lease never
released; session.delete of a live session not refused.
(cherry picked from commit fe078a891231d3ba459da4ca46bf2a06979f4444)
Class C7 (multiplex / tenancy isolation). The real app backend, 'hermes serve'
with HERMES_DESKTOP=1 (in-process cron ticker), driven over /api/ws exactly like
the Desktop: session.create {profile}, interleaved prompt.submit turns, a cron
tick per profile, session-bound and profile-bound config.set, session-bound
model.save_key, profiles.create + 'hermes profile delete' next to the live
backend (PID unchanged), restart + session.resume + a second cron tick. Same
canary invariant as the gateway suite plus: each settings write changes only
the addressed profile's config.yaml/.env, and no RPC reply/event for a session
carries another tenant's canary.
Red on the parents of 6952f1d5425 (lazy gateway.run import latched a secondary's
terminal.* into the launch env), 5a14abea326 (save_key wrote the launch .env) and
1e3727666e5 (secondary session reported/persisted the launch model).
(cherry picked from commit 522858661a60abc47ece47d977d73c54067789bc)
Class C7 (multiplex / tenancy isolation). A real 'hermes gateway run' with
gateway.multiplex_profiles serves default + alpha + beta, each with its own
loopback provider (accepting only its own key) and distinct canaries: provider
key under the same env var name, API-server key, .env marker, model, MEMORY.md,
SOUL.md, terminal cwd, cron prompt and an exported shell variable. Interleaved
api_server turns (/v1, /p/<name>/v1) each run 'env | sort; pwd' in the terminal
tool; a cron tick fires one job per profile; cross-profile API keys; profile
create/attach/delete next to the live host (PID must not change); SIGTERM
restart + a second cron tick. After every phase: no request, tool-subprocess
env snapshot, profile-home file (state.db + WAL, logs, sessions, config, .env)
or gateway output carries another tenant's canary.
Red on the parent of a445f346a8b (cron shared the launch profile's terminal
environment); see REPORT.md for the red-proof table.
(cherry picked from commit ba5bdae9aa9900d719be36bf9efa40037860cf0b)
Class C11 (provider/model routing + credential resolution). One real
loopback OpenAI-compatible host per provider identity, each accepting only
its own key; every scenario configures all identities and only changes the
selection, and one invariant runs after every leg: requests land only on the
selected hosts, each with only that host's key, the user sees that host's
answer, and nothing egresses to a real inference API (CONNECT trap).
Real `hermes chat -q` / `hermes -z` subprocesses cover custom base_url
(key_env / literal), bare custom fail-fast, named providers: and legacy
custom_providers entries, startup -m aliases (#103933/#107191/#109440),
fallback after 401/429, credential pool rotation / single-key exhaustion,
per-profile routing, aux title + delegation routing. The real stdio
tui_gateway walks mid-session /model switches incl. pool recovery without
restart.
fake_llm_provider: api_key may be a list of accepted keys (pool on one
host) and opt-in record_get; defaults unchanged.
(cherry picked from commit bf1f07571d7777fe91b16f386e026c12bdbfe73b)
Register fr/de/es in the Locale type, catalog and Appearance picker, map
display.language values and regional aliases (fr-CA, de-AT, es-419, ...)
to them, and add behavioral coverage: catalog key/kind/placeholder parity
with English, config round-trip for every picker option, display.language
loading, intro rotation, and slash descriptions after a locale switch.
Fixes#75282Fixes#89980
Full Translations object for es (neutral tú imperatives), reusing the
current-key strings from the community Spanish locale PRs and translating
the rest, plus the display-only intro rotation.
Co-authored-by: Kiro 有点Yes <246816394+sdyckjq-lab@users.noreply.github.com>
Co-authored-by: Jordi <satiro_jordi@hotmail.com>
Co-authored-by: Tormen28 <enriqueyepez28@gmail.com>
Co-authored-by: Eduardo Christopher López Acevedo <echla@gmail.com>
Full Translations object for de in the formal Sie register, reusing the
current-key strings from the community German locale PR (rewritten from du
to Sie) and translating the rest, plus the display-only intro rotation.
Co-authored-by: Oliver Hees <oliverhees@gmail.com>
