Mechanism: Git for Windows on arm64 ships native git.exe but an x86-64 MSYS
usr/bin/bash.exe, which runs under the WoA x64 emulator. With the os-tests
lane's 16 parallel workers, a replayed step's bash dies before running a line
with 0xC000026F (STATUS_WX86_INTERNAL_ERROR) or 0xC0000005 and empty
stdout/stderr, so an unmodified baseline replay fails as a bare
AssertionError (jobs 112635777217 and 112764629268, two different nodeids).
Repro (throwaway workflows on the real runners, previous commit's diagnostics):
- windows-latest-32-arm-core, 16 copies of the file's replay tests at once,
4 rounds: 21/64 copies red, every failure 0xC000026F.
- Same load, windows-latest-32-core (x64): 0/64.
- Passing SYSTEMROOT/WINDIR or the full Windows env to the step: 12/48 and
17/48 vs 15/48 base, so the stripped env is not the cause.
- One process, 32 threads, 600 replays: 0 crashes; it takes concurrent
processes, i.e. host-wide emulator load.
Seam: _NATIVE_WINDOWS_TOO keeps platforms("linux", "windows") and adds a
skipif for win32 + native_arch() == arm64. The x64 Windows row still runs the
replay on the Windows layout (python.exe-only venv, Git Bash), Linux arm64
keeps it (native bash), and the routing guard that pins the marker passes.
workflow_steps.py is a helper module, not a test module, so pytest does not
rewrite its asserts: a replayed bash step that exits non-zero with empty
stdout/stderr fails as a bare 'AssertionError' with no exit code. Two
Windows arm64 os-tests runs (jobs 112635777217, 112764629268) failed this way
on unmodified baseline replays, and the NTSTATUS needed to tell a crashed
child (0xC0000142 / 0xC0000005) from a bash failure was lost. Put rc (decimal
and hex), the bash used, and both streams in the message.
The resolution-allowlist ratchet flagged the lsof lookup (moved from
_held_open to _held_open_lsof) and the new ps lookup in _ps_git_holder:
both are the stdlib-only launch-time repair asking the OS's own tools,
with an absent tool meaning "unknown" and the lock kept. Rows renamed
and added with that justification.
_still_running also imported psutil inside the launch-time repair, which
runs from hermes_bootstrap before app dependencies can import; it is now
a POSIX os.kill(pid, 0) probe (the lsof/ps branch never runs on Windows).
The launch-time repair runs with whatever PATH the launcher got; the
macOS-only lane runs it with a PATH holding no git (as on the Windows
install that motivated the repair), so the bare "ps" call failed, the
scan answered "unknowable" and the dead index.lock of a killed update
was kept: test_a_merge_killed_writing_hermes_constants_is_repaired_by_the_next_launch
stayed red. ps is now resolved like lsof already is (which, then /bin/ps,
/usr/bin/ps). Probe: dead lock, PATH with neither git nor ps, at
ba2965261a0 the scan returned None (lock kept); now False (reclaimed).
The lsof branch's ps scan counted every lock-keeping git on the machine,
because ps cannot say where a process works. A git commit open in any
other repository (or a sibling test worker's git on CI) therefore kept a
dead index.lock and blocked the launch-time repair after a killed update:
the macOS-only lane failed
test_a_merge_killed_writing_hermes_constants_is_repaired_by_the_next_launch
with "a running git holds the index".
Each candidate git's cwd now comes from `lsof -d cwd`, compared to the
checkout by path components (as on Linux); a candidate whose cwd cannot
be read while it still runs makes the answer unknowable, so the lock is
kept. Probe with real lsof/ps and a dashed git-commit waiting in its
editor: in another repo, 4b3c28c816c named it a holder and the fix does
not; in the checkout, both keep the lock.
Review round 2 on #133077. A killed update's receipt plus a younger lock let
release_dead_index_lock delete the lock of a live `/usr/lib/git-core/git-commit -a`
waiting in its editor: the lock fd is closed there, and the Linux holder scan only
knew `comm == git` with an argv subcommand, so the dashed form read as "no holder".
A competing `git add` then got in and the user's commit died "unable to write new
index file".
- The receipt gate stays necessary, never sufficient. The reclaim now also needs
proof that no git of ANY form works in this checkout (`_held_open(any_git=True)`):
cwd, path arguments (relative ones against the git's cwd) or GIT_DIR/GIT_WORK_TREE/
GIT_INDEX_FILE inside the checkout or its git dir, compared by path components.
A git we cannot place (unreadable cwd/environ, hidepid) keeps the lock.
- Git forms: `git`, a dashed `git-<sub>` from git-core, `git.exe`/`git-<sub>.exe`,
`-C`/`-c`/`--git-dir` globals, aliases (an unknown subcommand counts). The
launch-time repair's own scan inverts its list: only known readers are exempt.
- macOS: an empty lsof is no proof any more; `ps` lists every git (comm, so dashed
forms too) and any lock-keeping one keeps the lock.
- Windows: the psutil scan matches dashed and .exe gits, reads environ, and compares
by path components (`hermes-backup` was read as inside `hermes`).
- A failed retry no longer erases the evidence: `_killed_update_owns` reads the durable
per-run records (read-only) and asks about the last run that STARTED before the lock
was written, so a later failed run cannot discharge the killed one.
- The FIFO test helper kills its blocked git when its own setup assertion fails.
A lock nobody holds open is not proof its git is dead: a git that just created it may not have
opened it yet. The start-of-run reclaim now takes .git/index.lock only when the latest update run
died unfinished (receipt running/interrupted, owner gone) and the lock is newer than that run's
start. Every other lock is left to the age-floor sweep, and the update refuses on it truthfully
(e2e-upgrade test_live_index_lock_is_refused_truthfully went red on fd2214b0ad6).
Windows has no /proc to name a lock's holder, and its unlink-as-probe cannot see a
lock-keeping git with its fd closed (a `git commit` in the editor), so the reclaim keeps the
lock while a git.exe has its cwd or a path argument inside the checkout, and whenever the
scan cannot tell (no psutil, an uninspectable git). Any other git on the machine no longer
blocks it, unlike the age-floor sweep's machine-wide check.
`.git/index.lock` left by a killed git refused every later merge with
"File exists": the stale-lock sweep keeps any lock younger than 10 minutes,
so a `hermes update` killed mid-run (Windows crash cell `mid_fetch`, run
37573293632) or a version probe killed by its own 3 s timeout (#132089)
failed the next update at "Pulling updates...".
- `gitlock.release_dead_index_lock`: drop the lock as soon as the launch
repair's ownership proof (`_early_recovery._release_dead_index_lock`:
/proc fds + lock-keeping gits on Linux, lsof on macOS, the unlink itself on
Windows) shows no live git holds it. An interrupted tree move keeps its own
lock judgement. `_cmd_update_impl` calls it first, under the update lock.
- `version_info._git_version_info`: the read-only status probe runs as
`git --no-optional-locks status`, so a timed-out probe strands no lock.
Part of #132089 (the stale index.lock half; naming why a pre-apply exit fired is separate work)
The stale-lock / aborted-pack sweep ran only in the apply path, after the
snapshot. A run that died in that window left `.git/index.lock` behind with
nothing in the next run to remove it, so the following update failed with
"File exists" until an operator cleaned it up by hand. The sweep moves to the
top of `_cmd_update_impl`, ahead of the plan and the backup;
`clear_git_debris` already wraps locks and aborted-transfer pack temps (and
the partial-clone maintenance keys), and `_sweep_stale` skips everything
while a git process holds it, so the earlier position is safe.
(Salvaged from #132089 work; the receipt stop-reason half of the original
commit is now owned by the overhaul's receipt API and is not carried here.)
Run History only rendered sessions the backend had already materialized,
so an accepted Trigger now was invisible for the tens of seconds the
scheduler took to create the run session (#70826). The queued row is
painted from the click, settles when Run History observes a run that
started after it (any load — a fast 1s poll while queued, the cron.changed
broadcast on event-capable backends, or the visibility poll), and drops
at a bounded 90s timeout so a never-materializing run cannot wedge the
button. A failed request drops the row immediately. Pause deliberately
does not settle it: the pause response only persists enabled=false and
never cancels an already-claimed execution.
Queued design (queued-run row + polling + settle semantics) originates in
#70840 by @nv-cho; this ports it onto current main's
createCronTriggerController wiring.
Fixes#70826
Windows-only tests job 112764655375 (run 37612501107, a YAML-only catalog PR)
went red with "667 passed, 0 failed": test_desktop_update_windows_marker.py
hit the runner's 300s per-file cap mid-file ("(300s exceeded; process tree
SIGKILL'd)"), so its 19 tests were never counted and it was listed under
"no tests ran". Run 37612584210 died the same way. Clean runs of this file
take 220-300s (arm64 hit 296.5s and 299.8s), so it is at the cap on every run.
Two defects:
1. The desktop_updater lane gate never worked. tests-os.yml passes
--ignore-glob='*test_desktop_update_windows_*.py' after `--`, and
run_tests_parallel.py forwards it to `pytest <file>`. pytest applies
--ignore/--ignore-glob only while recursing directories, never to an
explicit file argument, so the flag did nothing. These hand-off tests ran
on every PR, including ones that never touched the updater, even though
the job printed "desktop_updater lane off: skipping ...". The runner now
applies --ignore/--ignore-glob to its own file list (pytest's fnmatch on
the absolute path, relative patterns anchored at the repo root, which is
the per-file pytest's cwd) and says how many files it excluded.
2. test_desktop_that_never_exits_is_not_relaunched_over waited out the
production 150s Desktop-exit ceiling. That one wait is half the file's
budget. The script already honours HERMES_UPDATE_DESKTOP_EXIT_SECONDS
("so the self-tests need not sit it out"), and the posix twin sets it to
2. Set it to 5 and assert that the script's refusal names 5s, so the
override is proven to reach the wait. The verdict under test (refuse
with 4, never relaunch) is the same at any ceiling.
Repro (Linux, CI-shape args):
python scripts/run_tests_parallel.py --files "<windows_marker>:<windows_cwd>:<other>" \
-- --ignore-glob='*test_desktop_update_windows_*.py' -m "platforms and not integration"
base: "Running 3 test files", both gated files run
fixed: "note: --ignore/--ignore-glob excluded 2 test files", "Running 1 test files"
New test_passthrough_ignore_drops_files_the_runner_hands_pytest_explicitly
(--files and discovery x glob/glob-spaced/path) fails 6/6 on base and
passes 6/6 fixed.
Mechanism: the detached build writer in the two "keeps custody until a
detached writer is gone" tests rewrote its heartbeat file in place with
pathlib.write_text, which opens with O_TRUNC and then writes. Custody
SIGKILLs that writer at an arbitrary instant; when the kill lands between
the truncating open and the write, `beat` is left empty and the teardown's
`beat.read_text().split()[0]` raises IndexError (suppress only covered
ProcessLookupError/ValueError). CI hit it on
test_a_ctrl_c_keeps_custody_until_a_detached_writer_is_gone[verbose-slow-cleanup]
(job 112764854735).
Repro: scratch copy of the writer with a 0.1 s sleep between open() and
write() -> both ctrl-c cases and the group-kill sibling fail with
IndexError at the teardown read on every run (5/5).
Fix: one shared _heartbeat_writer helper writes each beat to `<beat>.tmp`
and os.replace()s it into place, so `beat` always holds a complete
"<pid> <n>" once it exists. With the same injection moved onto the temp
write, all three cases pass 5/5. The ValueError suppress that papered over
a partial read is dropped.
hermes.update.run blamed 971 failed CLI runs in Sep 28 - Oct 5 on the apply
stage while hermes.update.stage held zero failed apply rows. Stage marks are
END marks: every pre-apply exit (fetch, channel, branch, merge, HEAD checks)
leaves no mark for the stage it died in, so the run row inferred the stage and
no stage row said it failed. A failed run whose failed_stage names a stage it
never marked now gets one failed row for it.
Rebuilt on the updater overhaul's followups model (#132386): a committed run
that owes follow-ups is a run-level success and its failed stage already has
its own mark, so only failed runs get the row; refusals never do.
Root's login for nous, openai-codex and xai-oauth can live only in its
auth.json `providers.<id>` block, with no pool rows yet. In a profile
that borrows from root, load_pool() seeds it as a `device_code` entry
through the root fallback. The write-time claim in add_entry() kept
every entry except root's pool rows, so the profile's first `auth add`
wrote root's single-use refresh token into the profile's auth.json: a
forked grant (#100339) that the heal cannot match, because root has no
pool row for it.
- add_entry() writes only the added row when a profile claims a
single-use provider. Rows the profile seeds from its own sources are
seeded again on the next load.
- _seed_tokens_singleton() (openai-codex, xai-oauth) no longer seeds
root's providers block into a profile that owns its own rows, the
rule _seed_nous_singleton already applies. On current main such a
profile copied root's refresh token into its own pool on every load.
`hermes -p <profile> auth add anthropic --type oauth` saves a
`manual:hermes_pkce` row in the profile. On the next load_pool(), the
forked-grant heal took every row whose source ends in `hermes_pkce` for a
copy of root's .anthropic_oauth.json whenever root's auth.json held no
Anthropic OAuth row. It moved the profile's token pair into root's file
and deleted the row from the profile: the CLI printed "Added", the login
was gone on the next turn, and root's file now held the profile's
account.
Only a `hermes_pkce` row comes from that file. `manual:hermes_pkce` is
owned by the pool and never written to the singleton
(_commit_anthropic_rotation matches the exact source for the same
reason), so the heal now matches the exact source too. The heal's own
docstring already promises to leave an independent `auth add` grant
alone.
This covers the empty-root case the previous commit enables, and main's
existing variant where root holds only an Anthropic API-key row next to
its .anthropic_oauth.json.
Co-authored-by: Enough <10966420+Enough1122@users.noreply.github.com>
- Decide the profile claim when the row is written (the check from #101827)
and drop the None sentinel from #103712. On current main load_pool() leaves
a profile that borrows from an empty root at the default, so the sentinel
alone never reached the claim path (its test stays red).
- add_entry() re-reads the store after writing. A missing row raises
CredentialNotSavedError; `hermes auth add` exits with that message instead
of printing "Added", and the dashboard pool endpoint returns it as a 400.
- Tests: the two contributor tests become one invariant over every
SINGLE_USE_REFRESH_POOL_PROVIDERS provider (the row is readable, root
auth.json is byte-identical), plus the CLI success-line contract.
- Docs: profiles page, EN and zh-Hans.
Fixes#103694
(cherry picked from commit 0e250a005a3353564642ce70f100d89174a583f2)
Fixes#103694: When a named profile runs auth add for a single-use
refresh provider (e.g. Anthropic, Codex, xAI) and the root store has
no existing rows for that provider, load_pool() sets _borrowed_root_ids = set().
Previously, add_entry() checked if borrowed_ids:, which evaluated to
False on an empty set. It fell back to _persist() -> persist_pool_entries(),
which routed to the update-only _update_root_pool_rows() on root. Because
root had no rows to update, the credential was dropped with no error and
never written to the profile's auth.json.
Checking if borrowed_ids is not None: ensures that profile-scoped pools
persist their new credential directly to the profile store via write_credential_pool(),
transitioning the profile to owning rows for that provider.
(cherry picked from commit b7467a2999832749f540d86530cf463ca0945b04)
(cherry picked from commit 9c604999e233cf6d74a83e4d7c141785b341b0d6)
Follow-up to the cherry-picked #119201 (@KoNit-K): keep main's relocated
test file, flip the one invariant that pinned the refusal (a foreground
call with heartbeat>0 now runs with heartbeat=0 while notify=true stays
refused), reword the refusal so it no longer names heartbeat, and say in
the schema that heartbeat is ignored on foreground commands.
Why: the schema fix (4317ed0e71) only helps providers that materialize the
schema default. Models that copy a background call shape still send
heartbeat=60 on ordinary commands; this week 46 subagent sessions hit the
refusal 4 times each (173 refusals), 38 of them never ran a single shell
command, and the agents burned $316 / 1,498 model calls on retries.
The #115692 reclaim released any running attempt whose claim was older than
max(3 x HERMES_CRON_TIMEOUT, script timeout, 2 h), even when the owner was
healthy and busy. Every cron job that legitimately ran longer than that was
marked `unknown` mid-run, its fire claim yanked from under it and the run
aborted with "lost its durable fire claim ownership" (two weekly jobs on one
install in a single day).
The run monitor now stamps `progress_at` on the execution row every 60 s
while the agent reports recent activity, and the sweep measures the stale
bound from that stamp (claim age for legacy rows). A wedged worker stops
stamping and is still reclaimed once the bound passes.
Shape: the stamper and the inactivity watchdog loop (which reads the same
idle signal) live in a new topical sibling cron/scheduler_liveness.py, so
cron/scheduler.py shrinks (4457 -> 4436 lines) instead of growing.
The build-fail spec polls 30 s for posix.sh to exit after the relaunched
app reports the outcome. That outcome is "manual" (the Desktop build is
owed), and finish() then runs launch_app (1.5 s acceptance) and
stop_ui leave-window, which sleeps the 15 s shim grace before tearing the
UI down: ~13 s of margin, and a slow Linux runner overran it (posix.sh
still alive at the 30 s mark, run 37619760207). The same code passed this
spec on bb177627828 and c4c2d9d8b54; the head that failed differs only by
a Windows test helper. 90 s keeps "no updater is left running" a real
check without racing the grace window.
test_script_killed_before_publishing_the_delegate_runs_no_update failed on
the Windows arm64 lane with PermissionError reading
logs/desktop-update-handoff.log: windows.ps1 appends each line with
Add-Content, and a poll that lands while that write holds the file is a
Windows sharing violation, not a missing line. The poll helper now
retries the read (5 s budget) instead of failing the test on it.
Same test is green on main's latest arm64 run; nothing in this PR
touches the desktop-update scripts.
- record_stop_without_receipt reads the update marker raw and tags initiator=desktop only when the
marker names this run's hand-off (delegate line = us, or HERMES_UPDATE_HANDOFF_PID / our parent),
so a Desktop-started early refusal reads kind=desktop and a CLI run refused by someone else's
update stays cli.
- _move_checkout_to classifies its git error with the shared classifier (git_output_stop_class):
index.lock 'File exists' -> git_index_locked, 'Permission denied'/EACCES -> permission_denied
(now a stop class), else checkout_move_failed.
- _publish_shared_metrics: an interpreter that cannot import the config reader (the -I -S bootstrap
that finalizes a dependency-preparation failure) never emits; it parks the bounded receipt in
pending_updates unless it can tell collection is off (loaded config says off, or no config.yaml),
and the next normal start applies the existing gate (report_pending_updates / begin_process purge).
- The parked copy keeps a stop reason's label only when it is an exception type name (plus its
errno token) or one of Hermes' two fixed phrases; any other text before a colon reads '-'
(a free phrase like 'my project name:' used to survive).
- _exit_after_failed_branch_switch's branch_missing/checkout_move_failed probe cannot raise, so
the exit it names is unchanged even when git refuses.
- hermes update --plan/--check/--list-venv-holders on a managed install records no update run.
- Docs/contract say 'before the apply stage mark' and name the tokens that fire after git moved
and restored the tree; tests drive git_error_stop_class and _zip_stop_class directly.
- test_shared_metrics_update_stop_reasons: every stop token reads as its own class in-process
and through the parked copy (no free text kept), unknown tokens and post-apply tokens fall
back, a committed run that only owes parked changes is partial; the lock and Git-operation
refusals each record one row and leave the holder's latest.json byte-identical.
- test_shared_metrics_install_failures: a PermissionError stop reason now reads its own class
permission_denied (was os_error); ENOSPC reads disk_full; other OSErrors keep os_error.
- relay-shared-metrics.md: the new classes, partial outcome and parked fields.
- relay smoke: the failed update receipt names its fetch failure and the row asserts fetch_failed.
Every pre-apply exit of `hermes update` records one closed token on the
receipt at the exit itself (update_receipt.record_stop_reason ->
stop_class), and shared_metrics_update.update_failure_class maps it to
the run's failure_class. aborted_before_apply stays only as the fallback
for an exit with no recorded reason.
Exits that fire before the receipt opens (update-lock refusal, Git
operation in progress, managed install) write no receipt, as before, but
now record a metrics-only row with the same closed class.
The parked receipt copy keeps stop_class, exit_code, the stop reason's
leading label and the restart/user-action flags, so a parked run
classifies the same as the in-process one. A run that committed and is
owed only an interrupt or the user's parked changes reads outcome
partial (failure_class interrupted / local_changes_parked), never failed.
Two profiles whose cron/ resolves to one store share a store key; when one left, its departure dropped the degraded record the remaining profile still needed (gauge flipped to writable, a second outage notice, and the earlier outage start lost for one-shot cover). Departed stores now exclude the ones still ticked. The notices test pins it (red without the set difference).
The tick-lock case only asserted probe_store() returned an error, which any other
probe failure would also satisfy. Assert the error names .tick.lock so the case
proves that branch fired.
Gate r1 (PM12, quality) Low finding.
forget_homes rebuilt each store path from a departed home KEY
(Path(key) / "cron"). cron/AGENTS.md forbids rebuilding a home from a key
(hermes_home_key normcases), and the realpath only follows a symlinked cron/
while the home still exists: a profile whose home was deleted kept its degraded
record, holding the host-wide hermes.cron.store.writable gauge at 0.
register_ticked_homes now records each home's resolved store (new public
store_health.store_key) while the home exists and hands the departed stores to
forget_homes. gateway/cron_store_notices.py uses the same store_key against
record.store (already resolved), so one normaliser decides "is this the served
home's store" everywhere.
Gate r1 (PM12, 2c + quality + 2ab toothless) Low finding.
clear_run_claim (the dispatch-failure path: interpreter shutdown, execution
creation failure, pool.submit failure) set run_claim to None. The claiming scan
had already ended the recovery window, and a restart drops the in-memory outage
records, so the next scan saw a past-grace one-shot with no claim and retired it
as missed: the job the outage skipped never fired.
Keep {"outage": True} instead. With no "at" it is a stale claim, so the job is
re-dispatched on the next tick, and _retire_expired_oneshot keeps it. At-most-once
is unchanged: the fire_claim CAS still fences every re-dispatch.
Gate r1 (PM12, 2c) Low finding.
Follow-up to #133283 (greptile review). Store records are keyed by the
resolved store path, but the notice sender rebuilt the owning profile from
that path's parent. A profile whose cron/ is a symlink to another disk has
a foreign parent, so neither its outage nor recovery notice reached any
home channel; forget_homes had the same mismatch and never dropped such a
departed profile's record.
Both now match by resolving each served home's own cron/ path.
Follow-up to #133283 (greptile review). When only cron/.tick.lock is
unwritable (e.g. root-owned after a sudo run, in a writable cron dir) the
tick degrades and returns 0, so the ticker records a success; probe_store
checked only jobs.json, so `hermes cron status` said jobs would fire and
`hermes doctor` reported a writable store while nothing ran.
probe_store now treats an existing tick lock it cannot write the same way
as a read-only jobs.json target: both status and doctor surface it with the
usual fix hint.
Follow-up to #133283 (greptile review). The due scan stamps a one-shot's
run_claim; when the later fire-claim save fails the run never starts. The
next saved scan skips it on that live claim and ends the recovery window,
so once the claim expires _retire_expired_oneshot saw a stale claim past
grace and skipped it forever: neither fired nor retired.
The scan now marks a run claim it stamps past grace (only an outage lets a
one-shot through there) with outage=true; such a claim without a fire claim
keeps the job covered, so it fires once the claim expires. A fire claim
still means the run may have started, so at-most-once is unchanged.
Gate rQ quality Lows: the placeholders module imports nothing, so conversation_loop can import it at module level like turn_api_call; the terminal-approval comment now says callers re-queue _interrupt_message, so a system stop must not set one.
The verbatim "{x}" wording check sat between the batch-guard timeout and
the user-stop scenario, mutating _tool_interrupt_reason mid-story. Move
it to the end, alongside the other wording assertions.
Gate rP finding 9 (quality).
The _append_skipped_tool_results docstring still said content "is
formatted with {name}", but the body substitutes with str.replace so a
recorded interrupt reason containing braces renders verbatim. State
that, so nobody "simplifies" it back to str.format.
Gate rP finding 8 (quality).
agent_runtime_helpers (two sites) and message_sanitization imported
_INTERRUPTED_PLACEHOLDER inside the function body. The placeholders
module is a leaf with no imports, so there is no cycle to dodge; import
it once at module level (each module still imports cleanly in a fresh
interpreter). conversation_loop keeps its local import.
Gate rP finding 7 (quality).
message_sanitization.close_interrupted_tool_sequence and the
turn_api_call repetition-loop exit each spelled the same hidden
assistant row literal (content "", display_kind hidden, api_content
_INTERRUPTED_PLACEHOLDER), and the conversation loop's redirect built it
field by field. Add hidden_interrupt_placeholder_row() to the leaf
placeholders module and use it at all three sites so the shape cannot
drift between them.
Gate rP finding 6 (reuse).
_USER_STOP_WORDING restated the three USER_INTERRUPT_REASONS keys with
their values capitalised, so a new human-stop reason had to be added in
two places or it would render as "Turn aborted — …". Drop the mirror
and capitalise the recorded reason; output is identical for all three
keys (probe-B02-5.py).
Gate rP finding 5 (reuse).
fed76c1df2 made both KeyboardInterrupt sites publish
tool_reason="user interrupt", but only the sequential one was covered:
reverting the concurrent worker's tool_reason left the suite green
(the hook read "User sent a new message"). Parametrize the existing
cancelled-hook test over the concurrent path with two calls.
Gate rP finding 4 (2ab).
