A local-only rc.* tag (a hand-made ref or a claim that never pushed) counted
as an outstanding attempt, so release refused and abandon reported 'no
outstanding attempt'. The remote is the only authority on claims now.
The pipeline seeded its version line at 0.21.4, but the legacy CalVer line
already published "Hermes Agent v0.21.5" as v2026.9.24. A first `--bump patch`
cut therefore derived 0.21.5 again: installs already reporting 0.21.5 would
compare equal and never see the new build as an update, and the release page
would carry two v0.21.5 entries.
Seed at 0.21.5 so the first pipeline patch is 0.21.6 and the first minor stays
0.22.0. Entrypoint fixtures now pin their own published head instead of
tracking the production seed.
#133931 made source_build.py import hermes_cli/web_build_limits.py but did not add it to
_UPDATE_DEPENDENCIES, so tests/ci/test_update_ci_routing.py has been red on main and on
every PR whose merge ref includes it: a change to the module would skip the
e2e_upgrade and e2e_desktop_update lanes that run the build it shapes.
The dashboard build (Vite 8 + Rolldown + Tailwind v4) saturates every CPU
on small hosts (200%+ top readings, OOMs and frozen SSH sessions on 2–4
vCPU VPS boxes). Rolldown exposes no JS-level worker knob, but its native
binding parallelizes with rayon, which honours RAYON_NUM_THREADS.
Cap both sides of the build, everywhere it runs:
- scripts/build/web.mjs (the npm run build / packaging entry): limit the
native bundler's rayon pool to half the available cores before vite is
imported, and re-exec once with a --max-old-space-size cap (sized from
the cgroup memory limit, 75%, floored at 1 GB, ceiling 4 GB) unless
the user already set one.
- hermes_cli/web_build_limits.py + build_source_web(): the same caps for
the install/update/dashboard-triggered source build path.
User env always wins (existing NODE_OPTIONS heap flag or
RAYON_NUM_THREADS is preserved); HERMES_WEB_BUILD_MAX_OLD_SPACE_SIZE /
HERMES_WEB_BUILD_THREADS tailor the caps, and HERMES_WEB_BUILD_LIGHT=1
tightens them (1 thread, 1 GB heap) for hosts that cannot spare full CPU
during the build — the light-build idea from the closed PR #78131.
Windows-only tests job 112764655375 (run 37612501107, a YAML-only catalog PR)
went red with "667 passed, 0 failed": test_desktop_update_windows_marker.py
hit the runner's 300s per-file cap mid-file ("(300s exceeded; process tree
SIGKILL'd)"), so its 19 tests were never counted and it was listed under
"no tests ran". Run 37612584210 died the same way. Clean runs of this file
take 220-300s (arm64 hit 296.5s and 299.8s), so it is at the cap on every run.
Two defects:
1. The desktop_updater lane gate never worked. tests-os.yml passes
--ignore-glob='*test_desktop_update_windows_*.py' after `--`, and
run_tests_parallel.py forwards it to `pytest <file>`. pytest applies
--ignore/--ignore-glob only while recursing directories, never to an
explicit file argument, so the flag did nothing. These hand-off tests ran
on every PR, including ones that never touched the updater, even though
the job printed "desktop_updater lane off: skipping ...". The runner now
applies --ignore/--ignore-glob to its own file list (pytest's fnmatch on
the absolute path, relative patterns anchored at the repo root, which is
the per-file pytest's cwd) and says how many files it excluded.
2. test_desktop_that_never_exits_is_not_relaunched_over waited out the
production 150s Desktop-exit ceiling. That one wait is half the file's
budget. The script already honours HERMES_UPDATE_DESKTOP_EXIT_SECONDS
("so the self-tests need not sit it out"), and the posix twin sets it to
2. Set it to 5 and assert that the script's refusal names 5s, so the
override is proven to reach the wait. The verdict under test (refuse
with 4, never relaunch) is the same at any ceiling.
Repro (Linux, CI-shape args):
python scripts/run_tests_parallel.py --files "<windows_marker>:<windows_cwd>:<other>" \
-- --ignore-glob='*test_desktop_update_windows_*.py' -m "platforms and not integration"
base: "Running 3 test files", both gated files run
fixed: "note: --ignore/--ignore-glob excluded 2 test files", "Running 1 test files"
New test_passthrough_ignore_drops_files_the_runner_hands_pytest_explicitly
(--files and discovery x glob/glob-spaced/path) fails 6/6 on base and
passes 6/6 fixed.
- test_shared_metrics_update_stop_reasons: every stop token reads as its own class in-process
and through the parked copy (no free text kept), unknown tokens and post-apply tokens fall
back, a committed run that only owes parked changes is partial; the lock and Git-operation
refusals each record one row and leave the holder's latest.json byte-identical.
- test_shared_metrics_install_failures: a PermissionError stop reason now reads its own class
permission_denied (was os_error); ENOSPC reads disk_full; other OSErrors keep os_error.
- relay-shared-metrics.md: the new classes, partial outcome and parked fields.
- relay smoke: the failed update receipt names its fetch failure and the row asserts fetch_failed.
Nothing tested mac_bundle_recover(), the start-of-run repair that finishes or
rolls back a Hermes.app swap a killed hand-off left half done.
posix.sh --self-test-swap-interrupt (body in swap-selftest.sh) runs the REAL
mac_swap on a fixture bundle tree in its own process group. mac_swap now
names its step boundaries (swap_checkpoint start/staged/aside/installed); the
self-test makes the swapper SIGSTOP itself at one of them and SIGKILLs the
whole group there, or resumes it and SIGKILLs it mid-ditto / mid-cleanup rm.
It checks the wreck has the expected shape, runs the REAL mac_bundle_recover,
and asserts the target is the complete old or complete new bundle (whole-tree
checksum, version marker included), no .old/.new/.hermes-update-* is left,
and a second recovery changes and logs nothing. Seven cells: before-stage,
mid-stage, staged, aside, installed, mid-cleanup, complete (control).
The seams are inert in a real run: SWAP_PAUSE_AT is set only inside the
self-test, and MAC_DITTO stays /usr/bin/ditto unless the self-test runs where
ditto is absent (Linux: cp -R).
test_desktop_update_mac_swap_interrupt.py drives the arm under
platforms("posix"): the Linux suite and the macOS lanes of tests-os.yml
(real /usr/bin/ditto, macOS /bin/bash 3.2).
Get-DefaultBrowserExe read only the legacy UserChoice ProgId, which Windows
11 25H2 Settings no longer updates, so the update hand-off fell back to the
WinForms card (or a stale browser) on machines whose real default is
Chrome/Edge. Read UserChoiceLatest\ProgId first, legacy UserChoice second.
Fixes#108051. Follows #108062 / #108064.
Co-authored-by: KoNit-K <konit.block@protonmail.com>
Co-authored-by: Konstantin Khlopkov <konstantin.khlopkov93@gmail.com>
Importing a/b/c.py runs a/__init__.py and a/b/__init__.py first. The
update-lane routing listed hermes_cli/local_runtime/processes.py (the
v2026.9.24 updater imports it after the pull) but not the package init
that import runs. #134116 changed what that init loads, its PR never ran
e2e-upgrade, and main went red until #134242.
classify_changes now routes the package inits above every module the
e2e_upgrade and e2e_desktop_update lanes own, and above each shared hub,
because a hub's inits run whenever the hub is imported. An init that is
itself a hub stays out. The routing guard's import walker counts the
package inits a module's imports run, so the next unrouted one fails it.
This routes eight inits that were missing: local_runtime, gateway, cron,
tools, tools/environments, plugins, hermes_platform,
hermes_cli/observability.
HERMES_PREVIEW_FULL_CONNECTORS=true|false makes the free-tier mint send
{"preview_full_connectors": <bool>} in the /api/anonymous/create body, so
the account service can record the cohort on the account and gate
toolkits from it. Any other value (unset included) keeps the body {}.
The variable joins the desktop bundle-env allowlist (release script,
banner writer, channel request decoder, BUILDING.md) so a bundle can
bake it in. It is self-reported and visible in the packaged JS: the
service must treat it as a preference, not proof.
tirith ran `tirith check` on every terminal command, default-on with fail-open, as a
pinned PM binary auto-downloaded at startup. Measured against the shipped 0.4.2 binary
it caught 14/16 attack commands but flagged 12/24 everyday agent commands
(`echo $((6*7))`, `"$VAR/x.sh"`, `gh api | python3`, LAN curls, `.dev` hosts,
`pip install` via analysis_incomplete), and in cron/-q deny mode each of those is a
hard block. It only ever saw terminal strings (execute_code, writes and the browser
bypass it), fail-open meant no protection wherever the binary could not install
(Windows, Termux, x64 musl), and it produced a steady issue/PR stream (fail_open
semantics four times, install loops filling disks, a SIGSEGV hang, session-approval
scoping, third-party promo text on approval cards).
Removed: tools/tirith_security.py, the tirith branches of the command guard (the
is_tirith tuple, session-only "always" downgrade, import-error/fail-open paths,
unattended tirith pass), the CLI and gateway startup installers, the PM package and
lock pins, the nix runtime dep, security.tirith_* defaults, the CLI warning string
and three tirith tips (gaps refilled from the tail so translations survive), plus
the gateway's "no automated assessor" warning that only existed to point at tirith.
Config v50 drops security.tirith_* and enables nothing in its place. Hermes ships
no tirith plugin; if tirith upstream publishes one, it goes through normal catalog
admission under their name.
Two ratchets this suite arms for its own siblings fired once they landed:
- the Windows pause (#132338) now imports gateway/status_inline_source.py
to recognise inline-bootstrapped gateways, so a change there starts the
e2e_upgrade suite like gateway/status.py (only two importers: not a hub)
- the marker-corpus consumers #132345 and #132354 ship are on main, so
_LANDS_IN_SIBLING_PR empties as its comment asks
`hermes_cli/_early_recovery_zip.py` (split out of `_early_recovery.py` on the
commit-point PR) is imported by the update path, so a change to it must start
the update pipeline and install E2E suites. Both lists and the install-e2e path
filter now match the `_early_recovery` prefix, which also routes any later
sibling. `remote-lifecycle-v2-marker.test.ts` reads the update-marker corpus
and is now a listed consumer; `remote-update-marker-programs.test.ts` exists in
this tree, so its sibling-PR allowance is gone.
Replaying the last 131 non-merge main commits (4787e4d56f) through the
classifier: main selects e2e_upgrade 5 / e2e_desktop_update 3, the CI branch
12 / 10. Four of the seven new hits per lane touched only plugins_cmd.py or
plugins_cmd_install.py. migrate_all_homes reaches them only when a home's
configured memory provider left core (the catalog install); no upgrade or
Desktop update journey configures one, so those suites cannot observe the
change, and the unit lane already runs on every .py edit. Measured on a
fresh home: memory_provider_migration, pm/plugins_state, agent/memory_provider
and plugins/memory run; plugins_cmd_install executes only its module body,
plugins_cmd is never imported.
Kept, because a fresh-home update runs them: memory_provider_migration.py
(migrate_all_homes on every source build), plugins/memory/__init__.py (the
post-update Honcho profile sync), pm/install.py (sealed() /
lazy_installs_allowed() gate every update's default-tool install), and
scripts/build/ (frontend-common.mjs is imported by freshness/web/tui/desktop).
After: 8 / 6. A guard test fails if the journey-home migration starts running
plugin-install code, the signal to route those modules again.
update_cmd_commit._admitted_python now resolves python3.N through hermes_platform.resolver
(cd471c9c55a), so the update path imports the resolver package and a change to it must start the
upgrade E2E, like hermes_platform.host already does.
What: scripts/ci/classify_changes.py names hermes_cli/git_pack_tidy.py in the update pipeline, so a
change to it runs e2e_upgrade.
Why: the derived routing guard (test_every_module_an_update_entry_point_imports_starts_its_suite)
found it imported by the update path and unrouted: main's partial-clone tidy (a0f47657b9) runs in
every update's pre-fetch via update_cmd_check.
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
The stack's health refactors carved main_desktop_tcc.py out of main_desktop.py
and status_inline_source.py out of gateway/status.py; both are imported on the
update path, so an edit to either must start e2e_upgrade (and, for the desktop
module, e2e_desktop_update). Route each facade by prefix so its <stem>_*
siblings are covered without naming files this branch does not have yet.
The desktop gate's SSH-v2 marker fix added remote-update-marker-programs.test.ts,
which reads the shared marker corpus: list it as a consumer, allowed absent until
#132345 lands, like the other gate-PR consumer.
Route the exercised subprocess and provider-migration dependencies to both
real-update journeys. Keep backup routing under an explicit dispatch test.
Use the aggregate's existing detect outputs to reject skipped or missing
update consumers while retaining legitimate unselected PR skips.
Verified with the real classifier and aggregate subprocesses against
b9dec76a042, composite ff5c8f1a64b, and fixed code. Canonical runner:
571 passed, 0 failed, 1 native-Windows skip across CI and marker corpora.
(cherry picked from commit a082ca3c62ed49382f173a07882d4280d0ba7c9c)
(cherry picked from commit ed9f2028256b8b1d6af81ef97cd472f30d2cb042)
(cherry picked from commit 6d5cc740b6a75d476802222c2f4754855a5712b0)
tests/fixtures/update_marker_corpus.json is read by pytest (update_lock),
cargo (marker_tests.rs), vitest (update-marker-corpus.test.ts) and the
marker.sh / marker.ps1 corpus tests. By path it selected only python, so a
corpus edit never ran the Rust, Electron or Windows PowerShell readers.
_SHARED_FIXTURE_CONSUMERS routes a fixture with its consumers' test lanes;
the routing test derives the consumers from the tree and replays the
classifier through the workflow files to each lane's job (review D14).
(cherry picked from commit 4893ef31833b5fcbdfc527beb70724ea63a3eb8f)
(cherry picked from commit 20c04a27b2a04ebaabb9a9df3693339243d9380e)
main added hermes_cli/left_core_migration.py, imported by source_build.py; the
R15 import-graph test caught it unrouted on the composite of all seven PRs.
(cherry picked from commit 338aabcdda316a92b8bfcae10f52730fd6fcb344)
(cherry picked from commit 5a839bb475d08f82293f07bcdb60ded859814476)
known_failure XFAILs a gap while its fix is an open PR, which is right
on a PR and wrong on the acceptance run of the integrated batch: there
it accepts exactly the gap the batch claims to close (the final
composite's Windows run passed with `test_crash_cells.py (3✓ 1xf)`).
HERMES_E2E_STRICT_ACCEPTANCE=<owner>[,<owner>] (or 1 = every gap) makes
the matching known_failure / known_gate blocks re-raise the gap's own
assertion. It is set only from the `strict_acceptance` dispatch input of
ci.yaml (threaded through tests.yml / tests-os.yml to every E2E step)
and of windows-install-update-e2e.yml; pull requests never set it.
run_tests.sh forwards it through its env -i allowlist.
Red on 8cc78279a36: test_pending_fixes_strict.py (a real pytest process
per mode) — the batch-owned gap XFAILs under strict mode.
(cherry picked from commit ca1e7e749a61263181e89aa687dec366d95fda18)
(cherry picked from commit 7768bf3ef8e15fb7f228ebad619d6a9a85ad4100)
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
Offset the FILE_LINES growth the moa preset fixes add to hermes_cli/config.py
(4283 > its 4274 cap): the version-stamp trio (raw read of _config_version
plus check_config_version) moves verbatim into topical sibling
hermes_cli/config_version_stamp.py; config.py re-exports the two public
names (callers and tests import them from hermes_cli.config), the internal
_read_config_version_stamp callers now use read_config_version_stamp, and
scripts/docker_config_migrate.py imports from the new module directly.
- A "stdin" string used as a value (for example **{"cwd": "stdin"}) no longer marks a call safe; only a stdin keyword, dict key, subscript store or setdefault counts.
- An inline **dict(stdin=...) splat is read directly instead of being looked up as a helper named dict.
- age-keygen writes to <identity>.partial and moves it into place, so a timed-out run leaves nothing the next exists() check would accept.
Greptile 4198572811: the guard now finds subprocess/os/asyncio calls as ast.Call nodes, so multi-line calls are checked, and the 27 calls it newly flagged (pixel-art ffmpeg and 26 others) pass stdin=DEVNULL.
Optional-skill scripts can run inside the backend process, where a child
that inherits fd 0 can end the TUI/desktop JSON-RPC session with stdin EOF
(#14036). Add optional-skills/ to the guard's context dirs.
SKIP_DIRS used to match any part of the absolute path, so a context dir
named optional-skills (and any subdir named scripts, skills, gateway, cron
or hermes_cli) was never scanned. Match only "tests", relative to each
context dir.
The wider scan flags three calls; they now pass stdin=subprocess.DEVNULL:
comfyui auto_fix_deps.run_cmd, unbroker crypto.ensure_identity and
ast-grep cmd_install (its installers do not read stdin). Touching those
lines makes the code-health ratchet treat their missing timeout as new:
age-keygen gets timeout=30; the comfyui node/model installs and the
ast-grep installer download for as long as the network needs, so they
carry a reviewed HX006 allow comment instead.
The ladder's INT/TERM traps wrote the receipt and exited as soon as the
foreground stage returned, even when the stage's child caught Ctrl-C and
succeeded (setup's provider picker prints "Provider setup skipped." and
carries on). gateway and complete were skipped, the bootstrap marker was
never written and rc went 0 -> 130: a telemetry write changed installer
control flow.
install.sh:
- The ladder traps (INT, TERM and now HUP, a closed terminal) only note the
signal while a stage runs. rc==0 clears a Ctrl-C and the ladder carries on,
as bash's cooperative exit did on main; rc!=0 with a signal noted writes
`interrupted` and re-raises the signal. Outside a stage (platform check,
banner, between stages) the signal stops the run at once with a receipt.
The traps are installed with the EXIT trap, before check_platform.
- Inside a ladder stage, a signal marks STAGE_INTERRUPTED; stage_result
writes `interrupted` over the stage's own class, so pm.cli /
source_completion / `hermes setup` catching KeyboardInterrupt and exiting 1
no longer reads deps_install_failed / setup_failed. A child the signal
killed still ends the stage at once (status 128+N).
- A TERM/HUP sent to the installer alone takes effect when the running stage
returns (documented; the stage stays in the foreground because a
background stage's children ignore Ctrl-C under bash 3.2), and the receipt
names the stage the run stopped before, not a finished one.
- A stage killed outright (SIGKILL/OOM) now leaves a `failed/<stage>/other`
receipt; a failed write (full disk) removes its .tmp; the fallback id uses
$(( RANDOM*32768+RANDOM )) (no octal parse); no clock -> no receipt.
install.ps1:
- A native child that ended on Ctrl-C (130 or STATUS_CONTROL_C_EXIT) reads
`interrupted` over the stage's class; the receipt write removes its .tmp
on failure; the dead `other` class on the pre-ladder Initialize-ResolvedPaths
Fail is dropped (it runs before the ladder's try, so it never wrote one).
Tests (real install.sh main block with stub stage bodies, real SIGINT to the
process group): a handled Ctrl-C finishes the install with a success
receipt; a child that turns Ctrl-C into exit 1 reads interrupted. Both red
at 6246dd0f69d.
install.sh: a run id per ladder, first writer wins (a parent-only SIGTERM no longer
leaves two receipts); Ctrl-C / SIGTERM write failure_class=interrupted.
install.ps1: same once-only guard, finally{} records an interrupted run.
Reporter: latch a receipt id only after its row is saved, so a short-lived start
whose daemon reporter is cut off at exit re-reports on the next start.
install.sh / install.ps1 leave one closed-token receipt per full ladder run
(installer, outcome, failed_stage, failure_class, start/finish epoch) under
$HERMES_HOME/telemetry/shared_metrics/pending_installs. Every fail()/Fail call
site passes a closed failure class; the installer never sends anything.
The next Hermes start records each receipt once (latched by id, deleted after
the row is saved) only while collection is on; a start with collection off
purges the receipts unreported. Out-of-contract receipts are dropped.
Select the expected archive and embedded app version from the admitted
channel request, or the release tag for tagged builds. Channel builds
have no tag, so indexing it crashed before macOS packages could stage.
Keep filename, identity and provenance validation intact.
Regression coverage reproduces the crash on both architectures and checks
that wrong filenames, embedded versions and commits remain rejected.
plugin-catalog/mem0.yaml (#131267) points at mem0ai/mem0's own
integrations/hermes-plugin-mem0 at a reviewed pin. The entry stays dormant
while a bundled copy exists (bundled providers win in find_provider_dir), so
deleting the in-tree copy is what lets hermes_cli/memory_provider_migration.py
install it for every home whose memory.provider is "mem0". The vendor copy
keeps the provider name, mem0.json, the MEM0_* keys and the setup wizard.
Also dropped: the hermes-agent[mem0] extra, its win-arm64 platform row and
exclude-newer row, the pm/extras.py anchor (uv lock removes only mem0ai,
posthog, qdrant-client and sqlalchemy), the mem0-specific doctor check (the
generic check now names `hermes plugins install mem0`), and the tests that
imported plugins.memory.mem0. Prose that lists the bundled set now names mem0
as a catalog provider. MEM0_* env hints and redaction stay: the catalog copy
still reads them.
While this branch was in review, #132386 landed on main: a committed
`hermes update` that still owes work prints one whole "Desktop app build
owed:" line and one "Update follow-up '<step>' did not finish:" line per
owed step, and both Desktop hand-offs turn those into a manual outcome
naming the step (never a plain "Update complete."). This branch rewrites
both hand-off scripts, so the rebuild took its versions whole and this
commit carries main's behaviour into them, through this branch's own
post-commit design:
- posix.sh: owed_followup_steps() (unchanged from main: the parser contract
test runs it), the build-owed and owed-steps checks on a zero exit, each
setting DONE_NOTE (finish() makes it manual) and an add_warning entry so
the step also reaches the result's warnings; the Linux relaunch gate now
appends to DONE_NOTE instead of replacing an owed-step note.
- windows.ps1: the same two checks as Add-Followup -Manual calls before the
runtime verify, with main's exact [regex]::Matches expression.
Already equivalent here: the mac swap is skipped when the rebuild failed,
and finish() relaunches the kept app for a manual note unless the Linux
gate says skew/manual.
Before: 6 of main's contract tests fail on the rebuilt branch
(test_every_owed_followup_reaches_the_desktop_result x4, the truncated
build-owed case, the parser contract). After: tests/scripts/desktop_update/
plus test_update_post_commit_followups.py pass, 113 passed, 0 failed.
Within moments of its claim the daemon hands marker line 1 to the custodian it
forked (fdab580431e). The launcher's start check (contract C2) still waited for
the daemon's own pid, so on a host where the handover beat its next 0.1 s poll
it never matched, and once the daemon finished it reported a running update as
a failed launch (exit 70). `marker_names_handoff` now accepts line 1 naming the
daemon or a process the daemon forked; a grandchild or an unrelated pid still
does not count.
Sabotage A/B (daemon held alive 2 s after the custodian takeover, real
hand-off via the shim-progress harness): old acceptance exits 70, new exits 0
and publishes the stage. Test: the predicate over custodian/daemon/grandchild/
unrelated, red on the previous marker.sh.
Same gap as the POSIX hand-off: the -MarkerOp custody watcher took line 1
over only after windows.ps1 exited (WaitForExit, then the marker lock), so
an old packaged Desktop reading in between (lines 1 and 2 only) deleted the
marker while the resumed `hermes update` or a completion survivor still
held the checkout.
Start-MarkerCustodian now names the watcher on line 1 under the marker lock
right after starting it, before any update work, and the hand-off judges
its claim as that watcher from then on ($script:MarkerOwner in
New-MarkerContext): delegate publication, heartbeat and release keep
working. Invoke-MarkerRelease stops the watcher only after the release, so
line 1 stays live until then. Invoke-MarkerCustody keeps a claim that
already names it (judged by the creation time the hand-off recorded) and
falls back to the old takeover only when the handover failed.
Tests (Windows CI lane): the kill cell holds the marker lock across
taskkill /F and runs the shipped legacy reader at once; the pre-publication
kill cell and the A7 race expect the custodian on line 1.
The previous fix let the line-2 refresher take the marker over only AFTER
the hand-off died: it polls once a second and then waits for the A7 lock,
and until it rewrote line 1 the marker named the dead hand-off. An old
packaged Desktop (judges lines 1 and 2 alone, ignores A7 and the delegate)
reading in that gap deleted the marker and booted its backend while the
`hermes update` the hand-off started, or a completion survivor, still held
the checkout lock -- and the custodian then found no marker to keep.
Now the custodian is named on line 1 BEFORE any update work starts:
marker_refresher_start forks the refresher, then, under A7, rewrites the
claim's line 1 to the refresher's exact pid + creation time and adopts that
identity (MY_PID/MY_CT) for every later marker mutation (delegate, refresh,
release). posix.sh checks its own liveness with $$. The refresher keeps
line 2 young; if the hand-off dies it keeps the marker until the delegate
and the checkout lock are gone, then releases it. The hand-off releases the
marker before it stops the custodian, so line 1 stays live until the
release. If the handover fails (A7 busy), the old takeover-on-death path
still applies.
Test: the orchestrator-killed cell now holds the A7 lock across the SIGKILL
and runs the shipped legacy reader at once (no takeover is possible), then
keeps the post-takeover phases. Red on 5f855a2392c: {'live': None,
'kept': False}. Tests that asserted line 1 == the hand-off pid now expect
the custodian (that assertion encoded the vulnerable state).
marker.sh::checkout_lock_held and marker.ps1::Test-CheckoutLockHeld answered
"not held" when the lock file exists but cannot be opened (root-owned after
`sudo hermes update`, or held open without sharing on Windows), while the
Desktop's own probes (checkout_held / WINDOWS_CHECKOUT_LOCK_PS) answer "held".
The script helper's reclaim then deleted a dead marker the Desktop's probe
keeps. Both scripts now fail closed on every open/probe error except a
missing file: reclaim needs a provably free checkout. Parity tests: the
POSIX probe with perl, python3 and flock(1), plus an end-to-end reclaim; the
Windows reclaim against a lock file held with no sharing (Windows CI lane).
Same shape as the POSIX hand-off: windows.ps1 heartbeats line 2 from its
own process, so `taskkill /F` (no finally runs) left line 1 naming a dead
pid while the resumed `hermes update` (the line-4 delegate) or a completion
survivor still held the checkout, and an old packaged Desktop - which judges
lines 1 and 2 alone - deleted the marker and booted its backend.
After its claim the hand-off starts a hidden watcher (-MarkerOp custody
-CustodyOf <pid>, Start-MarkerCustodian). Once the hand-off exits with its
claim still published and the delegate running or the checkout lock held,
the watcher names itself on line 1 under the marker lock, heartbeats line 2
and releases the marker when both end (bounded like the R6 wait).
Invoke-MarkerRelease stops the watcher once the hand-off's own release is
decided. The Windows kill-cell test now also runs the reader be3fd671d7
shipped; that reader runner moves to a helper shared with the POSIX test.
