tirith ran `tirith check` on every terminal command, default-on with fail-open, as a
pinned PM binary auto-downloaded at startup. Measured against the shipped 0.4.2 binary
it caught 14/16 attack commands but flagged 12/24 everyday agent commands
(`echo $((6*7))`, `"$VAR/x.sh"`, `gh api | python3`, LAN curls, `.dev` hosts,
`pip install` via analysis_incomplete), and in cron/-q deny mode each of those is a
hard block. It only ever saw terminal strings (execute_code, writes and the browser
bypass it), fail-open meant no protection wherever the binary could not install
(Windows, Termux, x64 musl), and it produced a steady issue/PR stream (fail_open
semantics four times, install loops filling disks, a SIGSEGV hang, session-approval
scoping, third-party promo text on approval cards).
Removed: tools/tirith_security.py, the tirith branches of the command guard (the
is_tirith tuple, session-only "always" downgrade, import-error/fail-open paths,
unattended tirith pass), the CLI and gateway startup installers, the PM package and
lock pins, the nix runtime dep, security.tirith_* defaults, the CLI warning string
and three tirith tips (gaps refilled from the tail so translations survive), plus
the gateway's "no automated assessor" warning that only existed to point at tirith.
Config v50 drops security.tirith_* and enables nothing in its place. Hermes ships
no tirith plugin; if tirith upstream publishes one, it goes through normal catalog
admission under their name.
test_crash_cells carried its own marker parser and identity rule, and it
decides the orphaned_update verdict. It disagreed with the product on three
inputs: a float started_at (product: malformed, copy: live), a create time
1.5 s off (product: live within its 2.0 s tolerance, copy: dead at 1.0 s) and
a run: line before delegate: (product: delegate live, copy: dead, it read
line 4 only). The self-check claimed parity but never called update_lock and
used the test's own pid, which the product judges by exact incarnation.
_marker_live now asks update_lock._parse_marker(...).owner_live() /
delegate_live(); the crash cells run in a pytest process on the same Windows
host as the marker's pids, so the product's process table reads them as the
updater does. The self-check marks a separate live process with an integer
started_at, compares the oracle to judge_marker, and adds the three rows.
The ORPHAN_RELEASE_GRACE poll is unchanged.
Round 5's Start-MarkerCustodian takes a killed hand-off script's claim over and releases it on its
next 1 s poll after the update ends. Strict run 37393368977 read the marker ~2 s before that
release (custodian removed it at 01:01:32) and called it a surviving marker. The cell now waits up
to 30 s for the release; a genuinely stuck marker lives until the heartbeat ceiling, minutes.
run_tests.sh runs one pytest per file and reports a file whose tests were
all deselected (pytest exit 5) or all skipped as passed. 8ce4ce35fc2 made
the -m selection and platforms('windows') marker a tests/ci check, but any
other zero-execution path (the HERMES_E2E_WINDOWS_INSTALL opt-in lost from
the step env, a new skip condition) still ran zero crash cells green.
The crash journey now writes crash-cells-ran.txt (each journey step and
ok/failed) into HERMES_E2E_ARTIFACTS once it has run, and a new required
step "Every crash cell ran" after the run step throws when the manifest
is missing. The manifest is uploaded with the other artifacts.
Test (tests/ci, every PR): the step exists after the run step, is neither
disabled nor advisory (workflow_steps.required), reads the same
HERMES_E2E_ARTIFACTS and names the module's manifest constant; and
_record_cells_ran writes exactly the steps that ran, nothing without the
env. Red on the start workflow (no step) and with the step made
continue-on-error.
Review 5411223855 (F52: collected > 0 / executed-cells assertion).
All six crash cells share one module-scoped machine. A cell whose
follow-up update failed (torn checkout, .hermes-update-in-progress or
.git/index.lock left, or a step that raised) handed that state to every
later cell, which then reported the first cell's breakage as its own.
_run_cells now runs the cells in order and, before each one, checks what
the previous cell left: the checkout exactly its target, no marker, no git
lock, follow-up rc 0. When it is not, the harness stops the machine's
processes, drops the lock and the marker and resets the checkout with the
harness's git to that cell's target (else to the HEAD it left), and notes
the restore and its reason in machine.timings, so the evidence of any
later failure names the culprit. The broken cell keeps its own verdict:
its result was captured before the restore. If the harness cannot reset,
the next cell fails "not run: <cell> left the shared machine unsound
(...)" instead of running on it. This is harness plumbing like the stale
index.lock clean-up _crash already did; every target only appends
statements to modules, so the venv still runs the restored tree.
Tests: a broken cell, a raising cell and a sound cell in sequence (each
later cell sees the reset checkout, no marker, no lock, and the timings
name the culprit), and an unrestorable machine failing the next cell
with the culprit's name. Both fail on the start sha (no _run_cells).
Review 5411223855 (crash-cell cascade).
The mid_git cell names an unmodelled git (the hold filter ran, but no
merge/reset/checkout with .git/index.lock appeared) after 30 s of wall
time. While the filter holds, git, its op and index.lock cannot move: the
state is static and the poll only has to observe it, but on a starved
runner a tree walk can take seconds, so 30 s could be a handful of looks
and a slow runner could false-fail the cell. The verdict now also needs
HOLD_SETTLE_LOOKS (60) complete looks; a normal runner passes 60 looks
well inside 30 s, so its behaviour is unchanged, and the wait is still
bounded by _kill_when's UPDATE_TIMEOUT.
Test drives _GitHold.point with a 10 s-per-look clock: the start code
raised on the 4th look (40 s), now it raises only after 60 looks.
Review 5411223855 (crash-cell timing).
What: _kill_when judges the kill by `_kill_delivered`: taskkill rc 0, or its stdout carrying the
update root's own SUCCESS line ("with PID <pid> (child process of ...").
Why: the update's git now runs inside the update's kill-on-close job. Killing the update closes the
job, a member dies while `taskkill /T /F` is still walking the tree, and taskkill exits 128 even
though it terminated the update (rc 1 is the code /F forces). The mid-fetch and tree-moved cells
then reported "the update was not running when killed" on the strict composite run (artifact
receipt: outcome interrupted, transcript stops at the fetch). The mid-git cells passed only because
their git is frozen by a hold. "Not found" for the root stays a refusal.
Red: test_kill_point_counts_only_an_update_killed_while_running[None-128-...SUCCESS...-True-True]
fails on the old rc-only check. Green: tests/ci + tests/e2e/core/windows_update 546 passed.
ORPHAN_MARKER_GAP excused the orphaned_update cell's marker assertion with no
expiry. PR CI never runs that cell (Windows, opt-in) nor strict acceptance,
so after #132354 (hand-off script names its update child as the line-4
delegate) and #132365 (the judge honours a live delegate) land, a regression
of exactly that contract would still pass as an expected failure.
Now the fix's footprint is read from the tree (ORPHAN_MARKER_FIX):
- while any half is absent, _orphan_gap_excuse applies known_failure as before;
- once every half is present the excuse is off, so the cell is strict on any
runner whatever the wrapper says;
- tests/ci/test_windows_update_crash_oracles.py (runs on every PR) fails with
"delete ORPHAN_MARKER_GAP ..." once the fix is in the tree, the same expiry
shape as _LANDS_IN_SIBLING_PR, and fails if the Python half already on this
branch (update_lock.delegate_live) loses its footprint, so a rename cannot
keep the excuse alive.
Checked against the scripts branch: this tree + its marker.ps1/windows.ps1
reads no half missing.
Review CI1.
_git_op and _update_child skip any child whose name/argv read raises. Both
catches were broad with no (or a one-word) reason. The skip is fail-closed:
an unreadable child is never counted as the kill point, and _kill_when's
bounded wait turns a point never seen into a harness verdict, never a pass.
Kept broad on purpose (psutil on Windows raises more than psutil.Error for a
racing child) and now say so with a BLE001 waiver naming the boundary.
Review CI4.
What: _orphan baselines <HERMES_HOME>/logs/update_receipts before the hand-off starts; the cell passes on rc 0, or on rc 120 only when the newest receipt written since says outcome success.
Why: CPython replaces ANY exit status with 120 when the final flush fails, so a failure exit after the success line read as a completed update.
Red: pre-fix cell -> [120-failed] and [120-None] accepted; receipt helper missing (9 passed, 3 failed).
Green: tests/ci/test_windows_update_crash_oracles.py 12 passed, 0 failed.
What: the test's _identity_live takes the marker age and, for an identity without a creation time, is live only within hermes_cli.update_lock.UPDATE_MARKER_MAX_AGE_SECONDS.
Why: the oracle read a ct-less (v1) marker for a live pid as LIVE past the product's 20-minute ceiling, so it could disagree with the verdict update_lock gives every other updater.
Red: pre-fix test_crash_cells.py -> [1260-False-False] FAILED (6 passed, 1 failed).
Green: 7 passed, 0 failed.
What: _kill_when requires proc.poll() is None after the point is seen and taskkill_tree's rc 0 (it now returns the CompletedProcess); otherwise a harness AssertionError naming the point.
Why: tree_moved's point (HEAD at target) persists after a clean exit, so the cell could pass on an update that was never interrupted.
Red: pre-fix _kill_when/_machine -> exited-0 + truthy point and taskkill rc 128 cases accepted (2 passed, 2 failed).
Green: tests/ci/test_windows_update_crash_oracles.py 4 passed, 0 failed.
The launch repair's own entry module, unlinked by a merge held at git's smudge filter and
taskkilled with index.lock held. The next launch must restore the pre-update tree and run a
turn; the follow-up update must complete. Runs after mid_git: a launch that cannot repair this
tree leaves no runnable install for later cells.
(cherry picked from commit 1999a3868dc573dde1ec97745988823b9be1c30e)
(cherry picked from commit 232936de515c094292a56a9e2f6631457bc9d53d)
(cherry picked from commit 8c22ecbd465f80771eeb6d87ddea2be21e06ad55)
(cherry picked from commit a5ff463ce85b2b372661856525aff27d230f9226)
(cherry picked from commit 9f7a4d177929ca32cf1d25afd32dbb5005338db0)
The known_failure on the mid-merge launcher gap is gone: #132361's launchers import
hermes_bootstrap (the interrupted-pull repair) before any other checkout module, so the
merge-time kill must leave a runnable install with no gate.
(cherry picked from commit 96fd6724a86aa9b48ac7f88b56230e94f3447d7e)
(cherry picked from commit f975552309b97b7b2e4ac97b63ad278d185e2369)
Native run 37190464303 (first merge-time kill): git merge was held writing
hermes_constants.py with index.lock held and killed there. Every next
launch then died with ModuleNotFoundError: the launcher
(hermes_cli/_launchers.py::_launcher_script) imports hermes_constants from
the checkout before import hermes_bootstrap, whose
restore_interrupted_pull is the repair, so the repair never runs and even
hermes update fails. The bricked install then failed every later cell.
mid_git now runs last, and its recovery assertions sit under a
known_failure keyed on exactly that traceback (owner upd-txn: strict
acceptance fails on it). The hold precondition (index.lock left by the
kill) is asserted outside the gate.
(cherry picked from commit 7aca86f2341f6135b4da2666d58e24e3e14b7945)
(cherry picked from commit 95595e12f6a0e6ddc022a43e871c8e654149fbbe)
Every green native run killed the mid_git cell during git fetch (fetch was
in GIT_OPS and is the long step), so the merge-time index.lock schedule the
cell is named for never ran. A smudge filter the harness names in the
install's .git/config + .git/info/attributes (git's own filter hook; no
product change, no env var) now holds git merge while it writes
hermes_constants.py: index.lock held, the cell's new file and
hermes_cli/main.py already at the target, hermes_constants.py unlinked.
The kill point requires the hold, a merge/reset/checkout git child by argv
and index.lock; the cell asserts the lock was left. The fetch-time kill
keeps its own cell (mid_fetch).
(cherry picked from commit 592d57c3a9f78e04d617be659342e5338e080381)
(cherry picked from commit ba54f871bd580bb3fe9de7f840a7a21baf564dd2)
Moving the marker known_failure after the other assertions exposed a
harness race that the early XFAIL had hidden. PR run 37180721655 failed
the orphan cell with "did not finish (rc=None, 'Update complete!'
logged=True, tree at target, clean)". psutil.Process.wait returns None
for a non-child that has already exited. The orphaned update exits
between two 0.25 s samples, so its rc was lost.
_ExitCode opens PROCESS_QUERY_LIMITED_INFORMATION|SYNCHRONIZE on the
child before the kill and waits/reads GetExitCodeProcess through that
handle. The exit code stays readable after the process is gone, and a
reused pid cannot answer for it.
(cherry picked from commit 85488842f43b2154340231e2dd1a5e7900622487)
(cherry picked from commit 05c47385379ecda49e717e517425f5a9f559d76e)
- _assert_recovered compared HEAD SHAs only. A tree with HEAD at the target
and half the files from the old commit passed. Each target now appends a
statement to two modules every launch imports (hermes_constants.py,
hermes_cli/main.py), and right after the first launch the checkout must be
exactly the pre-update commit or the target: HEAD, `git status
--porcelain --untracked-files=no` clean, `git diff --quiet HEAD`, and the
target-only file present iff HEAD is the target. Same after the follow-up
update (target only) and after the orphaned update.
- The orphan cell's two known_failure blocks came first, so pytest.xfail
ended the test before its finish / first-launch / follow-up assertions.
Every unrelated assertion now runs first; the marker contract is one
block at the end (strict acceptance fails it: upd-txn owns the gap).
- _orphan took its completion-banner baseline after taskkill + wait, so a
completion logged during those waits counted as "not fresh". Both
baselines are taken before the script starts.
Linux probe of the oracle (real git, minted target, torn tree):
.../r5-ci/tree_probe.out — torn tree (HEAD=target, old hermes_constants.py)
matches neither commit; the old SHA-only check accepted it.
(cherry picked from commit 72308a85965635d9b746e52eccf35d91fcca75cf)
(cherry picked from commit 11df688a5831f2ed2e4a88b6a1fed5e7c9234a02)
known_failure XFAILs a gap while its fix is an open PR, which is right
on a PR and wrong on the acceptance run of the integrated batch: there
it accepts exactly the gap the batch claims to close (the final
composite's Windows run passed with `test_crash_cells.py (3✓ 1xf)`).
HERMES_E2E_STRICT_ACCEPTANCE=<owner>[,<owner>] (or 1 = every gap) makes
the matching known_failure / known_gate blocks re-raise the gap's own
assertion. It is set only from the `strict_acceptance` dispatch input of
ci.yaml (threaded through tests.yml / tests-os.yml to every E2E step)
and of windows-install-update-e2e.yml; pull requests never set it.
run_tests.sh forwards it through its env -i allowlist.
Red on 8cc78279a36: test_pending_fixes_strict.py (a real pytest process
per mode) — the batch-owned gap XFAILs under strict mode.
(cherry picked from commit ca1e7e749a61263181e89aa687dec366d95fda18)
(cherry picked from commit 7768bf3ef8e15fb7f228ebad619d6a9a85ad4100)
Run 37149266852 (delegate writers merged): the marker stayed LIVE via the delegate,
the update logged "Update complete!", its receipt says success and the checkout is at
the target, but the process exits 120: CPython's status when the final stdout flush
fails, and its stdout reader is the killed script. Require the checkout at the target
plus a fresh "Update complete!" in update.log, and rc 0 or 120.
(cherry picked from commit d9d965fc1e6f1ab3f5bcdfafe044e1439cf7d3da)
(cherry picked from commit e0ef639ebcc797bd4ff8f28ec1d6d8fd9d3cddcf)
Run 37148409574 (with the delegate writers merged) killed windows.ps1 0.4 s after its
`hermes update` child appeared, before that child reached UpdateLock.acquire(), so it
measured python start-up instead of the delegate contract. Wait for the child's
"Updating Hermes Agent..." banner in update.log (printed only after acquire) too.
(cherry picked from commit 274861c4ac40802afa03389a668d08e9c568d92c)
(cherry picked from commit 1f4d1ecd724285d00f898603f85fbf491b00900f)
- mid_git: the xfail is gone; #132361 (this branch is stacked on it) fixes the stale
index.lock + bare-git interrupted-pull repair.
- orphaned_update: kill ONLY windows.ps1 (no /T) once its `hermes update` child runs
under the claimed marker; the update must finish, the marker must read LIVE (owner or
line-4 delegate, C1) for as long as it runs and be gone after. The marker assertions
are a message-gated known_failure until #132354 + #132365 land the delegate line.
- _marker_text parses the line-format marker (it ran json.loads: always <unreadable>).
- tree_moved reads HEAD from the ref files instead of spawning git rev-parse every poll.
(cherry picked from commit aa8a8734785fdcc2d7fe958ab15ae002afdd12d9)
(cherry picked from commit eaf540f555cc936e798edc84ee1fd18fa802beac)
wine2e run 37139409703: the mid-git kill leaves .git/index.lock that gitlock only
sweeps after 10 minutes and the launch-time interrupted-pull repair fails with
WinError 2 on a machine whose only Git is the installer's copy, so the next
update refuses. That cell is xfail until LP-COMMIT lands; the later cells now
remove the earlier cell's lock the way the refusal tells the user to.
(cherry picked from commit f711518268fe3b94c3a08772ddfc6e1e7fc5f46f)
(cherry picked from commit 76263c968bfa589300679f3c9417062a23bb3ba6)
taskkill /T /F the real update tree a fresh install.ps1 machine started:
(a) while its git child (fetch / merge / reset) runs, (b) right after the
checkout reached the target but before the update finished, (c) the
Desktop hand-off script windows.ps1 while its `hermes update` child
runs. Each cell then asserts what the user is owed: the next launch runs
a turn, the checkout is at the pre-update commit or the target, and the
next `hermes update` completes with no .hermes-update-in-progress marker
left behind.
Machine helpers: mint/publish a per-cell commit, spawn with a streamed
transcript, descendants, taskkill_tree.
(cherry picked from commit cd608f71fe84920d587dc9174dc770a02f415219)
(cherry picked from commit ddf8f6ec3887a4a2a7c4eb2401a3066f522bf5c0)
The rollback cells blind the pre-pull syntax check so the broken release reaches the post-pull
guard. That check now reads the target in one git cat-file --batch (c8291c00b6b), so the git
shim also answers the broken file's batch request with HEAD's copy. The parked-branch cell
kills the checkout that moves the tree; since review O2 the -B form names the resolved commit,
so -B main anywhere is the move, not only a last argument of main.
The sync now merges the SHA it resolved from refs/remotes/upstream/main, not a
ref name, so the hostile git shim matches that commit.
(cherry picked from commit a18bd61d477cfb9763a4fa7c971239f8c6a4e4d3)
(cherry picked from commit acc97addd46cde31e15e6a817b777acffcd10e8a)
The updater's fork sync fast-forwards with `git -c ... merge --ff-only
refs/remotes/upstream/main` (custody flags before the subcommand, the full
ref name): match the subcommand and the ref, not one exact argv string.
(cherry picked from commit 4d8f81926eac53216a81437568b29a0b4136dfe1)
(cherry picked from commit 310010b489ed22d042cdb7b15b98f3b42675b362)
git_killed_inside_rollback_reset: a broken release passes the preflight, the
post-pull guard rolls back, and its `git reset -q <pre>` is SIGKILLed the moment
inotify sees index.lock (HEAD still on the broken release, the lock left). The
updater must keep the interrupted-pull marker and never say "Rollback complete";
the next real launch must reclaim the dead lock, redo the rollback and land on
the pre-update commit with no tracked change, no lock and no marker.
Red on a58135be6c6: the updater printed "✓ Rollback complete" with HEAD on the
broken release (the in-process settle deleted the marker). The broken module is
model_tools.py: a broken hermes_constants.py dies in the launcher before any
repair can run (documented limit, NOT_COVERED).
(cherry picked from commit 592e0e5955d986875d20c5aa556c35a587c9668a)
(cherry picked from commit 6618e14ee22fd70f1744ca3071a75c19827789bb)
The takeover's new fleet-obligation arm imported `update_cmd_fleet`'s writer,
which imports `update_cmd` -> `hermes_cli.config` -> ruamel. `prepare()` runs
in the HISTORICAL interpreter before PM syncs the new dependencies, so every
update from a release whose venv has no ruamel (the installer-script and
e2e-upgrade core/handoff/pm legs) exited 1 with
`Update preparation failed: No module named 'ruamel'`. The arm now writes the
host obligation through the stdlib-only `update_host_obligation`, with the same
per-home breadcrumb fallback.
Also:
- The syntax rollback's mixed reset un-tracked the files the broken commit
ADDED, and `reset --hard` never touches untracked files, so they stayed in
the tree (`?? hermes_cli/`). They are now removed by name after the reset.
- A git that EXITED on another git's `index.lock` no longer has that lock
deleted by the in-process settle: the lock is foreign by construction, the
scan only reads, and a tree with writes to put back keeps its marker.
- `_early_recovery`'s git runs carry `cwd` as well as `-C`.
- `_early_recovery._git_executable` gets a reasoned resolution-allowlist row
(stdlib-only launch repair; cannot import `hermes_platform`).
Tests rewritten to the new contract:
- test_source_release_channels zip fallback: the download stub takes the
target SHA the ZIP swap now journals, and asserts it is passed through.
- test_update_target_identity: the fault injection keys the upstream sync by
its remote (`upstream`/`upstream/main`), since the sync now merges the
fetched ref instead of a second `pull`; the origin-target syntax case
expects the pre-move preflight refusal.
- test_hostile_commit ZIP cell kills right after a tracked file landed, not
at the N-th swap (listing order made it land on unchanged entries only).
(cherry picked from commit b6adf96ca6988262fe7c408ea652d3b390c7be8d)
(cherry picked from commit 028cb933271626d21003c255bb397deb8998b7da)
The syntax rollback's first step was `reset --soft` (or `update-ref` for a
parked/detached checkout), which leaves the index on the broken commit. A
kill during the following `reset --hard`, after it had written a good file,
left that file staged as the broken version, so the next launch's restore
landed on the pre-update commit with a dirty index (`MM`).
The first step is now a mixed reset (`reset -q <pre>`), after `update-ref
--no-deref HEAD` for the detach mode. HEAD and the index are back on the
pre-update commit before any file is written. The marker's `rollback=` field
is `branch`/`detach`, and recovery redoes the same steps when the kill came
before them. A failed rollback step now goes to the in-process restore every
time. Before this, a failure in the first step dropped the marker, even
after `update-ref` had already moved HEAD.
Also: the old branch-switch hostile cell triggers on `checkout [--no-guess]
main`, since CP0 now passes `--no-guess`. The fork cell makes origin read as
a fork through the git shim, because the sandbox otherwise reports the
official URL and the upstream sync never runs.
(cherry picked from commit 23f9658868b9c1f639ed972e59251fe154372929)
(cherry picked from commit 5fd6de2e62152076d6703e5ff70250ec1ece71b9)
Review round 2 on the LP-COMMIT commit point:
- A failed upstream fork fast-forward after a committed origin pull no
longer hands back the obligations the origin pull armed.
`disarm_commit_obligations` now refuses unless HEAD is the commit the
run STARTED from (`record_run_start`), so every caller is covered; the
restore of the snapshot is an atomic replace.
- The ZIP swap journal counts a leftover `<entry>.hermes-update-old` as
"existed" (`_stage_replacement` puts it back before the swap), and the
`swapping` recovery treats any backup as proof the entry existed, so a
kill mid-swap can no longer delete both copies. A pre-run journal the
start-of-run settle could not clear refuses the swap instead of being
overwritten.
- CP0 runs `git checkout --no-guess <branch>` only when the local branch
exists; otherwise only the marked `checkout -B <branch> origin/<branch>`
runs. git's checkout guess used to create the branch and rewrite the
tree while the code believed the checkout could not move.
- The syntax rollback moves HEAD back to the pre-update commit first
(`reset --soft`, or `update-ref --no-deref` for a parked/detached
checkout), then `reset --hard`. The marker names pre = the good commit
and `rollback=<mode>`, so a kill anywhere lands the next launch on the
pre-update commit; recovery redoes the HEAD step if the kill came first.
- The syntax guards (preflight `git show`, ZIP compile gate, post-pull
backstop) skip a target whose `requires-python` excludes the running
interpreter: its syntax may be a newer Python's. Unknown specs keep
checking.
- The no-op exit in `_verify_head_after_pull` disarms (self-verified).
(cherry picked from commit c787a8c3f9823be8639f6f38efbbdff2b78a97cd)
(cherry picked from commit d785ad1c7e919bd7d64d17f2e9d664b84a7e8982)
The swap renames each live top-level entry aside and then its replacement in. A kill between the
two left the entry missing; for hermes_cli/ (the recovery code itself) or for the root modules every
launcher imports first (hermes_constants, hermes_bootstrap) the next launch died on
ModuleNotFoundError and the journal-driven restore never ran.
- update_cmd_zip: a root file swaps in by os.replace over a hardlinked backup, so it is never
absent (old or new bytes); a filesystem without hardlinks keeps the move-aside. Rollback and the
journal restore put a file back with os.replace too.
- hermes_bootstrap: settles a torn update (restore_interrupted_pull: ZIP journal + interrupted-pull
marker) right after the import path is hardened, before PM activation, launch preparation (which
could run the owed completion tail against the torn tree) or any other Hermes import. When
hermes_cli/ is missing and the journal exists, the restore runs from the moved-aside
hermes_cli.hermes-update-old/_early_recovery.py: the code that wrote the journal.
- Real-process cell: the ZIP swap SIGKILLed right after hermes_cli/ was renamed aside; the next
`hermes -z` must restore the old tree and run, and the follow-up update must succeed.
(cherry picked from commit c14cd236586309811ded28751b945a9173e6b770)
(cherry picked from commit a3e84c6db8cd8ebe01178da37ec90858513a63f4)
Every cell plants its schedule with the installed pause-record module (a driver stops itself on the named source line) and recovers through a real hermes.exe launch that really restarts the real gateway. test_pause_durable reads the checkout-keyed record name.
(cherry picked from commit f25dbc85cb23b17a9427218d367385d06958ad19)
(cherry picked from commit 488a7f25f0d58ad27df709858bbb37ed49943a9c)
The held-file cell asserted that a failed fast-forward leaves git's half-written tree
behind. With the commit-point lane (#132361) the updater puts HEAD/index/files back to
the pre-update commit itself, so that tree never exists and the premise failed.
The premise now proves the pull failed on the held file (rc != 0 + git's unlink error).
Then exactly one of two end states is accepted:
- torn (HEAD == pre, git's early file dirty): gateway stays stopped, record kept;
- restored verbatim (HEAD == pre, no tracked dirt, no pull marker, restore reported):
the paused gateway runs again on the old code, outlives the updater, record discharged.
Anything else (HEAD moved, restore claimed over a dirty tree) fails the cell.
(cherry picked from commit 0b5c6b9a041bc0c389673652e98a02396dd6c4d0)
(cherry picked from commit d0b04565d9633960133ba3b4f1483268e5ad51cf)
The cell review r1 found missing (m5/B1): after a normal Windows update the
resumed gateway must still be alive 30 s after `hermes update` exits, and the
record must be discharged.
(cherry picked from commit 59319ec14371e88c121c98dc404a61da4ef40bac)
(cherry picked from commit b3f98041c4741c60fc9dd318ed617323fd2e4ff1)
- unit: a launch SIGKILLed mid-recovery leaves the set to the next launch (and
it is resumed exactly once); runtime-host refusal for --profile / -p /
hermes_cli/main.py / dashboard argv; a committed update with a rewritten
tracked file is judged on dependencies only.
- e2e (real clone checkout): an update child running under an orchestrator's
marker adopts and merges an orphaned pause record.
(cherry picked from commit 67f2dd2c4cbeb0719b76d965d7e5270895c2663a)
(cherry picked from commit 33820b493a5bf4c9cd629da2e9d3511021bac5fb)
plugin-catalog/mem0.yaml (#131267) points at mem0ai/mem0's own
integrations/hermes-plugin-mem0 at a reviewed pin. The entry stays dormant
while a bundled copy exists (bundled providers win in find_provider_dir), so
deleting the in-tree copy is what lets hermes_cli/memory_provider_migration.py
install it for every home whose memory.provider is "mem0". The vendor copy
keeps the provider name, mem0.json, the MEM0_* keys and the setup wizard.
Also dropped: the hermes-agent[mem0] extra, its win-arm64 platform row and
exclude-newer row, the pm/extras.py anchor (uv lock removes only mem0ai,
posthog, qdrant-client and sqlalchemy), the mem0-specific doctor check (the
generic check now names `hermes plugins install mem0`), and the tests that
imported plugins.memory.mem0. Prose that lists the bundled set now names mem0
as a catalog provider. MEM0_* env hints and redaction stay: the catalog copy
still reads them.
The kill-on-close job that binds the completion child to `hermes update` allowed no breakaway,
so the gateways the completion restarts or resumes (gateway_windows._spawn_detached, which asks
for CREATE_BREAKAWAY_FROM_JOB) fell back into the job and were killed the moment `hermes update`
exited, on success too. Allow BREAKAWAY_OK (not SILENT: ordinary descendants stay bound).
A job that cannot be set up no longer raises out of run_completion after the commit point: it is
logged and the completion runs unbound; the bind sits inside the try whose cleanup kills the
child, so nothing can orphan it.
Windows cell: a real install, a real gateway, a real `hermes update`; the restarted gateway must
still be alive 30 s after the updater exited.
