Every pre-apply exit of `hermes update` records one closed token on the
receipt at the exit itself (update_receipt.record_stop_reason ->
stop_class), and shared_metrics_update.update_failure_class maps it to
the run's failure_class. aborted_before_apply stays only as the fallback
for an exit with no recorded reason.
Exits that fire before the receipt opens (update-lock refusal, Git
operation in progress, managed install) write no receipt, as before, but
now record a metrics-only row with the same closed class.
The parked receipt copy keeps stop_class, exit_code, the stop reason's
leading label and the restart/user-action flags, so a parked run
classifies the same as the in-process one. A run that committed and is
owed only an interrupt or the user's parked changes reads outcome
partial (failure_class interrupted / local_changes_parked), never failed.
`record_staged` rewrote and fsynced the whole swap journal once per entry:
124 durable writes for this repo's 123 top-level entries, ~8.6 ms each on
ext4/NVMe, ~1 s of a ~1.8 s stage (review K132361 P3, efficiency).
The write that has to happen is the one that makes a directory's staging
id durable before copytree fills it: a kill mid-fill must leave a partial
tree that recovery can prove is the update's and delete. A file entry is
already fully copied when its hook runs, so nothing waits on its record.
Its id now rides the next directory's write, or the last entry's, which
still lands inside `_stage_entries` before anything else runs. The journal
format, the reader and the whole-record temp+fsync+replace are unchanged.
Real 123-entry stage of this repo's tree: 124 -> 31 durable writes,
median 1859 -> 979 ms. All 93 top-level files copy in ~5 ms, under one
fsync, so a crash window that leaves a copy unrecorded (it is kept aside
and reported, never deleted) gets no materially wider.
b9ae435d78e let recovery delete <entry>.hermes-update-old.<gen>.tmp when its
bytes were a prefix of the live file. That is a necessary property of a
killed backup copy, not proof: a file created there after the journal (a
prefix, a full copy, or an empty file) was deleted and the journal retired.
The swap now journals each backup temp's identity (dev:ino:type: its size
and mtime move while it fills) the moment _copy_file_exclusive creates it,
before its first byte; the journal's new "temps" map is keyed by the temp's
file name. Recovery deletes a temp only when that recorded identity matches,
and still requires the prefix: FAT, where this copy path runs, hands a
recreated name its old inode. Anything else is kept aside and reported, as
before. The committed journal drops the map: every temp is a backup by then.
A journal without the map (an older writer) vouches for no temp.
Tests: the temp-name regression now covers user-made prefix, full-copy and
empty files (red before this change); the C4 control kills the real writer
inside the copy (subprocess, os._exit) and recovery still clears its temp.
(review F78-R on #132346)
Pure move, behaviour-identical. The ZIP swap journal, owner lock and
journal-driven recovery (~250 lines) are unrelated to the git-marker
restore, yet had grown _early_recovery.py from 684 to 1455 lines. They now
live in the topical sibling hermes_cli/_early_recovery_zip.py.
restore_interrupted_pull imports it only when the journal exists, so the
git-marker repair and its published closure (RECOVERY_CLOSURE) still need
only _early_recovery.py; ZIP_SWAP_JOURNAL stays there for that fast-path
stat. write_durable_text stays too (the marker and fleet writers use it).
Importers and the one monkeypatch seam (_lock_fd, read by the lock) now
point at the module that reads them. (review K132361 shape gate)
_stage_replacement dropped `<entry>.hermes-update-staging` and
`<entry>.hermes-update-old` unconditionally after the clean-tree preflight,
so a user's file created there during the download was deleted before the
exclusive create (and the pre-swap recheck then took the replacement for
this run's own staging copy). _hardlink_backup likewise removed whatever sat
at the backup path between the recheck and the swap.
Staging now keeps any such entry aside (`.hermes-update-kept`, reported)
instead of deleting it: on a git checkout the pre-swap recheck then
refuses with the user's bytes intact; a non-git install proceeds. The swap
refuses (and rolls back) when a backup path exists that staging did not
clear, instead of replacing it. _stage_entries lists an entry for
_discard_staged only once its staging copy is this run's, and a stage that
fails after creating its copy (including in the journal hook) drops only
that identity-checked copy.
The recovery-side arm of the finding (_settle_zip_entry dropping
`<old>.<gen>.tmp` by name) lives in _early_recovery.py and is left to that
module's lane.
(review F78 on #132346)
`target_sha or _archive_commit(zip_path)` left target_sha None for a ZIP
without a 40-hex comment, and _journaled_stage_and_swap then armed the
completion tail and fleet restart for "" -- a debt no later check can ever
prove discharged -- and installed the unidentified bytes. A pinned target
was trusted without looking at the bytes at all.
The installed identity now always comes from the downloaded archive
(`git archive` stores the commit as the ZIP comment; GitHub archives carry
it): no comment, a malformed one, or one naming another commit than the
pinned target raises before staging, so the all-or-nothing boundary exits 1
with no live rename and nothing armed. The check runs after extraction so
the symlink/zip-slip rejection still reports first.
(review S4 on #132361)
_commit_staged_replacements drops the `<entry>.hermes-update-old` backups
best-effort; the success path then unlinked the committed journal
unconditionally. A backup the cleanup could not remove (a read-only
directory in the old tree, an AV scan holding it on Windows) was left with
no ownership record: the next launch's recovery had nothing to settle, and
the next ZIP update's dirty-tree gate refused on the orphan forever.
The success path now applies the recovery path's own terminal check: the
journal (phase "committed") stays while any declared staging, backup or
`<backup>.<gen>.tmp` sibling is still on disk, and the next launch's
recovery drops the backup by its recorded identity, keeps the new tree,
then retires the journal. The update itself still reports success.
(review Q1 on #132361, F79 on #132346: same root cause)
What: the critical inventory now holds every module the launcher and hermes_bootstrap import
before application code (hermes_bootstrap.py, _parser, _launchers, the recovery closure,
update_handoff, venv_sync, pm/environments), pinned by a test that derives that set from the
real launcher text and bootstrap imports. One startup_syntax_error() serves the preflight, the
post-pull guard and the ZIP gate: a target whose requires-python excludes this interpreter is
compiled by an installed interpreter it admits (uv python find, then python3.N on PATH); with
none installed this interpreter still compiles it, and a refusal names the Python to install.
The preflight reads pyproject + the inventory in one `git cat-file --batch`.
Why: a broken bootstrap passed both guards and failed at the first import (G3); a Python bump
reduced the check to a conflict-marker scan, so `def broken(:` landed (N15, the class the
00f0e40b6ce marker scan left open); preflight cost one git spawn per file (kshitijk4poor).
Changed test: test_syntax_guards_skip_a_target_that_requires_a_newer_python pinned the N15 bug
(`def newer_syntax(:` admitted under >=3.99); it is replaced by the no-interpreter refusal and
the admitted-interpreter verdict tests.
ZIP gate (update_cmd_zip.py, czip's file): its compile loop is replaced by the shared helper.
The ZIP swap journal named entries by path only: recovery took any
<entry>.hermes-update-old for the swap's backup and any entry at a
journaled name for the one the swap installed. A suffix a user created
after the kill was installed over the live file (whose bytes were lost),
and a file that replaced an installed entry was deleted.
The journal now carries provenance: each entry records its staging copy's
identity as that copy comes into existence (a directory before it is
filled; the journal is rewritten per entry) and, at the swapping phase,
the live entry's identity, which its backup keeps (hardlink or rename).
Identity is the entry's own lstat dev:ino:type, plus size and mtime for a
non-directory (a recreated file often reuses the inode). The run's random
gen tags its no-hardlink backup temps (<old>.<gen>.tmp) so recovery owns
those by name. Recovery deletes only what it can prove is its own; any
other entry is renamed aside (.hermes-update-kept) and reported, never
deleted; restoring a backup needs no proof because it destroys nothing.
A dst that is still the recorded live entry is left alone (nothing moved).
The ZIP journal is new in this PR, so the 2-field schema needs no
compatibility path.
Changed fixtures (schema, not outcome): hand-written journals in
test_early_recovery.py, test_update_zip_fallback_guards.py and
test_update_zip_two_phase.py now carry gen + identities as the real writer
does; the C4 killed-copy temp is <old>.<gen>.tmp. Their assertions are
unchanged.
Review Z1.
_stage_replacement swept the fixed <entry>.hermes-update-staging name,
checked it was gone, then shutil.copy2'd to that same pathname: copy2 opens
the destination following symlinks, so a link planted between the check
and the copy carried extracted bytes outside the install before the
journaled swap. _file_backup's no-hardlink copy to <backup>.tmp had the
same shape.
Both now go through _copy_file_exclusive: O_CREAT|O_EXCL|O_NOFOLLOW create,
bytes and mode (and times where supported) through the held fd, our own
half-written create removed on failure. A planted entry makes the stage
fail closed (no swap; the journal path settles it). Directory staging was
already exclusive: copytree's os.mkdir never follows and fails on any
existing entry (pinned as the dir control).
test_zip_recovers_crashed_backup_before_failed_copy_and_retry injected its
copy fault through shutil.copy2; the file arm now injects it at
_copy_file_exclusive (the primitive that replaced copy2); its assertions
are unchanged.
Review Z2.
restore_interrupted_zip_swap judged the backup with Path.exists(), which
follows symlinks: a swap killed after it moved a tracked dangling symlink
aside and installed a regular file read as "no backup", the leftover sweep
(_drop_path, symlink-aware) then deleted the only copy and the journal was
retired, leaving the tree type-dirty (T alpha). _stage_replacement's
leftover-backup restore had the same exists() test.
Presence is now the entry's own (os.path.lexists) in every restore and
cleanup decision; os.rename moves the symlink itself, never its target. The
journal is retired only on a verified terminal state (no staging copy,
backup or backup temp left), not merely an exception-free loop.
Review Z3 (revalidated at fd5c37f17bf in 5414411301).
write_durable_text unlinked a fixed (ZIP journal) or pid-named temp before
its exclusive create, and the ZIP dirty-tree gate exempted the journal's
fixed .hermes-update-zip-swap.tmp name as Hermes' own before any journal
established ownership: a user's file by that name was admitted by the gate
and then erased by the journal writer.
The temp is now an unpredictable random name created O_CREAT|O_EXCL
(|O_NOFOLLOW): a pre-existing name is neither written through nor deleted,
and a collision draws a new name. The gate exempts only the writer's own
random-temp pattern (never deleted, never touched by the swap); a file at
the old fixed name is user work and refuses. Invariant 5 (unpredictable
exclusive temp + fsync + os.replace) now holds for every caller, including
the interrupted-pull marker's foreign-lock record.
Review Z5 (also hardens Z4).
What: for a target whose requires-python excludes this interpreter, the ZIP gate now checks
every startup-critical file with conflict_marker (raising SyntaxError like a failed compile,
before the commit point) instead of iterating over nothing; other targets still compile.
Why: target_syntax_error and _rollback_if_pulled_syntax_error already hold such targets to the
conflict-marker check (a newer Python's syntax cannot be judged by this compile(), a merge
marker is broken under every Python), but the ZIP gate's `for rel in () if newer_python`
checked NOTHING, so a release with an unresolved conflict in hermes_cli/main.py was swapped in
(review C5, N15 partial).
Test (RED at 5dfc384669e for >=3.99): a conflict-marked main.py is refused before the arm and
the live tree is untouched, for a current and a newer requires-python.
preflight_refusal: any error from the foreign-owned-venv probe (pm's
RuntimeError refusal, an unreadable facts record or venv dir) must refuse
the update before the first tree move; narrowing would let an unforeseen
error escape as a traceback instead of a refusal.
_download_and_swap_zip: the all-or-nothing boundary; any failure must
settle the swap journal, disarm the obligations and exit 1 with the
retry advice. Both catch-alls stay, now carrying `# health: allow BLE001`
waivers; behaviour is unchanged.
What: the post-commit journal unlink is best-effort (suppress OSError); a journal left behind
says "committed", which the next launch's recovery settles by keeping the new tree.
Why: a PermissionError there (AV/indexer on Windows) reached the failure handler, which
disarmed the new tree's completion obligations and printed "Your existing install was left in
place" over a fully swapped install.
Red (9ff1653fde6 update_cmd_zip.py): PermissionError escapes _journaled_stage_and_swap.
Green: returns, new tree in place, obligations still armed.
What: with no target SHA, _download_and_swap_zip reads the 40-hex commit GitHub stores as the archive's ZIP comment and uses it for arm_commit_obligations and the completion's expected_sha (returned to _update_via_zip); a missing/odd comment keeps today's behaviour.
Why: a branch ZIP armed expected_sha '' which can never be proven discharged (sticky warning / extra restart).
Red: pre-fix test_unpinned_zip_update_owes_the_restart_for_the_archive_commit -> armed == [''] . Green: armed == [sha], completion expected_sha == sha.
What: the initial gate first settles an interrupted swap's journal (restore_interrupted_zip_swap), then ignores only Hermes' fixed root breadcrumbs; the pre-swap re-check also ignores <entry>.hermes-update-staging for the entries this run staged (staged=...), nothing else.
Why: ignoring every *.hermes-update-staging/-old admitted a user's own file by that name, which _stage_replacement then deleted; journal-owned leftovers are told apart only by the journal, so it is settled before the gate.
Red: pre-fix test_zip_gate_refuses_and_keeps_a_user_file_named_like_an_update_artifact[both suffixes] -> DID NOT RAISE SystemExit (admitted). Green: refused, user bytes kept; interrupted-journal retry admitted with leftovers settled.
What: _commit_staged_replacements backs up a regular file as a hardlink or, where links are unsupported, a copy2 to <backup>.tmp + fsync + os.replace to <backup>, then os.replace(staging, dst); only directories move aside. The docstring's 'never absent' claim now holds.
Why: the no-hardlink fallback renamed hermes_bootstrap.py aside before renaming the new one in; a kill between left it absent, so every launcher failed before reaching the journal restore.
Red: pre-fix test_root_files_never_go_missing_mid_swap[False] -> root modules absent mid-swap: hermes_constants.py, hermes_bootstrap.py. Green: both params pass.
What: _journaled_stage_and_swap no longer discards staging copies when the swap fails; the journal stays and the caller's restore_interrupted_zip_swap drops them (no litter for the retry's free-space check).
Why: discarding first erased the proof (staging still on disk) that an absent entry was never installed, so journal recovery deleted the user's file at that path.
Red: pre-fix test_a_failed_swap_keeps_a_file_the_user_made_at_a_never_installed_entry -> live/brand_new deleted. Green: user file kept, a/ old, no staging/backup litter.
What: _stage_replacement removes leftover <entry>.hermes-update-staging/-old with _early_recovery._drop_path (no ignore_errors) and raises if either still exists.
Why: rmtree(ignore_errors) left a partly-deleted stale backup; the swap's move-aside then failed and journal recovery renamed that remnant in as the live entry.
Red: pre-fix test_a_stale_backup_never_stands_in_for_the_live_entry -> live pkg is only ro/f, pkg/m.py gone. Green: passes (pkg/m.py present).
R10. _stage_entries recorded an entry only after its copy succeeded, so a copy
that died partway (a real unreadable source file) left a partial
`<entry>.hermes-update-staging` nobody discarded, and _journaled_stage_and_swap
then deleted the journal: recovery had no record of it (reviewer probe).
Now the entry is recorded before its copy starts (the failure path drops it like
the finished ones), and the journal is removed only once every declared staging
path is gone; otherwise it stays and the next launch's recovery removes the
stage (making the update's own read-only copied directories writable first).
Tests: real-process/real-file invariants for R9 (three processes on one lock
inode; a root that cannot hold the lock refuses with a reason) and R10 (an
unreadable source file leaves no stage; a cleanup that cannot finish keeps the
journal and recovery cleans it). All six R1/R9/R10 invariants fail on a58135be6c6.
(cherry picked from commit 1d03d36ac88d50d56f99d5af65fafc5fb9b6fe95)
(cherry picked from commit 3fbc1d442495edd8da8d07b439c93adbfcf771dc)
R9. zip_swap_owner_lock unlocked, closed, then unlinked its path: a waiter that
won the lock in between held the old inode while a newcomer created and locked a
fresh one (reviewer: B on inode 9572101, C on 9612620, both "owning"). Its
open-error branch also yielded True with no lock at all.
The lock file is now a stable sidecar (never deleted; gitignored and already on
the ZIP dirty-tree allowlist), and the context yields a ZipSwapLock verdict:
truthy only while the kernel lock is held, otherwise a named reason ("busy", or
why the file could not be opened/locked). The swap refuses with that reason;
recovery says why it cannot settle. Test oracles that counted every
*.hermes-update-* name now exempt the sidecar.
(cherry picked from commit 93db2b7d75c03caa787dc4e5295a4e6f73f3cee1)
(cherry picked from commit d39812fbea263f407b82865b0cbae235e51e0251)
Review round 2 on the LP-COMMIT commit point:
- A failed upstream fork fast-forward after a committed origin pull no
longer hands back the obligations the origin pull armed.
`disarm_commit_obligations` now refuses unless HEAD is the commit the
run STARTED from (`record_run_start`), so every caller is covered; the
restore of the snapshot is an atomic replace.
- The ZIP swap journal counts a leftover `<entry>.hermes-update-old` as
"existed" (`_stage_replacement` puts it back before the swap), and the
`swapping` recovery treats any backup as proof the entry existed, so a
kill mid-swap can no longer delete both copies. A pre-run journal the
start-of-run settle could not clear refuses the swap instead of being
overwritten.
- CP0 runs `git checkout --no-guess <branch>` only when the local branch
exists; otherwise only the marked `checkout -B <branch> origin/<branch>`
runs. git's checkout guess used to create the branch and rewrite the
tree while the code believed the checkout could not move.
- The syntax rollback moves HEAD back to the pre-update commit first
(`reset --soft`, or `update-ref --no-deref` for a parked/detached
checkout), then `reset --hard`. The marker names pre = the good commit
and `rollback=<mode>`, so a kill anywhere lands the next launch on the
pre-update commit; recovery redoes the HEAD step if the kill came first.
- The syntax guards (preflight `git show`, ZIP compile gate, post-pull
backstop) skip a target whose `requires-python` excludes the running
interpreter: its syntax may be a newer Python's. Unknown specs keep
checking.
- The no-op exit in `_verify_head_after_pull` disarms (self-verified).
(cherry picked from commit c787a8c3f9823be8639f6f38efbbdff2b78a97cd)
(cherry picked from commit d785ad1c7e919bd7d64d17f2e9d664b84a7e8982)
The swap renames each live top-level entry aside and then its replacement in. A kill between the
two left the entry missing; for hermes_cli/ (the recovery code itself) or for the root modules every
launcher imports first (hermes_constants, hermes_bootstrap) the next launch died on
ModuleNotFoundError and the journal-driven restore never ran.
- update_cmd_zip: a root file swaps in by os.replace over a hardlinked backup, so it is never
absent (old or new bytes); a filesystem without hardlinks keeps the move-aside. Rollback and the
journal restore put a file back with os.replace too.
- hermes_bootstrap: settles a torn update (restore_interrupted_pull: ZIP journal + interrupted-pull
marker) right after the import path is hardened, before PM activation, launch preparation (which
could run the owed completion tail against the torn tree) or any other Hermes import. When
hermes_cli/ is missing and the journal exists, the restore runs from the moved-aside
hermes_cli.hermes-update-old/_early_recovery.py: the code that wrote the journal.
- Real-process cell: the ZIP swap SIGKILLed right after hermes_cli/ was renamed aside; the next
`hermes -z` must restore the old tree and run, and the follow-up update must succeed.
(cherry picked from commit c14cd236586309811ded28751b945a9173e6b770)
(cherry picked from commit a3e84c6db8cd8ebe01178da37ec90858513a63f4)
Arm the source-completion tail and the fleet obligation in the parent before
any tree move; cover the CP0 branch switch, the upstream fork ff and the syntax
rollback with the interrupted-pull marker; settle (never drop) the marker on a
non-kill git failure; refuse foreign-owned venvs and uncompilable targets before
HEAD moves; push the fork only after a validated update; ZIP fallback only from
an untouched checkout; journal the ZIP swap so start-of-run recovery finishes or
rolls it back and no staging/backup sibling wedges the next run.
(cherry picked from commit 584acad44f3d6c552b7c2767d12c612de0436498)
(cherry picked from commit c6567ac2d28c421f384d95e1a6fc28912cb30f86)
R2: hermes_cli/update_custody.run_git is THE updater git runner;
update_cmd._git_run, update_cmd_git._git_run, update_cmd_stash,
update_cmd_check._git, gitlock, update_cmd_zip and the fleet ancestry probe
all go through it.
- every updater git call carries -c gc.autoDetach=false
-c maintenance.auto=false: no detached gc/maintenance child ever forks
under (POSIX) or outlives (Windows) the checkout lock;
- POSIX: only local mutators (reset, checkout, merge, stash, rebase,
update-ref, clean, apply, commit, ...) inherit the lock fd, with
core.fsmonitor=false; fetch/ls-remote/credential and readers run without
it, so a credential-cache daemon never holds the checkout (reviewer: up to
2 h of installer waits). On Linux those fd-less children die with the
updater (PR_SET_PDEATHSIG) instead of rewriting refs under the next owner;
- Windows: inside an update every child is created suspended, assigned to
the owner's kill-on-close job and only then resumed; the Node build
(pm.progress.run_contained) joins it through a stdlib launcher.
The fork sync fast-forwards locally (merge --ff-only
refs/remotes/upstream/main after its fetch) instead of a networked pull
under the mutator fd.
The update pipeline already writes one machine-readable receipt per run, so the
run/stage counters are derived from it in the process that finalizes it instead
of instrumenting stages for metrics. The receipt gains minimal stage END marks
(plan, snapshot, apply[mode], deps, build, restart; verify is inferred from the
fleet matrix), an initiator fact (desktop when a live orchestrator claim
names another pid) and the pre-update commit date, which is all the derivation
needs for outcome, failed stage, duration and from-version-age buckets.
A pre-pull interpreter must never import pulled code: when it is the finalizer
(same pid, checkout sha moved) it parks the receipt (stdlib only) and the next
Hermes start records it. Collection off => nothing imported beyond a config read.
The completion-process test stub gains the new record_stage API the real
module now exposes (no assertion changed).
Resolved toward the branch: PM provisions uv/python (main's install.ps1 uv-shim
salvage + its test and workflow steps dropped), the shim re-exec stays retired,
package.json carries no electron-builder block (afterExtract identity stamp wired
into electron-builder.config.cjs instead; after-pack.mjs keeps signing only),
Desktop workspace-deps helpers stay retired. Main's scratch-dir bootstrap
(export_scratch_tmp_env) is taken and re-run after profile resolution.
The guard still blocked every real install: `.bytecode-fingerprint`,
`.hermes-bootstrap-complete`, `.install_method` and `hermes_agent.egg-info/`
sit gitignored at the checkout root of every installer-made install, and
`ui-tui/dist`, `ui-tui/node_modules`, `web/node_modules`, `apps/desktop/build`
and `scripts/whatsapp-bridge/node_modules` under shipped dirs — none in the
admitted set, so the ZIP fallback and the nested graft stayed unreachable
(fed the live install's 147-line status through the predicate: 21 lines
still blocked).
Classify a `!!` line by whether the swap would destroy it: the swap only
replaces the ZIP's top-level entries, so an ignored path under a root entry
the ZIP does not ship is admitted (tracked root entries stand in for the ZIP
set before the download; the pre-swap re-check passes the extracted entry
set), nested `__pycache__`/`node_modules` are regenerable, and the remaining
build outputs join `_ZIP_PRESERVED_NESTED` so the graft keeps them. Other
ignored files under shipped dirs, tracked edits and untracked files still
block.
Real-git fixture seeded with a real install's ignored set: guard blocked on
the PR head, None now; a shipped-name collision and ignored user data under a
shipped dir still refuse. Docs: Desktop troubleshooting entry for the
rebuild-after-lost-app behaviour (#90495).
The GitHub source ZIP ships only source. The two-phase swap replaces whole
top-level entries, so the gitignored outputs nested under them were destroyed:
apps/desktop/dist, apps/desktop/node_modules (electron itself) and
hermes_cli/web_dist — only apps/desktop/release was grafted. Worse, the
dirty-tree guard (--ignored=matching) reported every one of those as a `!!`
line and refused the swap outright, so an install with a built Desktop or
dashboard (or any `__pycache__/`, i.e. every real install) could not take the
ZIP fallback at all.
Generalise the release-only graft into `_ZIP_PRESERVED_NESTED`, a
{top-level entry: nested paths} table shared by the graft and the guard:
the guard admits those `!!` lines (and `__pycache__/`, regenerated at launch),
`_stage_entries` clones the live outputs into the staged copy by hardlink
(apps/desktop/node_modules is hundreds of MB; a link stays valid once the old
tree is unlinked and, on Windows, succeeds on a locked Hermes.exe where copy2
raises), falling back to a byte copy. A path the ZIP ships always wins over
the live copy, and a never-built install grafts nothing.
Salvaged from #104298; the symlinked-source branch and five of the seven tests
were dropped (no such install shape; the surviving test covers the swap, the
ZIP-wins rule and the guard admission/refusal matrix).
Part of #90495
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
A directory plugin's pyproject.toml [project].dependencies (or manifest
python_dependencies) are now installed into the Hermes venv on install/enable/
update, resolved under a constraints file built from Hermes' own pinned
dependencies together with every enabled peer's declarations. A candidate that
cannot resolve is refused before its tree is moved into place; nothing is
installed and no other plugin is touched. --no-deps opts a single install out.
hermes update rebuilds the venv from Hermes' lock and strips everything else, so
its git, zip and both repair paths now re-apply the union of every profile's
enabled plugins. When the union no longer resolves, each plugin is resolved on
its own so only culprits are dropped (never an alphabetical neighbour); a
plugin-vs-plugin conflict peels non-memory plugins first because a Hermes that
boots without memory reads as data loss. Dropped plugins are disabled through
the real config writer with a loud message naming the fix.
python_runtime: external lets sidecar-venv plugins (Mnemosyne's shape) opt out
of the union; hermes-agent self-dependencies and direct-URL requirements are
never installed (the latter are surfaced for the user to install by hand).
Independent review of the post-swap hand-off found state that did not cross or was
handled by the wrong side:
- update_cmd_config._LAST_SIBLING_SNAPSHOTS is rebound by the pre-update backup in the
parent; the child read an empty dict and the sibling-profile cron/model-settings safety
nets silently did nothing. It now rides in the payload.
- The ZIP path handed off without the Windows pause token, and the parent's finally/atexit
resumed the paused gateways while the child was reinstalling the venv. The token now
crosses on both paths; the parent flips resume_needed off only after a child actually ran.
- The child inherits HERMES_HOME but re-read the sticky active_profile, finishing a
`-p default` update inside another profile's home. _apply_profile_override keeps the
parent's resolved home for the post-swap child.
- The child env now also sets HERMES_UPDATE_REEXEC so cmd_update's Windows hard-exit tail
covers it and the shim re-exec can never fire a second time from inside it; the
hand-off pid is only claimed when no upstream updater (Tauri/Electron) already named
one.
- Popen + wait instead of subprocess.run: Ctrl-C no longer kills the child (which owns the
receipt and the gateway resume) 250 ms after the interrupt.
- A child that cannot start: the parent takes the receipt back, records the failed step,
writes .update-incomplete and the gateway exit code, exits 1.
- The child consumes the hand-off file and builds git_cmd without re-running the checkout
preflight (no second fork banner).
`hermes update` started in an interpreter that had imported the PRE-pull tree, then
kept running every post-swap phase (dependency sync, Node/web/Desktop builds,
maintenance, config migration, fleet restart, verification, receipt) in that same
process, lazily importing NEW source into an OLD `sys.modules` graph. Any rename
between the two commits surfaced as an ImportError/AttributeError inside the updater
after the code swap had already succeeded (#87134, #111271, #112465, #112558, #112604).
Each incident added another purge, reload list or per-step isolation, and each moved
the crash to the next module nobody had listed.
The pre-pull process now stops at the swap: it writes the open receipt, the pre-update
fleet plan, the pre-update version/active features and the Windows pause token to a
hand-off file and re-executes `hermes update <same flags> --post-swap <file>` under the
venv interpreter. The child imports exclusively from the pulled tree, resumes the
receipt and owns the rest of the run; the parent relays its exit code. Git and ZIP paths
both hand off. On Windows, when the updater runs from `hermes.exe`, the child is spawned
detached exactly as the shim hand-off already did (the shim cannot be awaited while the
sync must replace it).
With no pulled code ever executing in a pre-pull interpreter, the stale-module layer is
dead and removed: `_purge_stale_hermes_modules`, `_stale_purge_prefixes`,
`_evict_module`, `_STALE_PURGE_*`, `_reload_updated_runtime_modules`,
`_reload_process_scan_modules`, `_reload_config_modules`, `_UPDATE_RUNTIME_RELOAD_MODULES`
and their tests. `_run_config_check_fresh` / `_run_migrate_config_fresh` keep their names
and simply call the config API.
Tests: the hand-off boundary (child argv/env, detached receipt + plan in the payload,
exit-code relay) and the child side (receipt resumed with its history, plan rebuilt,
pre-update snapshots taken from the payload). Mocked updater flows run the tail
in-process through the same payload round-trip (autouse fixture; opt out with
`@pytest.mark.real_post_swap_handoff`).
Follow-up to the cherry-picked #112966 so the uv-default `.venv` layout is
supported end to end, not only at the lookup sites:
- `_ZIP_PRESERVED_TOP_LEVEL` gains `.venv`. The dirty-tree guard runs
`git status --ignored=matching`, so a gitignored `.venv/` surfaced as
`!! .venv/` and refused every ZIP fallback on such installs ("the working
tree has uncommitted changes or untracked files") — the live runtime was
being treated as user data the overlay would destroy.
- `_repair_venv_on_current_checkout` recreates the venv at the resolved
directory instead of a literal `venv`, so a broken `.venv` is rebuilt in
place rather than growing a second environment that `project_venv_dir()`
then prefers while `bin/hermes.cmd` still launches the old one.
- `_refuse_update_if_venv_foreign_owned` scans the resolved venv (the only
remaining `PROJECT_ROOT / "venv"` literal on the update path).
- windows.ps1 names the actual shim path in the lock-timeout message.
- Tests: extend the real-git ZIP guard test with the `.venv` case (red
before this commit); the holder-guard test now uses a kernel-runner child
whose cmdline lacks `hermes_cli.main`, so only the venv-prefix arm can match
it (red on origin/main); drop the `process.platform`-override vitest case,
which exercised the same resolver as the `.venv` case with a different
directory string.
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Run historical updater completion in a fresh interpreter so cached imports
cannot revive retired dependency installers. Share Git and ZIP completion,
carry receipt and recovery state, and preserve child exit status.
Route plugin admission, binary acquisition, desktop launch and build paths
through PM. Replace redundant helpers and tests with real worker, package,
publication and launch checks. Keep the shipped compatibility surface fixed.
Targeted Python and desktop checks pass. Native update journeys and fresh
production image qualification remain pending. This is a checkpoint before
those acceptance runs.
PM owns Python dependency generations. Shared frontend builders own Node
preparation and compilation. Route source updates and launchers through
these owners instead of separate repair ladders.
Remove obsolete live-venv holder gates and soft build-failure plumbing.
Preserve source validation, staged publication, fleet outcomes, and
historical relaunch hooks.
Verification: 956 tests passed in the combined focused run, with 43 skips.
After the final ZIP exit fix, 583 focused tests passed. Shared JavaScript
builder tests, Ruff, and diff checks passed. Native Windows/macOS and full
packaged-app builds were not run.
The ZIP fallback selected main, while Git reused a mutable local tag.
Select the remote release commit before applying either transport.
If a server requires a named-ref fetch, check FETCH_HEAD.
Keep local tags intact. Refuse a moved fallback tag before touching the tree.
Gitless Windows installs resolve the official release through the API.
Git-error fallback keeps the selected SHA and resumes paused gateways.
Stable updates retain their branch and do not sync upstream main.
Verification: 356 tests passed across eleven files, including real Git,
loopback archive swaps, dirty-tree refusal, local-ahead preservation and
import-surface checks. No production test hook or runner override added.
No installed environment, remote release, or live gateway was updated.
Prepare dependency generations before selecting them. Keep shipped tool
bytes separate from writable additions, and store facts beside their entries.
Validate proposed plugin sets before config publication. Restore the previous
config if the facts write fails.
Consolidate duplicate updater, backup, setup, and voice helpers. Repair
launcher selection, dependency consumers, download ownership, update feeds,
and native Windows process and file handling.
Verification: 206 changed/prior-failing Python files reported 4630 passed,
one failed, and 330 skipped. Fix the remaining Hindsight fixture boundary.
The final targeted rerun reported 234 passed and two skipped. The store
review regression batch reported 83 passed and one skipped. Desktop
TypeScript checks, 56 selected Electron tests, 24 release tests, and the
removed-import/compatibility guards passed.
This is an integration checkpoint, not full audit acceptance. The complete
Python suite has not run on this fixed tree. Crash-atomic plugin publication,
generation cleanup, receipt correlation, and packaged lifecycle acceptance
remain open in docs/pm-audit-status.md.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
