What: _update_takeover starts update_finish via update_custody.popen_post_commit and _old_updater
starts the takeover via the already-loaded update_lock helpers: created suspended, bound to the
lock owner's kill-on-close job, then resumed (POSIX keeps the lock fd). A refused bind still runs
(post-commit, D2) but is printed/receipted and the child joins the lock holding its own lease (R5b).
Why: both hops were plain unsuspended subprocess runs on Windows, so killing the takeover (or the
old updater) freed the checkout lock while the unbound child kept syncing and building.
Red/green: Windows-native cells added (refused bind -> receipt step / notice; pre-fix none); POSIX
takeover/finish/custody suites pass (144 passed).
R3 / A7 rule 1: every marker decision-and-mutation (reclaim of a dead claim,
adopt + delegate append, release, ownership hand-over) now runs while holding
the kernel lock on the never-deleted sidecar `<marker>.lock` (POSIX flock;
Windows: the file opened share-none, the same lock the PowerShell hand-off
takes), bounded to 10 s. Only the exclusive first publish stays lock-free
(rule 2). Python (update_lock.marker_mutex) and Rust (marker.rs lock_marker)
interoperate on the same lock. A stale reader paused before its unlink can no
longer delete a claim published after its read.
A7 rule 4 / R12: one incarnation rule. Our own pid is ours only within 5 ms of
our creation time; a no-ct claim naming our pid is a previous incarnation
(dead) — in Python (`update_lock.incarnation_live`, exported for the pause
record) and in marker.rs (no more 2 s self tolerance; Electron's v1 own-pid
pre-write is reclaimed and republished as v2; the exit-2 heal requires our
exact incarnation).
A7 rule 5: release is identity-based (`_release_decision` / `release_decision`).
The owner removes its claim regardless of delegate lines unless a LIVE
delegate other than itself exists, which then inherits the claim (rewritten
with the delegate as owner, started_at and run lines kept). A delegate drops
its line while the owner lives, else deletes. This is the native Windows
`orphaned_update` root cause: the hand-off script named `hermes update` as
delegate before it claimed, the child adopted without writing, and its exit
released nothing, leaving '<dead owner> ... delegate:<exited child>'.
Parser (A7 rule 7): lines 4+ are tagged — first well-formed delegate line,
every `run:<id>` line; pid must fit u32. Both consumers run
tests/fixtures/update_marker_corpus.json (judge + release cases).
R2: checkout mutators keep custody. `_git_run` passes the held checkout lock
fd to git (and its hooks); `run_source_script` passes it to the Node build;
`complete_source_checkout` ACQUIRES `UpdateLock(install_root=root)` instead of
claiming only its home's marker; venv_sync's interrupted-update finish acquires
the checkout lock before mutating instead of sampling it and hands the fd to
its completion child; the historical takeover chain passes it down. On Windows
a child of the update joins the lock when its holder record names a live
ancestor (it runs in that owner's kill-on-close job).
R11: a live pre-root-marker claim under <root>/profiles/*/ (the old per-profile
marker) is observed: a new update refuses while one is live and not a partner.
Real-process tests: tests/hermes_cli/test_update_lock_protocol.py (each fails on
6ff4055cd74), corpus consumers in Python and cargo.
A Desktop built before the blobless fix keeps walking history during
`hermes update`. On a treeless (tree:0) checkout every walk downloads trees
again, and #129514 filled a 434 GB disk during the minutes of dependency work.
The one-time conversion (#133016) ran at the very end of the update, after that
window. Both update hand-offs (update_completion._prepare and an older
updater's _update_takeover.prepare) now run it first, in the bootstrap
interpreter, and the late call is gone.
Re-running install.sh / install.ps1 over a treeless checkout now converts it
too. The filter is recorded only after the refetch succeeds, so a failure is
retried by the next update instead of leaving a blob:none filter over a
history that has no trees.
The pack cleanup also sweeps tmp_* transfer files that killed fetches left
behind. They are more than an hour old, a storm leaves gigabytes of them,
and git itself only prunes them after two weeks. The docs gain the recovery
steps for a disk that the storm already filled.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: Yuan Li <dskwelmcy@163.com>
An update resolves the enabled plugin union against the NEW core. A plugin
admitted against the old core can stop fitting when core moves (managed
Python 3.13 -> 3.14 vs a member's requires-python <3.14, a requires_hermes
upper bound, a bumped pin), and the whole update then died after the
source swap with a non-resolver InstallError whose 'retry' hint failed the
same way every time.
Update syncs now pass evict_incompatible_plugins=True (update completion,
historical takeover, launch-time completion, venv_sync, post-update
drift). PM screens statically first (requires-python vs the target
interpreter, manifest/requires_hermes), then, if the rest still fails,
builds core alone to prove the plugins are the cause and re-adds members
in config order, disabling each one that breaks the build. Misfits land in
plugins.disabled (memory.provider cleared) in every home that enables
them, published through the existing journaled change hook (the journal
now carries several configs), and are reported on stderr + receipt
warnings. Admission and ordinary syncs still refuse; only a core that
cannot build on its own fails an update.
A normal `hermes update` only re-synced the venv. The sync pulls uv/python
in through its own dependency, but a ripgrep/ffmpeg/node/npm pin bump in
pm/lock.json was never installed, so PATH activation warned and skipped the
managed tool dirs on every CLI start and gateway boot. Only the takeover
route for historical releases ensured the tool roots.
Move the takeover's loop into pm.client.ensure_tools_for_sync() and call it
from both routes before the sync: update_completion._prepare (the CLI and
Desktop route, running from the new tree so the new lockfile applies) and
_update_takeover.prepare. It uses explicit=True like the takeover (an update
is an explicit user action) and a failed download fails the update.
post_update.step_provision_runtimes / MACHINE_STEPS stay: `python -m
hermes_cli.post_update --scope machine` and tests still reference them.
The ffmpeg docstring no longer claims that step re-ensures it.
Main-era installs selected [all] and then lazily installed opt-in extras
(FAL, messaging SDKs, ...) into the checkout's venv with no ledger. The
legacy -> PM migration (launch completion and the historical updater
takeover) selected only [all], so the first launch after `hermes update`
prompted "Extra 'fal' requires: fal_client" for a feature that already
worked.
pm.extras.legacy_selection reads the old venv's site-packages (never
imports it) and selects every non-umbrella, platform-supported extra whose
anchors it held. The install prompt that remains for genuinely new
features names the feature instead of Python module names.
Windows PowerShell 5.1 returns every match from Get-Command. .Source on
that array joins the paths with a space, and the call operator then treats
the joined string as one program name. Git for Windows ships git.exe in
cmd\ and bin\, so setup died with CommandNotFoundException before pm
install ran.
setup-hermes.ps1 now installs the tool closure first (`pm install
--tools-only`), then prepares the ARM64 compiler environment, then syncs
the venv. The sync inherits that compiler environment. A bare `pm install`
and the update takeover path publish tools and put them on PATH before
uv sync. A missing tool stops the sync. A missing venv does not.
Verified: scripts/run_tests.sh on test_install_default_closure.py,
test_install_extra.py, and test_windows_build_deps.py — 13 passed.
hermes_cli.runtime_paths (venv generations, selection, activation) moves to
pm.environments, and gains venv_bin_dir / venv_python / project_python. Every
in-tree caller asks pm for an interpreter now; pm no longer reaches back into
hermes_cli for its own environment layout (pm.packages, pm.extras, pm.ensure,
pm.paths imported hermes_cli.runtime_paths). The three open-coded
"Scripts/python.exe or bin/python" ladders in pm collapse onto venv_python.
hermes_constants.venv_python_path / venv_bin_dir and hermes_cli.runtime_paths
stay as frozen-updater-surface shims only (tests/compat/old_updater_surface.json).
To keep the boot path light, pm/__init__ resolves its facade lazily (PEP 562)
and pm.registry loads the built-in package definitions on first read instead of
at import: `import hermes_bootstrap` now loads pm + pm.environments only (25ms,
was 37ms with the eager facade dragging in the downloader). The stripped-payload
fixtures that ship only pre-import files keep working for the same reason.
Also restores two frozen-surface re-exports the F401 sweep dropped
(banner._github_compare_behind, cua_backend.resolve_cua_driver_cmd).
The shim renders progress from its status JSON file, but everything
after 'Updating code and dependencies' — the PM dependency sync, Node
deps, the TUI/web/desktop builds — ran for minutes without touching
that file, so the window froze on a stale stage (or showed nothing at
all when the old shim skipped its browser window).
hermes_cli/update_stage.py publishes stages to the watching UI,
std-only and never raising. Two discovery paths: the exported
HERMES_UPDATE_STATUS_FILE (current shim), and, for an OLD shim that
never exported it (every old-to-new checkout transition), the marker's
owner pid, which names the shim whose status file is deterministically
/tmp/hermes-update-status.<pid>. On macOS the takeover child also pops
update-panel.applescript from the freshly pulled tree when the old
shim's log shows it started no renderer.
Publishes: PM sync (_update_takeover.prepare, venv_sync.sync), Node
deps and each product build (source_build.build_update_products).
Only 'running' stages are ever written — terminal states stay the
shim's.
Run historical updater completion in a fresh interpreter so cached imports
cannot revive retired dependency installers. Share Git and ZIP completion,
carry receipt and recovery state, and preserve child exit status.
Route plugin admission, binary acquisition, desktop launch and build paths
through PM. Replace redundant helpers and tests with real worker, package,
publication and launch checks. Keep the shipped compatibility surface fixed.
Targeted Python and desktop checks pass. Native update journeys and fresh
production image qualification remain pending. This is a checkpoint before
those acceptance runs.
