Merge ethie/shared-product-builders with the CI dependency cache and native Windows setup work. Preserve UTF-8 diagnostics in the shared Python environment runner. Pass a persistent cache through isolated native staging and PM-runtime construction. Reuse one Windows prerequisite installer from source setup, native adapters, and CI, preserving Rust homes across HOME isolation.
Verified 85 targeted Python tests (5 host skips), 18 JavaScript tests, workflow validation, and scoped lint/typecheck. On native Windows ARM64, five prerequisite contracts passed and the actual shared provider reused OpenSSL, compiled its header with MSVC, and retained Rust under isolated HOME. Full signed distribution builds and live Actions cache transfer remain CI verification.
Activation reaches plugin discovery before the application dependencies
exist. Give PM its own locked Python project and runtime so it can install
or repair the application without importing that dependency tree.
Keep PM outside the application workspace. A shared uv workspace resolves
the application graph and cannot provide this isolation. Route mutations
through an isolated worker and preserve transaction callbacks, cancellation,
custom package registrations, and correlated receipts.
Use the same runtime builder for source installs and packaged payloads.
Keep offline wheelhouse support in that builder. Nix builds the independent
PM lock as a separate derivation. Refuse lazy-disabled bootstrap before
installing tools or dependencies.
Move first-party YAML readers and writers to ruamel. Keep the application
lock's transitive PyYAML requirements for third-party packages.
Verification:
- Focused canonical Python suite: 177 passed, 1 host-gated skip.
- Electron backend probes: 12 passed. Electron typecheck passed.
- Both uv locks, scoped lint, Bash syntax, and whitespace checks passed.
- Cold activation, corrupt-app repair, offline staging, and relocation ran.
- Built and exercised the Nix PM runtime and standalone YAML merge script.
Six broader caller test files retain the same 24 failing test IDs as an
archive of HEAD. The existing real-home guard blocks those tests before
they can exercise the affected paths. No full-suite pass is claimed.
Native Windows signing and full Bionic package execution remain unverified.
BtbN publishes new FFmpeg builds without changing the version number.
The shared-minor comparison therefore reported stale artifacts as current.
Compare advertised artifact URLs during resolution and hash changed URLs
when applying the update. Keep dry-run checks metadata-only and preserve
pins for targets without an update source, including Termux.
Verification: 41 focused tests passed. The regression exercises real
archive downloads, installation, retained target pins, and a second
update that performs no writes. Native ARM64 FFmpeg also passed a real
16 kHz audio encode after installation.
The uv step used a nonexistent command. Both dependency steps used the
caller directory instead of the PM repository. Run uv lock --upgrade
and the installed npm executable in the correct project. Require the
installed tool closure without installing a fallback.
Reuse npm environment sanitization for unpack and update. The separately
pinned npm keeps its Node dependency on PATH without changing the parent.
Missing tools and failed commands return failure before venv sync.
Real uv and npm commands ran in guarded temporary projects. The test
removes Node's bundled npm before update and preserves unrelated files.
The missing-Python test checks the actual refusal and unchanged facts.
All 87 focused tests pass. Lint passes. No project pins or locks changed.
A failed lookup was reported as no change and returned success.
Track failures where exceptions occur rather than parsing status text.
Report independent successful lookups, then stop before any pin,
install or dependency refresh if one lookup failed.
Verification: the real PM CLI exercises check and apply with failed,
mixed, current and manual-source results. Mutation boundaries stay
unreached and the temporary lock stays unchanged. The focused gate
passed 36 tests with no failures. Lint passed.
No upstream package, real pin table or installed environment changed.
Keep downloads bound to their remote representation and publish through
atomic destination-local staging. Serialize shared partial ownership.
Keep explicit CA trust scoped to provider probes. Preserve checkpoint
history and edited files, validate all profile inputs before dependency
publication, and separate data removal from installed runtime ownership.
Exclude machine-specific PM state from portable transfers. Keep plugin
files and nested skill tools intact. Preserve native test isolation.
Focused native Windows receipts cover the individual repairs and their
integration. This commit does not claim a full-suite or release build.
Pin uv and uvx to the PM interpreter instead of ambient Python discovery.
A matching dependency stamp cannot prove that installed files still exist.
Repair now rebuilds the recorded workspace and lock in a fresh generation,
checks startup imports, and publishes the selection only after success.
Run startup recovery before dependency activation. Keep manual PM repair
reachable when the selected environment is damaged. Preserve plugin
selection, retry ownership, and the previous generation on failure.
Remove the separate pip, ensurepip, per-extra, and install-time quarantine
ladders. Keep orphan launcher restoration.
Verification: 717 targeted tests passed on native Windows ARM64, with
56 skipped. Ruff, diff checks, and the source-scoped compat check passed.
A disposable real Hermes install recovered deleted YAML and dotenv files,
then printed CLI help with exit 0. Its lock and stamp stayed unchanged.
The full suite and a release build were not run for this change.
Finish bootstrap uv before PM replaces its store entry. Keep failure
receipts stdlib-only and align the cryptography requirement and override
with the locked version.
Let bundle builders declare launch paths and update ownership. Remove
payload discovery, Store probing, and the unused develop command.
Derive Nix Python from the PM lock and share its provenance stamp.
Document setup, activation, optional dependencies, and distribution
ownership. Targeted Windows tests, relocated runtime launches, Electron
bundling, and bilingual docs builds pass. Native Nix and signed-package
acceptance remain CI gates.
Merge ethie/cli-bundles at 0765ad689b.
Keep PM runtime publication, install identity, TLS policy, and module
boundaries from pm-clean.
Resolve the Node version-discovery method in its owning class. Preserve
staged tools if a repin download or publication fails. Carry extra-only
memory-provider setup through PM and retain restart-required reporting.
Keep target-specific TUI path assertions and discard obsolete self-lock
fixtures and the orphaned Windows service handler.
Verified locally with the canonical Python runner, root JS checks, TUI
checks/build, shell parsing, and workflow YAML parsing. Existing platform
pins and executable modes are unchanged. No full-suite CI, new bionic
bundle, or phone acceptance is claimed for this merge.
Dependency publication now recovers interrupted config/facts changes before
activation and leases live generations during collection. Receipts retain
update correlation and failed steps across nested command boundaries.
Doctor and desktop surfaces report those failures through shared owners.
Move checkout updates out of the desktop facade. Stage a detached Windows
relaunch waiter before shutdown, with bounded handshake and process-birth
checks. Keep packaged lifecycle tests isolated from the installed app.
Native verification exposed two production races: cron maintenance imported
the interactive CLI and rewrote TERMINAL_CWD, and install-ID reads collided
with first publication. Use the existing owners and locks. Plugin checks
now run at startup and each due-gated housekeeping tick, not after 60 ticks.
Share updater-test mutation boundaries and remove collection-root fixtures.
Separate cold MCP startup from command latency and give the real HTTP drip
test enough time to reach body handling.
Root npm check passed, including packaging. The fixed-tree Windows Python
run reported 44557 passed, one failed, and 1404 skipped, plus one retry-only
HTTP test. Those final failures now pass in a 35-test bounded batch. A real
isolated gateway wrote startup and periodic plugin-check receipts.
Full final-tree CI, bundled Sandbox deployment, and actual App Installer
relaunch remain unverified. docs/pm-audit-status.md records these limits.
Prepare dependency generations before selecting them. Keep shipped tool
bytes separate from writable additions, and store facts beside their entries.
Validate proposed plugin sets before config publication. Restore the previous
config if the facts write fails.
Consolidate duplicate updater, backup, setup, and voice helpers. Repair
launcher selection, dependency consumers, download ownership, update feeds,
and native Windows process and file handling.
Verification: 206 changed/prior-failing Python files reported 4630 passed,
one failed, and 330 skipped. Fix the remaining Hindsight fixture boundary.
The final targeted rerun reported 234 passed and two skipped. The store
review regression batch reported 83 passed and one skipped. Desktop
TypeScript checks, 56 selected Electron tests, 24 release tests, and the
removed-import/compatibility guards passed.
This is an integration checkpoint, not full audit acceptance. The complete
Python suite has not run on this fixed tree. Crash-atomic plugin publication,
generation cleanup, receipt correlation, and packaged lifecycle acceptance
remain open in docs/pm-audit-status.md.
A seventh pm target (linux-arm64-bionic) stages the TUR python3.11
.deb, the termux nodejs/uv .debs, and their runtime-lib deps into the
payload -- same pm-consumer shape as the desktop legs: pm owns the pin,
the hardened download (redirect-safe, retry-wrapped), the ar+tar
DebPackage extraction, and file-evidence verify for binaries the
staging host cannot execute.
The win32-arm64 bundle stage deleted vcruntime140_1.dll (the x64 VC
runtime python-build-standalone ships beside ARM64 Python; it cannot
load there and would fail the arch guard) AFTER store.publish() — but
facts.record(digest=tree_digest(entry)) had already hashed the entry
WITH the dll. pm doctor's re-hash then flagged python on every
bundle: 'realized bytes do not match recorded digest' — red from the
commit that introduced both the drop and the digest check (3d12e86ef1),
masked until the smoke test's earlier dead-shim failure was fixed.
The drop belongs in Python.stage(), next to the macOS signing hook —
both are post-extract, pre-publish mutations, so the recorded digest
covers the shipped bytes. The cli-level _drop_unloadable_runtime_files
bundle hook is removed in full (its two tests now pin the stage()
contract). The pm-store CI cache key rotates to v2: failed arm64 runs
saved poisoned facts (dll-less entry + with-dll digest) under the old
key and would restore forever.
Every pm venv sync — startup, plugin install, update rebuild — now
writes a receipt with the SAME schema the updater's receipts use,
into the same <HERMES_HOME>/logs/update_receipts/ dir, separated by a
'kind' field (settled 2026-09-02 plan, task 8):
- pm/receipt.py: begin/record_step/record_venv_rebuild/
record_bisect/record_feature_list/finalize + rotation (keep 20) +
latest.json pointer. snapshot() lets the updater EMBED the sync
sections into its own receipt (one schema, one dir, one reader).
Exception-swallowing throughout — receipt machinery can never break
a sync.
- Venv.apply union path: records feature_list, venv_rebuild, and
bisect decisions; outcome 'ok' | 'bisected' (failures raise before
the receipt block and surface via the update receipt's error path).
- `hermes pm status` (new verb): prints the latest receipt as JSON —
the CLI/TUI reader. Desktop IPC reads the same latest.json (the
hermes:version/syncStatus wiring lands with the desktop branch's
About/update surface, which already consumes update receipts).
tests/pm/test_receipt.py: 5 tests (roundtrip, latest-pointer,
no-begin no-op, snapshot lifecycle, empty home). tests/pm: 180
passed, 0 failed.
The blow-the-venv-on-update contract needs rebuilds to be cheap, and
sealed installs need a writable venv at all (settled 2026-09-02 plan,
task 7):
- uv_cache_dir(): hermes-owned machine cache at
<default hermes root>/cache/uv — content-addressed, shared across
profiles. uv_env() ALWAYS pins UV_CACHE_DIR there (ambient UV_*
stripped), so the cache that ships is the cache that gets used.
First call on a sealed install seeds it from the payload's shipped
uv-cache/ (read-only payload can't serve uv's working cache); the
.seeded marker makes it once-only and non-clobbering.
- pm bundle stages the warmed cache into the payload after the venv
sync (uv-cache/ beside manifest.json) — warm 'uv sync --offline'
rebuilds probed at 0.4s vs 1.2s cold.
- Venv.venv_dir(): sealed installs resolve the MUTABLE venv to the
machine hermes root (<root>/venv), not the read-only payload;
dev/source installs keep the repo-local venv unchanged.
- Venv.seed_mutable_venv(): the bootstrap seed — lazy-off installs
copy the payload's shipped venv out as the starting point; lazy-on
installs skip the copy (first sync builds fresh from the shipped
cache). adopt() triggers it (KeyError-guarded, failure reported
never fatal — a cold sync still converges).
tests/pm/test_uv_cache.py: 6 tests (env pinning + ambient strip,
payload seed + marker once-only, cold machine, sealed venv_dir, seed
copy idempotence, lazy-on skip). tests/pm: 175 passed, 0 failed.
Lazy installs OFF is now a real contract, not just 'refuse
everything': the bundle's EXACT extras list is the frozen feature set
(settled 2026-09-02 plan, task 5).
- pm/features.py: write_features()/read_features() over
enabled-features.json at the payload root (bundle-written, beside
manifest.json; at runtime store_root().parent — the same relative
location on both install kinds). installed_extras() records what
`uv sync --all-extras` ACTUALLY installed on the target: every
declared extra whose pm anchor resolves in the staged venv —
marker-gated extras show up as missing anchors, the honest
per-platform record.
- cmd_bundle: after the staged venv sync, write the features file into
the payload ('✓ enabled-features.json (N extras recorded)').
- sync_venv: when security.allow_lazy_installs is false and the file
exists, requested extras OUTSIDE the frozen set are refused with a
message naming the policy.
- Venv.apply: plugin members + lazy-off + frozen file = loud InstallError
(the bundle IS the install; never union plugin deps into it).
tests/pm/test_features.py: 7 tests (roundtrip, absent/garbage reads,
payload-root path, anchor-truth installed_extras, frozen refusal,
in-set pass). tests/pm: 162 passed 0 failed.
Each Package subclass now declares how to find its own latest via a
latest_versions(target) hook (empty = no auto source). `hermes pm update`
intersects those candidate lists across every target the package serves,
compares against the lockfile, and re-pins + reinstalls the changed ones.
Version styles (Package.version_style):
- semver (default): one shared version across targets; update = highest
version every relevant target serves (node, uv, gh, ripgrep, git,
cua-driver, llama.cpp, agent-browser, npm).
- minor (ffmpeg): posix martin-riedl and win32 BtbN autobuilds have no
shared release cadence, so the lockfile version label is major.minor
and each target's exact patch lives in ITS artifact urls. An update
moves to the highest major.minor every target serves; within it each
target pins its own newest patch. A patch-only drift inside the shared
minor does not move the label.
Resolvers hit the real upstream indexes (GitHub releases with prefix
strip + version-shape filter that drops sandbox/experimental tags,
nodejs.org index.json, npm dist-tags, martin-riedl root page parse for
per-platform epochs, BtbN autobuild assets, python-build-standalone
filtered to the locked 3.11 line — a major bump is never automatic).
CLI:
- `pm update [names...]` — re-pin + install changed packages, sync venv
- `pm update --check` — dry-run report (exit 1 if updates exist), never
writes the lockfile, store, or venv
- `pm update --target T` — check-only cross-resolution for another target
- `pm update --uv` / `--npm` — also refresh uv.lock + venv / package-lock
chromium + chromium-headless-shell have no resolver (they follow
agent-browser, which pins its own browser pairing); venv is a state
package. Tests are pure (monkeypatched candidate lists) — the index
helpers are network I/O by design.
Verified: 136 pm tests pass; live `pm update --check` resolves node
26.7.0 → 26.8.1, uv 0.12.3 → 0.12.9, ffmpeg up to date, and reports
`no source` for chromium. GitHub unauthenticated rate limits (60/hr)
degrade to an honest per-package `resolve failed`.
store_root() resolves into the read-only sealed payload on MSIX installs
(WindowsApps/agent-payload/tools), so the default partials area there
failed with WinError 5 on local-model downloads. Partials are mutable
machine-scoped state keyed by sha256(url); anchor them to the default
hermes root (cache/partials) instead of the immutable byte store.
- pm/paths.py: add partials_root(); drop the _default_partials indirection
- pm/downloader.py: Download resolves paths.partials_root() directly
- pm/cli.py: pm gc sweeps the new partials area (store root no longer
contains them); guard bails only when both roots are absent
- tests: point resume + gc tests at the new root
The store's fetch-<sha> download-cache archives are install-time only —
dead weight once a package is published, in a staged payload AND in the
CI cache that restores apps/desktop/build/agent-payload/tools. They were
never in facts.entries_in_use(), so gc already dropped them, but nothing
proved it and nothing ran the sweep during the bundle, so the cached
store accumulated orphaned versions from older locks.
- extract the sweep core into _gc_store() (shared by pm gc and the
bundle command).
- run it at the end of pm bundle, before venv sync and packaging, so a
staged payload (and the CI cache) ships only live entries.
- test that gc removes fetch-<sha> dirs while keeping the live package
entry.
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
