27 Commits

Author SHA1 Message Date
teknium1
ee0ad5adb2 refactor(pm): expose MUSL_TARGETS publicly; drop the per-URL pin hash cache
packages.py and security_packages.py imported the private _MUSL_TARGETS
across modules; make it a public store constant next to ALL_TARGETS.
The _pin_tool hash memo is an unrelated optimisation, out of scope here.
2026-09-27 03:24:39 -07:00
JoaoMarcos44
24487f3db6 fix(pm): keep musl installs on compatible runtime artifacts
Skip glibc-linked FFmpeg on musl in both default install and update roots.
Select musl uv during the standalone shell bootstrap, and let the native
userland resolve libc before bootstrap Python build metadata.

Add regression checks for the closure, target precedence, and installer pins.
2026-09-27 03:24:39 -07:00
JoaoMarcos44
8371dd17f6 fix(pm): select native musl artifacts on Linux 2026-09-27 03:24:39 -07:00
ethernet
6dc1288dd1 refactor(pm): route .deb payloads through the one tar extractor
DebPackage kept its own 116-line member walker next to
pm.store.extract_tar. Both enforced the same containment rules, so the
.deb data.tar now goes through extract_tar (which accepts an open
stream) and FilterError maps to InstallError. Link fallbacks on hosts
without symlink support come from tarfile's own copy fallback, which
also resolves chained aliases. The Termux mode normalization stays in
DebPackage, since it is .deb policy and not extraction.

The per-member FilterError/OSError swallow in pm/packages.py was already
removed; Git's MSYS /proc link exemption is unchanged.
2026-09-24 14:08:21 -04:00
ethernet
b728fdd627 refactor(pm): own the lock/atomic-write primitives in pm.filesystem
pm imported runtime_state's private helpers (_lock, _atomic_bytes, _bytes,
_digest) at a dozen sites while runtime_state imports pm.environments at
module top. The primitives are pm's: move them into the stdlib-only
pm.filesystem as lock_fd, durable_write_bytes, read_bytes_or_none and
file_digest, and repoint every pm caller.

runtime_state keeps the private names only as import aliases: it still
calls them through its own globals, and pre-PM updaters load them by these
names mid-swap (tests/compat/old_updater_surface.json).

Boot-subset test fixtures now copy pm/filesystem.py, since runtime_state
imports it at process boot; worker-injection tests patch the name
pm.publication now reads.
2026-09-24 14:08:03 -04:00
ethernet
f17e27b1ce fix: ignore MSYS mount-table symlink in bundled Git archive 2026-09-24 02:50:25 -04:00
ethernet
bd392ba922 fix(pm): contain review security and dependency verification regressions 2026-09-24 02:03:59 -04:00
ethernet
053ec28542 fix(pm): say which lock a queued store writer is waiting on
A second `hermes pm install` behind a long sdist build blocked with no
output for minutes. After 2s without the lock, print the lock path to
stderr, then keep waiting as before.
2026-09-21 18:53:32 -04:00
ethernet
f9ea87ed48 fix(pm): extract relative tar symlinks under the 3.10 bootstrap interpreter
python-build-standalone ships share/terminfo/1/1178 -> ../a/adm1178. The
3.10.12 backport of the 'data' filter resolves a symlink target from the
archive root, not the link's directory, and rejects it, so PM could not
install Python on the ubuntu-22.04-arm docker runner. Resolve symlink targets
ourselves (from the link's own directory, still confined to the destination)
and keep the stdlib filter for everything else.
2026-09-19 02:01:48 -04:00
ethernet
a00ca233f8 fix(pm): voice enables audio-io through PM; public plugin-state readers; PM tests isolate the machine home
- voice_mode: `_import_audio` asks `pm.ensure_import("audio-io")` before importing
  sounddevice, so `/voice on` turns the feature on (or reports exactly why PM
  refused: lazy installs off, platform gate, restart needed) instead of printing a
  `python -c "from pm import sync_venv…"` one-liner for the user to run.
- pm.plugins_state: `read_home_selection` and `dependency_homes` are the public
  readers; memory_provider_migration and plugins_cmd stop importing underscored names.
- pm.store: the `sha256_file` shim is gone; the authority test asserts the Store has no
  file hash of its own rather than patching one.
- tests/pm/conftest: `isolated_machine_home` is autouse (Path.home, HOME, USERPROFILE,
  HERMES_HOME all under tmp_path); `@pytest.mark.real_machine_home` opts a module out
  — the four suites that spawn children with a home they build themselves.
- activate / activate.ps1 / Dockerfile followed hermes_cli.runtime_paths to
  pm.environments (the earlier sweep only covered .py).
- pm.registry loads builtins through importlib so a test patching `pm.packages` in
  sys.modules still registers the security packages.
2026-09-19 00:48:11 -04:00
ethernet
f6a30447de refactor(pm): one install_root() in pm.paths replaces six HERMES_INSTALL_ROOT ladders
post_update._install_root, boot_bootstrap.default_project_root,
update_channel._default_root, version_info._CODE_ROOT + _resolve_stamp_file,
_launchers.publish_launchers and pm.runtime each re-derived "HERMES_INSTALL_ROOT
or the code root". pm.paths.install_root() is the one place now; pm.store reuses
the downloader's User-Agent instead of carrying a second one.
2026-09-18 20:08:16 -04:00
ethernet
bbec973514 refactor(pm): pm owns the dependency-environment layout and interpreter paths
hermes_cli.runtime_paths (venv generations, selection, activation) moves to
pm.environments, and gains venv_bin_dir / venv_python / project_python. Every
in-tree caller asks pm for an interpreter now; pm no longer reaches back into
hermes_cli for its own environment layout (pm.packages, pm.extras, pm.ensure,
pm.paths imported hermes_cli.runtime_paths). The three open-coded
"Scripts/python.exe or bin/python" ladders in pm collapse onto venv_python.

hermes_constants.venv_python_path / venv_bin_dir and hermes_cli.runtime_paths
stay as frozen-updater-surface shims only (tests/compat/old_updater_surface.json).

To keep the boot path light, pm/__init__ resolves its facade lazily (PEP 562)
and pm.registry loads the built-in package definitions on first read instead of
at import: `import hermes_bootstrap` now loads pm + pm.environments only (25ms,
was 37ms with the eager facade dragging in the downloader). The stripped-payload
fixtures that ship only pre-import files keep working for the same reason.

Also restores two frozen-surface re-exports the F401 sweep dropped
(banner._github_compare_behind, cua_backend.resolve_cua_driver_cmd).
2026-09-18 20:02:36 -04:00
ethernet
3d0b6f9a0c refactor(pm): remove redundant work and test layers
Keep desktop rollback in staged publication instead of restoring backups
inside a candidate directory that will be discarded.

Use the npm tag endpoint, let the downloader own archive verification,
and ensure CI toolchain roots without repeating dependency verification.
Remove an unused resolver input and replace overlapping tests with
fault-proven lifecycle coverage.

Verified with the scoped Python gate and before-pack tests. Native
Windows/macOS update execution and the full suite were not run.
2026-09-12 18:34:01 -04:00
ethernet
3af8084671 fix(pm): support junction checks during legacy Python bootstrap
Python 3.11 fails while PM records the replacement interpreter's tree digest. The bootstrap cannot require Path.is_junction before it installs the managed Python.

Use Windows reparse metadata for junction detection in the store and data cleanup. Retain junction target protection without requiring the newer pathlib API.

Verified the exact failure on real Python 3.11 before the fix. Cold provisioning and source-update relaunch pass afterward. Targeted tests: 16 passed, one Windows-only test skipped on Linux. Ruff passed. The full suite and native Windows test were not run.
2026-09-12 11:01:12 -04:00
ethernet
c8a9682505 fix(pm): preserve pinned binary inputs in R2
Termux removes old package files, so a pinned URL and hash do not keep
build inputs available. Preserve the exact bytes without changing pins.

Archive every PM HTTP artifact and the Termux runtime inputs by SHA256.
CI reads R2 first. Only a missing object permits an upstream download,
hash verification, immutable upload, and verified readback. Seed the
actual toolchain and payload stores before their consumers run.

Use the public archive as a pinned fallback in PM, bootstrap installers,
and Nix fetchers. Keep network retries bounded and report attempted URLs.
Keep publication credentials in protected CI jobs, not installed clients.

Verification:
- 283 targeted tests passed; five POSIX tests skipped on Windows.
- All 87 preserved Termux packages passed local archive miss/hit checks.
- Native ARM64 ripgrep installed through the mirror and ran successfully.
- Wheel import, workflow lint, Python lint, shell syntax, and pins passed.

Live R2 publication, POSIX tests, and Nix builds remain for native CI.
The real-byte archive checks used loopback HTTP, not the live bucket.
2026-09-10 18:17:08 -04:00
ethernet
949a508308 fix(pm): remove download archives after package publication
Retain archives through extraction, verification and publication so failed
or paused installs can retry. After success, delete only the package's
archives while holding the store lock. Normal installs commit facts first.
Cross-target staging follows the same cleanup rule.

Keep unrelated archives and resumable partials. Docker uses this PM behavior
without a separate cleanup command. Later repairs may need a new download.

Verified 64 focused PM tests and 9 Docker tests. Failure-injection tests
cover extraction, verification, publication and facts writes. The rebuilt
linux/amd64 image contains no archives in any layer and remains 1.073 GB
compressed. Real Chromium navigation, clicks and screenshots pass.
2026-09-10 16:08:03 -04:00
ethernet
afa48bd413 fix(pm): hash directory links without following them
Directory symlinks contributed no bytes to realized-tree digests.
Windows junctions instead exposed their external target contents.
Hash both as link-target text and remove them from traversal.

Native directory-link and junction tests failed on the base. They now
verify retarget detection, unchanged digests after external writes,
and cycle termination. The existing PM authority and store suites
also pass. No payload facts or live stores were rewritten.
2026-09-09 18:46:37 -04:00
ethernet
bb824c381f merge: combine network retries with downloader safety
Keep the retry branch's bounded network policy under installation.
Preserve destination-local atomic publication, partial ownership,
representation checks, aggregate progress and operation-owned pause.
Keep the serial CDN fallback active across subsequent retry attempts.

Retain exact-origin authorization for index reads and safe redirects.
The retry policy does not retry hash, disk or certificate failures.

Verified with canonical native Windows ARM64 tests for both branches,
local-model download consumers, PM core and update resolution.
The commit-build draft remains uncommitted and unchanged.
2026-09-09 17:40:19 -04:00
ethernet
bdb52ce711 fix(local-models): use pm pins and resumable component downloads
Use PM for engine binaries, dependent runtime archives, and model files.
Keep one operation-owned pause event through installation and download.
Report whole-plan bytes and retain paused jobs across desktop remounts.

Preserve the completed lm-pm worktree as its own integration parent.
The owning session verified focused Python and desktop tests, actual
Windows ARM64 CUDA downloads, and rendered pause/resume controls.
Combined verification with the safety repairs follows in the merge.
2026-09-09 15:17:09 -04:00
ethernet
8e4a804f53 fix(pm): retry transient network failures
A transient HTTP 500 during the range probe aborted bundle staging.
Retry PM-owned requests and body transfers with one bounded policy,
instead of retrying complete installs or stacking caller retry loops.

Keep partial ranges across attempts and interrupt backoff on pause.
Keep the CDN serial fallback active until its source finishes. Do not
retry permanent HTTP errors, certificate failures, bad hashes or disk
errors. Use the same policy for metadata reads and artifact hashing.

Verified: 217 focused tests passed, 1 skipped, and Ruff passed on the
changed Python files. The pinned Windows ARM64 uv archive also survived
an injected HTTP 500 through the PM CLI and its installed binary ran.
The full suite and desktop release build were not run.
2026-09-09 12:54:08 -04:00
ethernet
0b30c2484d merge: integrate termux cli bundles into pm-clean
Merge ethie/cli-bundles at 0765ad689b.
Keep PM runtime publication, install identity, TLS policy, and module
boundaries from pm-clean.

Resolve the Node version-discovery method in its owning class. Preserve
staged tools if a repin download or publication fails. Carry extra-only
memory-provider setup through PM and retain restart-required reporting.
Keep target-specific TUI path assertions and discard obsolete self-lock
fixtures and the orphaned Windows service handler.

Verified locally with the canonical Python runner, root JS checks, TUI
checks/build, shell parsing, and workflow YAML parsing. Existing platform
pins and executable modes are unchanged. No full-suite CI, new bionic
bundle, or phone acceptance is claimed for this merge.
2026-09-06 20:13:45 -04:00
ethernet
7f6d63bcbb fix(termux): classify bionic from its interpreter and run the Linux tests
The POSIX runner had a blank line after exec env, so it printed the environment and never ran pytest. Keep the command attached and prove failure propagation. The actual payload records ANDROID_API_LEVEL in sysconfig; use that instead of looking for text in a guessed libc file. Rebuild unproven runtime-library extracts from verified archives.
2026-09-06 14:12:12 -04:00
ethernet
3c08d16ba7 fix(pm): close runtime publication and updater audit gaps
Dependency publication now recovers interrupted config/facts changes before
activation and leases live generations during collection. Receipts retain
update correlation and failed steps across nested command boundaries.
Doctor and desktop surfaces report those failures through shared owners.

Move checkout updates out of the desktop facade. Stage a detached Windows
relaunch waiter before shutdown, with bounded handshake and process-birth
checks. Keep packaged lifecycle tests isolated from the installed app.

Native verification exposed two production races: cron maintenance imported
the interactive CLI and rewrote TERMINAL_CWD, and install-ID reads collided
with first publication. Use the existing owners and locks. Plugin checks
now run at startup and each due-gated housekeeping tick, not after 60 ticks.

Share updater-test mutation boundaries and remove collection-root fixtures.
Separate cold MCP startup from command latency and give the real HTTP drip
test enough time to reach body handling.

Root npm check passed, including packaging. The fixed-tree Windows Python
run reported 44557 passed, one failed, and 1404 skipped, plus one retry-only
HTTP test. Those final failures now pass in a 35-test bounded batch. A real
isolated gateway wrote startup and periodic plugin-check receipts.

Full final-tree CI, bundled Sandbox deployment, and actual App Installer
relaunch remain unverified. docs/pm-audit-status.md records these limits.
2026-09-06 11:45:41 -04:00
ethernet8023
0ae85925f3 feat(termux): pinned termux toolchain via pm (linux-arm64-bionic)
A seventh pm target (linux-arm64-bionic) stages the TUR python3.11
.deb, the termux nodejs/uv .debs, and their runtime-lib deps into the
payload -- same pm-consumer shape as the desktop legs: pm owns the pin,
the hardened download (redirect-safe, retry-wrapped), the ar+tar
DebPackage extraction, and file-evidence verify for binaries the
staging host cannot execute.
2026-09-05 20:00:00 -04:00
ethernet
578a15a190 fix(pm): anchor downloader partials to a writable shared root
store_root() resolves into the read-only sealed payload on MSIX installs
(WindowsApps/agent-payload/tools), so the default partials area there
failed with WinError 5 on local-model downloads. Partials are mutable
machine-scoped state keyed by sha256(url); anchor them to the default
hermes root (cache/partials) instead of the immutable byte store.

- pm/paths.py: add partials_root(); drop the _default_partials indirection
- pm/downloader.py: Download resolves paths.partials_root() directly
- pm/cli.py: pm gc sweeps the new partials area (store root no longer
  contains them); guard bails only when both roots are absent
- tests: point resume + gc tests at the new root
2026-09-01 14:44:19 -04:00
ethernet
a9be133aea merge: local-models (upstream/feat/local-models) onto the pm-clean stack 2026-08-31 18:00:49 -04:00
ethernet
3d12e86ef1 feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.

Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-31 18:00:48 -04:00