d2639c0dc8 started probing plugins/<name> with is_junction before the
first install of a plugin. On Windows lstat() of that missing slot raised
FileNotFoundError, so every fresh plugin install crashed (caught by the
MSIX smoke's plugin step on rc.7-v0.21.7). os.path.isjunction answers
False for a missing path; match it, and let POSIX's absent reparse tag
fall into the same False.
The image bakes eight extras into /opt/hermes/.venv but never recorded
them as PM's dependency selection. The first generation a container built
(a lazy extra such as fal, a plugin's dependencies) started from an empty
selection, so it carried only what was requested and dropped aiohttp, the
messaging SDKs and mcp. After the next restart the API server, platform
adapters and MCP servers failed to start.
- pm/build_env.py, Dockerfile: --record-selection writes the frozen
build's --extra list to the store facts as the venv selection, as a
bundle build does. enabled-features.json is not written, so the frozen
lazy-install policy stays out of Docker.
- pm/recovery.py: stage2's refresh restores extras the image ships that a
volume's recorded selection lacks, through the normal explicit sync. On
failure the image venv boots and the missing extras stay recorded for
the next attempt. A malformed recorded fact is refused, as
venv_is_current does.
- pm/install.py: share the PEP 685 extra-name key between
_still_declared and the boot reconciliation.
Fixes#135329
Follow up the shared-cache copy observation on PR #135458 with behavioral regression coverage and a sibling staging file. Make the lock-change fixture independent of timestamp granularity.
Prepared with OpenAI Codex assistance; no human review claimed.
A macOS release leg could not resolve ffmpeg.martin-riedl.de, and PM
stopped there. A name lookup failure is not a retry case, but the R2
mirror is a different host and the pinned SHA256 still checks the bytes.
Let that failure fall through to the mirror.
uv_cache_dir() wrote a bare ".seeded" marker after the first seed and
never compared against the payload again, so an updated install never
received the new payload's wheels. Enabling a plugin re-resolves without
--offline, so uv fetched the new pins from PyPI instead; for
cryptography 50.0.1 on Windows ARM64, which has no wheel on PyPI, that
is an sdist Rust build.
The marker now records the running uv.lock's digest, so the first call
under a new lock merges the new payload into the existing cache.
uv_cache_dir() copies a sealed payload's uv-cache into the machine cache and
writes .seeded only after a clean copy. The retry after a failed or killed
copy skipped every top-level entry that already existed, so the copy that had
created archive-v0/ but not finished it was recorded as complete. uv then
fails on the unfinished bucket, e.g. "The wheel is invalid: Missing
.dist-info directory".
The retry now merges into the existing tree (dirs_exist_ok) and copies each
file that is missing or whose size differs from the payload's, leaving
finished files alone.
Enabling a plugin with Python dependencies, and `hermes pm install`, failed
in the packaged Windows app: uv lock/sync/pip check exit 101 with "Unable to
create process using ...\venv\Scripts\python.exe".
uv queries a venv by running its Scripts\python.exe. That redirector lives
outside the package and CreateProcess's the interpreter named by
pyvenv.cfg `home`. 6a031bf1b3 moved generations onto the packaged
interpreter, so `home` names a file under WindowsApps that a process without
package identity may not start (exit 101 is the redirector's
RC_CREATE_PROCESS). Restore the verified writable copy of the pinned Python
in the user store as the build interpreter for sealed Windows payloads.
venv_command (also from 6a031bf1b3) stays: Hermes still enters those venvs
on the packaged interpreter, so its own children keep package access to the
bundled git, rg and PM worker.
The previous commit pre-checked the closure with a hand-rolled walk + is_installed before ensure, which repeated ensure's own check and left a window between them. pm.ensure already has the knob: verify=False trusts the recorded digest (as startup does) and an explicit ensure still installs a missing tool. Thread it through pm.client and source_build_env; the old comment claimed a verification that the interrupted-update finish path never ran.
The payload pinned libc++ 29 while the build container's apt cmake needs 30,
so the two disagreed and the shim in the previous commit papered over it. The
pool has since retired 17 of our pins, so repin them all with
`hermes pm update --termux`, which fixes the mismatch at the source.
Repinned: libc++ 29 -> 30 plus 14 more runtime libs (fribidi, glib, harfbuzz,
libaom, libbluray, libexpat, libheif, libmp3lame, libpng, libvidstab, openssl,
pcre2, readline, vulkan-loader-generic) and the bionic lock rows for uv
0.12.15 -> 0.12.23 and ffmpeg 8.1.2-5 -> 8.1.3. The pool serves ffmpeg 8.1.3
only, so that row moves back from 9.0.1 as the pool has it.
Staging with the new table succeeds: 84 packages, 269 .so, same counts as
before. The libc++ SONAME is unchanged. The one unresolved DT_NEEDED
(libharfbuzz-cairo -> libcairo) is present on the old table too.
The shim is removed. What stays from it is the in-container
`cmake -E capabilities` check, which fails in 2 minutes if the table drifts
from the pool again, instead of after 30 minutes of compiling.
Conflicts, all in apps/desktop/src/i18n:
- de/en/es/fr/zh.ts, types.ts: import blocks only. Main added the
*_projects section split, this branch the *_onboarding one; kept both.
- en_shared_metrics, zh_shared_metrics, ar_common, zh-hant (usage-stats
consent copy): kept this branch's rewritten copy. Main's only change on
its side since the merge base was a prettier reflow of
collectedReliability (2943ee6f19), which the branch's rewrite replaces;
types_shared_metrics on HEAD already matches the branch's keys.
Follow-through: main's Undo-hide toast (sidebar.projects.hiddenFromSidebar /
undoHide) arrived without ar or zh-hant strings, which this branch's
overlay-completeness test rejects. Added both to ar_chrome / zh-hant_chrome
and regenerated locales/_keys.desktop.json.
The startup reclaim runs on a daemon thread, so a hermes exit can cut a removal
short, and Windows refuses to unlink open files. shutil.rmtree takes children in
scandir order, so inputs/.project-root (the only record naming the checkout)
often went first and the remainder was invisible to every later pass.
Everything else is removed first; the record goes only once nothing else is left.
Follow-up to the review of this PR:
- The startup pruner returned early (no .worktrees/, or no tree past 24h) before the
reclaim ran, so a tree removed by hand kept its state until some other tree aged out.
_prune_stale_worktrees now runs the reclaim in a finally around the prune phases.
- A missing recorded path is not proof of deletion: a container sharing the data root
(-v ~/.hermes:/opt/data) cannot see the host's checkout, and gc there deleted the
host's live dependency environment. A state is an orphan only when its checkout sat
under the data root that owns installs/, or directly in a .worktrees/ dir that still
exists.
- PM runtime generation leases (pm-runtime/generations/*/.leases) now count as held,
so a worker still running from a deleted checkout's runtime keeps it.
Every checkout that boots Hermes commits its own dependency state under
<home>/installs/<sha256(path)[:16]>/ (venv, test venv, PM runtime; ~200 MB
each) and nothing removed it when the checkout went away. On a host that
runs `hermes -w` campaigns the directory held 499 entries / 92 GB, of which
402 (80 GB) belonged to worktrees and scratch clones that no longer exist.
The state dir already records its checkout in inputs/.project-root. A new
pm/install_states.py lists entries whose recorded checkout is gone and
removes them unless something could still be reading them (install lock
taken, or a generation lease held). `pm gc` runs it and reports the count;
the startup worktree pruner runs it after its own reaping so a removed
worktree's state follows the tree instead of outliving it.
A sealed Windows payload copied its pinned Python to %LOCALAPPDATA% so
uv-built venvs had a reachable redirector target. Every process started
from that copy then sat outside the package and could not start the
package's executables (WinError 5): bundled git, rg, the PM worker.
Build generations on the bundled interpreter instead and never run a
venv's own executables there. pm.environments.venv_command returns the
argv that enters a venv: on a sealed payload, the payload Python with -S
plus pm/_venv_entry.py, which attaches the venv's site-packages (and its
.pth files) and runs a script, -c or -m; elsewhere the venv interpreter.
Recovery's startup validation and the plugin capability probe use it.
A sealed payload's venv Scripts dir stays off PATH.
Verified (macOS): new venv_command contract tests run scripts, -c and -m
through a payload layout whose venv interpreter is a trap that exits 99.
tests/pm + plugin/source/update-completion suites: 1175 passed; the 5
failures also fail on origin/main in this environment.
Not verified: a real MSIX. uv execs the venv interpreter to query it
during sync/pip (confirmed on macOS by trapping venv/bin/python), so
plugin installs depend on uv, a child of the in-package PM worker,
keeping package access to that redirector.
The long-path commits now on onboarding/next spell PM's store root extended-length (\\?\) on Windows, and _toolchain returned uv and Python under that root. Its callers run and compare those paths outside PM's own file calls, and on main they got the ordinary path. tests/pm/test_uv_python.py::test_bundled_uv_uses_a_verified_writable_python_without_changing_runtime fails on both Windows lanes for that reason. Return native() spellings; the existence checks inside still use the store's.
(cherry picked from commit acad756e33)
Some PM code reaches a store root from paths.store_root() or
writable_store_root() directly instead of through Store, so its file
calls kept the ordinary spelling and stay cut at MAX_PATH on Windows
with LongPathsEnabled=0. Worse, its comparisons against a Store root
(now \\?\-spelled) never matched.
Each such root now gets long_root() once, where the variable is born:
- Facts.refresh_digests digests whole payload tool trees;
- verified_tools walks and digests the source store's tool trees;
- the uv cache seed copies the payload's deep uv-cache tree out;
- _toolchain's writable store, so `location.root != writable` compares
like with like and the copy store is built from the same spelling;
- _installed_location passes store.root to Facts.installed;
- ensure() picks the shipped facts by comparing two Store roots;
- the corrupt-facts guard compares both sides spelled long.
Returned and exported paths stay ordinary (previous commit).
Store now spells its root \\?\C:\... on Windows so PM's own file calls
under it pass MAX_PATH. That spelling must not escape: the same folder
spelled two ways hashes and compares differently, and children such as
cmd.exe, gpg or uv refuse or record the prefix (uv would write it into
pyvenv.cfg).
native() now converts store-derived paths where their text leaves PM:
- process argv and cwd: uv via PythonEnvironment._run (its one exit),
binary version probes, npm self-install, the PortableGit extractor,
gpg provenance checks, the plugin-eviction interpreter probe;
- environment values: compose_env (PATH and package env diffs), the
activation PATH dirs, HERMES_RUNTIME_DIR;
- hashed identities: the PM runtime inputs and isolated-environment
identity, so existing records still match and nothing rebuilds;
- API results callers execute or compare: installed_package,
uv_launcher, verified_tools entries;
- user-facing verification messages.
Facts records already store {STORE}-templated env, so no absolute path
reaches them. Tests pin that identities are spelling-independent and that
records, composed env and uv invocations from a verbatim root carry no
prefix (both fail on e8c97320ac).
Windows cuts plain file calls at 260 characters when LongPathsEnabled
is 0, which is the default. PM's copy, walk, digest, rename and delete
calls under the store then fail or silently skip files (#130242).
Store now spells its root in the extended-length form (\\?\C:\...).
Every path built from it with entry() or / inherits that spelling, so
the file calls beneath the root no longer hit the limit. Relative
paths, and so digests, are unchanged. On POSIX the root is untouched.
native() returns the ordinary spelling for text that leaves PM's own
file calls (argv, environment values, hashed identities, records and
messages). The call sites adopt it in follow-up commits.
tirith ran `tirith check` on every terminal command, default-on with fail-open, as a
pinned PM binary auto-downloaded at startup. Measured against the shipped 0.4.2 binary
it caught 14/16 attack commands but flagged 12/24 everyday agent commands
(`echo $((6*7))`, `"$VAR/x.sh"`, `gh api | python3`, LAN curls, `.dev` hosts,
`pip install` via analysis_incomplete), and in cron/-q deny mode each of those is a
hard block. It only ever saw terminal strings (execute_code, writes and the browser
bypass it), fail-open meant no protection wherever the binary could not install
(Windows, Termux, x64 musl), and it produced a steady issue/PR stream (fail_open
semantics four times, install loops filling disks, a SIGSEGV hang, session-approval
scoping, third-party promo text on approval cards).
Removed: tools/tirith_security.py, the tirith branches of the command guard (the
is_tirith tuple, session-only "always" downgrade, import-error/fail-open paths,
unattended tirith pass), the CLI and gateway startup installers, the PM package and
lock pins, the nix runtime dep, security.tirith_* defaults, the CLI warning string
and three tirith tips (gaps refilled from the tail so translations survive), plus
the gateway's "no automated assessor" warning that only existed to point at tirith.
Config v50 drops security.tirith_* and enables nothing in its place. Hermes ships
no tirith plugin; if tirith upstream publishes one, it goes through normal catalog
admission under their name.
The agent's terminal puts a sealed payload's launcher dir first on PATH even when the inherited PATH already lists it behind another install.
The macOS removal instructions name the running .app bundle and its folder instead of a fixed Hermes.app.
A manifest.json with a mistyped runtime, commands, venv or store reads as not a payload, and a missing venv or store is treated as unset, so startup no longer raises.
Inside a packaged desktop build, bare `hermes` in the terminal tool resolved
to <payload>/venv/Scripts/hermes.exe. That venv was built on CI, and its
Windows redirector reads pyvenv.cfg `home`, which still names the CI path,
so every call failed with "did not find executable at C:\a\hermes-agent\...".
The working launcher in <payload>/bin carries the channel name and was never
on the agent's PATH.
- activate_dependencies puts the payload's own launcher dir (from the
manifest's runtime.commands) first on PATH, and on Windows no longer adds
the shipped venv's Scripts dir: every redirector in it names the CI
interpreter.
- _resolve_hermes_bin_dir answers with that launcher dir in a sealed payload,
so terminal and cron children do not depend on PATH order.
- stageDesktopLaunchers copies the canonical launchers to their qualified
names instead of renaming them. Only the qualified names are published.
- The "plugin already exists" hint names the real CLI command.
- Every manifest-relative path (venv, store, hermes launcher) goes through one
resolve-and-contain check, and a manifest.json that is unreadable or not a
payload manifest reads as "not a payload" instead of raising, since every
install now asks.
Source and dev installs are unchanged: they have no payload manifest.
plugin-catalog/mem0.yaml (#131267) points at mem0ai/mem0's own
integrations/hermes-plugin-mem0 at a reviewed pin. The entry stays dormant
while a bundled copy exists (bundled providers win in find_provider_dir), so
deleting the in-tree copy is what lets hermes_cli/memory_provider_migration.py
install it for every home whose memory.provider is "mem0". The vendor copy
keeps the provider name, mem0.json, the MEM0_* keys and the setup wizard.
Also dropped: the hermes-agent[mem0] extra, its win-arm64 platform row and
exclude-newer row, the pm/extras.py anchor (uv lock removes only mem0ai,
posthog, qdrant-client and sqlalchemy), the mem0-specific doctor check (the
generic check now names `hermes plugins install mem0`), and the tests that
imported plugins.memory.mem0. Prose that lists the bundled set now names mem0
as a catalog provider. MEM0_* env hints and redaction stay: the catalog copy
still reads them.
pm/ is bundled package-manager code outside this lane. pm.receipt.latest()
(hermes pm status) reads the same dir pm writes, so nothing is lost: the
updater embeds the pm sync section in memory (last_for_update) and its own
receipts stay under the root home.
The writers moved to the root home; the readers did not. `hermes logs update` /
`handoff`, `hermes logs list`, the debug bundle, the dashboard's durable
update-completion marker, the post-swap hand-off file and pm's sync receipts
(same update_receipts dir) now use get_default_hermes_root(), so a sticky
profile no longer hides the update log.
An install still on its legacy 3.11 venv runs its first `hermes update`
from that interpreter, which downloads uv and Python 3.14 with plain
urllib before PM's worker (and truststore) exists. Windows can cache Let's
Encrypt "Root YR" in its intermediate store without ISRG Root X1; GitHub's
asset CDN now chains through Root YR, so 3.11 rejected every download with
"unable to get issuer certificate" while PowerShell and 3.14 accepted it.
Python 3.13+ sets VERIFY_X509_PARTIAL_CHAIN by default. Set it on the PM
downloader's opener so the legacy interpreter applies the same rule;
hostname, expiry and pinned sha256 checks are unchanged.
Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com>
Follow-up to the salvaged per-package pin isolation (#125400). Once a
failing pin no longer aborts the run, the surviving pins still run the
venv sync, whose receipt finalizes `ok` as latest.json. `hermes pm
status` and the desktop sync card read latest.json, so a run that left a
package on its old pin and exited 1 was recorded as a clean success.
Write a kind=update `failed` receipt after the sync, with one failed step
per pin, whenever any pin failed (also when every pin failed, which
previously wrote no receipt at all). Trim the tests to the two
invariants and assert the receipt in the mixed case.
_apply_pins let any exception from _pin_artifacts propagate: one broken
pin - a package's resolution bug (gh's 3-part target split) or one
target's upstream pool skew (Termux rolling pool 404 while nodejs.org
is ahead) - aborted the entire run, blocked every unrelated package,
and left lockfile.save() unreached, so any failure meant zero progress.
Catch per package, pin everything that succeeds, install the pinned
set, print a 'pinned X, failed Y' summary, and exit non-zero when
anything failed. A run where every pin fails still leaves the lockfile
untouched.
Fixes#125386
flatten_single_dir renames a tree extracted seconds earlier, while
Defender or another scanner still holds it, and ffmpeg/agent-browser
installs died on the package/bin rename with [WinError 5] (#131884).
The entry displace/restore renames in install.py had the same bare call.
pm.filesystem.retry_held retries EACCES/EPERM/EBUSY for about 3s and
raises anything else immediately; the hoist, merge_tree and the entry
swaps use it.
Co-authored-by: bighamx <10262931+bighamx@users.noreply.github.com>
Co-authored-by: lzmuhioin <1814826+lzmuhioin@users.noreply.github.com>
The root file is project context and was 38.7k chars, past the context-file budget
(20k floor, 30,720 on a 128k window), so every session lost its middle: routing table,
testing rules, code-shape rules. It now keeps the two invariants, a condensed rubric
(every rule kept), environment, structure, one-line rules that apply everywhere and the
routing table (11.8k).
Moved verbatim to where they load: Testing -> new tests/AGENTS.md; dependency pinning and
PM ownership -> new pm/AGENTS.md; machine facts -> new hermes_platform/AGENTS.md;
session-scoped tool availability -> tools/AGENTS.md; the full rubric with its real-close
examples -> developer-guide/contributing.md. Pointers in lint, checker messages, docs
and test comments name the new homes.
Moves llamacpp-cpu, -cuda, -vulkan, -hip and -metal from b11146 to
b11370 (bed0a8566). Only digests change: every pinned asset keeps its
name, no command-line option or server route Hermes uses went away, and
the Windows cudart zips are byte-identical, so they keep their digests.
Installed b11146 engines read as outdated and offer the update.
The Linux builds keep the same glibc floors (2.34 for x64 CPU/Vulkan,
2.38 elsewhere), so the bundled jammy libgomp still covers them.
The libgomp .deb pinned for Linux engines came from Ubuntu 24.04 and
needs glibc 2.38. llama-server's $ORIGIN RUNPATH loads that copy ahead
of the host's, and the dynamic loader stops at the missing symbol
version, so the x64 CPU and Vulkan engines (glibc 2.34, built on 22.04)
failed PM's --version verify on Ubuntu 22.04, Debian 12, RHEL 9 and
Amazon Linux 2023.
Ubuntu 22.04's libgomp1 (gcc-12 12-20220319-1ubuntu1) needs glibc 2.34
and provides GOMP_4.5, the newest version any b11146 engine references.
The arm64, CUDA x64 and ROCm x64 builds already need glibc 2.38 from
upstream.
Verified through pm.install.ensure() in containers: the old pin fails
verification on Ubuntu 22.04; the new pin installs and runs on Ubuntu
22.04/24.04/26.04, Debian 12/13/testing and Fedora (x64, CPU and
Vulkan) and on Ubuntu 26.04 arm64, loading libgomp from the engine
directory.
run_completion runs in the OLD process after the checkout swap, and its
contract is never to import application helpers from the replacement tree.
Importing pm.libatomic there crashed the update when the new tree lacked
the module (downgrade or older pin) and on architectures where
current_target() raises. Run the pre-install as its own process from the
new tree, ignore its outcome, and let the in-lock repair report.
Also: gate on an openable /dev/tty first (cheapest check, and the detached
child exits immediately), treat an unsupported architecture as "skip",
bound the sudo -n probe, and drop the timeout on the interactive install
so a slow prompt or mirror is never killed mid-dpkg (it runs outside PM's
lock). Found in the final-head review of #132664.
hermes update's completion runs PM in a start_new_session child with no
controlling terminal, so the in-child sudo -v warm-up could never read a
password and the cold password-sudo update from #124926 still failed. A
cached ticket was also the wrong currency: under sudo's default per-tty
timestamps a ticket minted in one session is not visible to another.
Install the package itself, interactively, at the last point that still
owns the terminal: run_completion() before it detaches, plus the existing
ensure_tools_for_sync() entry for the other update paths. Gate on an
openable /dev/tty instead of isatty() (a detached child inherits a tty
stdin and a forwarded-isatty stdout). Inside PM's lock the repair is
unchanged: sudo -n, stdin closed. Reported by @JoaoMarcos44 on #132664.
The update warm-up resolved sudo on its own, so on a glibc host with no
supported package manager it still asked for a password the repair then
never used. Derive the prompt from the same _host_install_command and
_command_plan the repair runs, compute the target once, and correct the
module docstring now that there are two entry points.
The libatomic repair can only use sudo -n under PM's install lock, and
hermes update enters PM directly without install.sh's prerequisites
warm-up, so a cold password-sudo host still failed the update from
#124926. Before publishing tools, ask once on an interactive glibc host
that still lacks libatomic.so.1 and whose sudo would prompt. Reported by
@JoaoMarcos44 on #132664.
The hook glued the remedy into the reason, so the error ended with two
fixes, the second ("retry, or run hermes pm doctor") wrong for a missing
host library. Return (reason, remedy) and pass the remedy through. When the
package installs but Node still cannot load it, say that instead of telling
the user to run the command that just succeeded.
Every root that depends on node (npm, browser tools) re-stages it after a
failure, so a host without libatomic ran the package manager under PM's
install lock once per dependent root. Cache the outcome for the process.
Also use platform.freedesktop_os_release() instead of a hand parser, and
run apt-get update once when a minimal image has empty package lists.
facts.json under a candidate root only says that root synced dependencies
for the checkout, and a borrowing launch writes that too. Ranking the root
the checkout sits in first fixed a default-root borrower of a custom-root
checkout but not the reverse: a checkout the default root keeps in
`install.sh --dir <dir>`, launched with HERMES_HOME=<dir>, made <dir> the
owner after its first sync, so the next launch rebound the shared launchers
to <dir>'s interpreter.
Only the owner publishes the checkout's launcher, and it execs the owner's
store Python. Among the roots holding state, the one whose tools/ holds that
live interpreter is the owner. Without a live launcher, the parent root
still outranks the platform default.
Co-authored-by: FroRaut <79215537+Froraut@users.noreply.github.com>
