- A selection that resolves to no entries (e.g. ":(exclude)*") is refused
before the pre-rollback snapshot. Checkout with an empty pathspec file
switches the store's HEAD instead of restoring nothing.
- Remove the nested repository with utils.rmtree_readonly so the
deleted-repo test runs on Git for Windows (read-only loose objects).
- Check out the entries resolved during inspection as literal paths,
instead of re-evaluating the pathspec after the pre-rollback snapshot
restages the index (attribute pathspecs could select an uncaptured
gitlink the inspection never saw).
- Put a multiline reason on one line before appending the nested-repo
note; listings read %s, which stops at the first blank line.
- Gate host-specific test rows with platforms() and decode raw-byte
filenames inside the test, so the file collects on Windows, passes on
macOS and is selected by the OS lanes.
A nested repository is stored as a gitlink, so its files are not in the
checkpoint. Say so when the checkpoint is taken: its message gains
"[nested git repos not captured: <paths>]", which every /rollback listing
(CLI, gateway, TUI, Desktop) already shows. The gitlink parse is shared
with the rollback guard.
rollback.list sent the TUI an always-empty message (it read a "message"
key the manager calls "reason"), so TUI and Desktop listings never showed
what a checkpoint was; map it from reason.
Co-authored-by: FroRaut <79215537+Froraut@users.noreply.github.com>
The scoped restore guard resolved the requested path through the live
filesystem, so a captured file later replaced by a symlink into a nested
repository was refused even though the checkpoint holds its bytes.
Normalize the selection lexically; resolved-path confinement stays in
_validate_file_path. A recorded gitlink replaced by an unrelated symlink
is still refused.
Resolve selective restores against a disposable index before any checkpoint mutation, preserving unrelated-file recovery and refusing uncaptured gitlinks. Keep NUL-delimited filenames lossless, including surrogateescaped POSIX bytes. Add real-Git regression coverage and document the capture limitation.
Fixes#127073
Salvages #127077
Co-authored-by: Yuan Li <dskwelmcy@163.com>
Move retention, orphan pruning, status and clear operations into a topical sibling and route callers and tests directly to their defining module. Keep the shared store paths and git execution in checkpoint_manager; shorten the local child-env WHAT docstring.
shutil.rmtree stops at the first entry it cannot unlink, and Git leaves loose object files read-only on Windows (r--r--r-- trees on POSIX package installs behave the same). Checkpoint clear, MCP catalog uninstall/reinstall and git-installed plugin removal all deleted clones with a bare rmtree, so each aborted mid-tree and left partial state.
Add utils.rmtree_readonly: clear the write bit on the failing entry and its parent, retry that one operation, and keep rmtree semantics for every other failure (both the 3.11 onerror and 3.12+ onexc callback shapes).
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).
Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.
uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
A git gc killed by _run_git's timeout strands tmp_pack_* files in the bare
store's objects/pack; gc.auto=0 means git itself never reclaims them (~16 GB
observed on one host). clear_stale_tmp_packs() already sweeps this debris
class for the update checkout/worktree paths — teach it to resolve a bare
repo's objects/pack (no .git/ layer) and call it from prune_checkpoints(),
unconditionally under _store_has_head so the daily auto-prune and
'hermes checkpoints prune' both reap it even when no ref moved (a sweep is a
cheap directory listing, unlike the pack-rewriting gc that stays gated on
refs having moved).
checkpoint_manager_profile_rename.py (merged from main) reaches for
_store_has_head/_ref_tip/_list_project_refs/_unlink_quiet, which main
extracted from checkpoint_manager after this branch diverged; a profile
rename with checkpoints raised AttributeError. Add the same helpers here.
Also: tests follow this branch's contracts (model_metadata_http.get instead
of a requests module, no agent.ssl_guard, backup excludes PM trees too,
_safe_restore_plan), and run_tests.sh forwards SSL_CERT_FILE/DIR so network
tests verify TLS with the pinned interpreter on hosts without a compiled-in
bundle path.
Port of main's 804707bea6 onto the Pruner. On the checkpoint-take path a count
trim or a size round rewrites the ref and marks the store .gc-pending; the full
repack (tens of seconds on a GB store — it was holding write_file for the whole
gc) runs only in prune_checkpoints, which reclaims when it deleted something or
the marker is set. Over the cap a checkpoint drops ONE round of oldest snapshots
per project; the 20-round loop with gc per round stays in the periodic prune,
where the measurement actually moves.
The two size-cap tests pin the deferred contract now: no gc call inside
ensure_checkpoint, marker present, store under cap after the prune. A failed
rewrite still aborts the take-path pruning before the size round with history
intact. clear_legacy counts the deletion failures it already reported.
`container_backend_for_task` already catches everything inside `_terminal_env_type_for_task`
and `_uses_container_paths` and returns defaults, and every CheckpointManager has
`unsupported_backend_reason`: call the method directly (no getattr fallback), drop the
try/except in `unsupported_backend_reason` and the suppress() around the import+call in
turn_explainers (the one around record_agent_write stays). The rollback.restore fakes in the
server tests gain the method they now must provide.
Review follow-up (#113530): the manager no longer records the first container
backend, so a session whose terminal backend changes is answered by the backend
configured now, not by the first one seen. The checkpoint hooks simply skip
container-backed tasks; unsupported_backend_reason() classifies at call time.
The docs state what /rollback and the rollback.* RPCs do for container sessions.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 2dc4c0d2b17d0f478118eca55742001eeed8bdb0)
With a container terminal backend (docker, singularity, modal, daytona,
vercel_sandbox, container plugins) file-tool paths keep container semantics,
but the checkpoint hook handed them to the host-side CheckpointManager: a path
that does not exist on the host produced a useless snapshot attempt, one that
happens to exist on the host snapshotted the wrong tree, the destructive
terminal branch did the same with the container cwd, and the post-write ledger
hashed the container path on the host so safe restore could trust unrelated
host content. Every failure was swallowed, so a docker user saw "No checkpoints
found for /home/admin" with nothing behind it.
Classify the task's backend the way the file tools do (_uses_container_paths)
and, for container-backed tasks, take no checkpoint and record no ledger entry;
/rollback prints the reason and refuses diff and restore for that session (a
host checkpoint that predates it belongs to another tree), and the
rollback.restore RPC returns the same reason as a failed restore. The refusal
classifies the session's configured backend directly (in the gateway under the
session's own identity and profile scope, as a turn binds them), so it holds
before the first mutation of the session. Local and ssh backends are untouched. This stops the
false protection; it does not add rollback support for containers (translating
bind mounts is a separate contract).
Tests: eight cases in tests/agent/test_tool_executor_checkpoint_paths.py through
the production classifier (a fake docker environment registered for the task, or
the configured backend): missing host path, colliding host tree (POSIX),
destructive terminal command, post-write ledger on a real host file, /rollback
and rollback.restore refusal in a fresh session, the local session still
restoring, and unchanged local behavior. Five fail on main on Windows, where the
collision case is skipped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 97a5709e4980c8f85a5a640e6e5e11fe8f94affa)
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
record_agent_write and safe_restore_plan each spelled out
_project_hash(get_working_dir_for_path(...)); the bug being fixed was those two
sites disagreeing on the key. Name it once (_ledger_key) and have the reader
use only that key: the writer has keyed by the marker walk since the ledger
was introduced (bd4b709258), so an exact-dir ledger only exists where the two
keys coincide.
record_agent_write keys the agent-write ledger by the marker-walk result of
get_working_dir_for_path(), while safe restore reads the ledger by the exact
hash of the directory named in the restore. The two keys diverge whenever
the walk lands somewhere other than that directory:
- a markerless project dir under an ancestor that carries a generic project
marker (e.g. /tmp with a stray package.json) -> the ledger is filed under
the ancestor's hash and restore reads an empty ledger;
- restoring a repo subdir while the write was recorded at the repo root.
An empty ledger makes restore() silently degrade to a full restore, which
overwrites files the user hand-edited after Hermes' last write — the exact
outcome safe restore exists to prevent.
safe_restore_plan now falls back to the walked key when the exact-key ledger
is empty, so the recorded protection is honored regardless of where the walk
landed. Verified: 9 previously failing TestSafeRestore tests now pass, plus
a new regression test that seeds an ancestor marker and asserts the user
hand-edit survives the restore.
Checkpoints were enabled by default from 9e845a6e (2026-03-16) until #20709
(2026-05-06) flipped the default back to off; the migration of that window
wrote `checkpoints.enabled: true` into user configs, where a later default
flip cannot reach it. Users who never type /rollback have carried a GB-scale
`~/.hermes/checkpoints/store` since (one live install: 1.2 GB across 250
projects, mostly disposable worktrees and /tmp dirs), and the cap cannot
bring it down because every project keeps at least one snapshot.
Silently flipping the key back is indistinguishable from overriding a real
opt-in, so this surfaces it instead: `checkpoint_footprint_notice()` returns
one line when checkpoints are enabled AND the store is at or above
`max_total_size_mb`, naming the opt-out (`hermes config set
checkpoints.enabled false` + `hermes checkpoints clear`) and the
retention knob. `hermes update` prints it with the post-update notices;
`hermes doctor` reports it as a warning after the state.db check.
Symptom: `hermes update` sat for ~40s after "Refreshing cua-driver" and ended
with "Fleet version check returned no rows" (exit 1); the restarted gateway
took 26s to reach "Starting Hermes Gateway" instead of the usual 3s. The
gateway constructor was running `maybe_auto_prune_checkpoints` synchronously,
before the control socket, adapters and the code_sha stamp, and on a 1.2 GB
store its `git gc --prune=now` (a full repack) takes 20-28s — twice, because
the size-cap shrink gc'd again even when it could drop nothing.
The same defect sat on the tool-call path: `CheckpointManager._take` ran
`_enforce_size_cap`, whose `_shrink_store_to_cap` returned True without
dropping anything and triggered a 20-28s gc on the first file-mutating tool
call of every turn once the store was over the cap. That loop also re-measured
a pack size that cannot move without a gc, so a single over-cap checkpoint
dropped 20 rounds of history and flattened every project to one snapshot.
- `_take` never gcs: `_prune` and `_enforce_size_cap` rewrite refs (cheap),
drop at most one snapshot round, and mark the store `.gc-pending`.
- `prune_checkpoints` gcs only when a ref moved (project deleted, or the
pending marker), and its cap loop is drop -> gc -> re-measure.
- `maybe_auto_prune_checkpoints` claims the interval marker before the run
so a failing prune costs one day, not a gc per housekeeping tick.
- `auto_prune_from_config` is the one config-driven entry point; the gateway
calls it from the housekeeping tick (last chore), the CLI from a daemon
thread. Nothing on either startup path waits for git.
Live A/B on a copy of a real 1.2 GB / 224-ref store: checkpoint 20.5s ->
1.2-1.6s (0 inline gc); the single repack (19.6s) now runs in the prune.
Delete consumer-side Git discovery caches and layout reconstruction. Share the selected environment while preserving PM acquisition policy, system Git fallback and downstream config isolation. Route Matrix binding through its existing dependency operation and explicit setup through sync_venv.
Real Git regression tests cover absent ambient PATH, changed selection, unsupported target and failed acquisition. Independent final review: 100 focused tests pass, including malicious Git config and Matrix setup contracts. Native Windows execution was not run.
tools/checkpoint_manager.py's CHECKPOINT_BASE and gateway/sticker_cache.py's
CACHE_PATH are resolved once at import time via get_hermes_home(), which is
a context-local ContextVar under the multiplexed gateway (multiple profiles
sharing one process). Freezing the path at import time pins every later
checkpoint/cache read-write to whichever profile's HERMES_HOME was active
when the module was first imported -- the same bug class already fixed for
cache dirs, skills_hub, rich_sent_store, and (this session) the OAuth/auth.json/
sessions.json paths.
CheckpointManager is "owned by AIAgent" per-instance, but its methods read
the frozen module constant directly instead of taking the store root from
the instance, so a profile's CheckpointManager can read/write code-edit
checkpoints into a different profile's store.
Add a per-call resolver for each path, following the established "respect
an existing test monkeypatch of the constant, otherwise re-resolve through
get_hermes_home()" pattern so the extensive existing test seams in
tests/tools/test_checkpoint_manager.py and tests/gateway/test_sticker_cache.py
keep working unmodified.
(cherry picked from commit 03ae075d969094cb584e6ab38d2a773d15ff875c)
(cherry picked from commit b850c4b18e2ae2158a97c6cb87bd2057918b8170)
Keep downloads bound to their remote representation and publish through
atomic destination-local staging. Serialize shared partial ownership.
Keep explicit CA trust scoped to provider probes. Preserve checkpoint
history and edited files, validate all profile inputs before dependency
publication, and separate data removal from installed runtime ownership.
Exclude machine-specific PM state from portable transfers. Keep plugin
files and nested skill tools intact. Preserve native test isolation.
Focused native Windows receipts cover the individual repairs and their
integration. This commit does not claim a full-suite or release build.
Merge upstream b1f003e186 while preserving PM runtime ownership and
Python 3.14 worker startup, Windows signing, and macOS wait recovery.
Keep retired runtime modules deleted. Port upstream updater preflight
checks into the checkout strategy and preserve live build logging.
Carry checkpoint filename handling and process recovery into the current
module layout. Regenerate locks and adapt incoming platform test markers.
Focused Python and JavaScript tests, desktop and root-test typechecks,
conflict-path lint checks, lock validation, and retired-import checks pass.
The full test suite and packaged release builds were not run.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
The scope-normalization sweep only fixed the read paths
(list_providers/get_provider/registry_generation) in most registries;
the write paths (register/snapshot/restore) still stored under the raw
scope string while reads resolved through hermes_home_key — so a
scoped registration was written under one key and read under another
(agent/terminal_env_registry.py test_scoped_registration_isolated
failed on both Linux and Windows CI).
- add hermes_constants.normalize_scope(scope): hermes_home_key for
non-None, None preserved for the process-global layer — one named
function instead of the expression repeated at ~20 call sites
- apply it on both sides of all nine scoped registries
(browser, terminal_env, image_gen, transcription, tts, video_gen,
web_search, secret_sources, dashboard_auth, tools.registry)
- agent/deadline: MAX_SAFE_TIMEOUT_S on win32 leaves 60s headroom under
the DWORD-ms ceiling so margin-adding consumers (human-wait +60s)
stay platform-safe; tools/approval fail-closed fallback mirrors it
- tools/approval: strip shell escapes everywhere except inside
drive-anchored paths (the Windows shell is bash, so r\m still spells
rm; only C:\... backslashes must survive deobfuscation)
- consolidate the duplicated _rmtree_force into hermes_cli/fs_utils
and repoint both callers
- hermes_cli/kanban + disk-cleanup: shlex.split(posix=False) keeps
wrapping quotes in tokens; strip them before dispatch/matching
- test gating: SIGKILL durability conformance cells are linux_only
(TerminateProcess is not SIGKILL), macOS TCC anchor symlink-tree
classes are linux_only, Linux chromium profile tests resolve the
fixture home via expanduser patch + posix-joined expecteds so they
run identically on every host
Gate the POSIX-only and symlink-only tests with the linux_only and
require_symlinks markers. Fix the real cross-platform bugs:
- file_operations: use the translate_path flag, send snippets as base64, and
use sys.executable (the MS Store python3 stub and the list2cmdline
backslash collapse both broke snippets)
- approval: treat backslash as a Windows path separator, not an escape
- registry, browser_registry, secret_sources, plugins: normalize scope-key case
- checkpoint_manager, plugins_cmd: clear read-only bits before delete
- deadline: make MAX_SAFE_TIMEOUT_S fit the Windows limit
- image_routing, acp, cua_backend, daytona: fix Windows and POSIX paths
- hermes_state: match backslash in the retag LIKE clause
- kanban, disk-cleanup: match drive paths and split command arguments
- scripts: emit host separators through as_posix
207 test files are gated or isolated.
Folds review findings: surface failed_deletes in CLI and gateway
/rollback output (new gateway.rollback.failed_deletes locale key, 17
locales), emit skipped_oversize on the nothing-to-restore early return
too, document all three report keys in the restore() docstring, and pin
the failed_deletes contract from both sides in tests.
Follow-up to #95491. The restore result dict had two inconsistent
reporting surfaces: skipped_oversize was only present when non-empty
(unlike skipped_user_edits), and failed_deletes was filtered from
restored_files but never surfaced to the user at all (debug-level log
only). Both are the same silent-omission class #95491 fixed for
oversize files; this completes the cleanup.
Follow-up to the salvaged #95207 fix, completing the misreport bug class:
- restore() now also drops delete_targets whose unlink failed (OSError
swallowed) from restored_files — the sibling of the kept-oversize
misreport the salvaged fix closed.
- /rollback output in the CLI (cli_commands_mixin) and gateway
(slash_commands + gateway.rollback.kept_oversize locale key in all 17
catalogs) now tells the user which files were kept because the size
cap excluded them from every checkpoint; previously the file was
correctly preserved but the user got no notice it was not reverted.
- Regression test for the failed-unlink misreport.
Review feedback on #95207: `_exceeds_size_cap` and `_drop_oversize_from_index`
each computed the byte cap and compared against it. Both used `> cap`, so they
agreed, but only by coincidence of two independent expressions — nothing held
them together.
The coupling is the whole point of the fix. The checkpoint decides what to
store and safe restore decides what may be deleted; a threshold that drifted
between them would produce a file both absent from the checkpoint and not
recognised as capped at restore, which is exactly the deletion this branch
exists to prevent. `_drop_oversize_from_index` now calls the predicate.
Added a boundary case to TestSafeRestore that pins the round trip from both
ends: a file at exactly the cap is stored, so it must revert; one byte more is
excluded, so it must be kept. Mutation-checked — moving either side to `>=`
fails it, including the re-inlined-with-`>=` shape the reviewer described.
No behaviour change: the byte cap, the strict comparison and the
unstattable-path result are all as before.
Regression: the 10 test files covering checkpoint_manager / rollback, against
current main (1fe0f2f3a, 134 commits newer than the base measured on the first
commit) — 130 passed on main, 135 here (+5 new), zero failures either side.
`/rollback <N>` runs `restore(..., safe=True)` — safe mode is the default,
`--all` opts out. Safe mode splits the changed files into two groups: those
present in the checkpoint are checked out, and those absent from it are treated
as files Hermes created during the turn and deleted, since deleting them is
what restores the pre-turn state.
Absence from the checkpoint is not proof of authorship. `max_file_size_mb`
(default 10) keeps large files out of every checkpoint via
`_drop_oversize_from_index`, so a file the agent appended to — a dataset, a
corpus, an export, a log — is absent for a completely different reason. Safe
mode deleted it. No checkpoint held a copy, so nothing could bring it back, and
`restored_files` listed the path, so the user was told it had been restored.
Reproduced on main with shipped defaults:
corpus.jsonl (2 MB), agent appends to it, then /rollback 1
safe_restore_plan restore=['corpus.jsonl', 'notes.py']
restore ok=True restored_files=['corpus.jsonl', 'notes.py']
notes.py exists=True content="v1 = 'original source'"
corpus.jsonl exists=False <- deleted, was in no checkpoint
Scope: this needs an agent write to the capped file. A large file Hermes never
touched is not in the ledger, lands in `skipped`, and was already safe.
The delete branch now asks whether the path is one the cap would have excluded,
using the same test `_drop_oversize_from_index` applies when building the
checkpoint, so "kept out of the checkpoint" and "refused deletion at restore"
share one definition. Such a path is reported under a new `skipped_oversize`
key and dropped from `restored_files`.
The classification keys on "absent from the checkpoint", not on "large now".
A file small enough to be checkpointed and later bloated past the cap does have
a stored version, and reverting to it is exactly what was asked for — it still
restores, and a test pins that.
The ledger records a content hash, not whether a write created or modified the
file, so an oversize path cannot be proven agent-created. Leaving one behind
costs a stale file the user can delete; removing it costs the file.
Tests: 4 cases in tests/tools/test_checkpoint_manager.py::TestSafeRestore. Two
fail on main — the deletion and the misreport. Two are guards: the
grew-past-the-cap revert, and the small agent-created file that must still be
removed.
Regression: the 10 test files covering checkpoint_manager / rollback —
130 passed on main, 134 with this change (+4 new), zero failures either side.
Copilot CLI 1.0.78 reworked /rewind to restore only the files the agent
changed, 'skipping any file whose contents no longer match what Copilot
last wrote'. This ports that protection to Hermes checkpoints:
- tools/checkpoint_manager.py: per-project agent-write ledger
(sha256 of every landed write_file/patch), safe_restore_plan()
classifier, and restore(safe=True) that reverts only agent-authored
changes, deletes agent-created files, and preserves user hand-edits.
Empty ledger (pre-existing stores) falls back to the classic full
restore.
- run_agent.py: feed the ledger from _record_file_mutation_result on
every landed mutation (zero new hooks; rides the existing verifier).
- CLI + gateway /rollback: safe mode is the default; --all/--force
restores everything; skipped files are reported with a hint.
- 17 locales: new gateway.rollback.kept_user_edits key.
- Docs: checkpoints-and-rollback.md updated.
- Tests: 7 new cases incl. user-edit preservation, post-agent user
tweaks, agent-created file removal, empty-ledger fallback.
