Reverts the nine commits merged by #132260 (merge 05ab8766e7), from
31194ea00e "fix(cli): revalidate worktree cleanup gates before removal"
through 05ab8766e7, restoring hermes_cli/worktree_gc.py,
hermes_cli/worktree_ops.py and tests/security/test_gitspawn_config_injection.py
to their pre-merge state and removing
tests/hermes_cli/test_worktree_reclaim_revalidation.py.
Maintainer decision: back the behaviour change out (trees kept when their
state changed since the audit, plain `git worktree remove` under the tree's
git env, dirty submodule trees kept). No commit on main touched these files
after the merge, so this is the exact inverse of the merged diff.
The contributor mapping for @rudycelekli stays (attribution record only).
Gate findings (Medium efficiency + Low reuse, convergent): the startup
sweep revalidated each reap tree with a fresh _classify_prune_candidates
call. Each call reloaded (and could rewrite) the on-disk merge cache and
built its own ls-remote memo, so every pushed-tier tree paid another
network ls-remote. A 3-tree sweep made 4 loads and 4 ls-remotes; it now
makes 1 of each. The attended reclaim seeded an empty cache and re-ran
`git cherry` for verdicts the audit had just computed.
_classify_prune_candidates takes keyword merge_cache / get_remote_heads.
When the caller supplies them it uses them and leaves the save to the
caller. _prune_stale_worktrees loads once, threads both through
_reap_prune_verdicts, and saves once. reclaim_worktrees seeds from
_load_worktree_merge_cache(). Cache keys include both SHAs, so post-audit
commits still miss. Revalidation still runs per tree, right before
removal. The dead `if not refreshed` guard goes away: a one-item list
always yields one verdict.
Gate finding (Medium, 2c + quality convergent): plain `git worktree
remove` always refuses a tree with a checked-out submodule ("working
trees containing submodules cannot be moved or removed"). Once --force
was dropped, clean merged trees like that got a reap verdict on every
startup and every `hermes worktree prune`, and were never removed.
Both paths still try the plain remove first. Only when its stderr names
submodules and an immediate re-check finds the tree still clean
(_worktree_is_dirty False on startup; _dirty_split == (False, []) in the
attended path) do they retry with --force, under the same safe env.
Adds a submodule row to the existing parametrized startup test.
Gate finding (High, 2ab): dropping --force from the startup reap's
`git worktree remove` makes git read the tree's index, which executes the
repository's core.fsmonitor (GHSA-7x36-8jrh-v4pw) on the unattended
startup sweep. tests/security/test_gitspawn_config_injection.py went red.
Route the remove through noninteractive_repo_git_env(entry) +
stdin=DEVNULL, the same pattern as _worktree_is_dirty; when the env
cannot be built, keep the tree. Extend the security test to also drive
the attended reclaim_worktrees remove (worktree_gc._git already uses the
safe env) so both plain-remove paths are pinned.
The startup pruner now runs `git worktree remove` without --force so a
tree that gained work after classification is refused. But git treats
our own .worktreeinclude directory symlinks (node_modules, .venv) as
untracked content, so every include-symlink-only tree that the
classifier already reaps was refused and kept forever.
Unlink exactly the paths _include_symlink_paths() recognises (the same
classification _worktree_is_dirty uses) right before the remove; any
other untracked file still blocks the removal.
The reclaim sweep (_reap_prune_verdicts), the orphaned-branch pass
(_prune_orphaned_branches) and the cleanup after a failed worktree add
(_cleanup_failed_worktree_add) ran `git branch -D` with the inherited
environment. A ref update runs the repository's reference-transaction hook
through core.hooksPath, so the unattended sweep executed whatever the
checkout's own config named. The three calls now use the same hardened env
as the other worktree_ops calls, with stdin closed.
(cherry picked from commit 31eec2f67e47a39c1b715f55d4fa649208076473)
The reclaim sweep's ls-remote, the shallow-repo fetch --unshallow and both worktree prune calls ran with the inherited environment, so a repository-level core.sshCommand or credential.helper ran unattended. They now use the same pinned env as the base fetch.
(cherry picked from commit a0d8e2a592790a90f6b666445017ea73f88cbc63)
finalize_subagent_worktree's status probe re-hashes files the child touched, so it ran a
repository-named clean filter after every delegated subagent. worktree_ops._worktree_is_dirty, the
dirty probe kanban teardown and the stale-worktree sweep run unattended, called git status with no
hardened env at all: core.fsmonitor ran as well. worktree_ops._worktree_add (hermes -w) had the
fixed pins only, so its checkout ran a named smudge filter.
All three now use noninteractive_repo_git_env. When filter discovery fails, finalization keeps the
worktree as unproven, the dirty probe reports dirty (the reclaimers keep the tree), and hermes -w
refuses to create the worktree. worktree remove --force at prune runs no filter once the status
probe is neutralized.
(cherry picked from commit e727281137a495602457fe4a86b51212af2fdbb7)
Every caller (cli._cleanup_worktree, _prune_stale_worktrees, kanban
teardown) already passes the main checkout, so the git-discovery fallback
(_main_checkout_root) was dead code that could silently disagree with the
caller's root. Make repo_root a required positional and drop it.
Also normalise .worktreeinclude entries with Path(...).as_posix() so an
entry written with backslashes on Windows still matches git status
output, which always uses "/".
_include_symlink_paths took the main checkout as the parent of
--git-common-dir. For a submodule (common dir .git/modules/<name>) or a
--separate-git-dir repo that is not the checkout, so no .worktreeinclude
symlink matched and every include-only worktree was kept forever as
"dirty" — a regression against BASE, which removed it.
_worktree_is_dirty / _include_symlink_paths now take repo_root, and the
CLI exit cleanup, the startup pruner and kanban teardown pass the one
they already hold. When it is None, fall back to the first
`git worktree list --porcelain` entry resolved through
`rev-parse --show-toplevel` (a submodule lists its git dir there).
Also share the .worktreeinclude parsing between the copy step and the
symlink check (_worktreeinclude_entries), compare with rstrip('/') only,
drop the backslash rewrite, and drop the redundant inner try/except —
_worktree_is_dirty's outer handler already fails safe to dirty.
_copy_worktree_includes symlinks included directories (node_modules,
.venv) into the worktree. A trailing-slash gitignore pattern never
matches a symlink, so `git status --porcelain` lists each one as `??`
and _worktree_is_dirty reported the worktree dirty forever. With exit
cleanup now honouring dirtiness, every session in such a repo kept its
worktree; the startup pruner and kanban teardown had the same bug.
Ignore an untracked entry only when it is a symlink at a path listed in
the main repo's .worktreeinclude that resolves to that same entry in the
main checkout. Any other change, a non-zero git exit, or an error still
counts as dirty (fail-safe). Fixes all three callers at once.
- _deepen_shallow_repo marks packs in its finally without a git config probe
that could itself raise and turn a finished unshallow into False; markers
are inert in a clone without a promisor remote.
- Installers warn instead of aborting the stage when a marker cannot be
written, matching the Python helper.
- Docs: the manual commands follow HERMES_HOME, and the missing-object
backfill feeds IDs through `fetch --stdin` so a large list cannot exceed
the Windows command-line limit.
- Test and docstring comments describe the contract, not the incident.
git 2.53+ `index-pack --promisor` hands every object outside the promisor
packs to pack-objects, which BUG()s (should_include_obj) on the missing
objects those objects lead to. A partial clone with unmarked packs therefore
keeps crashing its fetches.
The retry added for this ran `git -c remote.origin.promisor= fetch`. That
override does nothing: git registers promisor remotes additively, so the
repo's own `true` still wins (GIT_TRACE on 2.50 and 2.55 shows
`index-pack --promisor` and `filter tree:0` under the override). The retry
only passed when the crashed fetch had already stored the pack; a retry that
has to transfer again crashed again.
- On the crash, give every pack without a `.promisor` marker one, then retry
the same fetch once. The clone stays partial, no refetch, no config change.
If git still crashes, the updater points at the docs section.
- The two fetches that turn a full clone partial now mark the clone's old
packs afterwards: the depth-1 `tree:0` unshallow in
`fetch_full_commit_graph`, and `worktree_ops._deepen_shallow_repo`
(`--filter=blob:none`). git writes the partial-clone config before it
fetches, so this also runs when the fetch fails. Without it, a converted
depth-1 install crashed every later fetch on git 2.55 (3 of 3 rounds in a
local repro).
`_resolve_worktree_base` fetched by name, which on a tag-pinned narrow
clone succeeds without creating origin/<branch>. It then reported
"origin/main (fetched)" for a ref that did not exist. Fetch with the
updater's tracking refspec so the ref it names is the remote tip.
Conflicts resolved toward the branch: PM owns dependency preparation, the
Windows shim re-exec/hand-off path stays retired (main's shim-parent wait,
gateway-resume env token and update_cmd_deps tests dropped), docs describe
the PM update flow. The docker workflow parks install-stamp.json around the
toolchain step instead of deleting it so tests/docker can compare provenance.
`LSPService` kept one client per `(server_id, workspace_root)` for the life
of the process. In a long-running gateway that outlives its coding sessions
the client for a removed worktree stayed registered with its stdio pipes
held open — tsserver heaps of several GiB pointed at trees that no longer
existed (#102345). The idle reaper (d7578018c5) does not cover this: a
client whose root vanished is not idle from the server's point of view.
- `LSPService.release_workspace(path)`: detaches every client whose folders
live under `path` under `_state_lock`, waits for in-flight spawns so a
concurrent `_get_or_spawn` cannot reinsert a client after release, prunes
the delta baselines and broken-set entries beneath the path, and shuts the
clients down on the service loop. Multi-root servers (pyright) only drop
the folder (`LSPClient.remove_workspace_folder`) so siblings keep their
shared process. Idempotent; best-effort; returns the count.
- The reaper sweep now also uses that primitive for clients whose every
workspace folder no longer exists (externally deleted roots).
- `agent.lsp.release_workspace()` reaches every started service without
creating one; `hermes_cli.worktree_ops.release_lsp_clients()` is called
from both worktree-removal paths (`cli._cleanup_worktree`, kanban
`_cleanup_worktree_workspace`) BEFORE `git worktree remove`.
Salvages the direction of #102381 (@Sahilvishnaliya, release_workspace +
cli hook) and the deleted-root eligibility of #95047 (@israellot); both
matched on `key[1]`, which is `""` for multi-root servers and would have
reaped pyright on every sweep — the primitive here matches on
`client.workspace_folders` instead.
Co-authored-by: Sahil Vishnalya <222165401+Sahilvishnaliya@users.noreply.github.com>
Co-authored-by: Israel Lot <840042+israellot@users.noreply.github.com>
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
Every `hermes -w` launch on a clone past the pack-sprawl threshold started
its own `git repack -a -d` of the whole object store in a daemon thread.
On a multi-agent box that meant dozens of concurrent multi-GB repacks of
the same repo, each too starved (nice 19, under the others) to finish
inside the 1800 s timeout. subprocess.run's timeout killed only `git
repack`, so the `pack-objects` grandchild kept running with ppid 1 for
days; the CLI exiting orphaned it the same way. Observed: 51 pack-objects
processes, load 200 on 20 cores, 143 GB swap, 29 GB of `.tmp-*-pack`
debris, and every `hermes` invocation taking 5-7 s of wall clock for
0.6 s of CPU.
- `_claim_repack_slot`: one repack per clone per 6 h across processes
(`.git/hermes-repack.lock`, mtime = stamp; stale takeover via `replace`
so only one of N racers wins).
- `_run_bounded_repack`: own process group + `kill_process_tree` on
timeout and at exit, so the whole tree dies with the launcher.
- incremental `git repack -d --geometric=2 --write-midx` instead of a
full `-a` rewrite: consolidates sprawl in seconds instead of rewriting
10 GB per run.
- `.tmp-<pid>-pack*` (pack-objects debris) joins gitlock's stale tmp-pack
sweep and is swept before repacking.
`hermes worktree prune` (and the startup/cron pruner) classified every clean tree in a
repository with no remote as "clean and fully merged/pushed" and force-deleted its branch,
even when the branch carried commits that exist nowhere else. `audit_branches` returned []
in the same repos, so a unique local-only branch was invisible to the audit as well.
Root cause: `_worktree_has_unpushed_commits` answered False when `refs/remotes` was empty
("nothing to be unpushed against") and both consumers — `worktree_gc._classify_tree` and
`worktree_ops._classify_prune_candidates` — read False as "safe to reap".
The preceding commit (#111897) flips that branch to True, which is safe but also means a
no-remote repo can never reclaim anything (a tree sitting at trunk, or squash-merged into
it, stays "unpushed" forever because `_worktree_commits_all_merged_upstream` finds no
origin/* base). This commit replaces the unconditional True with a real baseline:
- `_worktree_local_trunk`: `main`/`master`, else the branch checked out in the main
worktree; None when no trunk exists.
- `_worktree_merge_base_ref`: origin/HEAD|origin/main|origin/master, falling back to the
local trunk ONLY when the repo has no remote-tracking refs at all. Single resolver used
by `_worktree_commits_all_merged_upstream` and `worktree_gc.audit_branches`.
- `_worktree_has_unpushed_commits`: with no remote refs, `git log HEAD --not <trunk>`;
no trunk -> True (preserve), matching the docstring's fail-safe promise.
Net effect in a no-remote repo: unique work is kept ("unpushed commits not found
upstream"), trees at/merged into the local trunk still reap, branch audit reports unique
branches as keep and merged ones as delete, and `git branch -D` can only run on a branch
whose every commit is reachable from or patch-equivalent to the trunk.
Tests: the salvaged no-remote keep test now uses a shared `local_repo` fixture; a control
test proves trunk-merged trees still reclaim and the branch audit reports in the same
repo; `test_merged_predicate_fails_safe_without_upstream` now pins both halves of the
contract (trunk resolves -> merged; no trunk at all -> False/preserve).
Each copy re-implemented temp+replace by hand and lacked one or more of
fsync, symlink preservation, atomic_replace's Windows-contention retry and
EXDEV/bind-mount fallback, mode preservation, or interrupt-safe temp
cleanup. Three (gateway/session_persistence, cron/suggestions,
agent/shell_hooks) were verbatim inlines of utils._atomic_write; two
modules defined their own directory-fsync helper, now utils.fsync_directory.
plugins/google_meet/_jsonfile.write_json_atomic is deleted (callers use the
canonical helper directly).
Behavior change: every one of these writers now fsyncs the payload, keeps a
pre-existing target's mode, cleans its temp file on BaseException, and
survives Windows AV/indexer contention and cross-device renames the way
config writes already did. cron/suggestions.json is 0600 from creation
(previously chmod'ed after the replace). Skipped on purpose: cron/jobs.py
two-phase staging, gateway/status._write_json_excl (create-only lock),
kanban_transfer staging (not atomic writers); tools/skill_usage.
_write_suppressed_names lives inside a PLUGIN-COMPAT block.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
Repo scanners (check_subprocess_stdin, check-windows-footguns --all) flagged 21 sites where
the r3 single-line collapses lost stdin=DEVNULL, encoding='utf-8'/errors='replace', the
'# windows-footgun: ok' same-line marker, or the getattr(os, 'geteuid') gate. Each guard is
restored at the call site (real portability/hang fixes, not suppressions).
