Review round 2 on #133077. A killed update's receipt plus a younger lock let
release_dead_index_lock delete the lock of a live `/usr/lib/git-core/git-commit -a`
waiting in its editor: the lock fd is closed there, and the Linux holder scan only
knew `comm == git` with an argv subcommand, so the dashed form read as "no holder".
A competing `git add` then got in and the user's commit died "unable to write new
index file".
- The receipt gate stays necessary, never sufficient. The reclaim now also needs
proof that no git of ANY form works in this checkout (`_held_open(any_git=True)`):
cwd, path arguments (relative ones against the git's cwd) or GIT_DIR/GIT_WORK_TREE/
GIT_INDEX_FILE inside the checkout or its git dir, compared by path components.
A git we cannot place (unreadable cwd/environ, hidepid) keeps the lock.
- Git forms: `git`, a dashed `git-<sub>` from git-core, `git.exe`/`git-<sub>.exe`,
`-C`/`-c`/`--git-dir` globals, aliases (an unknown subcommand counts). The
launch-time repair's own scan inverts its list: only known readers are exempt.
- macOS: an empty lsof is no proof any more; `ps` lists every git (comm, so dashed
forms too) and any lock-keeping one keeps the lock.
- Windows: the psutil scan matches dashed and .exe gits, reads environ, and compares
by path components (`hermes-backup` was read as inside `hermes`).
- A failed retry no longer erases the evidence: `_killed_update_owns` reads the durable
per-run records (read-only) and asks about the last run that STARTED before the lock
was written, so a later failed run cannot discharge the killed one.
- The FIFO test helper kills its blocked git when its own setup assertion fails.
A lock nobody holds open is not proof its git is dead: a git that just created it may not have
opened it yet. The start-of-run reclaim now takes .git/index.lock only when the latest update run
died unfinished (receipt running/interrupted, owner gone) and the lock is newer than that run's
start. Every other lock is left to the age-floor sweep, and the update refuses on it truthfully
(e2e-upgrade test_live_index_lock_is_refused_truthfully went red on fd2214b0ad6).
Windows has no /proc to name a lock's holder, and its unlink-as-probe cannot see a
lock-keeping git with its fd closed (a `git commit` in the editor), so the reclaim keeps the
lock while a git.exe has its cwd or a path argument inside the checkout, and whenever the
scan cannot tell (no psutil, an uninspectable git). Any other git on the machine no longer
blocks it, unlike the age-floor sweep's machine-wide check.
`.git/index.lock` left by a killed git refused every later merge with
"File exists": the stale-lock sweep keeps any lock younger than 10 minutes,
so a `hermes update` killed mid-run (Windows crash cell `mid_fetch`, run
37573293632) or a version probe killed by its own 3 s timeout (#132089)
failed the next update at "Pulling updates...".
- `gitlock.release_dead_index_lock`: drop the lock as soon as the launch
repair's ownership proof (`_early_recovery._release_dead_index_lock`:
/proc fds + lock-keeping gits on Linux, lsof on macOS, the unlink itself on
Windows) shows no live git holds it. An interrupted tree move keeps its own
lock judgement. `_cmd_update_impl` calls it first, under the update lock.
- `version_info._git_version_info`: the read-only status probe runs as
`git --no-optional-locks status`, so a timed-out probe strands no lock.
Part of #132089 (the stale index.lock half; naming why a pre-apply exit fired is separate work)
#132927 deleted gitlock.consolidate_lazy_fetch_packs, the one updater git
that ran under its own bounded runner with spawn_kwargs custody. Every
updater git now goes through run_git, so the import and the docstring
exception go with it.
m3: installer checkouts are partial clones, so a move (reset --hard, merge
--ff-only, checkout) lazily fetched the target's missing objects as a child
holding the checkout lock fd, with the user's credential helper; a
credential-cache daemon it started held the checkout for its lifetime.
Mutators now run with -c credential.helper=, and before a move run_git
fetches what it will read (git diff <HEAD or empty tree> <target>) without
the fd and with the helpers, so a private fork still authenticates.
m4: consolidate_lazy_fetch_packs ran gc --auto with neither the fd nor a
death signal; gc/repack/pack-objects outlived a killed owner and repacked
under the next one. gc (it packs refs) is a local mutator and its bounded
runner gets run_git's custody via update_custody.spawn_kwargs (POSIX: the
lock fd its children inherit). pull is dropped from LOCAL_MUTATORS (unused;
its network half must not hold the fd). The module doc names the exception
and the read-only source_releases/source_check readers.
R2: hermes_cli/update_custody.run_git is THE updater git runner;
update_cmd._git_run, update_cmd_git._git_run, update_cmd_stash,
update_cmd_check._git, gitlock, update_cmd_zip and the fleet ancestry probe
all go through it.
- every updater git call carries -c gc.autoDetach=false
-c maintenance.auto=false: no detached gc/maintenance child ever forks
under (POSIX) or outlives (Windows) the checkout lock;
- POSIX: only local mutators (reset, checkout, merge, stash, rebase,
update-ref, clean, apply, commit, ...) inherit the lock fd, with
core.fsmonitor=false; fetch/ls-remote/credential and readers run without
it, so a credential-cache daemon never holds the checkout (reviewer: up to
2 h of installer waits). On Linux those fd-less children die with the
updater (PR_SET_PDEATHSIG) instead of rewriting refs under the next owner;
- Windows: inside an update every child is created suspended, assigned to
the owner's kill-on-close job and only then resumed; the Node build
(pm.progress.run_contained) joins it through a stdlib launcher.
The fork sync fast-forwards locally (merge --ff-only
refs/remotes/upstream/main after its fetch) instead of a networked pull
under the mutator fd.
The convert-first step runs in the bootstrap interpreter, which cannot import
update_cmd (ruamel), so it carried its own raw os.environ copy (the spawn-env
guard flagged it) and lacked the Windows console-hide flags (#117781).
Move the body unchanged to _subprocess_compat.no_prompt_git_kwargs: prompts
off, credential helpers kept, consoles hidden. update_cmd keeps the
_no_prompt_git_kwargs name as an import because old updaters load it after
the checkout swap (tests/compat/old_updater_surface.json); existing callers
are untouched.
A Desktop built before the blobless fix keeps walking history during
`hermes update`. On a treeless (tree:0) checkout every walk downloads trees
again, and #129514 filled a 434 GB disk during the minutes of dependency work.
The one-time conversion (#133016) ran at the very end of the update, after that
window. Both update hand-offs (update_completion._prepare and an older
updater's _update_takeover.prepare) now run it first, in the bootstrap
interpreter, and the late call is gone.
Re-running install.sh / install.ps1 over a treeless checkout now converts it
too. The filter is recorded only after the refetch succeeds, so a failure is
retried by the next update instead of leaving a blob:none filter over a
history that has no trees.
The pack cleanup also sweeps tmp_* transfer files that killed fetches left
behind. They are more than an hour old, a storm leaves gigabytes of them,
and git itself only prunes them after two weeks. The docs gain the recovery
steps for a disk that the storm already filled.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: Yuan Li <dskwelmcy@163.com>
Final-review findings on convert_treeless_checkout:
- A local tag that the --tags refetch would clobber made git exit non-zero
after the objects had landed, and check=True raised before the
completeness check ran, so every update warned and stayed tree:0. The
offline history check is now the verdict, not the fetch's exit status.
- Git older than 2.36 has no --refetch; such a checkout is left as it is
instead of printing git's usage text as a warning on every update.
Review on 6f9a3d6 found two gaps. The refetch covered only origin's
configured refspec with --no-tags, so a release checked out from a tag no
branch reaches, or a branch fetched outside a narrow refspec, was recorded
as blobless while its history still had no trees. The refetch now takes
the tags too, checks HEAD's whole history offline (rev-list --missing=print,
about 3 s on this repo) and refetches HEAD by commit when trees are still
missing; the new filter is recorded only once that history is whole.
A raw fetch also spawned git's detached maintenance, gc --auto and a full
cruft repack of the 120 MB refetch, outside the update's time limit. The
conversion fetch now runs with per-command gc.auto=0 and
maintenance.auto=false, leaving the user's own settings untouched.
Installers have cloned with --filter=tree:0 since c3e9ac87a7. A treeless
clone keeps commits only, and git requests a missing tree without telling
the server which trees it already has, so every checkout and every
path-filtered history walk re-downloads complete directory snapshots
(~0.58 MB per commit for this repo). GitHub's own guidance is that
treeless clones are for throwaway CI builds, not long-lived checkouts;
the blobless layout keeps every commit and tree local and fetches only
file contents on demand.
- install.sh / install.ps1 clone --filter=blob:none (same release tags,
--commit pins and ancestor checks the treeless switch was made for).
- hermes update (post-update step) converts an existing tree:0 checkout once with
`git fetch --refetch --filter=blob:none origin` (~120 MB, ~8 s
from GitHub) and records the new filter only after it succeeds, so a
failed refetch retries next update.
- The shallow-heal conversion unshallows as blob:none instead of tree:0.
- checkout_contains asks git with GIT_NO_LAZY_FETCH: a recorded SHA the
checkout lacks is "not contained", never a download of that commit's
whole history.
The fold added in #131679 runs `git gc --auto`, which on a large partial
clone is a full repack: a 9.6 GB checkout with 463 packs sat in it for
20 minutes per update, got killed at the cap, and started over on the
next update without ever finishing. It also never freed disk (on-demand
packs are kept, only merged). Turn it off until a bounded, progressive
cleanup replaces it.
The partial-clone keys that stop git writing a commit-graph (#127711)
stay: settle_partial_clone_maintenance re-applies them every update and
leaves full clones alone, and git's own automatic gc still folds packs.
73c17151f6 (Sep 29) persisted maintenance.auto=false, gc.auto=0 and
fetch.writeCommitGraph=false in treeless install checkouts. 6880ffc1af
stopped writing gc.auto=0 but never removed it, and `gc --auto` with
gc.auto=0 is a no-op (git skips every heuristic, gc.autoPackLimit included),
so the update's pack fold never ran on those checkouts: 203 -> 203 packs.
The previous commit's fingerprint required gc.writeCommitGraph=false, which
only the Oct 2 cut wrote, so the Sep 29 cohort also kept maintenance.auto=false.
Both cuts wrote maintenance.auto=false together with
fetch.writeCommitGraph=false and neither wrote
maintenance.commit-graph.enabled; that pair is now the fingerprint, and
gc.auto=0 is removed under it as well. An operator's own lone
maintenance.auto=false still never matches.
Real clear_git_debris() on a treeless clone with 203 lazy-fetch packs, git
2.53.0 and 2.55.0 alike:
Sep 29 keys: main 203 -> 203, #131741 203 -> 203, this 203 -> 1 (both keys gone)
Oct 2 keys: 203 -> 1, maintenance.auto gone
operator's own maintenance.auto=false: 203 -> 1, setting kept
_restore_auto_maintenance unset any local maintenance.auto=false, so an
operator's own setting on a partial-clone checkout was erased on every
update (nothing records who wrote it). Remove it only with the first cut's
fingerprint: maintenance.auto, gc.writeCommitGraph and fetch.writeCommitGraph
all false and maintenance.commit-graph.enabled not yet set. The first pass
of disable_tree0_auto_maintenance writes that key, so the migration runs once
and an operator's setting is left alone before and after.
Review point from @Godhet on #131718.
#131679 set maintenance.auto=false in the treeless install checkout to stop the
commit-graph lazy-fetch loop (#127711). That also switched off the post-fetch
`gc --auto` that git <= 2.53 runs after every lazy fetch, so packs piled up
between updates and only folded on a manual `hermes update` / `--check`
(nothing runs `--check` automatically).
The loop needs a commit-graph write, and gc.writeCommitGraph=false +
fetch.writeCommitGraph=false already stop gc and fetch from writing one.
maintenance.commit-graph.enabled=false closes the last writer (a commit-graph
maintenance task, e.g. from a user's global maintenance.strategy) while
leaving automatic maintenance, and its gc fold, on.
Checkouts that already got maintenance.auto=false from the first cut are
converged: the next update/--check unsets exactly that value.
Measured on a real tree:0 clone with a Bloom commit-graph, 100 lazy fetches
after one updater run:
git 2.53.0: main 101 packs, this change 1 pack, 0 commit-graph writes
git 2.55.0: 101 / 101 (stock 2.55 maintenance does not fold), 0 writes
The fold was killed after 120 s on the assumption it takes "well under a second". That holds
for ~100 small packs, not for the checkouts it exists for (#129712: 2,475 packs / 39 GiB): those
need a full repack, and a killed fold restarted from scratch on every update without a word,
so the worst-hit installs never converged.
- Bound raised to 20 minutes (measured: 750 packs / 530 MB of hermes-agent history fold in
5.8 s at 7.4 cores; a 39 GiB pack dir on a slow disk is a different order of magnitude).
- consolidate_lazy_fetch_packs returns None on timeout; the shared update_cmd_check reporter
(used by both `hermes update` and `hermes update --check`) prints the hand-fold command.
- on_fold_start announces a fold that will actually run (pack count past gc.autoPackLimit), so
a multi-minute first fold is not a silent hang.
- Test fixture disables auto-maintenance: each lazy fetch's detached gc otherwise raced the
test and folded the packs itself.
The two salvaged commits above disagree: #127721 persists gc.auto=0, which
turns the fold's `gc --auto` into a no-op (106 packs stay 106 on git 2.50.1,
2.53.0 and 2.55.0), and the fold itself is exposed to the loop #127721
guards against. Probed on all three gits against a blobless clone whose
commit-graph carries changed-path (Bloom) data:
- gc, `gc --auto`, `maintenance run --task=commit-graph` and
fetch.writeCommitGraph each lazy-fetch the trees of every commit the graph
has not seen (30 unseen commits: 30 fetches, 30 packs), and every lazy
fetch spawns `git maintenance` again. gc.writeCommitGraph=false stops it
(gc: 31 packs -> 1).
- Without a Bloom graph none of that happens, so the keys are a guard, not a
fix for every partial clone.
Changes:
- keys: maintenance.auto, gc.writeCommitGraph, fetch.writeCommitGraph; gc.auto
stays default. Idempotent (no rewrite when already set), `--local`, hidden
console on Windows. install.sh and install.ps1 set the same three keys.
- consolidate_lazy_fetch_packs applies the keys before the fold, so installs
that predate the installer change converge on their next update check.
- the fold runs under bounded_probe_run: subprocess.run(timeout=) kills only
`git gc` and leaves `pack-objects` running (reproduced), and with
GIT_NO_LAZY_FETCH.
- dropped the post-gc mark_unmarked_packs_promisor: gc keeps the .promisor
marker on 2.50.1, 2.53.0 and 2.55.0.
- docs: updating.md troubleshooting entry.
Every on-demand fetch on the promisor remote of a tree:0 installer
checkout writes its own small packfile, and no update-path step ever
consolidates them, so .git grows without bound (2,475 packs / 39 GiB
observed for a ~1 GiB repo, #129712).
Ride git's own threshold: run `git gc --auto` from the debris-cleanup
step both `hermes update --check` and `hermes update` already run.
Below gc.autoPackLimit (default 50) small packs it exits in
milliseconds; past it, the lazy-fetch packs fold into one. The folded
pack is re-marked .promisor so git 2.53+ fetches keep working (#124272),
and the gc does not write a commit-graph, which git <=2.50 misreads on
the next lazy fetch.
A console-less parent (pythonw backend, detached daemons) allocates a visible
console per bare spawn, so Windows startup flashes ~7-8 console windows
(#117781) and the GPU statusbar flashes two more every 5s (#101895). Threads
creationflags=windows_hide_flags() through every hermes_cli spawn site the
desktop reaches: gitlock (tasklist probe + git plumbing), the scheduled-task
PowerShell probe, the updater's _git_run/_no_prompt_git_kwargs/_GIT_TEXT_KW
runners, the safe-directory config probe, and consolidates the statusbar's two
nvidia-smi reads per poll into one cached hidden query. The statusbar also
stops polling while the document is hidden and resumes on visibilitychange
(#120262).
Supersedes #117805, #117833, #102047, #120285 (thanks @JoaoMarcos44, @Finn763,
@monerostar, @szzhoujiarui).
- _deepen_shallow_repo marks packs in its finally without a git config probe
that could itself raise and turn a finished unshallow into False; markers
are inert in a clone without a promisor remote.
- Installers warn instead of aborting the stage when a marker cannot be
written, matching the Python helper.
- Docs: the manual commands follow HERMES_HOME, and the missing-object
backfill feeds IDs through `fetch --stdin` so a large list cannot exceed
the Windows command-line limit.
- Test and docstring comments describe the contract, not the incident.
git 2.53+ `index-pack --promisor` hands every object outside the promisor
packs to pack-objects, which BUG()s (should_include_obj) on the missing
objects those objects lead to. A partial clone with unmarked packs therefore
keeps crashing its fetches.
The retry added for this ran `git -c remote.origin.promisor= fetch`. That
override does nothing: git registers promisor remotes additively, so the
repo's own `true` still wins (GIT_TRACE on 2.50 and 2.55 shows
`index-pack --promisor` and `filter tree:0` under the override). The retry
only passed when the crashed fetch had already stored the pack; a retry that
has to transfer again crashed again.
- On the crash, give every pack without a `.promisor` marker one, then retry
the same fetch once. The clone stays partial, no refetch, no config change.
If git still crashes, the updater points at the docs section.
- The two fetches that turn a full clone partial now mark the clone's old
packs afterwards: the depth-1 `tree:0` unshallow in
`fetch_full_commit_graph`, and `worktree_ops._deepen_shallow_repo`
(`--filter=blob:none`). git writes the partial-clone config before it
fetches, so this also runs when the fetch fails. Without it, a converted
depth-1 install crashed every later fetch on git 2.55 (3 of 3 rounds in a
local repro).
is_partial_clone_pack_objects_crash() required "index-pack failed" next to the
pack-objects assertion, but git 2.55 builds dropped that string: Homebrew git
2.55.0 (macOS) only reports "fatal: fetch-pack: invalid index-pack output", so
the recognizer always returned False there, fetch_with_partial_clone_recovery()
never retried with the promisor machinery disabled, and `hermes update` exited 1
on the crash (#125138 reports the same bare-assertion failure on 2.55.0.windows.5).
Fingerprint on the assertion text — the only version-stable part of the failure
— and accept the 2.55 wrapper as a terminator. The POSIX/Windows shapes and the
"fingerprint without a terminator" rejection behave as before.
(cherry picked from commit 74103b15ceb3eacb2346fca669c3a3dc81457201)
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule
The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:
- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
names, restored dead definitions) and the three re-export stub modules
(gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
`plugins.allow_deprecated_imports` escape hatch
An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.
hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).
In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.
* chore: retrigger CI (zero-job startup_failure phantom)
* test: drop resolution allowlist rows for the two deleted which() sites
hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
The pre-fetch heal (#123254) and the post-update tag fetch now share
fetch_full_commit_graph, so both follow one clone-mode rule: a partial clone
repeats its own filter, a full clone fetches unfiltered (#122353), and only a
depth-limited clone whose history is really missing unshallows as tree:0. A
full clone grafted by a user's `git fetch --depth 1` already has its history
locally, so it unshallows unfiltered and stays full.
Also: the network-timeout message no longer claims "no response from the
remote" (a large transfer on a slow link hits the same cap), without adding a
classifier rule that would assert the opposite.
A depth-1 installer checkout far behind main could never update: the plain
main fetch pulled nearly the full object graph (merge-heavy history drags in
side-branch ancestry past the shallow boundary), hit the 300s network cap
mid-transfer every time, and the unshallow step that makes later fetches
incremental only ran after a successful update. Unshallow the checkout first
(commit-graph-only fetch with the same 900s budget the post-update tag fetch
uses, pulling the update branch along), non-fatally, and stop reporting a
wall-clock kill as 'no response from the remote'.
Fixes#123254
prune_stale_shallow_grafts dropped grafts no live ref points at, but the
remote-tracking reflog still naming a dropped fetch tip made the --reflog
fail-safe walk fail (its parent was never fetched), rolling the prune back
on every run — grafts kept accumulating and the next update fell into
orphan divergence (#124645). Expire exactly the fetch reflogs that pin a
dropped graft before writing the trimmed file; user reflogs (HEAD, local
branches) are never touched, so the fail-safe rollback still guards them.
Fixes#124645
Git for Windows 2.54 reports the same partial-clone pack-objects BUG()
without the POSIX 'pack-objects died of signal 6' line (#124293 field
evidence): the BUG assertion is followed directly by 'fatal: could not
finish pack-objects to repack local links' and 'fatal: index-pack failed'.
The recognizer demanded the signal wording, so the recovery retry missed
this Windows occurrence entirely.
Require the BUG fingerprint plus EITHER terminator of the same abort —
the POSIX signal line or the Windows repack fatal — and pin both stderr
shapes plus a fingerprint-without-terminator negative in the regression.
On a partial clone (clone --filter=tree:0), git 2.53/2.54 crashes every
fetch: index-pack --promisor runs repack_local_links(), which feeds
pack-objects --exclude-promisor-objects-best-effort, and pack-objects
BUG()s (SIGABRT, 'should only be called on existing objects') when the
link traversal hits a legitimately missing promisor object. The crash is
deterministic, so every fetch — and with it the CLI's update/check flows
— stays broken until the user applies the manual workaround themselves.
The recovery is the reporter's verified workaround, wired in: on this
exact crash (all three stderr markers), retry the fetch once with
-c remote.origin.promisor= (per-invocation only; the user's filter
choice stays in their config). Wired into both fetch sites — the update
fetch and the check fetch (which also serves forks via upstream).
Unrelated fetch failures are returned untouched; if the retry still
crashes, the update path prints the manual heal command.
Fixes#124272
Hardcoding tree:0 for partial clones would silently tighten a checkout the
user cloned with --filter=blob:none. Read the existing
remote.origin.partialclonefilter instead and pass it back verbatim; full
clones still fetch unfiltered.
git fetch --filter=tree:0 writes remote.origin.promisor=true and
remote.origin.partialclonefilter=tree:0 even when those keys were
deliberately removed, so every hermes update converted a de-partialised
checkout back into a partial clone and re-armed the should_include_obj
fetch failure (#122353). Pass the filter only when the checkout already
is a partial clone; full clones fetch tags unfiltered.
On Windows, subprocess text=True without an explicit encoding decodes
child output with the ANSI code page (e.g. 'gbk'); non-ASCII bytes then
raise UnicodeDecodeError inside subprocess._readerthread, killing the
Hermes backend before it becomes ready and surfacing as the desktop boot
timeout.
Sweep every hermes_cli text=True subprocess call to encoding='utf-8',
errors='replace', and add an AST-based regression test that fails when a
future text-mode call omits the encoding.
Fixes#55658
Pre-PM installers cloned --depth 1, so an install updated onto this line
stamped baseVersion "unknown" and printed git.<sha>. The completion tail
(install, update, historical takeover, --finish-update) now turns a
shallow checkout into a treeless partial clone (fetch --unshallow
--filter=tree:0) before the completion line and the stamp read identity.
Commits only: 77M -> 119M in ~5s on the real repo. A failed fetch warns
and the update still completes; the next update retries.
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).
Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.
uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
A git gc killed by _run_git's timeout strands tmp_pack_* files in the bare
store's objects/pack; gc.auto=0 means git itself never reclaims them (~16 GB
observed on one host). clear_stale_tmp_packs() already sweeps this debris
class for the update checkout/worktree paths — teach it to resolve a bare
repo's objects/pack (no .git/ layer) and call it from prune_checkpoints(),
unconditionally under _store_has_head so the daily auto-prune and
'hermes checkpoints prune' both reap it even when no ref moved (a sweep is a
cheap directory listing, unlike the pack-rewriting gc that stays gated on
refs having moved).
The salvaged commit added a module-level `_IS_WINDOWS` flag "so tests can
flip it" and a test that monkeypatched it to True with a faked Path.unlink.
A flag flipped by tests is an OS fake (repo rule: never make the interpreter
believe it is on another OS), so use the inline `os.name == "nt"` check the
rest of this module already uses and replace the test with a
`@pytest.mark.windows_only` invariant: a real aged 0o444 tmp_pack_* under
.git/objects/pack is removed by clear_stale_tmp_packs on native Windows.
The WARNING-level caplog test stays (Linux-runnable, red on base).
Supersedes #116391 (@kokhlo), which carried the same inline-check shape.
Co-authored-by: kokhlo <konstantin.khlopkov93@gmail.com>
git renames its transfer temps (tmp_pack_* and friends) into place
read-only; Windows refuses to unlink a read-only file with EACCES, and
_sweep_stale swallowed that at debug level — so the sweep silently
removed nothing on real debris (#116384: 171 tmp_pack_* files / 8.8 GB
accumulated with zero sweep success lines in the logs).
Clear the write bit before unlinking on Windows and surface every skip
at warning: a cleaner that fails silently is worse than none.
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
Every `hermes -w` launch on a clone past the pack-sprawl threshold started
its own `git repack -a -d` of the whole object store in a daemon thread.
On a multi-agent box that meant dozens of concurrent multi-GB repacks of
the same repo, each too starved (nice 19, under the others) to finish
inside the 1800 s timeout. subprocess.run's timeout killed only `git
repack`, so the `pack-objects` grandchild kept running with ppid 1 for
days; the CLI exiting orphaned it the same way. Observed: 51 pack-objects
processes, load 200 on 20 cores, 143 GB swap, 29 GB of `.tmp-*-pack`
debris, and every `hermes` invocation taking 5-7 s of wall clock for
0.6 s of CPU.
- `_claim_repack_slot`: one repack per clone per 6 h across processes
(`.git/hermes-repack.lock`, mtime = stamp; stale takeover via `replace`
so only one of N racers wins).
- `_run_bounded_repack`: own process group + `kill_process_tree` on
timeout and at exit, so the whole tree dies with the launcher.
- incremental `git repack -d --geometric=2 --write-midx` instead of a
full `-a` rewrite: consolidates sprawl in seconds instead of rewriting
10 GB per run.
- `.tmp-<pid>-pack*` (pack-objects debris) joins gitlock's stale tmp-pack
sweep and is swept before repacking.
Simplify-pass follow-up on the repair rework: the self-check rev-list walks
and the rollback restore now run inside the same _ShallowLock hold (rev-list
never takes shallow.lock), so lock contention can no longer defeat a failed
self-check's rollback and leave a broken .git/shallow in place. The tmp-write
+ os.replace sequence shared by repair and prune moves into _write_shallow.
Scope note added: fetch-by-SHA install tips (HEAD-reflog-only) are not repair
candidates; corruption of that shape is prevented by the prune's reflog
fail-safe. 68 focused tests green; two-cycle repair->prune E2E re-verified.
Rework of the repair pass from #108361 (salvage) addressing the blocking
review findings, verified with real-git probes:
- Sequencing: prune_stale_shallow_grafts' fail-safe now also walks
rev-list --all --reflog, so a boundary the repair just restored (one a
reflog-only commit still needs) is never dropped again; previously the
production repair->prune sequence re-broke the repo on every update run.
- Header-only parent parsing: a "parent <sha>" line inside a commit
message body is prose; _batch_missing_parents stops at the blank line
ending the commit header, so healthy history is never truncated.
- Candidates restricted to fetch-recorded tips (refs/remotes/* reflogs),
not --batch-all-objects: unrelated object loss (a deleted parent of a
locally-created commit) is no longer re-labelled as shallow history;
fsck keeps reporting it.
- Concurrent-writer safety: both .git/shallow writers now hold git's own
shallow.lock, so a depth-1 fetch between read and write fails fast
instead of being clobbered (or clobbering us).
- Cheap gate: repair runs its subprocess fan-out only when
rev-list --all --reflog already fails; healthy updates pay one probe.
- --batch-check returncode is now checked; shared helpers
(_shallow_file_path, _ShallowLock) replace the copy-pasted plumbing;
test file footguns fixed (encoding=, as_uri()) and the missing
repair->prune end-to-end regression added, mutation-checked.
A reflog-only commit can remain present after stale-graft pruning drops the shallow boundary it needs, while its parent was never fetched. That leaves git gc, fsck, and rev-list unable to traverse the repository. Prevention alone is insufficient because a broken gc walk prevents reflogs from expiring.
Repair scans local commit objects without graph traversal, identifies commits with missing parents, and atomically restores their shallow boundaries. It only updates .git/shallow and never expires reflogs, prunes, or deletes objects, so the operation is non-destructive and idempotent.
This complements PR #108290, which owns the prevention half.
Refs #108286
Every 'git fetch --depth 1' in 'hermes update --check' (and the past
banner passive checks, before #107648 moved them to the GitHub API)
appends the fetched tip to .git/shallow as a new graft and git never
removes the previous one, so a long-lived shallow installer checkout
accumulates one graft per check (57 observed). The stale grafts break
merge-base and push 'hermes update' into the orphan-divergence reset
path with a rescue ref on every run.
prune_stale_shallow_grafts() now runs after each successful depth-1
fetch in 'hermes update --check' and clears the grafts already
accumulated by past checks: it keeps only the boundaries still
protecting referenced tips (HEAD, FETCH_HEAD, every ref tip) and
atomically rewrites .git/shallow, restoring the original file if the
trimmed set breaks history walking. The dropped commits are already
unreachable; their objects are left for git gc.
Rebased onto main after #107648: the banner.py hook is dropped (the
passive check no longer git-fetches); the update --check prune and the
cleanup of already-accumulated grafts are kept.
(cherry picked from commit 6174837fc5b9f4cc3d4d46dc1b2d9a2f6b83c120)
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:
git revert <this sha>
removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.
What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)
Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
