hermes-admin/hermes-agent
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

fix(update): make the partial-clone pack fold and maintenance keys work together (#129712, #127711)

Browse Source 
The two salvaged commits above disagree: #127721 persists gc.auto=0, which
turns the fold's `gc --auto` into a no-op (106 packs stay 106 on git 2.50.1,
2.53.0 and 2.55.0), and the fold itself is exposed to the loop #127721
guards against. Probed on all three gits against a blobless clone whose
commit-graph carries changed-path (Bloom) data:

- gc, `gc --auto`, `maintenance run --task=commit-graph` and
  fetch.writeCommitGraph each lazy-fetch the trees of every commit the graph
  has not seen (30 unseen commits: 30 fetches, 30 packs), and every lazy
  fetch spawns `git maintenance` again. gc.writeCommitGraph=false stops it
  (gc: 31 packs -> 1).
- Without a Bloom graph none of that happens, so the keys are a guard, not a
  fix for every partial clone.

Changes:
- keys: maintenance.auto, gc.writeCommitGraph, fetch.writeCommitGraph; gc.auto
  stays default. Idempotent (no rewrite when already set), `--local`, hidden
  console on Windows. install.sh and install.ps1 set the same three keys.
- consolidate_lazy_fetch_packs applies the keys before the fold, so installs
  that predate the installer change converge on their next update check.
- the fold runs under bounded_probe_run: subprocess.run(timeout=) kills only
  `git gc` and leaves `pack-objects` running (reproduced), and with
  GIT_NO_LAZY_FETCH.
- dropped the post-gc mark_unmarked_packs_promisor: gc keeps the .promisor
  marker on 2.50.1, 2.53.0 and 2.55.0.
- docs: updating.md troubleshooting entry.
This commit is contained in:
ethernet
2026-10-02 13:05:47 -04:00
committed by Teknium
parent 73c17151f6
commit 6880ffc1af
4 changed files with 74 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
hermes_cli/gitlock.py
View File

@@ -12,10 +12,18 @@ import time
from pathlib import Path
from typing import Callable, Iterable, List, Optional
from hermes_cli._subprocess_compat import windows_hide_flags
from hermes_cli._subprocess_compat import (
NO_LAZY_FETCH_ENV,
bounded_probe_run,
noninteractive_git_env,
windows_hide_flags,
)
logger = logging.getLogger(__name__)
# Folding hundreds of tiny packs takes well under a second; this only bounds a pathological repack.
LAZY_FETCH_GC_TIMEOUT_SECONDS = 120
# Files younger than this are presumed live (a fetch may be in flight) and are never removed. Lock
# files live for seconds and a healthy fetch completes in minutes; 10 minutes is abandoned.
STALE_LOCK_MIN_AGE_SECONDS = 10 * 60
@@ -126,31 +134,43 @@ def mark_unmarked_packs_promisor(repo_root: Path) -> int:
return marked
# A tree:0 partial clone that lets detached git maintenance auto write a
# commit-graph lazy-fetches every missing tree on demand, and each fetch
# re-triggers maintenance, looping without bound (#127711). A repo cloned or
# converted to tree:0 never needs automatic maintenance; explicit git gc and
# git maintenance run still work.
# In a partial clone whose commit-graph already carries changed-path (Bloom) data, any commit-graph
# write over commits it has not seen yet needs their trees: one lazy fetch (and one pack) per commit,
# and each lazy fetch spawns ``git maintenance`` again, which is the unbounded loop of #127711. gc,
# ``maintenance run --task=commit-graph`` and ``fetch.writeCommitGraph`` all write one (git
# 2.50.1, 2.53.0 and 2.55.0 alike). So a promisor checkout never writes the graph. ``gc.auto`` stays
# at its default on purpose: ``gc --auto`` is how consolidate_lazy_fetch_packs folds the lazy-fetch
# packs, and ``gc.auto=0`` turns that into a no-op.
_TREE0_MAINTENANCE_OFF = (
("maintenance.auto", "false"),
("gc.auto", "0"),
("gc.writeCommitGraph", "false"),
("fetch.writeCommitGraph", "false"),
)
def disable_tree0_auto_maintenance(repo_root: Path) -> None:
"""Turn off automatic maintenance commit-graph writes in a tree:0 clone.
"""Keep git from writing a commit-graph or running automatic maintenance in a partial clone.
Never raises: a read-only config must not turn fetch recovery into a
Idempotent (a value already in place is not rewritten, so concurrent git never meets a config
lock from this) and never raises: a read-only config must not turn fetch recovery into a
traceback, matching mark_unmarked_packs_promisor above.
"""
for key, value in _TREE0_MAINTENANCE_OFF:
try:
current = subprocess.run(
["git", "config", "--local", "--get", key],
cwd=str(repo_root), capture_output=True, text=True,
encoding="utf-8", errors="replace", timeout=30,
creationflags=windows_hide_flags(),
).stdout.strip()
if current == value:
continue
subprocess.run(
["git", "config", key, value],
["git", "config", "--local", key, value],
cwd=str(repo_root), check=True,
capture_output=True, text=True,
encoding="utf-8", errors="replace", timeout=30,
creationflags=windows_hide_flags(),
)
except Exception:
logger.warning("Could not set %s=%s in %s", key, value, repo_root)
@@ -566,23 +586,23 @@ def consolidate_lazy_fetch_packs(repo_root: Path) -> int:
blob by blob, so ``.git`` grows without bound (2,475 packs / 39 GiB observed for a ~1 GiB
repo, #129712). ``git gc --auto`` already knows when this is worth doing: it exits in
milliseconds while the small-pack count sits under ``gc.autoPackLimit`` (default 50) and
repacks them into one pack once past it. The gc does not write a commit-graph: on git ≤2.50
a freshly written one makes the next lazy fetch abort with "in the commit graph file but
not in the object database". gc's repack merges the marked packs into a fresh one; on gits
that do not carry the ``.promisor`` marker over, an unmarked pack is the git 2.53+ fetch
crash (#124272), so the merged pack is re-marked afterwards. Best-effort like every helper
here: never raises, returns 0 for a non-partial checkout or when nothing folded.
repacks them into one pack once past it, keeping the ``.promisor`` marker. The gc must not
write a commit-graph (see ``_TREE0_MAINTENANCE_OFF``): over a Bloom-carrying graph that is a
lazy fetch per unseen commit, so the same call that folds 100 packs would leave 30 new ones.
Runs under ``bounded_probe_run`` because ``subprocess.run(timeout=)`` kills only ``git gc``
and leaves its ``pack-objects`` child running. Best-effort like every helper here: never
raises, returns 0 for a non-partial checkout or when nothing folded.
"""
try:
if _partial_clone_filter(repo_root, creationflags=windows_hide_flags()) is None:
return 0 # only a promisor remote's on-demand fetches write these packs
disable_tree0_auto_maintenance(repo_root)
before = len(list(_pack_dir(repo_root).glob("pack-*.pack")))
subprocess.run(
bounded_probe_run(
["git", "-c", "gc.autoDetach=false", "-c", "gc.writeCommitGraph=false", "gc", "--auto"],
cwd=str(repo_root), capture_output=True, timeout=300,
creationflags=windows_hide_flags(),
timeout=LAZY_FETCH_GC_TIMEOUT_SECONDS, cwd=str(repo_root),
env={**noninteractive_git_env(), **NO_LAZY_FETCH_ENV},
)
mark_unmarked_packs_promisor(repo_root)
folded = before - len(list(_pack_dir(repo_root).glob("pack-*.pack")))
if folded > 0:
logger.info("Folded %d lazy-fetch pack(s) in %s", folded, repo_root)

12
scripts/install.ps1
View File

@@ -733,6 +733,16 @@ function Stage-Prerequisites {
Write-Ok "prerequisites ok (git)"
}
# A treeless checkout must never write a commit-graph: over a graph with changed-path
# data that lazy-fetches the trees of every unseen commit, in a loop (#127711).
# gc.auto stays on: `hermes update` folds lazy-fetch packs with `gc --auto`.
function Disable-TreelessGraphWrites([string]$Dir) {
foreach ($key in 'maintenance.auto', 'gc.writeCommitGraph', 'fetch.writeCommitGraph') {
Invoke-Native { git -C $Dir config $key false } | Out-Null
if ($LASTEXITCODE) { Write-Warn "could not set $key in $Dir" }
}
}
function Stage-Repository {
# Refuse an occupied non-checkout before provisioning Git. This check
# needs no tool download and must not overwrite a user's existing files.
@@ -779,6 +789,7 @@ function Stage-Repository {
catch { Write-Warn "could not mark $marker as a partial-clone pack" }
}
}
Disable-TreelessGraphWrites $InstallDir
}
Invoke-Logged "Fetching origin/$Branch" { git -C $InstallDir fetch origin "+refs/heads/${Branch}:refs/remotes/origin/${Branch}" }
if ($LASTEXITCODE) { Fail "git fetch failed" }
@@ -888,6 +899,7 @@ function Stage-Repository {
}
if (-not $cloned) { Fail "git clone failed; no checkout published" }
Move-Item -LiteralPath $tree -Destination $InstallDir
Disable-TreelessGraphWrites $InstallDir
Write-Ok "Hermes Agent cloned"
} finally {
Remove-Item -LiteralPath $staged -Recurse -Force -ErrorAction SilentlyContinue

15
scripts/install.sh
View File

@@ -476,8 +476,8 @@ stage_repository() {
# Existing treeless checkout: same commit-graph lazy-fetch loop guard (#127711).
git -C "$INSTALL_DIR" config maintenance.auto false \
|| log_warn "could not disable maintenance.auto in $INSTALL_DIR"
git -C "$INSTALL_DIR" config gc.auto 0 \
|| log_warn "could not disable gc.auto in $INSTALL_DIR"
git -C "$INSTALL_DIR" config gc.writeCommitGraph false \
|| log_warn "could not disable gc.writeCommitGraph in $INSTALL_DIR"
git -C "$INSTALL_DIR" config fetch.writeCommitGraph false \
|| log_warn "could not disable fetch.writeCommitGraph in $INSTALL_DIR"
fi
@@ -596,13 +596,14 @@ stage_repository() {
fail "cannot publish cloned checkout"
fi
rmdir "$staged"
# A treeless checkout plus detached git maintenance auto commit-graph
# writes lazy-fetch every missing tree in a loop (#127711). Automatic
# maintenance is never needed here; explicit git gc still works.
# A treeless checkout must never write a commit-graph: over a graph with
# changed-path data that lazy-fetches the trees of every unseen commit, in a
# loop (#127711). gc.auto stays on: `hermes update` folds lazy-fetch packs with
# `gc --auto`.
git -C "$INSTALL_DIR" config maintenance.auto false \
|| log_warn "could not disable maintenance.auto in $INSTALL_DIR"
git -C "$INSTALL_DIR" config gc.auto 0 \
|| log_warn "could not disable gc.auto in $INSTALL_DIR"
git -C "$INSTALL_DIR" config gc.writeCommitGraph false \
|| log_warn "could not disable gc.writeCommitGraph in $INSTALL_DIR"
git -C "$INSTALL_DIR" config fetch.writeCommitGraph false \
|| log_warn "could not disable fetch.writeCommitGraph in $INSTALL_DIR"
log_success "Hermes Agent cloned"

14
website/docs/getting-started/updating.md
View File

@@ -207,6 +207,20 @@ git -C $repo rev-list --objects --missing=print --all | Where-Object { $_.Starts
git -C $repo rev-list --objects --missing=error --all | Out-Null; $LASTEXITCODE # 0 = complete
```
### `.git` keeps growing in a partial clone
The installer's checkout is a partial clone: git downloads trees and blobs on demand, and each
on-demand download is written as its own small pack. `hermes update` and `hermes update --check`
fold them back together with `git gc --auto` (git's own `gc.autoPackLimit`, 50 by default), so a
healthy checkout pays a no-op. They also set `maintenance.auto`, `gc.writeCommitGraph` and
`fetch.writeCommitGraph` to `false` in that checkout, because a commit-graph write over commits the
graph has not seen yet downloads every one of their trees. Leave those settings alone, and leave
`gc.auto` at its default: `gc.auto=0` stops the fold. To fold by hand (with Hermes closed):
```bash
git -C "$repo" -c gc.writeCommitGraph=false gc --auto
```
### Updating against a non-default branch: `--branch`
On the default source channel, `hermes update` tracks `origin/main`. Use