Get-LiveProcessCt caches a pid's creation time for the script's lifetime and
forgets it only when a probe SEES the pid dead. A delegate (the `hermes update`
child) that exits and whose pid is reused between two probes keeps the old
incarnation's time, so the release judged the unrelated new process live and
handed the marker to it (A7 rule 5), parking every later update behind it.
Every judgement made through New-MarkerContext (claim, delegate, heartbeat,
release) and the delegate line Add-MarkerDelegate writes now pass -Fresh,
which re-reads the time. Pollers that only ask "still alive?" (Wait-DesktopExit)
keep the cache.
Test (Windows live cell): a release with the delegate's pid cached at the
recorded ct but really owned by another incarnation deletes the marker instead
of handing it over. marker.sh reads /proc or ps on every call: no cache.
review S6
marker.ps1::Test-CheckoutLockHeld probed only the owner byte at 1 MiB. Since R5b
every joiner the kill-on-close job refused also locks one of the 16 lease bytes
after it, and hermes_cli/update_lock.py::_try_lock treats any held lease as a
held lock. With the owner dead and a leased completion child still running, the
hand-off read the checkout as free, so the R6 release wait ended and the
Desktop's reclaim deleted the marker next to a live mutator.
Lock/Unlock now cover 1+16 bytes in one LockFile (it fails if any byte is held).
marker.sh needs no change: POSIX takes the checkout lock with flock on the whole
file and the lease bytes exist only on win32 (_take_lease).
Tests: a live Windows cell holds only a lease byte (1 and 16) and expects
`held`; a static cell, runnable on Linux, pins the probed range (RED at start:
(1048576, 1)).
review S1
What: run_bounded's no-perl branch starts the probe under bash job control (set -m), so it gets its own process group and the timeout's kill -KILL -- -pid reaches its children.
Why: without perl the probe stayed in the caller's group, the group kill failed, and a hung hermes --version / update --help left its python child running.
Red: pre-fix posix.sh fails test_timed_out_probe_is_killed_with_its_whole_process_tree[absent] (grandchild alive after rc=124). Green: both [present] and [absent] pass.
Review R8 m7: after the merge with main's 732b63d61a the probe decodes launcher
stdout as UTF-8 through ProcessStartInfo.StandardOutputEncoding and never touches
[Console]::OutputEncoding, so the test docstrings no longer claim the call sites
scope and restore it. runtime.ps1 keeps a one-line #124526 pointer at the
encoding line. Behaviour and assertions are unchanged.
The corpus now pins line 2 to digits that fit u64 (any count) and a pid to
digits that fit u32 after leading zeros. bash took a line 2 past u64 as a
far-future start (live); PowerShell rejected any line 2 over 18 digits
(so u64::MAX was malformed) and any pid over 10 characters (so a
zero-padded pid was malformed), and Windows PowerShell's [double]::Parse
throws on a creation time past double's range.
bash compares a 20-digit line 2 as two 10-digit halves (never in signed
shell arithmetic); PowerShell strips leading zeros before the width checks,
compares 20 digits as a decimal, and reads an out-of-range ct as +Infinity,
which never matches a live process. Rewrites carry line 2's exact digits;
only the age arithmetic clamps.
(cherry picked from commit 7208aa4af3c9e355dadee970dddb730fcfeb0301)
The two scripts disagreed on an old packaged Desktop's v1 bridge (round 5
D11): bash adopted a live launcher that is our parent but no longer the
Desktop's child when line 2 equals HERMES_UPDATE_STARTED_AT (the Desktop
quit first and the launcher was re-parented); PowerShell refused it, also
accepted a v2/delegate claim naming its parent, required a gone launcher to
be its recorded parent, and Trim()/TryParse'd the env value.
One rule now, as a pure function in both (marker_launcher_rule /
Test-MarkerLauncherRule) over the same facts, plus one env comparison
(plain ASCII digits of line 2's value). tests/scripts/desktop_update/
lineage_rule_cases.py is the shared table; the posix and Windows suites run
it, and a real-process cell per platform covers the divergent row.
(cherry picked from commit aa4b8ce704bc474d539690235c191edee3b637f2)
checkout_lock_held took the real checkout lock through flock(1) on a shell
fd and released it only when the shell closed that fd: the hold spanned
flock(1)'s exit, the wait and the close, so a concurrent `hermes update`
acquiring non-blocking was refused 223 of 3157 times in the reviewer's
tight-loop probe (round 5 D17). Now one process (perl, else python3) opens
the file, tries LOCK_EX|LOCK_NB and drops it in the next syscall: 0 of
3387. flock(1) stays the last-resort fallback; no tool still means held.
The old-Electron line-2 refresher ran only while `hermes update` ran, so
an old packaged Desktop could age-delete the marker 20 minutes into the R6
release wait (round 5 D18). The refresher now lives from the claim until
the release is decided (posix.sh), and the Windows heartbeat ticks inside
Invoke-MarkerRelease's wait. Tests shorten the cadence with the test-only
--self-test-refresh-every argument.
(cherry picked from commit 8339b25e9cfbd13f9f3e8f606e703d994b8b5410)
Release waits (bounded) while any process still holds the checkout lock's
byte (hermes_cli/update_lock.py takes it with msvcrt.locking at 1 MiB), and
-MarkerOp reclaim answers `held` instead of freeing a dead marker in that
state, so a Desktop gate never opens over a surviving completion. posix.sh
withdraw now keeps a dead adopter's claim (`foreign`), like marker.ps1.
(cherry picked from commit 8a5525d26f036a3c4b5176f11ee4a590c746e2db)
Invoke-HermesProbe handed a .cmd launcher to Process.Start with
UseShellExecute=false, letting CreateProcess pick the interpreter and
re-parse the arguments. Run it through ComSpec explicitly like the update
step does, and refuse arguments cmd.exe cannot be made to take literally.
(cherry picked from commit 0b24aab96962535268d71711c0ef564e01ac6e11)
Every marker mutation (reclaim, adopt, delegate, release/hand-over,
withdraw, line-2 refresh) now happens inside one hold of
[IO.File]::Open("<marker>.lock", OpenOrCreate, ReadWrite, None), retried
for 10 s then BUSY (claim refused). Read->judge->mutate all inside the
hold; the sidecar is never deleted. Replaces the compare-then-replace /
compare-then-delete helpers.
- parser/judge per tests/fixtures/update_marker_corpus.json (trimming,
BOM/CR, u32 pids, pid 0 dead, delegate/run on any line >= 4, own-pid
exact incarnation, 2 s tolerance, v1 1200 s ceiling)
- release per corpus: owner deletes or hands the claim to a live
delegate; a delegate releaser drops its line
- -HandoffRun <id>: adopt only the Desktop bridge carrying run:<id>
- -DesktopPid alone (old packaged Desktop): accept its cmd.exe wrapper's
v1 overwrite by lineage, never by dropping the check
- -MarkerOp reclaim|withdraw: one verdict line, exit 0 (64 usage)
- `# hermes-handoff-protocol: 2`
- the update child is created suspended in a kill-on-close job, named as
the delegate under the lock, then resumed: no update instruction runs
before (or without) the delegate line
- line 2 refreshed every 5 min while the child runs (old readers)
- the job helper moves to update-job.ps1 (windows.ps1 < 2000 lines)
(cherry picked from commit 8bdf467c768da314ce7817f114a6b1b154e11154)
marker.sh (sourced by posix.sh) owns the update marker: corpus parser and
judge, claim, delegate, release and the line-2 refresh, each read -> judge ->
mutate inside one flock(1) / perl Fcntl hold of <marker>.lock (busy 10 s
fails closed). Two live claimants of one dead marker no longer both win (R3).
Hand-off protocol 2 (`# hermes-handoff-protocol: 2`): --handoff-run R adopts
only the Desktop's live bridge carrying run:R; an older Desktop's bridge (its
launcher's pid, v1) is adopted by lineage -- the named pid is our parent and
the Desktop's child, or carries HERMES_UPDATE_STARTED_AT -- never by dropping
the check (R4). --marker-op reclaim|withdraw gives the Desktop one verdict
line; it never mutates the marker itself.
Release waits while any survivor still holds the checkout lock and the
reclaim helper answers `held` for a dead marker in that state (R6). The
update child is gated on a go-file created only after the delegate line is
published (pre-publication crash window), run_bounded kills the probe's
whole process group, and line 2 is refreshed every 5 min while ours so an
older Desktop's 20-minute age ceiling never deletes a live claim.
(cherry picked from commit f16dcc5583df8294cea96e6941cfdf5d4a7c8078)
A daemon killed in the second between spawning `hermes update` and that
child taking the update lock left a marker whose only identity was the dead
script: every reader judged it DEAD and a second update could start over the
first. The child now runs asynchronously just long enough for its pid to be
written as line 4 (`delegate:<pid> ct:<ct>`, compare-and-swap on our own
lines 1-3), then is waited on; HUP/INT/QUIT that land meanwhile are handled
after it exits, exactly as with the former foreground child.
(cherry picked from commit a09d2fb7cac0d730b491560ea0f970e5d227421e)
- marker helpers move to scripts/desktop-update/marker.ps1, dot-sourced
first (windows.ps1 2113 -> 1930 lines); a hand-off that cannot load it
exits 3 having changed nothing. It ships in the checkout next to
windows.ps1 and runtime.ps1, which is where the Desktop runs them from.
- creation time comes from Win32_Process.CreationDate (limited query
rights), cached per pid while it lives: Get-Process .StartTime is access
denied for SYSTEM/elevated/other-user pids under PowerShell 5.1, so a
pid reused by a service after a reboot read as a live owner forever.
Alive with an unreadable time is live only inside the 20-minute ceiling.
- -DesktopPid runs adopt the Desktop's live bridge claim only; a refused
run exits 2 without a result file and only reopens a Desktop that is gone.
- positional parse identical to the other readers; claims publish the
complete body with an exclusive hard link (rename fallback); a fresh
empty marker is a claim in flight; the delegate line always carries ct.
- a follow-up step whose process tree cannot be quiesced after the commit
point is ok:true + a manual warning (no relaunch, marker kept live),
not "still on the previous version".
- the interrupted post-update tail is always a manual warning: re-running
hermes update is the only remedy, so the copy says exactly that.
- the launcher probe bound is a -ProbeTimeoutSeconds parameter (default
60), not the HERMES_UPDATE_PROBE_TIMEOUT_SECONDS environment variable.
(cherry picked from commit 65663dc6a64f942c6385b7de3ba13cdecd393387)
Contract amendments A1-A4, posix side:
- --desktop-pid runs only adopt that Desktop's live bridge claim; with no
bridge (the Desktop gave up after its 20 s wait) or anyone else's marker
the script exits 2 without claiming, running, or writing a result
- a refused hand-off (exit 2) never writes .hermes-update-result.json; the
other update owns the outcome, and a stale "still on the previous
version" dialog could otherwise follow a successful CLI update
- positional parse: BOM/CRLF accepted, non-integer line 2 = malformed
(dead), bad ct line = v1, line 4 only as "delegate:<pid> ct:<ct>"
- a live pid whose creation time cannot be read is live only inside the
20-minute ceiling (unknown never means forever)
- an empty marker younger than 5 s is a claim in flight
- macOS ps lstart is rendered and parsed with TZ=UTC0 (DST-safe)
(cherry picked from commit 9d8b58063f0c2d8de6bc37b615e6e49f1abb7549)
windows.ps1 implements update-marker contract C1 v2 and the C3 outcome
vocabulary on the Windows Desktop hand-off:
- The marker claim is the script's first action (before any Add-Type, UI
or probe): exclusive create (tmp + File.Move, never overwrites) with a
v2 body "<pid>\n<started_at>\nct:<creation time>\n". A live marker owned
by exactly -DesktopPid is adopted by compare-and-swap keeping its
started_at; any other live owner refuses the run (exit 2, marker
untouched, "Nothing was changed"); a dead owner (pid gone, creation time
mismatch = pid reuse, v1 older than 20 min) is compare-and-deleted and
the claim retried.
- Release is compare-and-delete on lines 1-3 and keeps the marker while a
line-4 delegate is alive. When a stalled tree cannot be quiesced (exit 7)
the marker gains a delegate line naming a surviving job member
(JobObjectBasicProcessIdList), so it stays live exactly as long as the
tree does instead of naming the dying script.
- After `hermes update` exits 0 (or the 124 banner remap) the result is
ok:true, exit_code 0: verify failure (was exit 8), legacy rebuild
failure (was exit 6), a gateway restart miss, an interrupted post-banner
phase and any unexpected throw become `warnings` + manual, with a
"Hermes was updated, but ..." message; gateway start --all still runs
after a failed verify/rebuild.
- The idle watchdog counts job CPU/IO accounting deltas as progress, so a
pipe-silent but busy step (download, extraction) is not killed.
- Launcher probes (--print-runtime-command, --version, update --help) run
with a 60 s timeout and kill their tree on expiry.
- Desktop exit wait checks pid + creation time pinned at start, and a
still-open Desktop is never relaunched over (exit 4).
- The result file is written atomically, carries started_at (marker line
2) and warnings, and is no longer deleted at hand-off start.
- Stop-UiServer only removes its own profile dir and dirs whose pid is not
running (TEMP is shared across HERMES_HOMEs).
(cherry picked from commit af178fd695d7f94bf584bfabe9748cbb752fea13)
posix.sh now implements the update-marker contract (C1/C2) as a lock:
- the daemon claims `.hermes-update-in-progress` as its first action with a v2
body (pid, started_at, `ct:` creation time) via an atomic no-replace create
(link(2)); a LIVE foreign owner refuses with exit 2 regardless of age, a
marker owned by --desktop-pid is adopted keeping its started_at, a dead one
is compare-and-deleted; liveness honours a line-4 `delegate:` (the running
`hermes update`), so an orphaned update is never reclaimed;
- the launcher waits for the daemon's claim and exits non-zero when it never
lands, so the Desktop does not quit into nothing;
- release is compare-and-delete and keeps a marker whose delegate is alive.
Outcomes follow C3: a legacy rebuild failure, a mac bundle stage/swap failure
or a relaunch gate after `hermes update` exited 0 is ok:true + warnings (no
more exit 6 / exit 7 "still on the previous version"). The result carries
started_at and warnings and is written via a per-pid tmp + rename.
Crash safety: start-of-run recovery restores an interrupted .app swap from its
aside copy (both swappers' suffixes) and a killed TCC heal backup, and sweeps
UI/probe litter of dead pids; the two swap renames are a signal-deferred
critical section. `hermes desktop`'s Python swappers recover the same state
instead of deleting the only copy first.
Also: probes are bounded (60 s), `hermes update` streams into the log, the
update child gets default TERM and the daemon default HUP (no SIG_IGN leak into
gateways or the relaunched app), the relaunched Linux app drops the update env,
the Desktop wait is zombie/pid-reuse aware and never relaunches over a live
app, serve-ui.py exits with its orchestrator, and the job section is parsed as
one brace group so `hermes update` rewriting posix.sh cannot shift the rest.
(cherry picked from commit c7dd9fab00b8af09321ec982d07a62f03351355d)
The protect-main ruleset requires a check by NAME, and a green check run on a
PR head never expires. After the code-health ratchet joined the required set,
32 PRs whose last CI run predated it merged on their old "All required checks
pass" green without the ratchet ever running; 20 of them would have been
blocked (62 findings, incl. 15 new blind `except Exception`).
Renaming the gate to "All required checks pass (v2)" and pointing the ruleset
at the new name voids every older green: each open PR needs one fresh CI run,
which now includes the ratchet, before it can merge, whoever merges it.
The ci.yaml comment documents the bump as the standard step whenever the
required set gains a job every PR must pass.
live_comment.py keeps both names in its infra list (old runs still render),
and a test pins that the gate's current display name never shows as a job.
A committed update exits 0 and prints one whole "Update follow-up '<step>'
did not finish:" line per owed step (update_receipt.record_followup). The
hand-offs only reacted to the Desktop build line, so a gateway proven to
run old code (gateway_restart), a refused Windows resume, a lost
completion or a failed channel adoption ended as a plain
"Update complete.".
posix.sh collects the distinct owed steps from that line
(owed_followup_steps) and turns them into the manual result message
(DONE_NOTE, the only channel the main-based script has); a gateway_restart
adds the `hermes gateway restart` hint. windows.ps1 does the same through
its manual outcome. With #132354's add_warning protocol the same
owed_followup_steps output feeds one add_warning per step.
record_followup's docstring names the line as a protocol; a contract test
runs the real posix.sh parser and the windows.ps1 regex against it.
Review: kshitijk4poor 5411222454 regression 1 / open thread F6.
What: build_update_products acquires npm inside the "Node dependencies" attempt and, whenever the
desktop product was requested but not built, prints `Desktop app build owed: <step> failed`.
posix.sh matches that line instead of the truncated follow-up text; windows.ps1 now projects it as
a manual outcome (it had no projection of the current follow-up at all).
Why: the 500-char follow-up leads with whichever product failed first, and an npm acquisition
failure escaped un-named, so a committed update whose Desktop was not rebuilt said "Update complete."
Red: pre-fix source_build.py -> test_unbuilt_desktop_is_named_on_one_whole_line[both] fail; pre-fix posix.sh -> truncated-followup hand-off test reports manual=false.
Green: test_source_build.py 15 passed; test_desktop_update_gateway_flag.py 3 passed. windows.ps1 needs a native run.
Contract C3 keeps post-commit step failures at exit 0, but the parked stash is not a
step anything retries: the receipt records a user_action, finalizes partial and
hermes update exits 1 (#122557). The Desktop handoff reports an owed app rebuild as
an action ("could not be rebuilt ... hermes desktop --force-build"), and the pm e2e
follows A6 (dependency sync after the tree moved is an owed follow-up).
Every core spec drives open, resume and switch, yet changes under
apps/desktop/src/app/session/ skipped the suite on the PR; #132017
changed resume that way and main went red on remote-secondary.
The reuse receipt check in node-deps.mjs accepted a tree where every locked
directory existed but package contents had been stripped (partial cleanup or
snapshot restore). npm trusts the hidden lockfile and never repairs such
entries, so every 'hermes update' skipped npm ci, the TUI build died on the
missing esbuild manifest, and the stale receipt kept the loop closed (#128935).
Probe each locked entry's package.json — the same file product builders
resolve tools through — so a gutted tree falls back to a real npm ci.
Fleet telemetry shows built-in memory writes failing 34-47% and automatic
compression failing ~15%, and several merged fixes each target one cause
(#133038 for memory, #133054/#133040/#133051/#133032 for compression). Both
metrics only carried outcome, so the fleet could not tell which cause a fix
moved.
- hermes.memory.op.count and hermes.compression.count gain one closed-enum
dimension, failure_class; `none` on success.
- Memory: every refusal/failure return in tools/memory_tool.py and
tools/memory_tool_store.py names its class (no_match, ambiguous,
over_budget, would_empty, drift, read_failed, retry_cap, scan_blocked,
missing_content, missing_old_text, invalid_args, stale_entry, disabled,
staged, gate_refused) through a context variable; the result the model
sees is unchanged. Plugin providers report provider_error / exception.
- Compression: the attempt log's own failure_class, collapsed onto the
compressor's closed names (exception:<Type> / rollback:<Type> keep only the
prefix, anything else reads other, a classless failure unknown). Skipped
rows carry the skip reason, and the skip set is now defined once in the
contract.
- Package schema v4 / event schema v4; v3 field sets of both metrics stay
valid so pending rows drain. Doc table rows and the consent copy (CLI
setup and every Desktop locale's collectedActivity) name the new reason.
A Desktop built before the blobless fix keeps walking history during
`hermes update`. On a treeless (tree:0) checkout every walk downloads trees
again, and #129514 filled a 434 GB disk during the minutes of dependency work.
The one-time conversion (#133016) ran at the very end of the update, after that
window. Both update hand-offs (update_completion._prepare and an older
updater's _update_takeover.prepare) now run it first, in the bootstrap
interpreter, and the late call is gone.
Re-running install.sh / install.ps1 over a treeless checkout now converts it
too. The filter is recorded only after the refetch succeeds, so a failure is
retried by the next update instead of leaving a blob:none filter over a
history that has no trees.
The pack cleanup also sweeps tmp_* transfer files that killed fetches left
behind. They are more than an hour old, a storm leaves gigabytes of them,
and git itself only prunes them after two weeks. The docs gain the recovery
steps for a disk that the storm already filled.
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: Yuan Li <dskwelmcy@163.com>
skills.sh rows in the hermes-index carry the skills.sh slug as `path` and,
since mid-September, never a `resolved_github_id`: the scheduled index build
resolves paths AFTER the ClawHub walk, ~80 min into the job, when the 1 h
GitHub App token has expired. Every tree fetch 401s and the log says
"Resolved 0/20000 paths" (last good run 2026-09-15: 17917/20000). Installs
then work only where the repo dir happens to equal the slug, and everything
else is reported as a "stale index entry".
- build_skills_index: resolve skills.sh paths straight after the skills.sh
crawl, while the token is fresh.
- HermesIndexSource.fetch: when the indexed candidates miss, locate the skill
in the repo tree (slug-named dir, a lone SKILL.md in a generic dir such as
`skills/`, or the repo root) and install it under the slug, not as
`skills/`.
- _find_skill_in_repo_tree: a tree with exactly one non-root SKILL.md
resolves to it when no dir matches the slug (fails closed on 2+).
- GitHubAuth: once the credential chain came up empty, stay anonymous for the
instance instead of re-running `gh auth token` before every API call; with
gh installed but not logged in that cost up to 5 s per request and pushed an
anonymous install past four minutes.
Fixes#130129
Based on #130135 by @dskwe.
publishDirectory moved a finished product into place with one renameSync.
On Windows an antivirus scanner opens every freshly written file, so the
rename onto build/native-deps (right after hud-modifier-monitor.exe is
compiled) or hermes_cli/web_dist failed EPERM and threw away a build that
took minutes.
retryHeld retries EPERM/EACCES/EBUSY for about 3s and raises every other
code at once. All three publication renames use it, as does the HUD
helper's staging rename. The previous product's backup now sits beside
the output instead of inside withProduct's scratch tree, so a rollback
that also fails no longer has its only copy deleted by scratch cleanup;
the error names where it is.
Co-authored-by: dskwe <6286305+dskwe@users.noreply.github.com>
Co-authored-by: juliorobertoalvarez <39749815+juliorobertoalvarez@users.noreply.github.com>
Co-authored-by: bighamx <10262931+bighamx@users.noreply.github.com>
Co-authored-by: lzmuhioin <1814826+lzmuhioin@users.noreply.github.com>
Installers have cloned with --filter=tree:0 since c3e9ac87a7. A treeless
clone keeps commits only, and git requests a missing tree without telling
the server which trees it already has, so every checkout and every
path-filtered history walk re-downloads complete directory snapshots
(~0.58 MB per commit for this repo). GitHub's own guidance is that
treeless clones are for throwaway CI builds, not long-lived checkouts;
the blobless layout keeps every commit and tree local and fetches only
file contents on demand.
- install.sh / install.ps1 clone --filter=blob:none (same release tags,
--commit pins and ancestor checks the treeless switch was made for).
- hermes update (post-update step) converts an existing tree:0 checkout once with
`git fetch --refetch --filter=blob:none origin` (~120 MB, ~8 s
from GitHub) and records the new filter only after it succeeds, so a
failed refetch retries next update.
- The shallow-heal conversion unshallows as blob:none instead of tree:0.
- checkout_contains asks git with GIT_NO_LAZY_FETCH: a recorded SHA the
checkout lacks is "not contained", never a download of that commit's
whole history.
scripts/ci/check_agents_md_size.py (in scripts/check and lint.yml): root <= 12k, every
AGENTS.md plus its ancestors <= 30k (the joined root->cwd block on a 128k window, minus
the per-file labels), every area file <= _MAX_HINT_CHARS read from the source so the two
cannot drift. On origin/main it reports the root and all 11 root-to-area chains over.
The root file is project context and was 38.7k chars, past the context-file budget
(20k floor, 30,720 on a 128k window), so every session lost its middle: routing table,
testing rules, code-shape rules. It now keeps the two invariants, a condensed rubric
(every rule kept), environment, structure, one-line rules that apply everywhere and the
routing table (11.8k).
Moved verbatim to where they load: Testing -> new tests/AGENTS.md; dependency pinning and
PM ownership -> new pm/AGENTS.md; machine facts -> new hermes_platform/AGENTS.md;
session-scoped tool availability -> tools/AGENTS.md; the full rubric with its real-close
examples -> developer-guide/contributing.md. Pointers in lint, checker messages, docs
and test comments name the new homes.
On a case-insensitive filesystem (Windows, macOS) `scripts/check --staged`/`--commit`
and the hooks' judge() aborted with "checkout-index -a failed: pkg/Notes.md already
exists" when the judged tree held two paths that differ only by case, so the
case-collision guard never got to report them (exit 2 instead of FAILED:
case-collisions). Both snapshots now check out with -f into their empty temp dir; the
guards read the private index, which keeps both paths. Found on the Windows runner by
the case-collision test once that test staged its premise as blobs.
The pre-push hook judges each pushed branch tip, as CI judges the PR as a whole, not
every intermediate commit; CONTRIBUTING.md and the hook comments said "each commit you
push". CONTRIBUTING.md also said a moved function you lightly edit keeps its cap; that
holds only when no function of that name exists at the destination, and a rename plus
an edit is new code. Both limits are now stated.
bodyWithoutSelf treated a declaration of the function's name at any depth as a rebind,
so a renamed recursive function containing an unchanged `.map((legacy) => legacy)` kept
its old self-call text, no longer matched its base body, and lost its cap ("CC 22 >
target 20 (new code)"). Only the function's own scope (parameters, a named function
expression's name, declarations outside nested functions and classes) now stops the
normalization, and nested functions that bind the name keep their own reads, as the
Python measurer already does.
_Binder.visit_Global marked the name ambiguous in the module, so `global execute` in a
function hid `from subprocess import run as execute` and HX006 (missing timeout) never
fired on `execute(cmd)`; nonlocal did the same for an enclosing function's import. A
declaration selects the outer binding, it does not rebind it: declared names now resolve
in the scope they point at, and only an assignment (or import) through the declaration
rebinds that outer name.
The hooks' judge() built its private index and tree under `mktemp -d`, a POSIX /c/...
path in Git Bash, and passed it to native git.exe and python.exe through GIT_INDEX_FILE,
--prefix and the checker path. Hermes' Windows terminal exports MSYS_NO_PATHCONV=1 and
MSYS2_ARG_CONV_EXCL=*, so those paths arrived unconverted and the hook died
("Unable to create '/c/.../index.lock'") before any check ran. judge() now hands native
tools the `cygpath -m` spelling when cygpath exists; elsewhere nothing changes.
Measurer.measure wrote each blob's decoded text into the temp tree with write_text's
default newline translation. On Windows that turns a CRLF blob into CR CR LF, so ruff
counted lines differently from the AST and comment tables: hits moved to <module>,
their `# health: allow` waivers were missed, and valid code blocked. Writing with
newline="" keeps the blob's own line endings. The test forces the Windows write
translation so it also runs on Linux.
A manifest row whose range could not be built (base "", head "error: ...") or replayed
(an OID the clone does not have) was counted in the "N of M PRs had a blocking finding"
denominator with no findings, and the run exited 0, so a replay that measured nothing
read as evidence that a rule never fires. Such rows now carry an "error" in replay.jsonl,
stay out of the totals and the denominator, are listed by number, and fail the run.
Files past 2,000 lines stay strictly no-growth (tests included). The message now says how
to offset growth in the same PR: move an existing function into a topical sibling (moved
code keeps its cap) or put new tests in a new test file. Probed on a 2,128-line file: plain
growth blocks; the same growth plus moving a CC-22 function out passes; a new test file
passes; growing the old test file blocks.
HX001, HX003, HX010, HX012 and PS-P05/P06 now print as advisory findings instead of
blocking. Reviewers' samples put them between ~12% and ~60% precision, and the plan
ships weak rules as warnings first. Promote one by flipping it to blocking with replay
evidence in the PR. The ENFORCEMENT comment now says where the switch is read from.
Tests that used PS-P05 as a stand-in blocking violation now use a blind except.
compare.py's cross-file move matching slices the origin function by its end line;
without it the engine estimated the end from the line count. Pass ast's end_lineno through.
No behaviour change: line wrapping in the hook text, parse_switch docstring,
commit_or_none and the new tests; the test helpers lose an unused env parameter.
JoaoMarcos44 (inline scripts/check:40): the hooks began with
`[ -f "$repo/scripts/check" ] || exit 0`, before reading the pushed SHAs, so the
checked-out branch decided whether another revision was checked. Pushing an
invalid checker-containing branch while an older branch was checked out succeeded
without judging it.
Eligibility is now decided from the candidate tree: pre-push runs
`git cat-file -e <local_sha>:scripts/check` per pushed ref (a candidate that
predates the checker is skipped, deletions are still skipped), pre-commit does the
same for the index tree. An eligible candidate is checked out alone into a temp
dir (private index + checkout-index; the worktree, index and stash are untouched)
and judged by its own scripts/check with GIT_DIR pointing at the repo - with the
previous commit's snapshot-bound health job, implementation, job table, config and
policy all come from the judged artifact, whatever is checked out. pre-commit
exports an absolute GIT_INDEX_FILE because git may hand hooks a relative one.
unsupportedpastels M5: both hooks ran whatever python3 came first on PATH, but
scripts/check imports ruff_runner, which imports tomllib (3.11+, the project
minimum). With Ubuntu 22.04's python3 3.10 every hooked commit or push died with
"ModuleNotFoundError: No module named 'tomllib'".
The hooks now use the repo's virtualenv interpreter when there is one (.venv/ or
venv/, bin/python or Scripts/python.exe; this worktree first, then the main
checkout) and fall back to python3, then python. scripts/check checks
sys.version_info before importing the engine and exits 2 with a one-line message
naming the virtualenv instead of a traceback. Verified with a real CPython 3.10.21:
exit 2 and the message, where 96bb97a crashed in ruff_runner.
F20 (andrexibiza): in --staged/--commit mode the guard jobs ran inside the
snapshot root with the snapshot's git env, but the health job ran in the real
worktree. Its source was pinned to the index tree, yet its engine and policy were
not: Measurer reads scripts/ci/profile_scope_patterns.json from the repo it runs
in, so with an identical `git write-tree`, editing only the unstaged P05
path_regex turned PS-P05 / exit 1 into "ok" / exit 0 and back.
The health job now runs like every other job: cwd = the snapshot root with the
snapshot env, so `python -m scripts.code_health` imports the snapshot's own engine
and reads the snapshot's policy files and ruff pin. The ruff job resolves its pin
from the same root. Unstaged edits to the policy file or the engine no longer move
the verdict, and they stay untouched in the worktree; plain runs are unchanged
(root is the worktree).
unsupportedpastels m4: scripts/check says it runs the lint checks CI runs, but CI
also runs scripts/check-case-collisions.py, scripts/ci/check_lazy_deps_imports.py
and scripts/ci/check_profile_archive_boundary.py in workflows of their own
(case-collision-check.yml, lazy-deps-guard.yml, profile-artifact-check.yml). A
case-colliding pair passed scripts/check and failed CI.
All three are deterministic and local (git ls-files / an os.walk of the checkout;
measured here: 0.2 s, ~7 s, 0.1 s on the full tree), so they run as ordinary guard
jobs - in the snapshot root for --staged/--commit like the other guards - and are
selectable with --only case-collisions / lazy-deps / profile-archives.
unsupportedpastels m3: (a) the switch was found by a regex that accepted only the
exact `ENFORCEMENT = "..."` spelling, so `ENFORCEMENT: str = "off"` silently stayed
blocking; (b) local runs and both hooks read it from the merge-base with
origin/main, so a flip on main reached a branch only after a rebase, contrary to
"CI and local hooks alike".
(a) parse_switch() reads the module-level plain or annotated assignment with ast
(last one wins, as at runtime); missing, unparseable or unknown values are
blocking. (b) Without --base, the base is still the merge-base of the head (a
--head commit, else HEAD) but the switch comes from origin/main when it resolves,
falling back to the base. An explicit --base pins both (CI passes HEAD^1, the
target tip). scripts/check --commit now hands the commit itself to the ratchet so
it derives that commit's merge-base. The head under test never supplies the switch.
(c) A test pins enforcement() on this tree to config.ENFORCEMENT.
F21 (andrexibiza): PS-P05/PS-P06 ran their regexes over raw source lines, so a
comment such as `# Avoid env = os.environ.copy(); use the scoped builder.`, the same
sentence in a docstring, or a comment warning against `os.getenv("DISCORD_TOKEN")`
produced a blocking hit although nothing builds an env or reads a credential.
Before the regexes run, comment tokens (tokenize) and string statements (ast:
docstrings and bare string examples) are blanked in place, preserving every line
number and line break. Ordinary string literals inside code are kept, so the
env-key argument of `os.getenv("DISCORD_TOKEN")` still matches; the real
`env = os.environ.copy()` / `dict(os.environ)` / platform getenv stay flagged. If
tokenizing fails, the raw lines are used (more hits, never fewer).
banditburai (inline report.py:36): `# health: allow BLE001 S110 -- boundary` waived
neither rule and reported "allow comment ignored: it needs a reason", although the
reason is there. _ALLOW accepts spaces in the rule list, but apply_allows split it
on commas only, so "BLE001 S110" was looked up as one rule name.
The rule list is now split on commas and whitespace, so `BLE001 S110`,
`BLE001, S110` and `BLE001,S110` all waive both rules. A reason is still required.
