Why (cost, same concern as Adolanium's line-matching note): pairing repeated names by body is
all-pairs within a group, and the per-group lookup of base units scanned the whole base file,
so a codemod touching every function of a big file was quadratic (5,000 edited unique names:
5.4 s -> 0.04 s with a per-file name index). A group whose all-pairs check would exceed
250,000 pairs (e.g. hundreds of edited `it(...)` callbacks in one scope) falls back to the
ordinal pairing, which is bounded; measured 2,000-callback group: 0.05 s.
Why (M2 follow-up; invariant 4, no trading): the previous commit let a moved+edited unit's
new-looking hits spend any departed occurrence of its origin unit by line text, so a swallow
dropped at the top of `legacy` paid for one added at the bottom of the moved copy (exit 0;
red on the previous commit), while the same relocation in place blocks. A unit matched
across files (pass 5) now compares its lines with its origin unit's (SequenceMatcher, code
only), exactly like an edit in place: hits on unchanged lines are old, the rest are new.
The departed-occurrence pool is now module-level only.
Why (M1 follow-up; invariant 4, no trading): matching a departed module-level occurrence by
its line alone let `except Exception:` stand for any guard, so deleting
`try: import json / except Exception: pass` from one file paid for a new
`try: import yaml / except Exception: pass` in another (exit 0; red on the previous commit).
A module-level occurrence now moves only with its whole top-level statement unchanged
(found by indentation: a whole try/except block, or a one-line constant). Moving an unchanged
HX005 module constant to another file passes; a different constant still blocks.
Hits inside a unit matched across files keep line matching: only that unit's own departed
occurrences can pay.
Why:
- banditburai (inline, py_structure.py:97) and unsupportedpastels M3: repeated names (Python
`@method(...) def _`, `f#2`; TS `<anon>#n`) were paired by source-order ordinal, so a new
CC-22 `_` inserted above a trimmed one took its cap (exit 0), and inserting a small `_`
above an unchanged CC-22 one made the old one "new" (exit 1). Same-file name matching now
groups units by name without the ordinal and pairs the most similar bodies first
(difflib ratio >= 0.6 over code lines, comments/whitespace dropped); the last unit left
with a name is that name's unit, exactly as a unique name was before. Exact-hash passes
still run first. If a body span is unknown, the group falls back to the ordinal.
- unsupportedpastels M2: cross-file matching was body-hash only, so a function moved to
another file with one constant changed lost its cap (P1 exit 1; same edit in place exit 0).
New last pass: another file, same name, similar body (>= 0.6), only when the name is gone
from its origin file and exactly one unit of that name left and exactly one arrived
unmatched. A copy (origin keeps it), two arrivals, or an unrelated body reusing a deleted
name stay new code (andrexibiza: never arbitrary same-name credit). Its existing hits
follow it through the departed-occurrence pool (one-to-one), a second swallow is new.
Unit gains an optional end_line; until the measurers report it (py_structure: lane A,
measure.py: lane C) the span is estimated from the unit's own FUNC_LINES. Tests pass both ways.
Why (round-5 review, unsupportedpastels M1; Adolanium cost note):
- M1: code outside any unit is not a unit, so its hits could survive only through the line
diff of a file against its own base (or git-detected rename). Moving an unchanged
module-level `try/except Exception: pass` into pkg/a_compat.py reported S110 + BLE001 as new
(exit 1). Now a base occurrence whose line LEFT its file may pay, once, for an identical
occurrence (same rule, same normalized code) arriving in another file in the same diff, at
module level or inside the unit its own unit was matched to. One-to-one: the spent
occurrence no longer counts as credit at its origin. A copy (origin line survives) still
blocks; two arrivals for one departure block; a re-add in the same file stays new; a
module-level hit never pays for one inside a function.
- Adolanium §3: SequenceMatcher ran for every changed file, TS included (1.66 s on desktop
main.ts for a one-line change); line matching is now skipped when neither side has hits.
Fourth review round (F11-F18, N1-N4, kvnloo, Adolanium):
- Hits are old only where the base occurrence survives: a line the comment- and
whitespace-blind line diff keeps (blame semantics), or anything inside a unit whose
body is unchanged. An identical violation moved past unrelated code is new; removing
an allow comment or re-indenting is not. Hit fingerprints exclude comments (N4).
- Inline allows cover their own line; only a comment-only line covers the next (N3).
- Rules read a copy of the AST with import aliases resolved, so `sp.run`,
`from subprocess import run as execute`, aliased threading/asyncio/load_config are
seen (F12); deadline helpers must resolve to the real asyncio API, and
communicate()/wait() are judged only on receivers holding a spawned process (F13);
infinite deadlines are unbounded (F14).
- HERMES_* writes introduce a name too (N1); bool(os.environ[...]) is HX010 (N2);
HX009 needs the isinstance subject to come from gather(return_exceptions=True).
- Recursive self-references are normalised out of body hashes, so a recursive rename
keeps its cap, py and ts (F15). Decorators belong to their function's scope and
classes are matchable units, so their hits follow proven moves (F16).
- An over-cap file may not grow (F17, Teknium: strict improvement-only).
- A file the checker cannot parse or measure (ast/ruff/TS parse error) fails.
- scripts/check: --staged always lints an index snapshot (ignored files are invisible
to git status, not to guards) (F18); create the --prefix dir for Git for Windows
(F11); --only builds just the selected jobs, so it needs no ruff for non-ruff checks.
- CODEOWNERS covers scripts/check + scripts/code_health; frontend-only PRs skip the
Python-only lint jobs again and run only the ratchet.
Second-round review on #132646 (F1-F10), each reproduced on 2531fb2 first:
- Moves are matched one-to-one: unchanged units are reserved first, then
same-file body, then cross-file body only when the origin's name is gone,
then same name. A copy of unchanged debt is new debt; a function moved
over a same-name stub keeps its own cap (F1, F3).
- Every base hit is owned by exactly one head scope, so a file rename no
longer counts each old hit twice (F2).
- Ruff runs when either side has Python, so a delete-only .py diff with a
TS edit no longer crashes on a missing ruff result (F4).
- Deadlines must be finite: timeout=None, **{} and asyncio.timeout(None)
are no deadline; awaited calls inside `async with asyncio.timeout(x)`
are bounded (F5, F8).
- HX005 walks only what runs at import: the import-time branch of a
`__name__` guard (either operator), loops, match, try*, decorators,
class bases; a lambda value is deferred (F9).
- `health: allow` counts only in real comment tokens (Python tokenize, TS
comment trivia), never in string literals (F7).
- TS function lines are owned by the innermost function, so siblings on
one line give it up once (F10).
- The pre-commit hook runs `scripts/check --staged`: health compares the
index tree, and when the working tree differs from the index the other
checkers run on a checkout-index copy with a copied index, so commits are
judged on the staged bytes and unstaged work is never touched (F6).
Independent review of #132646 found five defects, each reproduced and fixed:
- Pattern hits from any vanished scope went into a shared (rule, text) pool, so deleting one
`except Exception:` paid for an unrelated new one elsewhere, and class scopes (never units)
always looked vanished. Hits now transfer only along a unit matched by body hash.
- A unit's prior is now looked up by body hash before qualname, so a TS anonymous callback
whose source-order ordinal shifted keeps its cap.
- The TS body hash included the function name; it is now parameters + body, so a rename keeps
its cap (as in Python).
- HX006 flagged `await asyncio.wait_for(proc.communicate(), timeout=...)`; calls passed to
wait_for/timeout are bounded. The fix text now names the asyncio form.
- scripts/check printed output only on failure, hiding advisories and comment allows; the
health report is now always shown.
Docs now say growth of a file already over 2,000 lines is a warning for now.
scripts/code_health measures the files a change touches on both sides
(base revision and head) and fails when:
- a function or file over target grows past its own value on the base
(CC 20, 300 own lines, nesting 6, file 2,000 lines);
- a new function or file is over target;
- a pattern rule gains a fingerprint (rule + enclosing function + line
text), so fixing one violation never buys another.
Code that moves keeps its cap: functions match by name-independent body
hash, pattern hits by (rule, line text) once their function has left the
old place. Nested functions are their own units, so a closure edit never
counts against the hook or function around it. Python CC is ruff's C901
minus the direct nested defs; TypeScript is parsed with the lockfile's
typescript (no type check).
Rules: swallowed exceptions (S110, BLE001, E722), correctness (F821/F823/
F811/F841, RUF006), hangs (ASYNC230/240, S113, subprocess/urlopen without
timeout, sync config I/O in async def, get_event_loop, gather results
checked against Exception), profile scope (hardcoded ~/.hermes, a new
HERMES_* env var, UnscopedSecretError -> os.getenv, import-time capture,
raw threading.Thread, P05/P06 from profile_scope_patterns.json), argv
process identity, bool() of an env string, and if/elif ladders.
Growth of a file already over 2,000 lines is advisory: replaying 300
merged PRs, a blocking file-size ratchet would have stopped 46 of them
before the 119 over-target files are split. Everything else blocks; the
same replay finds 18 PRs that would have needed a change, every finding
a real hit for its rule.
No baseline file: base is measured on the fly, so an improvement that
lands on main is the new cap at once and parallel PRs never conflict on
ratchet state. `# health: allow <RULE> -- <why>` is the reviewed escape.
