A header cut short raised struct.error, which no caller of
read_gguf_header catches (they skip a model on ValueError or OSError),
so one truncated file escaped the whole preset pass. A short read now
raises ValueError.
The status row sized a side-loaded split by its first file, which can
be a metadata stub of a few MB; it now sums every part on disk. The
split fixture writes model metadata into part 1 only, as gguf-split
does: a published Flash Next split carries 67 keys in part 1 and only
its 3 split.* keys in parts 2-3.
A split model was priced by its first shard alone, so the physics check
and the residency cap admitted giants the card cannot hold: on a 256 GB
box a 175 GB model priced as 6.7 GiB and the router booted with room for
three residents, two of which then co-resided and OOM'd every session.
Sum the tensor table, the host-side embedding duplicate and the per-block
FFN map across every part on disk; architecture metadata still comes from
the first part. The truncated FFN map also disabled spill placement,
which is the lever that would have relieved the pressure.
A part that cannot be parsed is priced out rather than fatal — refusing an
incomplete split to serve remains staged_in's job.
The journey-home guard counted every non-<module> frame in plugins_cmd*.py as
the migration running plugin-install code. d6813a5dd7 gave
plugins_cmd_install.py its first class statement (_ConsentRefusal), whose body
runs at import as a frame named after the class, so a plain import (via
plugins_cmd.py's module-level import) read as a call and the guard failed with
no change in what the update does: migrate_all_homes() still returns [] and
calls nothing in the module.
Frames called by their own file's <module> frame (class statements,
module-level comprehensions) are import-time execution and are skipped; a real
call into an install function is still recorded.
Two ratchets this suite arms for its own siblings fired once they landed:
- the Windows pause (#132338) now imports gateway/status_inline_source.py
to recognise inline-bootstrapped gateways, so a change there starts the
e2e_upgrade suite like gateway/status.py (only two importers: not a hub)
- the marker-corpus consumers #132345 and #132354 ship are on main, so
_LANDS_IN_SIBLING_PR empties as its comment asks
The replayed steps call git. On Linux os.defpath reaches it; on Windows the
replay's PATH only reached it through the located bash's directory, and Git
for Windows ships a bash.exe in usr\bin with no git.exe beside it. The
Windows arm64 lane resolved that bash and 13 replay cases failed with
"git: command not found" (1e547a814c3 had just put them on the native
Windows lanes). git's own directory now rides along like bash's.
`hermes_cli/_early_recovery_zip.py` (split out of `_early_recovery.py` on the
commit-point PR) is imported by the update path, so a change to it must start
the update pipeline and install E2E suites. Both lists and the install-e2e path
filter now match the `_early_recovery` prefix, which also routes any later
sibling. `remote-lifecycle-v2-marker.test.ts` reads the update-marker corpus
and is now a listed consumer; `remote-update-marker-programs.test.ts` exists in
this tree, so its sibling-PR allowance is gone.
Replaying the last 131 non-merge main commits (4787e4d56f) through the
classifier: main selects e2e_upgrade 5 / e2e_desktop_update 3, the CI branch
12 / 10. Four of the seven new hits per lane touched only plugins_cmd.py or
plugins_cmd_install.py. migrate_all_homes reaches them only when a home's
configured memory provider left core (the catalog install); no upgrade or
Desktop update journey configures one, so those suites cannot observe the
change, and the unit lane already runs on every .py edit. Measured on a
fresh home: memory_provider_migration, pm/plugins_state, agent/memory_provider
and plugins/memory run; plugins_cmd_install executes only its module body,
plugins_cmd is never imported.
Kept, because a fresh-home update runs them: memory_provider_migration.py
(migrate_all_homes on every source build), plugins/memory/__init__.py (the
post-update Honcho profile sync), pm/install.py (sealed() /
lazy_installs_allowed() gate every update's default-tool install), and
scripts/build/ (frontend-common.mjs is imported by freshness/web/tui/desktop).
After: 8 / 6. A guard test fails if the journey-home migration starts running
plugin-install code, the signal to route those modules again.
test_crash_cells carried its own marker parser and identity rule, and it
decides the orphaned_update verdict. It disagreed with the product on three
inputs: a float started_at (product: malformed, copy: live), a create time
1.5 s off (product: live within its 2.0 s tolerance, copy: dead at 1.0 s) and
a run: line before delegate: (product: delegate live, copy: dead, it read
line 4 only). The self-check claimed parity but never called update_lock and
used the test's own pid, which the product judges by exact incarnation.
_marker_live now asks update_lock._parse_marker(...).owner_live() /
delegate_live(); the crash cells run in a pytest process on the same Windows
host as the marker's pids, so the product's process table reads them as the
updater does. The self-check marks a separate live process with an integer
started_at, compares the oracle to judge_marker, and adds the three rows.
The ORPHAN_RELEASE_GRACE poll is unchanged.
tests-os[windows] imports only the files list_os_marked_tests.py emits and
keeps `-m "platforms and not integration"` tests its host admits. The replay
regression for the Windows interpreter layout (venv holding python.exe alone,
Git Bash) carried no platforms marker, so the Windows selector omitted the
whole file and green Windows jobs never ran it.
The replay test, the R15 dispatch chain and the disabled/advisory/no-command/
empty-selection controls now carry platforms("linux", "windows"): Linux keeps
its coverage, the Windows selector lists the file (194 -> 195 files) and the
Windows -m filter collects 35 of its nodeids. macOS is left out on purpose:
the replay has not been run on bash 3.2. A routing test pins both gates via
the real selector and its spec semantics.
Round 5's Start-MarkerCustodian takes a killed hand-off script's claim over and releases it on its
next 1 s poll after the update ends. Strict run 37393368977 read the marker ~2 s before that
release (custodian removed it at 01:01:32) and called it a surviving marker. The cell now waits up
to 30 s for the release; a genuinely stuck marker lives until the heartbeat ceiling, minutes.
workflow_steps replaced PATH with the interpreter dir plus os.defpath. A
Windows venv ships only python.exe and os.defpath carries no Git usr/bin,
so replaying the ubuntu `gate-lanes` step (`python3 - <<PY`) died with
`python3: command not found` and the selection steps lost sort/tr/paste/find;
on Linux a missing python3 silently fell through to the host interpreter.
The replay now resolves bash via hermes_platform.resolver, adds that bash's
dir (Git usr/bin on Windows, already on defpath on Linux) after os.defpath,
and drops a disposable python/python3 shim to REPLAY_PYTHON only for a name
the interpreter dir lacks, mirroring setup-python putting both on PATH.
Production YAML is unchanged; on a Linux venv no shim is written.
update_cmd_commit._admitted_python now resolves python3.N through hermes_platform.resolver
(cd471c9c55a), so the update path imports the resolver package and a change to it must start the
upgrade E2E, like hermes_platform.host already does.
run_tests.sh runs one pytest per file and reports a file whose tests were
all deselected (pytest exit 5) or all skipped as passed. 8ce4ce35fc2 made
the -m selection and platforms('windows') marker a tests/ci check, but any
other zero-execution path (the HERMES_E2E_WINDOWS_INSTALL opt-in lost from
the step env, a new skip condition) still ran zero crash cells green.
The crash journey now writes crash-cells-ran.txt (each journey step and
ok/failed) into HERMES_E2E_ARTIFACTS once it has run, and a new required
step "Every crash cell ran" after the run step throws when the manifest
is missing. The manifest is uploaded with the other artifacts.
Test (tests/ci, every PR): the step exists after the run step, is neither
disabled nor advisory (workflow_steps.required), reads the same
HERMES_E2E_ARTIFACTS and names the module's manifest constant; and
_record_cells_ran writes exactly the steps that ran, nothing without the
env. Red on the start workflow (no step) and with the step made
continue-on-error.
Review 5411223855 (F52: collected > 0 / executed-cells assertion).
All six crash cells share one module-scoped machine. A cell whose
follow-up update failed (torn checkout, .hermes-update-in-progress or
.git/index.lock left, or a step that raised) handed that state to every
later cell, which then reported the first cell's breakage as its own.
_run_cells now runs the cells in order and, before each one, checks what
the previous cell left: the checkout exactly its target, no marker, no git
lock, follow-up rc 0. When it is not, the harness stops the machine's
processes, drops the lock and the marker and resets the checkout with the
harness's git to that cell's target (else to the HEAD it left), and notes
the restore and its reason in machine.timings, so the evidence of any
later failure names the culprit. The broken cell keeps its own verdict:
its result was captured before the restore. If the harness cannot reset,
the next cell fails "not run: <cell> left the shared machine unsound
(...)" instead of running on it. This is harness plumbing like the stale
index.lock clean-up _crash already did; every target only appends
statements to modules, so the venv still runs the restored tree.
Tests: a broken cell, a raising cell and a sound cell in sequence (each
later cell sees the reset checkout, no marker, no lock, and the timings
name the culprit), and an unrestorable machine failing the next cell
with the culprit's name. Both fail on the start sha (no _run_cells).
Review 5411223855 (crash-cell cascade).
The mid_git cell names an unmodelled git (the hold filter ran, but no
merge/reset/checkout with .git/index.lock appeared) after 30 s of wall
time. While the filter holds, git, its op and index.lock cannot move: the
state is static and the poll only has to observe it, but on a starved
runner a tree walk can take seconds, so 30 s could be a handful of looks
and a slow runner could false-fail the cell. The verdict now also needs
HOLD_SETTLE_LOOKS (60) complete looks; a normal runner passes 60 looks
well inside 30 s, so its behaviour is unchanged, and the wait is still
bounded by _kill_when's UPDATE_TIMEOUT.
Test drives _GitHold.point with a 10 s-per-look clock: the start code
raised on the 4th look (40 s), now it raises only after 60 looks.
Review 5411223855 (crash-cell timing).
What: scripts/ci/classify_changes.py names hermes_cli/git_pack_tidy.py in the update pipeline, so a
change to it runs e2e_upgrade.
Why: the derived routing guard (test_every_module_an_update_entry_point_imports_starts_its_suite)
found it imported by the update path and unrouted: main's partial-clone tidy (a0f47657b9) runs in
every update's pre-fetch via update_cmd_check.
What: _kill_when judges the kill by `_kill_delivered`: taskkill rc 0, or its stdout carrying the
update root's own SUCCESS line ("with PID <pid> (child process of ...").
Why: the update's git now runs inside the update's kill-on-close job. Killing the update closes the
job, a member dies while `taskkill /T /F` is still walking the tree, and taskkill exits 128 even
though it terminated the update (rc 1 is the code /F forces). The mid-fetch and tree-moved cells
then reported "the update was not running when killed" on the strict composite run (artifact
receipt: outcome interrupted, transcript stops at the fetch). The mid-git cells passed only because
their git is frozen by a hold. "Not found" for the root stays a refusal.
Red: test_kill_point_counts_only_an_update_killed_while_running[None-128-...SUCCESS...-True-True]
fails on the old rc-only check. Green: tests/ci + tests/e2e/core/windows_update 546 passed.
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
ORPHAN_MARKER_GAP excused the orphaned_update cell's marker assertion with no
expiry. PR CI never runs that cell (Windows, opt-in) nor strict acceptance,
so after #132354 (hand-off script names its update child as the line-4
delegate) and #132365 (the judge honours a live delegate) land, a regression
of exactly that contract would still pass as an expected failure.
Now the fix's footprint is read from the tree (ORPHAN_MARKER_FIX):
- while any half is absent, _orphan_gap_excuse applies known_failure as before;
- once every half is present the excuse is off, so the cell is strict on any
runner whatever the wrapper says;
- tests/ci/test_windows_update_crash_oracles.py (runs on every PR) fails with
"delete ORPHAN_MARKER_GAP ..." once the fix is in the tree, the same expiry
shape as _LANDS_IN_SIBLING_PR, and fails if the Python half already on this
branch (update_lock.delegate_live) loses its footprint, so a rename cannot
keep the excuse alive.
Checked against the scripts branch: this tree + its marker.ps1/windows.ps1
reads no half missing.
Review CI1.
_git_op and _update_child skip any child whose name/argv read raises. Both
catches were broad with no (or a one-word) reason. The skip is fail-closed:
an unreadable child is never counted as the kill point, and _kill_when's
bounded wait turns a point never seen into a harness verdict, never a pass.
Kept broad on purpose (psutil on Windows raises more than psutil.Error for a
racing child) and now say so with a BLE001 waiver naming the boundary.
Review CI4.
The stack's health refactors carved main_desktop_tcc.py out of main_desktop.py
and status_inline_source.py out of gateway/status.py; both are imported on the
update path, so an edit to either must start e2e_upgrade (and, for the desktop
module, e2e_desktop_update). Route each facade by prefix so its <stem>_*
siblings are covered without naming files this branch does not have yet.
The desktop gate's SSH-v2 marker fix added remote-update-marker-programs.test.ts,
which reads the shared marker corpus: list it as a consumer, allowed absent until
#132345 lands, like the other gate-PR consumer.
What: a tests/ci guard evaluates the workflow's own -m expression (windows-install-update-e2e.yml) against every test in tests/e2e/core/windows_update/test_*.py and requires platforms('windows').
Why: run_tests.sh reports a fully deselected file as passed with 0 tests, so dropping the crash file's platforms marker would silently run zero crash cells.
Red: mutant test_crash_cells.py without platforms('windows') -> [test_crash_cells.py] FAILED 'the workflow deselects it' (19 passed, 1 failed).
Green: 20 passed, 0 failed.
What: _orphan baselines <HERMES_HOME>/logs/update_receipts before the hand-off starts; the cell passes on rc 0, or on rc 120 only when the newest receipt written since says outcome success.
Why: CPython replaces ANY exit status with 120 when the final flush fails, so a failure exit after the success line read as a completed update.
Red: pre-fix cell -> [120-failed] and [120-None] accepted; receipt helper missing (9 passed, 3 failed).
Green: tests/ci/test_windows_update_crash_oracles.py 12 passed, 0 failed.
What: the test's _identity_live takes the marker age and, for an identity without a creation time, is live only within hermes_cli.update_lock.UPDATE_MARKER_MAX_AGE_SECONDS.
Why: the oracle read a ct-less (v1) marker for a live pid as LIVE past the product's 20-minute ceiling, so it could disagree with the verdict update_lock gives every other updater.
Red: pre-fix test_crash_cells.py -> [1260-False-False] FAILED (6 passed, 1 failed).
Green: 7 passed, 0 failed.
What: _kill_when requires proc.poll() is None after the point is seen and taskkill_tree's rc 0 (it now returns the CompletedProcess); otherwise a harness AssertionError naming the point.
Why: tree_moved's point (HEAD at target) persists after a clean exit, so the cell could pass on an update that was never interrupted.
Red: pre-fix _kill_when/_machine -> exited-0 + truthy point and taskkill rc 128 cases accepted (2 passed, 2 failed).
Green: tests/ci/test_windows_update_crash_oracles.py 4 passed, 0 failed.
What: the shared-fixture guard fails when a _LANDS_IN_SIBLING_PR entry's file exists ('sibling landed: delete its allowance').
Why: the allowance never expired, so after #132345/#132354 merge a later deletion of a listed corpus consumer stayed green.
Red: guard without the check -> test_sibling_allowance_expires_once_its_consumer_is_in_the_tree DID NOT RAISE (5 passed, 1 failed).
Green: 6 passed, 0 failed (-k 'sibling_allowance or shared_fixture').
test_crash_cells.py runs one module-scoped journey (install, the shared
gateway phase, six crash cells) inside a single file. It took 1,750-2,090 s
on normal runners (87% of the 2,400 s per-file budget) and was killed at
2,400 s on a runner that was uniformly 1.3-1.8x slower (run 37243653456:
32 passed, 0 failed, crash cells SIGKILL'd mid-journey). Raise the per-file
budget to 3,600 s and the job to 75 min so runner variance is not a verdict.
The launch repair's own entry module, unlinked by a merge held at git's smudge filter and
taskkilled with index.lock held. The next launch must restore the pre-update tree and run a
turn; the follow-up update must complete. Runs after mid_git: a launch that cannot repair this
tree leaves no runnable install for later cells.
(cherry picked from commit 1999a3868dc573dde1ec97745988823b9be1c30e)
(cherry picked from commit 232936de515c094292a56a9e2f6631457bc9d53d)
The strict corpus guard requires every listed consumer of the update-marker
corpus to exist. On this stacked PR the Electron reader lands in #132345 and
the two hand-off script readers in #132354, which merge before this PR, so
the guard went red here although the composite has all five.
Absence is now admitted only for consumers named in _LANDS_IN_SIBLING_PR
with the PR that lands them. A missing consumer no sibling lands, a fixture
whose every listed consumer is absent, and an allowance entry that names no
listed consumer all still fail. Once the siblings are on main the table can be
emptied and the full strict check applies unchanged.
(cherry picked from commit cfdfa6beeb4a1d8f17eb0ef7b078547b3c39cd5d)
(cherry picked from commit 3fb57d4a4e8805da89338ac0f469924171bcc0d1)
Route the exercised subprocess and provider-migration dependencies to both
real-update journeys. Keep backup routing under an explicit dispatch test.
Use the aggregate's existing detect outputs to reject skipped or missing
update consumers while retaining legitimate unselected PR skips.
Verified with the real classifier and aggregate subprocesses against
b9dec76a042, composite ff5c8f1a64b, and fixed code. Canonical runner:
571 passed, 0 failed, 1 native-Windows skip across CI and marker corpora.
(cherry picked from commit a082ca3c62ed49382f173a07882d4280d0ba7c9c)
(cherry picked from commit ed9f2028256b8b1d6af81ef97cd472f30d2cb042)
(cherry picked from commit 6d5cc740b6a75d476802222c2f4754855a5712b0)
Execute gate-lanes and safe selection scripts instead of assuming copied outputs or counting reachable jobs. Check required step conditions, advisory flags, matrix cells, real Windows file enumeration and corpus graph existence. Keep pytest and Playwright at argv-receipt boundaries; no native suites are launched.
(cherry picked from commit d4da1a0266bf2a1ad12309fda35d24302b6b7106)
(cherry picked from commit a295255e8901c57d0c8a768c7b65677553e4f433)
(cherry picked from commit 4fcb270f842c26f05420c2bcfd2d8b69a2f42fa9)
(cherry picked from commit 8c22ecbd465f80771eeb6d87ddea2be21e06ad55)
(cherry picked from commit a5ff463ce85b2b372661856525aff27d230f9226)
(cherry picked from commit 9f7a4d177929ca32cf1d25afd32dbb5005338db0)
The known_failure on the mid-merge launcher gap is gone: #132361's launchers import
hermes_bootstrap (the interrupted-pull repair) before any other checkout module, so the
merge-time kill must leave a runnable install with no gate.
(cherry picked from commit 96fd6724a86aa9b48ac7f88b56230e94f3447d7e)
(cherry picked from commit f975552309b97b7b2e4ac97b63ad278d185e2369)
The shared-fixture test also failed when a listed consumer was absent while
the fixture was present. The batch lands them in separate PRs: this branch,
stacked on the commit-point lane, has the corpus and the pytest/cargo
readers but not the Electron or hand-off readers, so the PR's own Python
lane went red. Routing a path that is not in the tree yet costs nothing;
the check that loses coverage (a reader in the tree the table omits) stays.
(cherry picked from commit 8d387483ccc8a20cb82ff4b60a97c747184aab9c)
(cherry picked from commit 24518bc30e8d67af6e165f1a59d28a5f6a2978bf)
Native run 37190464303 (first merge-time kill): git merge was held writing
hermes_constants.py with index.lock held and killed there. Every next
launch then died with ModuleNotFoundError: the launcher
(hermes_cli/_launchers.py::_launcher_script) imports hermes_constants from
the checkout before import hermes_bootstrap, whose
restore_interrupted_pull is the repair, so the repair never runs and even
hermes update fails. The bricked install then failed every later cell.
mid_git now runs last, and its recovery assertions sit under a
known_failure keyed on exactly that traceback (owner upd-txn: strict
acceptance fails on it). The hold precondition (index.lock left by the
kill) is asserted outside the gate.
(cherry picked from commit 7aca86f2341f6135b4da2666d58e24e3e14b7945)
(cherry picked from commit 95595e12f6a0e6ddc022a43e871c8e654149fbbe)
Every green native run killed the mid_git cell during git fetch (fetch was
in GIT_OPS and is the long step), so the merge-time index.lock schedule the
cell is named for never ran. A smudge filter the harness names in the
install's .git/config + .git/info/attributes (git's own filter hook; no
product change, no env var) now holds git merge while it writes
hermes_constants.py: index.lock held, the cell's new file and
hermes_cli/main.py already at the target, hermes_constants.py unlinked.
The kill point requires the hold, a merge/reset/checkout git child by argv
and index.lock; the cell asserts the lock was left. The fetch-time kill
keeps its own cell (mid_fetch).
(cherry picked from commit 592d57c3a9f78e04d617be659342e5338e080381)
(cherry picked from commit ba54f871bd580bb3fe9de7f840a7a21baf564dd2)
tests/fixtures/update_marker_corpus.json is read by pytest (update_lock),
cargo (marker_tests.rs), vitest (update-marker-corpus.test.ts) and the
marker.sh / marker.ps1 corpus tests. By path it selected only python, so a
corpus edit never ran the Rust, Electron or Windows PowerShell readers.
_SHARED_FIXTURE_CONSUMERS routes a fixture with its consumers' test lanes;
the routing test derives the consumers from the tree and replays the
classifier through the workflow files to each lane's job (review D14).
(cherry picked from commit 4893ef31833b5fcbdfc527beb70724ea63a3eb8f)
(cherry picked from commit 20c04a27b2a04ebaabb9a9df3693339243d9380e)
main added hermes_cli/left_core_migration.py, imported by source_build.py; the
R15 import-graph test caught it unrouted on the composite of all seven PRs.
(cherry picked from commit 338aabcdda316a92b8bfcae10f52730fd6fcb344)
(cherry picked from commit 5a839bb475d08f82293f07bcdb60ded859814476)
Moving the marker known_failure after the other assertions exposed a
harness race that the early XFAIL had hidden. PR run 37180721655 failed
the orphan cell with "did not finish (rc=None, 'Update complete!'
logged=True, tree at target, clean)". psutil.Process.wait returns None
for a non-child that has already exited. The orphaned update exits
between two 0.25 s samples, so its rc was lost.
_ExitCode opens PROCESS_QUERY_LIMITED_INFORMATION|SYNCHRONIZE on the
child before the kill and waits/reads GetExitCodeProcess through that
handle. The exit code stays readable after the process is gone, and a
reused pid cannot answer for it.
(cherry picked from commit 85488842f43b2154340231e2dd1a5e7900622487)
(cherry picked from commit 05c47385379ecda49e717e517425f5a9f559d76e)
- _assert_recovered compared HEAD SHAs only. A tree with HEAD at the target
and half the files from the old commit passed. Each target now appends a
statement to two modules every launch imports (hermes_constants.py,
hermes_cli/main.py), and right after the first launch the checkout must be
exactly the pre-update commit or the target: HEAD, `git status
--porcelain --untracked-files=no` clean, `git diff --quiet HEAD`, and the
target-only file present iff HEAD is the target. Same after the follow-up
update (target only) and after the orphaned update.
- The orphan cell's two known_failure blocks came first, so pytest.xfail
ended the test before its finish / first-launch / follow-up assertions.
Every unrelated assertion now runs first; the marker contract is one
block at the end (strict acceptance fails it: upd-txn owns the gap).
- _orphan took its completion-banner baseline after taskkill + wait, so a
completion logged during those waits counted as "not fresh". Both
baselines are taken before the script starts.
Linux probe of the oracle (real git, minted target, torn tree):
.../r5-ci/tree_probe.out — torn tree (HEAD=target, old hermes_constants.py)
matches neither commit; the old SHA-only check accepted it.
(cherry picked from commit 72308a85965635d9b746e52eccf35d91fcca75cf)
(cherry picked from commit 11df688a5831f2ed2e4a88b6a1fed5e7c9234a02)
known_failure XFAILs a gap while its fix is an open PR, which is right
on a PR and wrong on the acceptance run of the integrated batch: there
it accepts exactly the gap the batch claims to close (the final
composite's Windows run passed with `test_crash_cells.py (3✓ 1xf)`).
HERMES_E2E_STRICT_ACCEPTANCE=<owner>[,<owner>] (or 1 = every gap) makes
the matching known_failure / known_gate blocks re-raise the gap's own
assertion. It is set only from the `strict_acceptance` dispatch input of
ci.yaml (threaded through tests.yml / tests-os.yml to every E2E step)
and of windows-install-update-e2e.yml; pull requests never set it.
run_tests.sh forwards it through its env -i allowlist.
Red on 8cc78279a36: test_pending_fixes_strict.py (a real pytest process
per mode) — the batch-owned gap XFAILs under strict mode.
(cherry picked from commit ca1e7e749a61263181e89aa687dec366d95fda18)
(cherry picked from commit 7768bf3ef8e15fb7f228ebad619d6a9a85ad4100)
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
Run 37149266852 (delegate writers merged): the marker stayed LIVE via the delegate,
the update logged "Update complete!", its receipt says success and the checkout is at
the target, but the process exits 120: CPython's status when the final stdout flush
fails, and its stdout reader is the killed script. Require the checkout at the target
plus a fresh "Update complete!" in update.log, and rc 0 or 120.
(cherry picked from commit d9d965fc1e6f1ab3f5bcdfafe044e1439cf7d3da)
(cherry picked from commit e0ef639ebcc797bd4ff8f28ec1d6d8fd9d3cddcf)
Run 37148409574 (with the delegate writers merged) killed windows.ps1 0.4 s after its
`hermes update` child appeared, before that child reached UpdateLock.acquire(), so it
measured python start-up instead of the delegate contract. Wait for the child's
"Updating Hermes Agent..." banner in update.log (printed only after acquire) too.
(cherry picked from commit 274861c4ac40802afa03389a668d08e9c568d92c)
(cherry picked from commit 1f4d1ecd724285d00f898603f85fbf491b00900f)
- mid_git: the xfail is gone; #132361 (this branch is stacked on it) fixes the stale
index.lock + bare-git interrupted-pull repair.
- orphaned_update: kill ONLY windows.ps1 (no /T) once its `hermes update` child runs
under the claimed marker; the update must finish, the marker must read LIVE (owner or
line-4 delegate, C1) for as long as it runs and be gone after. The marker assertions
are a message-gated known_failure until #132354 + #132365 land the delegate line.
- _marker_text parses the line-format marker (it ran json.loads: always <unreadable>).
- tree_moved reads HEAD from the ref files instead of spawning git rev-parse every poll.
(cherry picked from commit aa8a8734785fdcc2d7fe958ab15ae002afdd12d9)
(cherry picked from commit eaf540f555cc936e798edc84ee1fd18fa802beac)
Cell -> kill point -> test -> fixing lane, and when the real-update suites
are required on a pull request.
(cherry picked from commit 9a8d1406641ee1a66e755267677c2b029a38f067)
(cherry picked from commit 05d543c6a6e4da288939ab44b198cd3dd18a6f9a)
wine2e run 37139409703: the mid-git kill leaves .git/index.lock that gitlock only
sweeps after 10 minutes and the launch-time interrupted-pull repair fails with
WinError 2 on a machine whose only Git is the installer's copy, so the next
update refuses. That cell is xfail until LP-COMMIT lands; the later cells now
remove the earlier cell's lock the way the refusal tells the user to.
(cherry picked from commit f711518268fe3b94c3a08772ddfc6e1e7fc5f46f)
(cherry picked from commit 76263c968bfa589300679f3c9417062a23bb3ba6)
