install-e2e.yml only ran on the clock, so nothing in front of a merge
installed a release and updated it on a real OS. A pull_request trigger,
path-filtered to the install/update surface (derived from 60 days of
update/install/pm commits), runs the new `pr` route of
generate-e2e-matrix.mjs with only the newest release tag sampled:
linux installer-script -> hermes-update (newest release -> PR)
linux installer-script -> hermes-update (PR -> NEXT)
windows installer-script -> hermes-update (PR -> NEXT)
macos installer-script -> hermes-update (newest release -> PR)
The bundle-manifest validation job is skipped on PRs (bundled legs need
dispatch-only manifests). The full matrix stays on schedule and release.
route already means "which legs run"; a leg name is the most specific
route. Presets keep their meaning (all, the linux trio, windows-desktop,
macos-desktop, the bundled three); any other value selects legs by name,
so a leg name, a fragment of one, or the job name GitHub shows
("<leg> / e2e") runs just those legs. A route that selects nothing fails
instead of producing a green empty run.
The generator is now the one interpreter of route for source legs: the
linux/windows/macos jobs run when it selected legs for them, replacing
three hand-kept preset lists. Dispatch route becomes a string input (a
choice cannot take a leg name) and reaches the gen step through env, never
interpolated into the script.
Every leg installed a release tag and updated to HEAD, so nothing ran
HEAD's installer on an empty machine (where all four 2026-09-23
install.ps1 breaks lived) and nothing exercised the updater we ship
today -- tag legs run the OLD build's updater handing off to HEAD.
The generator appends a HEAD -> NEXT start after the sampled tags, so the
column runs wherever the update legs run (dispatch and stable-release).
NEXT is a reserved update ref: the drivers mint a child of the install
commit that adds one marker file, written to the object store only, which
the local bare clone carries into serve.git.
On Windows the HEAD leg takes every git.exe dir off PATH and installs no
remote get-url shim: Get-PinnedGit returns any git on PATH, so either one
skipped pinned-git staging. launch-from-spec's HEAD observer now uses the
driver's real git so it cannot poll '' forever on that leg.
Keep unknown failures red, rotate evidence per attempt, and emit receipts for signature-confirmed historical cases. Add CI-only diagnostics and an exact-tag input for the unresolved July hand-off.
Exercise a real child writing beyond pipe capacity on success and failure. Both tests fail on upstream/main and pass with the merged stdout routing. Annotate the matrix rank input for TypeScript 6 checkJs.
tag-count now reaches the shell via the environment, validated to 1-10
(an apostrophe in the raw interpolation could terminate quoting; above
~14 tags the expansion exceeds GitHub's 256-job matrix limit).
Result-chart cell ranking matches on the leading token: rendered
success/failure cells carry artifact links, so whole-cell indexOf
ranked them -1 and any skip in the map beat a real outcome.
README documents per-run cost, route slice sizes, the tag-count bound,
and a warning against running the GUI drivers outside a disposable VM.
Windows transcripts were ZERO bytes: ts-prefix.ps1 formatted with
{0:D2}, but Floor() returns a double and the D specifier is
integer-only - it threw per line, and under the driver's relaxed EAP
every line errored into the void. {0:00} fixes it (custom numeric
format works on doubles). Reproduced the exact pipeline locally
(empty file + Format specifier invalid), verified the fix produces
prefixed merged stdout+stderr with exit code intact. That is also
why the log timeline never auto-synced: there was nothing in the
files to sync.
The GitHub artifact URL 307s to /suites/... server-side and strips
the ?zip= query param. The player now reads the zip URL from a
#zip= HASH param (client-side, survives the redirect) with ?zip=
as fallback; the hash path was verified in a real browser against a
real leg zip (auto-fetch + boot).
Per ethie's design, one player artifact for the whole run: new
leg-player job uploads playback.html (archive:false) before the
matrix legs, the report job needs it, and each ran cell gets TWO
links - 📼 to the player with #zip=<that leg's logs zip> and ⬇️ to
the raw zip. Per-leg player uploads removed from all three run
workflows.
Debugged against run 31635036702 (real jobs + artifacts replayed
through the renderer). Two naming realities the renderer ignored:
1. upload-artifact with archive:false IGNORES the name: input and
registers the artifact under the FILE's basename - every leg's
player artifact is called 'playback.html' (all the same blob). The
renderer now uses any 'playback.html' artifact for the player half
of the 📼 link instead of install-e2e-player-<leg_id>.
2. The posix arms append -<sha> to the logs artifact name at upload
(install-e2e-logs-<leg_id>-<sha>), while windows does not. Match
by prefix instead of exact name.
Also fixed the pass/fail summary counters, which compared cells with
=== against '✅' - the appended reel link made every ran cell
count as neither passed nor failed (the summary said 0 passed while
the table was full of checks).
Replayed against the real run: 31 passed, 25 failed, 149 skipped,
24 📼 links on ran cells (both outcomes), none on skips.
Each leg uploads playback.html as a single-file artifact (archive:
false) before the driver runs, so it exists even on failure. The
results chart now links every leg that RAN (pass or fail, not skip)
to its player with ?zip= pointing at that leg's logs artifact.
Leg<->artifact mapping: the generator mints a leg_id per matrix entry
(sanitized matrix name, exported legId()), every run workflow names
its artifacts install-e2e-{player,logs}-<leg-id>, and the report job
feeds the run's artifact name->id list to the results renderer, which
rebuilds the leg id from the parsed job name. GitHub does not link
jobs to artifacts, so the deterministic name is the join key.
Empirical finding: GitHub artifact downloads are auth-gated (the
download URL 307s to /suites/... which is 404 anonymous), so a
locally-opened player page cannot fetch the zip cross-origin. The
player now degrades gracefully: ?zip= fetch failure renders a real
download link for the zip (a normal click carries the user's session)
plus a drag-and-drop / file-picker path, and no-param opens as a pure
drop target. Verified in a real browser against a real artifact URL.
Verified: generator emits leg_id, results renderer emits
✅/❌ [📼](...?zip=...) only on ran cells, npm run check PASS,
install tests 36/36, strict tsc PASS, actionlint x4 PASS.
Skipped cells split into their reason: pre-desktop (a desktop-surface
method against a tag that predates apps/desktop) vs TODO (declared,
no driver arm yet). The report job passes pick-releases' annotated
tags into --format results; the shared methodNeedsDesktop() is the
same predicate the plan chart uses, so plan and results agree about
what pre-desktop means. Without --tags the renderer keeps the flat
skip label (backward compatible).
Verified against run 31579084845 real job list: 82 legs, 44 skips
labeled correctly.
macos gains the desktop-installer@latest install method: the website's
Hermes-Setup.dmg (verified live), mounted with hdiutil and its app
binary run DIRECTLY - an open-launched app inherits none of the git
redirect env, so direct exec is what keeps the isolation honest while
staying the same binary and first-launch flow.
install-e2e-macos-run.yml takes the windows shape: one workflow, one
inner job per driver arm, native skips. Arm 1 delegates script installs
to the shared OS-agnostic run workflow; arm 2 stages, installs from the
dmg, and drives both app-update methods through launch-from-spec.mjs -
open-app-update launches the installed .app (the double-click surface,
env via Playwright), hermes-desktop-app-update captures the product's
own hermes desktop spawn. Both end on sha asserts, never version
strings.
windows-desktop-gui-e2e.ps1 and windows-installer-script-e2e.ps1 fold
into tests/install/windows-e2e.ps1 with orthogonal -InstallMethod and
-Route axes: the install phase dispatches on one, the update phase on
the other, and shared workroot state carries how OLD landed - so any
implemented update method can follow any implemented install method.
Implementing a new pair is now a driver function plus a gate edit,
never a new job.
The run workflow collapses to ONE inner job whose if: is the
implemented-pairs table. Newly cheap pairs go live with the merge:
desktop-installer@latest -> hermes-update / installer-script /
installer-script+desktop / hermes-desktop-app-update
installer-script(+desktop) -> hermes-desktop-app-update
installer-script+desktop -> open-app-update (the -IncludeDesktop
install registers real Start Menu / Desktop shortcuts)
Only desktop-installer@latest as an UPDATE method stays a declared
TODO. scripts/windows_e2e_harness.ps1 executes the parse/parameter/
dispatch checks under pwsh before any Windows runner spins up.
The one-liner with its desktop stage opted in (--include-desktop /
-IncludeDesktop) is a real install kind, distinct on both sides:
on windows the stage builds Hermes.exe AND registers Start Menu /
Desktop shortcuts - a second path to a hand-launchable app - while
on linux/macos it builds into the checkout and registers no OS
entry point.
Declared on every OS and driven by both script drivers: the drivers
pass the flag through (hard failure if the ref predates it - the
tag-has-desktop gate already skips pre-desktop tags upstream) and
assert the built app exists under apps/desktop/release afterwards.
The run-workflow gates run +desktop pairs only on desktop-bearing
tags; app-update pairs from +desktop installs stay declared TODOs.
The desktop app has two launch paths, so app-update becomes two
methods. open-app-update starts the app from the OS entry point the
desktop installer created (the installed exe / the .app), so it exists
only where a desktop installer does. hermes-desktop-app-update starts
the app via hermes desktop, which every install method provides on
every OS that ships the desktop app - on linux it is the only app
surface, since no desktop installer or packaged artifact exists there.
Both variants are desktop-surface methods on every OS, so the
tag_has_desktop annotation moves from windows-only to every matrix
entry, install-e2e-run.yml grows the input, and the plan chart marks
pre-desktop cells on all OSes.
The windows GUI arm's implemented pair renames to open-app-update;
every other new combination is a declared TODO that natively skips.
generate-e2e-matrix.mjs grows --format results: reads the run's own
job list as NDJSON {name, conclusion} on stdin (per-leg conclusions
are NOT reachable through needs - a matrix job collapses to one
aggregate result) and re-renders the plan chart with each cell's
outcome. Legs are recognized by the exact name shape buildMatrices
mints, so unrelated jobs fall out; duplicate leg names (one windows
job per driver arm, only one runs) merge by significance - real
outcomes beat skips, failures beat successes. A final report job
(if: always, needs all three OS jobs) appends the chart to its step
summary via gh api with the default token.
Verified against two real runs: 31536931863 renders 11 passed / 0
failed / 54 skipped all-green; 31557865241 (the pre-EAP-fix run)
renders its 4 real failures + cancellations over the sibling arm's
skips.
generate-e2e-matrix.mjs grows --format results: reads the run's own
job list as NDJSON {name, conclusion} on stdin (per-leg conclusions
are NOT reachable through needs - a matrix job collapses to one
aggregate result) and re-renders the plan chart with each cell's
outcome. Legs are recognized by the exact name shape buildMatrices
mints, so unrelated jobs fall out; duplicate leg names (one windows
job per driver arm, only one runs) merge by significance - real
outcomes beat skips, failures beat successes. A final report job
(if: always, needs all three OS jobs) appends the chart to its step
summary via gh api with the default token.
Verified against two real runs: 31536931863 renders 11 passed / 0
failed / 54 skipped all-green; 31557865241 (the pre-EAP-fix run)
renders its 4 real failures + cancellations over the sibling arm's
skips.
generate-e2e-matrix.mjs grows --format markdown: one row per
{os, install -> update} combination, one column per starting tag,
appended to GITHUB_STEP_SUMMARY by the expand job. Cells mark
dispatched legs; run-vs-grey stays the run workflows' call, so the
only special cell is pre-desktop (the one annotation the plan owns).
JSON mode unchanged.
The fake Internet (bubblewrap + slirp4netns + MITM proxy +
upload-pack shim, 883 lines across dev-sandbox.sh, stage2-run.sh,
proxy.py, ssh-shim.sh, openssl.cnf, install-update-e2e.sh) existed to
isolate install.sh's network. The GIT_CONFIG_GLOBAL insteadOf redirect
the windows driver introduced does the same job with a gitconfig file
and works on any OS, so:
* install-e2e-run.yml now runs tests/install/installer-script-e2e.sh
directly on the bare runner - no sandbox deps, no userns sysctls -
and takes a runner input;
* the macos matrix calls the SAME workflow on macos-latest, deleting
install-e2e-macos-run.yml: installer-script -> installer-script /
hermes-update flip from grey to live, app-update pairs stay TODO
inside the shared gate;
* install.sh is no longer curl'd through a fake CA - each leg runs
the copy from the ref a user of that version actually executed;
* scripts/dev-sandbox.sh becomes the minimal isolation sandbox from
ab6b9492f (separate HERMES_HOME / Electron userData / app name,
same CLI surface: --persistent, --from, --delete), keeping its
.hermes-sandbox dir name so gitignore and docs hold;
* nix/sandbox.nix drops the bwrap/proxy closure and keeps only the
Electron runtime LD_LIBRARY_PATH the desktop app needs.
Verified: nix build .#sandbox + smoke run (isolated HERMES_HOME
created, ephemeral cleanup), shellcheck/bash -n on both scripts,
actionlint on all three workflows, and the new driver ran the full
v0.20.2 -> HEAD hermes-update pass locally before this commit.
Per review the unions were overcomplicated. Install methods are now
just: installer-script (the platform one-liner - curl | bash on
linux/macos, irm | iex on windows), desktop-installer, and
packaged-app (declared, unused). Update methods are every install
method (re-run it over the existing install) plus hermes-update and
app-update. desktop-installer-rerun, desktop-app, curl-bash, and
irm-iex are gone as ids; the windows driver's ValidateSet, switch
arms, and both run workflows' gates renamed to match. tsc --checkJs
clean; generator output re-verified (4 linux / 16 windows / 6 macos
legs for 2 tags).
Per review: the method/version vocabulary is now closed TYPE unions
(@ts-check + jsdoc typedefs - InstallerVersion, InstallMethod,
UpdateMethod - checked with tsc --checkJs, which rejects a SPEC entry
outside the unions; verified by corrupting a copy: 6 errors) instead
of runtime KNOWN_METHODS/ALLOWED_VERSIONS sets. validateEntry and
routeWants are deleted with all the paranoia: the generator always
emits every OS matrix and the dispatch route filter moved to plain
job-level ifs in install-e2e.yml, where the OS jobs already live.
secondUpdate is typed never[] so declaring one is a type error until
a leg implements it. Anything types cannot catch is self-evident on
the next CI run.
Revert the hardcoded 16-job experiment: the combination spec belongs
in scripts/sandbox/generate-e2e-matrix.mjs (restored), not copy-pasted
YAML blocks. What survives from the experiment:
* leg names carry everything - 'os: install -> update (tag -> HEAD)' -
generated per entry, since slash-joined names are all the graph
renders;
* pick-releases annotates each tag ({ref, desktop}) and the generator
threads tag_has_desktop onto windows entries, so the windows run
workflow still gates pre-desktop tags without a probe job;
* the per-OS run workflows are untouched: single job, static 'e2e'
name, native skip gates own all capability knowledge.
install-e2e.yml is one generate job + three per-OS matrix fanouts.
Generator shape (4/16/12 legs for 2 tags), annotation threading, and
all six error paths verified; all four workflows pass actionlint;
driver parses clean pure-ASCII.
GitHub only draws matrix boxes for the PRIMARY workflow's matrices;
everything inside a called workflow flattens into slash-joined names.
The generator + per-tag sub-workflow therefore bought no structure in
the graph and hid the support matrix in a script.
Invert it: install-e2e.yml now declares one job per {os,
install-method -> update-method} combination (2 linux + 8 windows +
6 macos - same 16 the generator produced, verified by inventory
before/after), each a matrix over the picked release tags. The graph
now renders one titled box per combination whose legs read
'... from vX' - the tag axis inside the combo axis. The per-OS run
workflows are unchanged: they own capability knowledge and natively
skip unimplemented method pairs and pre-desktop tags.
install-e2e-tag.yml and generate-e2e-matrix.mjs are deleted; adding a
method is now adding one job block here, implementing one is flipping
the run workflow's gate.
Remove all capability knowledge from the combination generator: no
IMPLEMENTED table, no skipped matrix, no per-OS special cases. It now
only declares and expands - every {os, install-method, update-method}
combination is dispatched to its OS's run workflow, and each run
workflow natively skips (grey, job-level if on the method inputs) the
pairs its driver cannot run yet:
* install-e2e-run.yml gains install-method/update-method inputs,
gates on the supported pairs (curl-bash -> hermes-update/curl-bash),
and maps the method id to the sandbox script's --route internally;
* install-e2e-macos-run.yml is new - all pairs skip until a macOS
driver exists, and implementing one flips its job-level if;
* install-e2e-windows-run.yml already worked this way;
* install-e2e-skip.yml is deleted - nothing special-cases macOS
anymore, so the tag workflow is three identical OS fanouts.
Structure is now uniformly matrix(tag) -> matrix(combination) ->
run-or-skip, with capability knowledge living only next to each
driver. Generator shape/route filters/error paths re-verified; all
five workflows pass actionlint.
Two structural changes to the combination fanout:
1. Tags become the OUTER axis, as a sub-graph per starting version:
install-e2e.yml fans a plain matrix over the picked tags into a new
per-tag reusable workflow (install-e2e-tag.yml), which runs the
combination generator for that one tag and fans out one job per
{os, install-method, update-method}. The Actions graph now reads
'from vX -> windows: install -> update' per leg. Nothing is
hardcoded in the workflows: the tag workflow calls the generator
itself.
2. Native skips move to the point that owns the capability knowledge:
macOS combos (no driving workflow exists) grey out in the tag
workflow via install-e2e-skip.yml, untouched by the tag axis; ALL
windows combos dispatch to install-e2e-windows-run.yml, which takes
install-method/update-method inputs and natively skips the pairs
its driver cannot run yet - so implementing a windows method is a
change in the run workflow + driver only. The driver's -Route ids
now match the generator's method ids verbatim.
Generator output shape, route filters, and all error paths re-verified
locally; all four workflows pass actionlint; driver re-parses clean
pure-ASCII.
Replace the hand-enumerated update/installer/windows-desktop jobs with
a support-matrix generator (scripts/sandbox/generate-e2e-matrix.mjs).
The spec declares every {os, install-method, update-method} combination
a user could be on; generate-matrix expands it and fans out ONE JOB PER
COMBINATION:
* linux combos (curl-bash install x hermes-update/curl-bash rerun)
drive install-e2e-run.yml, still multiplied by the sampled release
tags from pick-releases;
* the windows combo (desktop-installer@latest -> desktop-app) drives
install-e2e-windows-run.yml - the real GUI flow;
* every declared-but-unimplemented combo (all of macOS, the remaining
windows methods) becomes its own visible skipped job, so the
coverage gap is enumerable from the Checks tab and implementing one
is a one-line move into IMPLEMENTED.
Strictness carried into the generator: method ids validate against a
closed set, installer 'versions' arrays only allow 'latest' until a
versioned archive exists, secondUpdate must stay empty until a chained
second-update leg is implemented, and unknown spec keys/routes throw.
Expansion, route filtering, empty-matrix gating, and all seven error
paths verified locally; the dispatch route choice keeps its exact
previous semantics (all/both/update/installer/windows-desktop).
allow you to simulate the whole official curl | bash installer,
and subsequent hermes updates.
Run development commands in a bubblewrap filesystem and network sandbox
with a local HTTPS MITM fixture server and a fake github
git-upload-pack transport.
Package the sandbox command and expose it from the nix devShell.
Stage the local installer at its canonical fake HTTPS URL and add a
persistent installation/update test path. Route root installs through
sandbox-owned filesystem locations and snapshot dirty source worktrees
into temporary fake commits so update tests can fast-forward without
changing the real checkout.
Includes a --install-ref sandbox installer mode that fetches any commit
(--from-main is a nice shorthand for local development) outside the
sealed sandbox, installs from that snapshot, and then promotes the fake
remote to the current worktree so update flows can be exercised with FF.
Notes on non-root sandboxes:
Giving a non-root sandbox a network is tricky.
slirp4netns joins the target userns and setuids to root before configuring the
netns, so the userns must map a uid 0; bwrap's --unshare-user maps exactly ONE
uid, so --uid 1000 leaves no root to become and slirp diedswith
`setns(CLONE_NEWNET): Operation not permitted`. Stage 1 builds the user+net
namespaces with `unshare` and two one-id ranges:
inner 0 -> a subuid, unused by the payload, present only so slirp can
become root
inner 1000 -> our real host uid
Mapping the payload to the *host* uid (not a second subuid) keeps everything the
sandbox writes owned by us, so `rm -rf` on a persistent sandbox still needs no
privileges. Stage 2 execs bwrap WITHOUT --unshare-user -- it only adds mount/pid
-- sidestepping bwrap's refusal to accept --uid outside a userns it created.
Costs a /etc/subuid range for the invoking user (we error with the exact line to
add) and util-linux `unshare`; `--root` needs neither.
