A red scheduled run blocked nothing and told nobody. install-e2e-red.yml
runs after every scheduled "Install & Update E2E" run: red opens one issue
labelled install-e2e-red (or rewrites the open one's body in place) with the
red legs grouped by failure class and linked; the first green run closes
it. No per-run comment, never a second issue.
It is its own workflow_run workflow because install-e2e.yml is also called
by stable-release.yml with read-only permissions, and a nested job asking
for issues: write would fail that call at startup. workflow_dispatch with a
dry-run default previews the change for any run id.
Every leg installed a release tag and updated to HEAD, so nothing ran
HEAD's installer on an empty machine (where all four 2026-09-23
install.ps1 breaks lived) and nothing exercised the updater we ship
today -- tag legs run the OLD build's updater handing off to HEAD.
The generator appends a HEAD -> NEXT start after the sampled tags, so the
column runs wherever the update legs run (dispatch and stable-release).
NEXT is a reserved update ref: the drivers mint a child of the install
commit that adds one marker file, written to the object store only, which
the local bare clone carries into serve.git.
On Windows the HEAD leg takes every git.exe dir off PATH and installs no
remote get-url shim: Get-PinnedGit returns any git on PATH, so either one
skipped pinned-git staging. launch-from-spec's HEAD observer now uses the
driver's real git so it cannot poll '' forever on that leg.
The install/update legs asserted plenty about the code -- the checkout
landed, the version bumped, the desktop artifact exists -- and nothing
about the user's own state. An upgrade that ate auth.json or truncated
state.db would have passed every leg.
Adds a read-only, stdlib-only verifier (snapshot/verify) plus the hooks
that drive it around the real upgrade, on the POSIX and Windows drivers.
The state it defends is produced through the ordinary CLI
(hermes chat -q / auth add / profile create), never seeded by the
harness, and each action asserts it actually landed so a leg cannot
'pass' while testing nothing.
Judged: config.yaml, .env, auth.json, state.db, gateway_state.json and
the user's trees. state.db is compared by row counts, not bytes -- a
live SQLite file moves for benign reasons. The bundled skills/ tree is
recorded but never judged (the product re-syncs it), and plugins/** is
left to verify-plugin-preservation.py.
The e2e-screen-record action installed ffmpeg through three different
OS package managers (apt, brew, winget). winget is the flaky leg on
windows-11-arm and serves an x64-gyan build that runs emulated on ARM;
choco's community package wraps the same gyan x64 zip, so a choco
fallback would not fix either problem. PM already ships a locked,
sha256-pinned ffmpeg (martin-riedl posix, BtbN win32 including a NATIVE
winarm64 build), so the recording action now verifies ffmpeg on PATH
instead of installing it, and the pinned binary rides the same
tools-cache as node/python.
- setup-pm learns a `packages` input (extra PM tools beyond the
toolchain roots, e.g. ffmpeg) threaded through setup_toolchain.py's
prepare/install/archive-inputs phases; the tools-cache key gains an
extra-packages fragment so existing keys stay byte-identical.
- The six chat-driver jobs pass `packages: ffmpeg` to setup-pm.
- e2e-screen-record drops the apt/brew/winget install steps, the
ffmpeg actions/cache steps and the save-cache input; Xvfb (headless
linux) and the macOS replayd-approval hack stay.
- The "Install locked chat driver dependencies" step installs only the
tests-js workspace with --omit=dev instead of the whole apps/desktop
tree: the drivers need @playwright/test, zod (previously a phantom
hoisted from @assistant-ui/react), js-yaml and semver only. 64 pure
packages in ~2s vs ~1800 including electron-builder and native
builds; the tree-shaken node_modules runs the real driver modules
(verified by importing desktop-chat-smoke.ts and update-window-chat
end to end).
- tests pin the new contracts; tests/install/README.md updated.
Share the real composer, provider-witness and completed-reply check across
post-build bundle smoke and desktop-bearing install/update checkpoints.
Keep native automatic-relaunch proof separate from post-update chat.
Download receipt-bound artifacts without release credentials and install
DMG, ZIP, MSIX and universal MSIXBUNDLE on each native architecture.
Split Windows assembly from feed publication; publish tested bytes only.
Bind candidate smoke results into the manifest used by stable promotion.
Verify historical/source provenance without assuming a version IPC commit,
strip CI identity from source build children, and use the actual Electron
PID rather than Playwright's Windows launcher wrapper.
Validation: real Linux Electron chat and sequential OLD/NEW source smoke
with preserved history; 145 targeted Python tests and 14 JS tests passed;
TypeScript, shell/PowerShell parsing and workflow checks passed.
Native macOS/Windows deployment and historical upgrades need Actions proof.
Run historical updater completion in a fresh interpreter so cached imports
cannot revive retired dependency installers. Share Git and ZIP completion,
carry receipt and recovery state, and preserve child exit status.
Route plugin admission, binary acquisition, desktop launch and build paths
through PM. Replace redundant helpers and tests with real worker, package,
publication and launch checks. Keep the shipped compatibility surface fixed.
Targeted Python and desktop checks pass. Native update journeys and fresh
production image qualification remain pending. This is a checkpoint before
those acceptance runs.
Finish bootstrap uv before PM replaces its store entry. Keep failure
receipts stdlib-only and align the cryptography requirement and override
with the locked version.
Let bundle builders declare launch paths and update ownership. Remove
payload discovery, Store probing, and the unused develop command.
Derive Nix Python from the PM lock and share its provenance stamp.
Document setup, activation, optional dependencies, and distribution
ownership. Targeted Windows tests, relocated runtime launches, Electron
bundling, and bilingual docs builds pass. Native Nix and signed-package
acceptance remain CI gates.
Add a Hermes-owned upgrade-preservation contract to the install E2E
harness: a tagged upgrade must not delete or modify anything under the
active home's plugins/** or any profile's plugins/<name>/plugins tree.
- tests/install/e2e-assets/verify-plugin-preservation.py: standalone
stdlib-only READ-ONLY verifier. snapshot records every entry (kind,
size + sha256, link target, recursive fingerprint of symlinked
external targets so the externally-owned sidecar witness is covered)
including empty dirs and the roots themselves (lstat, so dangling
root links are still scanned); verify fails on deletion or
modification, treats unreadable paths as hard errors, and refuses an
empty snapshot as inconclusive. Runs under python3/python on all
three driver platforms.
- tests/install/e2e-assets/preserve-plugins.sh: POSIX/macOS hooks.
After install: seed controlled non-dependency directory fixtures
(mnemosyne-wrapper marker + payload + symlinked runtime, second
profile plugin tree, external witness outside the home; no pyproject
in the scanned root, nothing downloaded) and snapshot. After update:
verify; violation fails the leg. Seeding never clobbers a populated
wrapper without the expected marker.
- installer-script-e2e.sh / macos-desktop-e2e.sh / windows-e2e.ps1:
wire before/after hooks into the update leg, and add --update-ref
(-UpdateRef on Windows) so a leg can target an actual stable tag
instead of HEAD; HEAD-upgrade legs keep the HEAD label. Matrix and
workflows unchanged.
- tests/scripts/test_verify_plugin_preservation.py: 17 unit tests on a
real temp filesystem covering clean survival, file/marker deletion
and modification, whole-root deletion, empty-dir deletion, symlink
repoint, external witness tamper, read-only guarantee, empty-snapshot
refusal, unreadable-path hard error (POSIX), and the exact CLI
round-trip the drivers use.
- tests/install/README.md: document the contract, the hooks, and the
stable-to-stable rule.
Keep unknown failures red, rotate evidence per attempt, and emit receipts for signature-confirmed historical cases. Add CI-only diagnostics and an exact-tag input for the unresolved July hand-off.
The harness asserted the driver still throws 'not implemented yet' for
the desktop-installer@latest update route; that arm is implemented now
(Invoke-PhaseInstallGui -Mode "update"), so the check failed against
its own tree. It asserts the implemented contract instead.
The 256-job cap wording in the workflow comment and README now states
the scope GitHub applies it at: each per-OS matrix separately, not the
combined leg count. At the 10-tag bound the largest matrix is windows
at 180.
e2e-screen-record comment: hhttps -> https.
^(10|[1-9])$ replaces the two-step guard: the [0-9]+ regex accepted
leading zeros that bash arithmetic then read as octal (010 passed as 8,
08 errored). README cost figures corrected to the generator's real
expansion: 41 legs/tag, 82 at the default 2 tags, update route 8/tag,
first matrix overflow at 15 tags (270 windows entries).
tag-count now reaches the shell via the environment, validated to 1-10
(an apostrophe in the raw interpolation could terminate quoting; above
~14 tags the expansion exceeds GitHub's 256-job matrix limit).
Result-chart cell ranking matches on the leading token: rendered
success/failure cells carry artifact links, so whole-cell indexOf
ranked them -1 and any skip in the map beat a real outcome.
README documents per-run cost, route slice sizes, the tag-count bound,
and a warning against running the GUI drivers outside a disposable VM.
The workflow input descriptions and the skips README still declared
open-app-update and the Setup.exe re-run as driver TODOs; both run now.
Skips have exactly two causes and the prose names them: no OS entry
point for the pair, or the starting release predates the surface. The
chart's TODO label itself stays until the n/a relabel lands with the
known-broken-OLD gate work.
macos gains the desktop-installer@latest install method: the website's
Hermes-Setup.dmg (verified live), mounted with hdiutil and its app
binary run DIRECTLY - an open-launched app inherits none of the git
redirect env, so direct exec is what keeps the isolation honest while
staying the same binary and first-launch flow.
install-e2e-macos-run.yml takes the windows shape: one workflow, one
inner job per driver arm, native skips. Arm 1 delegates script installs
to the shared OS-agnostic run workflow; arm 2 stages, installs from the
dmg, and drives both app-update methods through launch-from-spec.mjs -
open-app-update launches the installed .app (the double-click surface,
env via Playwright), hermes-desktop-app-update captures the product's
own hermes desktop spawn. Both end on sha asserts, never version
strings.
windows-desktop-gui-e2e.ps1 and windows-installer-script-e2e.ps1 fold
into tests/install/windows-e2e.ps1 with orthogonal -InstallMethod and
-Route axes: the install phase dispatches on one, the update phase on
the other, and shared workroot state carries how OLD landed - so any
implemented update method can follow any implemented install method.
Implementing a new pair is now a driver function plus a gate edit,
never a new job.
The run workflow collapses to ONE inner job whose if: is the
implemented-pairs table. Newly cheap pairs go live with the merge:
desktop-installer@latest -> hermes-update / installer-script /
installer-script+desktop / hermes-desktop-app-update
installer-script(+desktop) -> hermes-desktop-app-update
installer-script+desktop -> open-app-update (the -IncludeDesktop
install registers real Start Menu / Desktop shortcuts)
Only desktop-installer@latest as an UPDATE method stays a declared
TODO. scripts/windows_e2e_harness.ps1 executes the parse/parameter/
dispatch checks under pwsh before any Windows runner spins up.
Playwright must own the spawn (it needs the inspection pipe), but
hermes desktop is not just build+launch - stamp checks, integrity
gates, sandbox fixups, and a constructed child environment. So the
driver intercepts the product's own launch: a sitecustomize.py on
PYTHONPATH (opt-in via HERMES_E2E_CAPTURE_LAUNCH) wraps subprocess.run,
captures argv/cwd/env at the spawn site, and fakes success instead of
spawning; launch-from-spec.mjs then _electron.launch-es exactly that
spec and clicks Settings -> About -> Update now. Completion is product
state, not a Playwright event: the handoff result file or the checkout
reaching the expected sha (source installs write no result file).
Ships with the driver, so it works unchanged on every sampled OLD ref
- no product flag, no pre-flag fallback split. Both launch shapes are
matched (npm exec electron / packaged exe under apps/desktop/release);
npm BUILD calls pass through untouched. Exit 0 without a capture fails
the leg: a version that never reached its launch must not pass.
Probe-the-probe: scripts/launch_capture_probe.sh runs control rows
(no opt-in, non-launch argv) and both treatment shapes - all green
locally. Gate flips on the shared run workflow for linux/macos;
windows adopts the same path with the driver restructuring.
The composite action .github/actions/e2e-screen-record owns setup and
lifecycle on all three OSes: ffmpeg via apt/brew-verify/winget+cache,
capture via x11grab/gdigrab/avfoundation, mkv at 15fps stopped by 'q'
on live stdin with kill fallback. Linux runners have no display, so
start brings up a dedicated Xvfb :99 and exports DISPLAY - one display
serves both the recorder and any app a later step launches.
Recording moves out of the GUI driver into workflow infrastructure -
that is what makes it uniform - and a missing ffmpeg or a zero-frame
file now FAILS the leg instead of skipping silently: the graceful-skip
path is how the windows leg shipped no recording.mkv while green.
Lifecycle proven locally: start against lavfi testsrc, q-stop, ffprobe
duration check (record-start.sh/record-stop.sh under nix ffmpeg).
The one-liner with its desktop stage opted in (--include-desktop /
-IncludeDesktop) is a real install kind, distinct on both sides:
on windows the stage builds Hermes.exe AND registers Start Menu /
Desktop shortcuts - a second path to a hand-launchable app - while
on linux/macos it builds into the checkout and registers no OS
entry point.
Declared on every OS and driven by both script drivers: the drivers
pass the flag through (hard failure if the ref predates it - the
tag-has-desktop gate already skips pre-desktop tags upstream) and
assert the built app exists under apps/desktop/release afterwards.
The run-workflow gates run +desktop pairs only on desktop-bearing
tags; app-update pairs from +desktop installs stay declared TODOs.
The desktop app has two launch paths, so app-update becomes two
methods. open-app-update starts the app from the OS entry point the
desktop installer created (the installed exe / the .app), so it exists
only where a desktop installer does. hermes-desktop-app-update starts
the app via hermes desktop, which every install method provides on
every OS that ships the desktop app - on linux it is the only app
surface, since no desktop installer or packaged artifact exists there.
Both variants are desktop-surface methods on every OS, so the
tag_has_desktop annotation moves from windows-only to every matrix
entry, install-e2e-run.yml grows the input, and the plan chart marks
pre-desktop cells on all OSes.
The windows GUI arm's implemented pair renames to open-app-update;
every other new combination is a declared TODO that natively skips.
Each script-driver leg now proves the installed CLI can build the
desktop app, after the install phase and again after the update.
--build-only runs the full desktop pipeline and stops before the
launch - the same call hermes update makes. Old releases that predate
the flag skip the phase after a --help probe of the installed binary.
Actually launching the app is a TODO: it needs the spawn-interception
launcher and, on linux runners, a virtual display.
Also adds tests/install/README.md describing how the test family
works: the four layers, the git-redirect isolation, the phases, the
probe-do-not-assume rule for old versions, skips, triggers, artifacts.
