#125053 fixed the exe's receipt writer, but the installer .exe is not being
rebuilt, so every Desktop-installer machine keeps the old receipt
(completedAtUnix, and a null pinnedCommit without git on PATH) until
`hermes update` rewrites it. No reader depends on completedAt or
pinnedCommit: Desktop's launch gate is runtime usability, and every other
reader only checks that the file exists. So the stamp phase stops
presenting this as a gate that lifts when a fixed exe ships and asserts the
contract that holds with the released exe: only those two FAIL lines, an
integer completedAtUnix, and the checkout at the expected commit. Any
other FAIL line stays red.
Refs #124949, #125053.
With one PM store per leg the Desktop app no longer re-bootstraps on first
launch, so the receipt Hermes-Setup.exe writes survives to verify-stamp:
pinnedCommit null on a git-less machine and completedAtUnix instead of
completedAt, replacing install.ps1's correct receipt. Gate only that
writer's exact two FAIL lines, and then require the checkout itself to be
at the expected commit. Any other FAIL line stays red.
With one store per leg, HEAD's install.ps1 never finished on Windows: the
source completion relaunched its store Python into itself every ~19 s, 35
levels deep after 12 minutes (job 108573522351). The workflow's workroot is
<workspace>\..\hermes-desktop-gui-e2e, so the store path carried the "..";
Windows reports sys.executable normalized, and prepare_launch()'s
Path.absolute() comparison never matched (#122513, fix PR #122518). The
job's own store (setup-pm) had no "..", which hid this until now. Normalize
the workroot, as a user's HERMES_HOME is.
install.ps1 now runs under the same watchdog as `hermes update` (40 min),
and the watchdog adds a py-spy stack for every Python process of the leg
before it stops them. The workflow installs py-spy beside pywinpty.
`windows: {installer-script, installer-script+desktop, desktop-installer@latest}
-> hermes-desktop-app-update (v2026.7.1 -> HEAD)` failed with "Program
'git.exe' failed to run: The filename or extension is too long". A v2026.7.1
clone leaves hundreds of CRLF-churned paths, and Clear-HistoricalInstallerChurn
passed every one to `git checkout --` on the command line. Pass them through
a NUL-separated --pathspec-from-file with --literal-pathspecs instead.
`windows: desktop-installer@latest -> hermes-update (HEAD -> NEXT)` and
`installer-script+desktop -> hermes-update (HEAD -> NEXT)` failed with
"source launcher publication failed". install.ps1 and `hermes update` ran
with setup-pm's HERMES_RUNTIME_DIR, while the desktop smoke launched the app
without it (as a user's app runs) and settled the checkout onto a second
store at <HERMES_HOME>\tools. The update then re-pointed its own running,
locked hermes.exe.
windows-e2e.ps1 now drops HERMES_RUNTIME_DIR, HERMES_PYTHON and VIRTUAL_ENV
on entry, so every product step of a leg (installer, app, update, chat)
resolves the one store a user's machine has. The driver keeps setup-pm only
through $DriverPython (and PATH, which older installers need for uv and
ripgrep).
`hermes update` also runs under a 45-minute watchdog. Past it, the driver
writes every process (pid, parent, start time, command line) to
logs\update-hang-processes.txt, stops this leg's processes and fails with
that table, instead of being cancelled blind at the job cap.
This reverts commit c7883282c91. On real Windows (runs 36293787001 and
36293789061, c7883282 + #124702), dropping HERMES_RUNTIME_DIR for the update
turned a 40-second red ("source launcher publication failed") into a silent
40-minute hang: the update finished its code, Desktop rebuild and skills
steps, printed the curator notice, and never exited until the job was
cancelled. The store it then used (<HERMES_HOME>\tools) was created by the
desktop smoke and never received the install's PM tools, so the update's
post-update default-tool install ran against a half-populated second store.
Both one-sided variants (keep the variable in the smoke; drop it for the
update) fail. The real fix is one store per leg, and it touches every
Windows leg, so it goes in on its own with a full Windows run.
`windows: desktop-installer@latest -> hermes-update (HEAD -> NEXT)` and
`installer-script+desktop -> hermes-update (HEAD -> NEXT)` failed with
"source launcher publication failed". The desktop smoke runs the app
without the job's HERMES_RUNTIME_DIR, the way a user's app runs, so it
settles the checkout onto <HERMES_HOME>\tools and republishes .hermes\bin
for that store's Python. The driver's `hermes update` then ran from that
hermes.exe with the job's store (setup-pm\tools) back in scope. The update
re-pointed both launchers, and Windows refused to replace the running
hermes.exe.
Invoke-HermesUpdate now drops HERMES_RUNTIME_DIR while <HERMES_HOME>\tools
exists, so the update stays on the one store the install actually uses.
Keeping HERMES_RUNTIME_DIR in the smoke instead fixed these two legs but
broke eight others whose app never reached its backend (run 36288957697),
so that variant was reverted.
Once launch capture works, two Windows hermes-desktop-app-update failures
show up (run 36286580917):
- v2026.9.24 -> HEAD (installer-script, installer-script+desktop,
desktop-installer@latest): the app's update is correct but slow. From
v2026.9.24 it runs the historical venv->PM takeover and a full Desktop
rebuild (the `hermes update` alone took 9m43s). launch-from-spec's 10 min
default gave up 9m55s after the Update now click while the updater window
showed "Updating code and dependencies 9m 22s elapsed". windows-e2e.ps1
now passes --timeout-ms 1800000, in line with open-app-update's 35 min
wait. The driver's self-deadline is now measured after the update wait
instead of being a flat 20 min inside it.
- HEAD -> NEXT: Desktop offered real GitHub main (0f4a98f8) instead of the
staged NEXT, so assertStagedBranch refused to click. HEAD's Electron main
is one ESM bundle, and checkout-source.ts binds promisify(execFile) at
load, before installSourceBranchProbe patches execFile in the packaged
app. POSIX avoids this by wrapping the launcher script, which Windows
skips. The probe now also hooks ChildProcess.prototype.spawn, which every
child passes through, and reuses branchProbeArgs, which already knows both
the --run-module and cmd.exe .cmd shapes. Verified locally: a promisify
captured before the hook now gets `--git <real> --branch main`, and other
commands are untouched.
`windows: * -> hermes-desktop-app-update (HEAD -> NEXT)` failed with
"a launch was actually captured (exit 0 without a launch must not pass)":
`hermes desktop` built the app and launched the real Hermes.exe, and the
driver's hook never saw it. There were two gaps, and either one alone is
enough to miss the launch:
- HEAD installs publish the PM launcher (.hermes\bin\hermes.exe), which
runs its interpreter with -I and drops PYTHONPATH, so sitecustomize never
loads. windows-e2e.ps1 now does what installer-script-e2e.sh already does:
run launch-capture/pm-launch.py for a PM launcher, and keep PYTHONPATH
for pre-PM venv console scripts.
- Since 3ddf82fe24 the Windows packaged launch is a detached
subprocess.Popen, not subprocess.run. sitecustomize now also intercepts a
launch-shaped Popen: it records the same spec and stands in for a child
that already exited with 0. Every other Popen, including the one inside
subprocess.run, spawns for real.
Three `-> hermes-desktop-app-update (v2026.9.24 -> HEAD)` legs failed with
"installed source has only installer-generated changes (other: 1 .M N...
<same blob> <same blob> website/i18n/zh-Hans/.../user-stories.mdx)".
v2026.9.24's install.ps1 clones under Git for Windows' system
core.autocrlf=true and pins false only afterwards, so the ~1700 text files
.gitattributes does not pin to LF (*.mdx, LICENSE, ...) sit on disk as CRLF
over LF blobs. The stat cache hides them, except the last few the clone
wrote inside the index's racy timestamp window, which read as modified.
Which files show up varies (1 or 3 per leg); main's installer no longer
pins autocrlf.
Clear-HistoricalInstallerChurn now also restores a `1 .M` entry whose
index and worktree modes match and that has no `diff --numstat
--ignore-cr-at-eol` record (a line-ending-only difference). A real content
edit, an edit inside a CRLF file, a mode change, or an untracked file still
fails the assertion, listed.
A red scheduled run blocked nothing and told nobody. install-e2e-red.yml
runs after every scheduled "Install & Update E2E" run: red opens one issue
labelled install-e2e-red (or rewrites the open one's body in place) with the
red legs grouped by failure class and linked; the first green run closes
it. No per-run comment, never a second issue.
It is its own workflow_run workflow because install-e2e.yml is also called
by stable-release.yml with read-only permissions, and a nested job asking
for issues: write would fail that call at startup. workflow_dispatch with a
dry-run default previews the change for any run id.
Every Windows leg of every scheduled run since 2026-09-25 was cancelled at
the 60-minute job timeout in its Install step. The user-state phase runs
`hermes chat -q "..."` under pty-run.py (a ConPTY, so the CLI sees a real
TTY). Since a5c7eed (shipped in v2026.9.21) `-q` on a TTY seeds an
interactive session and only answers-and-exits with --oneshot; the turn
answered ("Hello from the mock inference server!") and then sat at the
prompt. pty-run.py's --timeout 900 never fired either: it checked the
deadline only between blocking PtyProcess.read() calls, and an idle prompt
never returns from read().
- windows-e2e.ps1 passes --oneshot when `chat --help` advertises it (older
tags have no flag and exit after -q on their own), lowers the turn budget
to 300s, and names a 124 as "never exited" instead of a generic failure.
- pty-run.py drains the pty on a daemon thread and waits on the queue with
the deadline, so --timeout holds whatever the child does, and writes a
TIMEOUT line into the captured log before killing it.
The driver passed --skip-browser whenever the installer's help listed it.
That was for historical installers' own Playwright step; on a PM installer
the flag is now a real opt-out and would hide the default users get.
The GUI driver refuses a dirty source tree. On Windows, v2026.7.1's
installer ran `npm install`, which rewrites package-lock.json (later
installers use `npm ci`), so its Desktop reported dirty and the upgrade
arm failed before clicking Update. Mirror the Linux harness: print the
status, restore only package-lock.json churn and exclude a generated
.install_method; any other change still fails with the list.
The v2026.7.1 app launched by the latest DMG was still starting its backend
and did not finish a normal Quit within 30s. Wait up to 120s; the quit must
still be the normal one (no force-kill).
The desktop feed base came from config.yaml's
updates.desktop_feed_base_url and then an undocumented env var. Config
already wins and is the documented override; a second env source only
lets a stray variable redirect where updates are fetched from. Remove
the fallback from resolveFeedBaseUrl, its caller, its test cases and
the Windows bundled E2E that blanked it.
PM launchers run python -I, which ignores PYTHONPATH, so the macOS
hermes-desktop-app-update route never loaded the sitecustomize capture hook.
Route PM launchers through pm-launch.py as the Linux driver already does, and
cover the packaged .app shape the macOS route launches.
The current source checker reports updateAvailable with behind null when GitHub
compare cannot count staged commits; the app-update predicate demanded an integer
behind and refused every HEAD->NEXT leg. Require behind > 0 only for the historical
shape without updateAvailable.
v2026.6.19's DMG bootstrap runs the same install.sh that writes .install_method,
so its macOS leg saw a dirty tree. Share the Linux driver's guarded exclude through
source-driver.sh and apply it before every app-driven macOS update.
The desktop-installer legs intermittently leave Hermes-Setup.exe on
LAUNCHING after the Launch click: no app window, no desktop.log, and the
installer never exits. Its tracing log is buffered and dies with the job,
so the stall left no trace of where it blocks.
On a driver failure, while the installer is still alive, record the
Hermes/webview/python/uv/git/node process table (was Hermes.exe ever
started, and by whom), the installer's thread states, and a full memory
dump of the installer into proof/driver-failure/. Capture failures are
logged and never replace the driver's own assertion.
Packaged Electron rejects NODE_OPTIONS --require, so the preload never reached the Desktop source check. Wrap only the E2E installation launcher and route its source-check call to the real staged Git main with an explicit branch; other launcher calls remain unchanged. Keep production channel resolution fail-closed.
Every leg installed a release tag and updated to HEAD, so nothing ran
HEAD's installer on an empty machine (where all four 2026-09-23
install.ps1 breaks lived) and nothing exercised the updater we ship
today -- tag legs run the OLD build's updater handing off to HEAD.
The generator appends a HEAD -> NEXT start after the sampled tags, so the
column runs wherever the update legs run (dispatch and stable-release).
NEXT is a reserved update ref: the drivers mint a child of the install
commit that adds one marker file, written to the object store only, which
the local bare clone carries into serve.git.
On Windows the HEAD leg takes every git.exe dir off PATH and installs no
remote get-url shim: Get-PinnedGit returns any git on PATH, so either one
skipped pinned-git staging. launch-from-spec's HEAD observer now uses the
driver's real git so it cannot poll '' forever on that leg.
dd86f27636 added the verify-stamp phase dispatch and the workflow step that calls it, but not the ValidateSet entry, so every Windows leg failed parameter binding after a successful update (fork E2E run 35933334927).
New `-Phase verify-stamp` in the windows install/update driver, wired into
install-e2e-windows-run.yml AFTER the update phase — including the new
runtime's launch and smoke checks — because the bootstrap marker can
complete on a later run than the install itself. Known-failure legs skip
it: they legitimately never reach the new HEAD.
The phase runs scripts/verify-bootstrap-version-stamp.py against the
installed checkout with --expect-commit from the staged serve state, so
a real windows-latest run now asserts both the bootstrap-complete
receipt and the checkout's install-stamp.json tell the truth about the
final HEAD (native coverage the Linux-only suite can't provide).
Validation: windows-e2e.ps1 parses clean under real pwsh; workflow YAML
parses; tests/scripts/test_powershell_script_syntax.py green.
