test(install): prove updating from a release reaches this commit
Nothing covered the update path, which is the worst thing to break: a broken updater strands users on the version that cannot fix itself. `hermes update` alone is ~2000 lines (hermes_cli/update_cmd.py) and had no end-to-end test. tests/install/install-update-e2e.sh installs a genuine earlier Hermes through the real one-liner (curl -fsSL https://…/install.sh | bash, served by dev-sandbox's MITM proxy at the canonical URL, cloning "github.com" through the upload-pack shim), which really installs uv, a managed Python, Node and the venv. It then applies ONE update route and requires the checkout to land on this commit with `hermes --version` still working -- so a pass means the venv and entry point survived, not merely that git moved. One route per run, each on a sandbox built from scratch. Sharing one install across routes -- or rewinding with `git reset --hard` between them -- leaves the second route running against a tree the first already updated (same venv, same console script, same __pycache__), which is not the state any real user is in: a route could pass only because its predecessor did the work, and a failure in the first left the second exercising something undefined. --install-ref chooses what to install first, so this covers "update from an older release", not just from the tip. Installer flags are probed against the target rather than assumed, because releases from months back predate flags current Hermes takes for granted: --skip-browser is read out of that ref's own install.sh, and `--yes` is asked of the installed `hermes update --help` (the update subcommand has lived in main.py, subcommands/update.py and update_cmd.py across the tags we sample, so a static parse rots silently -- and did). Without those probes, old releases die on "Unknown option: --skip-browser" and "unrecognized arguments: --yes" before doing any work. Installer output is streamed through tee rather than captured: a real install of uv, Python, Node and the venv IS the substance of this test, so it belongs in the job log, not only in an artifact. pipefail keeps the installer's exit status rather than tee's, so a failed install cannot look like a pass. The sandbox's own proxy log is printed in full on failure, since a rejected TLS handshake explains a failure that otherwise reads as a bare `curl: (35)`. Deliberately reuses dev-sandbox rather than adding a second harness. An earlier draft rewrote install.sh's hardcoded URLs with insteadOf and ran it against the host; that tested the installer LESS faithfully (bash install.sh instead of the real one-liner, host libs instead of a clean machine, ssh disabled to keep a failed rewrite from reaching real GitHub) while duplicating a fake Internet we already have. Shell, not pytest, so scripts/run_tests.sh and run_tests_parallel.py stay untouched: a pytest version needed an entry in the former's `env -i` credential allowlist and a _SKIP_PARTS exclusion in the latter, and every meaningful line was a command run inside the sandbox anyway. Two guards, both earned during bring-up. It prefers the `sandbox` wrapper and falls back to the raw script only when bwrap is on PATH (under Nix the wrapper supplies the PATH and DEV_SANDBOX_* vars, so the bare script exits 127). And it refuses to run on a dirty worktree: every dev-sandbox invocation re-derives fake main from the working copy, so uncommitted changes move the update target between the call that installs and the call that verifies -- a failure that looks like a broken updater but is a moving reference.
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -145,6 +145,9 @@ docs/superpowers/*
|
||||
|
||||
# Persistent dev sandbox dir (scripts/dev-sandbox.sh --persistent)
|
||||
.hermes-sandbox/
|
||||
# Sandbox dirs used by the install/update E2E (tests/install/). The suffix is
|
||||
# the route name, so each route gets its own tree and two can run at once.
|
||||
.hermes-sandbox-e2e*/
|
||||
|
||||
# Interrupted-update breadcrumb + recovery lock written next to the shared venv
|
||||
# by `hermes update` / launch-time self-heal. Runtime state, never a code change
|
||||
|
||||
Reference in New Issue
Block a user