36cb5ae553
Wires tests/install/install-update-e2e.sh into CI as a reusable workflow plus a
caller that fans out over real releases, because that is the question users care
about: can someone on a version they actually installed get to this commit?
install-e2e-run.yml takes `route` and `install-ref`, so the combinations that
matter are expressible without duplicating runner setup. Each leg is independent
-- its own runner, its own sandbox, its own install, nothing shared or rewound.
The starting versions are chosen at runtime by scripts/sandbox/pick-release-tags.sh:
newest, oldest, and an evenly spaced spread between (5 by default). Choosing at
runtime rather than hardcoding keeps the matrix honest -- a pinned list stops
covering the newest release the day after it ships, and pins an "oldest" long
after anyone still runs it. Newest catches "did the last release break
updating?", oldest is the longest upgrade jump still possible, and the spread
samples the migrations in between (config-schema bumps, venv layout changes,
dependency floors). Tags are read from the checkout with `git tag --list`, not
`git ls-remote`: the job has the repository already, so this needs no network,
works offline and on a fork, and takes 8ms. The repo is derived from the
script's own resolved path rather than $PWD, so a copy cannot silently report a
different checkout's tags. The pick-releases job takes the checkout that suits
it -- blob:none filter, sparse-checkout of just that script, and fetch-tags,
since tags are the entire input and the default shallow checkout has none.
Triggers match the shape of the work:
* every 12 hours, so upstream drift (a new uv, a Node bump, a PyPI change)
surfaces on a schedule instead of in someone's review cycle;
* on release tags, the moment the set of versions users can update FROM
changes and the moment a broken updater would strand them;
* manually, with the route and the sample size as inputs.
Not on pull_request: a leg is ~9 minutes of real toolchain installation and the
matrix multiplies it. fail-fast is off so one broken release does not mask the
others, and max-parallel caps the fan-out so a run does not hammer the runners
or PyPI. The tag list is resolved once and shared by both route matrices, so the
two routes cover the same versions.
Artifact names include the sanitized install-ref, since a matrix runs the
reusable workflow several times per route and same-named artifacts collide; that
name is built in a step because Actions expressions have no string-replace
function. The name step runs with `if: always()`, since a failing leg is exactly
when its logs are wanted.
.gitignore covers .hermes-sandbox-e2e*/ rather than the bare directory: the
per-route sandbox trees (-update, -installer) fell outside it, so the sandbox
made the worktree dirty and dev-sandbox reacted by snapshotting the working copy
into a fresh fake-main commit on every invocation.
111 lines
4.1 KiB
YAML
111 lines
4.1 KiB
YAML
name: Install & Update E2E
|
|
|
|
# Can a user on a released version get to this commit?
|
|
#
|
|
# For each release we sample, a leg installs that release through the real
|
|
# `curl | install.sh` one-liner (uv, a managed Python, Node, the venv) inside
|
|
# scripts/dev-sandbox.sh, then applies one update route and requires the
|
|
# checkout to land on this commit with a working `hermes`.
|
|
#
|
|
# The starting versions are chosen at runtime from the repo's release tags
|
|
# (scripts/sandbox/pick-release-tags.sh): newest, oldest, and a spread between.
|
|
# A hardcoded list would stop covering the newest release the day after it
|
|
# ships, and would pin an "oldest" that nobody still runs.
|
|
#
|
|
# Triggers:
|
|
# * every 12 hours, so upstream drift (a new uv, a Node bump, a PyPI change)
|
|
# surfaces on a schedule rather than in someone's review cycle;
|
|
# * when a release tag is created -- the moment the set of versions users can
|
|
# update FROM changes, and the moment a broken updater would strand them;
|
|
# * manually, where you can pick the route and how many releases to sample.
|
|
#
|
|
# Deliberately NOT on pull_request: a leg takes ~11 minutes of real toolchain
|
|
# installation, and the matrix multiplies that. Updating is release-shaped work,
|
|
# so it is gated on releases and the clock instead.
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
inputs:
|
|
route:
|
|
description: 'Which update route to exercise.'
|
|
required: false
|
|
type: choice
|
|
default: both
|
|
options: [both, update, installer]
|
|
tag-count:
|
|
description: 'How many release tags to sample (newest, oldest, and a spread between).'
|
|
required: false
|
|
type: string
|
|
default: '5'
|
|
schedule:
|
|
# Every 12 hours, off the hour to avoid the top-of-hour runner crunch.
|
|
- cron: '20 7,19 * * *'
|
|
push:
|
|
tags:
|
|
# Release tags only: the repo also carries backup/* and one-off tags.
|
|
- 'v[0-9]+.[0-9]+.[0-9]+'
|
|
- 'v[0-9]+.[0-9]+.[0-9]+.[0-9]+'
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: install-e2e-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
# Which released versions do we test updating FROM? Resolved once and shared
|
|
# by both route matrices, so the two routes cover the same set.
|
|
pick-releases:
|
|
name: Pick release tags
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
outputs:
|
|
tags: ${{ steps.pick.outputs.tags }}
|
|
steps:
|
|
# This job only reads tag names and runs one script, so take the cheap
|
|
# checkout: no blobs (filter), no other files (sparse), but DO fetch tags
|
|
# -- they are the whole input, and the default shallow checkout has none.
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
filter: blob:none
|
|
fetch-tags: true
|
|
sparse-checkout: scripts/sandbox/pick-release-tags.sh
|
|
sparse-checkout-cone-mode: false
|
|
- id: pick
|
|
run: |
|
|
set -euo pipefail
|
|
tags="$(scripts/sandbox/pick-release-tags.sh --count '${{ inputs.tag-count || 5 }}')"
|
|
echo "Testing updates from: $tags"
|
|
echo "tags=$tags" >> "$GITHUB_OUTPUT"
|
|
|
|
# `hermes update` -- the route most users take.
|
|
update:
|
|
if: github.event_name != 'workflow_dispatch' || inputs.route != 'installer'
|
|
needs: pick-releases
|
|
strategy:
|
|
# One release breaking is worth knowing about even if another already
|
|
# failed, so let every leg report.
|
|
fail-fast: false
|
|
matrix:
|
|
install-ref: ${{ fromJSON(needs.pick-releases.outputs.tags) }}
|
|
uses: ./.github/workflows/install-e2e-run.yml
|
|
with:
|
|
route: update
|
|
install-ref: ${{ matrix.install-ref }}
|
|
|
|
# Re-running the curl one-liner over an existing checkout: autostash + pull
|
|
# rather than the updater's own git handling.
|
|
installer:
|
|
if: github.event_name != 'workflow_dispatch' || inputs.route != 'update'
|
|
needs: pick-releases
|
|
strategy:
|
|
fail-fast: false
|
|
max-parallel: 3
|
|
matrix:
|
|
install-ref: ${{ fromJSON(needs.pick-releases.outputs.tags) }}
|
|
uses: ./.github/workflows/install-e2e-run.yml
|
|
with:
|
|
route: installer
|
|
install-ref: ${{ matrix.install-ref }}
|