Files
hermes-agent/.github/workflows/install-e2e-run.yml
ethernet 36cb5ae553 ci: test updating from sampled release tags, on tag + every 12h
Wires tests/install/install-update-e2e.sh into CI as a reusable workflow plus a
caller that fans out over real releases, because that is the question users care
about: can someone on a version they actually installed get to this commit?

install-e2e-run.yml takes `route` and `install-ref`, so the combinations that
matter are expressible without duplicating runner setup. Each leg is independent
-- its own runner, its own sandbox, its own install, nothing shared or rewound.

The starting versions are chosen at runtime by scripts/sandbox/pick-release-tags.sh:
newest, oldest, and an evenly spaced spread between (5 by default). Choosing at
runtime rather than hardcoding keeps the matrix honest -- a pinned list stops
covering the newest release the day after it ships, and pins an "oldest" long
after anyone still runs it. Newest catches "did the last release break
updating?", oldest is the longest upgrade jump still possible, and the spread
samples the migrations in between (config-schema bumps, venv layout changes,
dependency floors). Tags are read from the checkout with `git tag --list`, not
`git ls-remote`: the job has the repository already, so this needs no network,
works offline and on a fork, and takes 8ms. The repo is derived from the
script's own resolved path rather than $PWD, so a copy cannot silently report a
different checkout's tags. The pick-releases job takes the checkout that suits
it -- blob:none filter, sparse-checkout of just that script, and fetch-tags,
since tags are the entire input and the default shallow checkout has none.

Triggers match the shape of the work:

  * every 12 hours, so upstream drift (a new uv, a Node bump, a PyPI change)
    surfaces on a schedule instead of in someone's review cycle;
  * on release tags, the moment the set of versions users can update FROM
    changes and the moment a broken updater would strand them;
  * manually, with the route and the sample size as inputs.

Not on pull_request: a leg is ~9 minutes of real toolchain installation and the
matrix multiplies it. fail-fast is off so one broken release does not mask the
others, and max-parallel caps the fan-out so a run does not hammer the runners
or PyPI. The tag list is resolved once and shared by both route matrices, so the
two routes cover the same versions.

Artifact names include the sanitized install-ref, since a matrix runs the
reusable workflow several times per route and same-named artifacts collide; that
name is built in a step because Actions expressions have no string-replace
function. The name step runs with `if: always()`, since a failing leg is exactly
when its logs are wanted.

.gitignore covers .hermes-sandbox-e2e*/ rather than the bare directory: the
per-route sandbox trees (-update, -installer) fell outside it, so the sandbox
made the worktree dirty and dev-sandbox reacted by snapshotting the working copy
into a fresh fake-main commit on every invocation.
2026-08-04 17:36:26 -04:00

123 lines
4.9 KiB
YAML

name: Install & Update E2E (reusable)
# Runs ONE update route against ONE starting commit, in the dev sandbox, with a
# real install (uv, a managed Python, Node, the venv) behind it.
#
# Reusable so callers can fan out over the combinations that matter -- update
# from the tip vs. from an older release, `hermes update` vs. re-running the
# installer -- without duplicating the runner setup. Each leg is independent:
# its own sandbox, its own install, nothing rewound or shared.
#
# Call it:
#
# jobs:
# tip:
# uses: ./.github/workflows/install-e2e-run.yml
# with:
# route: update
# install-ref: refs/heads/main
on:
workflow_call:
inputs:
route:
description: 'Update path to exercise: update (hermes update) or installer (re-run install.sh).'
required: true
type: string
install-ref:
description: 'What to install before updating: a branch, a tag (v2026.7.7), or a SHA reachable from main.'
required: false
type: string
default: refs/heads/main
runner:
description: 'Runner label.'
required: false
type: string
default: ubuntu-latest
timeout-minutes:
description: 'Job timeout. A cold run installs real toolchains twice.'
required: false
type: number
default: 45
permissions:
contents: read
jobs:
e2e:
name: ${{ inputs.route }} from ${{ inputs.install-ref }}
runs-on: ${{ inputs.runner }}
timeout-minutes: ${{ inputs.timeout-minutes }}
steps:
# Full history: the sandbox fetches the starting commit and the test
# compares against this commit, so a shallow clone is not enough.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
# bubblewrap + slirp4netns are what the sandbox is built on; util-linux
# supplies the `unshare` that builds the multi-uid userns for the
# user-level (non-root) install.
- name: Install sandbox dependencies
run: |
set -euo pipefail
sudo apt-get update -qq
sudo apt-get install -y -qq bubblewrap slirp4netns uidmap util-linux
# Ubuntu 24.04 restricts unprivileged user namespaces through AppArmor,
# which is exactly what bwrap needs. Report the state before touching it
# so a future runner-image change is visible in the log rather than
# silently altering what this job proves.
- name: Permit unprivileged user namespaces
run: |
set -euo pipefail
echo "--- kernel userns settings (before)"
sysctl kernel.unprivileged_userns_clone 2>/dev/null || echo " (sysctl absent)"
sysctl kernel.apparmor_restrict_unprivileged_userns 2>/dev/null || echo " (sysctl absent)"
if sysctl -n kernel.apparmor_restrict_unprivileged_userns >/dev/null 2>&1; then
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
fi
echo "--- subuid/subgid for $(id -un)"
grep "^$(id -un):" /etc/subuid /etc/subgid || echo " (none — sandbox will say so)"
- name: Run install + update E2E
run: |
set -euo pipefail
tests/install/install-update-e2e.sh \
--route '${{ inputs.route }}' \
--install-ref '${{ inputs.install-ref }}'
env:
# Outside the workspace on purpose: the script creates this directory
# up front, and an untracked dir inside the repo makes the worktree
# dirty -- which dev-sandbox reacts to by snapshotting the working
# copy into a fresh fake-main commit on every invocation, moving the
# update target mid-run.
HERMES_E2E_LOG_DIR: ${{ runner.temp }}/e2e-logs
# Artifact names cannot contain '/', and install-ref may be a full ref
# like refs/heads/main. GitHub Actions expressions have no string-replace
# function, so build the safe name here. Runs even on failure -- that is
# exactly when the logs are wanted.
- name: Build artifact name
if: always()
id: artifact
run: |
set -euo pipefail
safe_ref='${{ inputs.install-ref }}'
safe_ref="${safe_ref//\//-}"
echo "name=install-e2e-${{ inputs.route }}-${safe_ref}" >> "$GITHUB_OUTPUT"
# The installer's own transcripts say far more than the assertion that
# tripped when a real install breaks.
- name: Upload installer logs
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Unique per leg: a matrix over releases runs this workflow several
# times per route, and same-named artifacts collide.
name: ${{ steps.artifact.outputs.name }}-${{ github.sha }}
path: ${{ runner.temp }}/e2e-logs
retention-days: 14
if-no-files-found: ignore