10f99bc15e
Every Linux lane that does real work ran on a 4-core `ubuntu-latest`. The Python suite and the JS checks were split into many small jobs to make that size usable. Each split job repeated the full setup. In most of the JS jobs the repeated setup cost more than the work. The work lanes move to larger runners. Then the splits that existed only to make small runners usable go away. Python tests: 12 slices become 1 job on a 96-core runner. Slicing cost a matrix job, a duration cache, a per-slice artifact and a merge job. 96 cores clear the floor that the slowest single test file sets, which is about 82s. A second slice divides work that is already at that floor, and adds a second setup. Duration data from run 32522943054 gives the numbers behind this: 3178 files, 11645s in series. The worker count is explicit, because `run_tests.sh` defaults to twice the core count. A later commit sets it from a measurement on this hardware. JS checks: 14 jobs become 1. The matrix paid about 371s of repeated setup to spread about 612s of work. One larger runner installs one time. The three UI shard scripts and `run-ui-shard.mjs` are therefore removed, because the unsharded `test:ui` covers the same tests. The unit of parallel work inside that job is a CHECK, and not a workspace. apps/desktop is most of the payload, and its own `check` is a serial && chain. A spread across workspaces alone therefore leaves that chain as the long pole. A package that declares `check:*` sub-scripts gives one unit for each sub-script. That is the same selection rule the matrix used. The loop lives in `.github/scripts/run-workspace-checks.mjs`, so the same sequence runs on a laptop. It runs 11 units together, buffers the output of each one, and fails at the end with the full list. Children that share one stdout interleave their lines and make a failure hard to read. `npm run --ws check` stops at the first workspace that fails. `check:test:plugins` joins the desktop `check` script. The matrix prefers `check:*` sub-scripts over the plain `check` script, so `check:test:plugins` ran only as its own leg. Without this change the merge drops that suite and the job stays green. node_modules is cached on the lockfile, and `npm ci` is skipped on an exact hit. The `cache: npm` option of `setup-node` caches only the ~/.npm tarball cache, which leaves the extract and the postinstalls to pay again. The arm64 image build stays on a native arm64 runner. A build of linux/arm64 on an x64 host uses emulation. The docker test lane caps its workers at the core count. Each of those tests drives a container, so the docker daemon sets the limit and not the processor. `.github/actionlint.yaml` declares the runner labels. actionlint knows the GitHub-hosted labels only, and an undeclared label reads as an error that hides the real findings. The `detect` job checks out one file through a sparse checkout, and its timeout drops to 1 minute. It reads `scripts/ci/classify_changes.py` and nothing else. Verification: - actionlint reports 9 findings across all workflows. An unmodified HEAD with the same config reports the same 9. This change adds none. - A wrong label still fails. actionlint reports `ubuntu-latest-32-cor` and `ubuntu-latest-32-arm-cores`. - Every changed workflow parses, and `name` parses as a string. - A replay of the `save-durations` merge step against a three-artifact layout returns all 3178 entries. - An expansion of the npm script graph gives the same leaf commands for the parallel units and for a plain `npm run check`, in both directions. Against the 13-leg matrix the count is 13 to 11, and the whole difference is the three UI shards that collapse into one unsharded `check:test:ui`. - `--list` reports the 11 units, and a full local run completes and reports the time of each unit. - The runner labels cannot be verified here. The first real run is the test.
84 lines
3.6 KiB
YAML
84 lines
3.6 KiB
YAML
# .github/workflows/js-tests.yml
|
|
name: JS Tests
|
|
|
|
on:
|
|
workflow_call:
|
|
|
|
jobs:
|
|
check:
|
|
name: JS & TS checks
|
|
# One 32-core job replaces a 14-leg matrix. The matrix spread about 612s
|
|
# of check payload over 4-core runners. It paid about 371s of repeated
|
|
# setup to do it: 14 checkouts, 14 node installs, 14 node_modules
|
|
# restores.
|
|
#
|
|
# One larger runner installs one time. vitest, tsc and eslint each size
|
|
# their own worker pool from the core count.
|
|
runs-on: ubuntu-latest-32-core
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
|
with:
|
|
node-version: 26
|
|
cache: npm
|
|
|
|
- name: grab npm 12
|
|
run: |
|
|
# No-op once the bundled npm is already 12.x — saves ~5-15s/job and
|
|
# keeps the installed major aligned with the npm12 cache-key tag.
|
|
npm --version | grep -q '^12\.' || npm i -g npm@12
|
|
|
|
# The ``cache: npm`` option of ``setup-node`` caches only the ~/.npm
|
|
# tarball cache. The job then extracts the full workspace node_modules
|
|
# again and runs the postinstalls again, which includes the Electron
|
|
# binary fetch. This caches the installed tree itself, keyed on the
|
|
# lockfile, and skips ``npm ci`` on an exact hit. There are no
|
|
# restore-keys: a partial hit leaves a stale tree, so anything other
|
|
# than an exact lockfile match reinstalls from the start.
|
|
#
|
|
# This install runs WITH scripts, so the tree holds the postinstall
|
|
# artifacts. The postinstall of electron unpacks its binary into
|
|
# node_modules/electron/dist, which is inside the cached tree.
|
|
#
|
|
# The ~/.cache/electron download cache stays out of the key on purpose.
|
|
# ``npm ci`` is skipped on a hit, so nothing reads that cache. It only
|
|
# makes the archive larger.
|
|
- name: Restore node_modules
|
|
id: node-modules-cache
|
|
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
|
|
with:
|
|
path: |
|
|
node_modules
|
|
apps/*/node_modules
|
|
ui-tui/node_modules
|
|
ui-tui/packages/*/node_modules
|
|
tests-js/node_modules
|
|
web/node_modules
|
|
key: node-modules-scripts-${{ runner.os }}-node26-npm12-${{ hashFiles('package-lock.json') }}
|
|
|
|
- uses: ./.github/actions/retry
|
|
if: steps.node-modules-cache.outputs.cache-hit != 'true'
|
|
with:
|
|
command: npm ci
|
|
|
|
# Every check runs at the same time. The step fails only after all of
|
|
# them finish. There are two reasons this is not ``npm run --ws check``.
|
|
#
|
|
# * ``--ws`` is serial and stops at the first workspace that fails. A
|
|
# run then reports one failure, where the matrix this replaced
|
|
# reported every failure together.
|
|
# * The unit of work is a CHECK, and not a workspace. apps/desktop is
|
|
# most of the payload, and its own ``check`` is a serial && chain.
|
|
# A spread across workspaces alone leaves that chain as the long
|
|
# pole. This expands the ``check:*`` sub-scripts of a package, so
|
|
# its lint, ui, electron and plugin suites all run together. That
|
|
# is the same selection rule the old matrix job used.
|
|
#
|
|
# Discovery is ``npm query .workspace``. A new package or a new
|
|
# ``check:*`` script needs no change here. An empty list is an error and
|
|
# not an empty run, because an empty run reports green after it checks
|
|
# nothing.
|
|
- name: Run all workspace checks
|
|
run: node .github/scripts/run-workspace-checks.mjs
|