686c34d3f6
The published image had no Xvnc/Xfce because nothing set the Dockerfile's HERMES_BOT_DESKTOP argument, and a hosted instance (unprivileged, no sudo, sealed /opt/hermes) cannot install at run time. The image layer is the only delivery path. - docker.yml: variant axis [slim, desktop]. :latest / :main / :v* stay the image they are today; :latest-desktop / :main-desktop / :v*-desktop carry the packages plus Playwright's headed Chromium. Slim owns the build cache scope; one manifest per variant so a desktop publish failure never skips slim's :latest. - Dockerfile / stage2-hook.sh: XDG_RUNTIME_DIR=/tmp/hermes-runtime seeded 0700 as hermes (containers have no logind; the $HOME/.cache fallback was the shared /opt/data volume), refused when foreign-owned; deterministic Chromium discovery exporting the headless shell for ordinary browsing. - bot_desktop: memory gate reads the cgroup working set (usage minus inactive_file) so it cannot tighten over uptime and refuse to restart a screen idle-stop just stopped; installable() gives three distinct dead-end messages instead of a sudo line nobody there can run; env_for_agent replaces a headless-shell pin so agent and dock share one Chromium. Squash of IAvecilla/hermes-agent:bot-desktop-cloud-image (#112381, 13 commits), which GitHub auto-closed when its base branch merged as #108914. Review fixes from pefontana (cache scope, per-variant merge, red browser test) are included. Co-authored-by: pefontana <pefontana@users.noreply.github.com>
369 lines
16 KiB
YAML
369 lines
16 KiB
YAML
name: Docker Build, Test, and Publish
|
|
|
|
on:
|
|
# This workflow owns its own triggers. ci.yml does not call it.
|
|
# A reusable-workflow call eeps the caller run in progress for that full time.
|
|
# GitHub refuses ``gh run rerun`` on a run that is still in progress.
|
|
# Thus one slow advisory job blocked every rerun of the fast required jobs. A separate
|
|
# run reruns and cancels independently.
|
|
#
|
|
# Trusted main pushes resolve the environment-scoped Docker Hub secrets in
|
|
# this same workflow, never across a workflow boundary.
|
|
pull_request:
|
|
push:
|
|
branches: [main]
|
|
release:
|
|
types: [published]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
# Concurrency: push/release runs are NEVER cancelled so every merge gets
|
|
# its own image. PR runs reuse a PR-scoped group with
|
|
# cancel-in-progress: true so rapid pushes to the same PR collapse to
|
|
# the latest commit.
|
|
concurrency:
|
|
group: docker-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
|
|
env:
|
|
IMAGE_NAME: nousresearch/hermes-agent
|
|
|
|
jobs:
|
|
# Classify the PR's changed files. ci.yml used to gate the docker call on
|
|
# its own ``detect`` outputs; now that this workflow triggers itself, it
|
|
# runs the same composite action. On push and release the classifier fails
|
|
# open (every lane true), so post-merge validation is never weakened.
|
|
detect:
|
|
name: Detect affected areas
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
outputs:
|
|
build: ${{ steps.gate.outputs.build }}
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Detect affected areas
|
|
id: classify
|
|
uses: ./.github/actions/detect-changes
|
|
with:
|
|
github-token: ${{ github.token }}
|
|
|
|
- name: Decide whether to build
|
|
id: gate
|
|
env:
|
|
# The docker lane derives from python_prod (not python: the image
|
|
# copies installed code, never tests/, so tests-only PRs skip the
|
|
# build), frontend and docker_meta. classify_changes.py owns the
|
|
# formula so this gate and the nix lane cannot drift apart.
|
|
DOCKER: ${{ steps.classify.outputs.docker }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$DOCKER" = "true" ]; then
|
|
echo "build=true" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "build=false" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
# Build and test the image for each architecture. This job runs PR code,
|
|
# so it must remain secret-free. Publishing happens in the separate,
|
|
# protected publish job after these tests pass.
|
|
build:
|
|
needs: [detect]
|
|
if: github.repository == 'NousResearch/hermes-agent' && needs.detect.outputs.build == 'true'
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
# Two images per arch. `slim` is what :latest has always been; `desktop`
|
|
# adds the Bot Screen packages and a headed Chromium (+1.4 GB) for the
|
|
# tier that offers a screen. Both are built on a PR so a change that only
|
|
# breaks the gated layers cannot reach publish.
|
|
arch: [amd64, arm64]
|
|
variant: [slim, desktop]
|
|
include:
|
|
- arch: amd64
|
|
runner: ubuntu-latest-32-core
|
|
platform: linux/amd64
|
|
# arm64 builds on the native arm64 larger runner. A build of
|
|
# linux/arm64 on an x64 host uses emulation.
|
|
- arch: arm64
|
|
runner: ubuntu-latest-32-arm-core
|
|
platform: linux/arm64
|
|
|
|
runs-on: ${{ matrix.runner }}
|
|
timeout-minutes: 45
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Reject profile exports in the build context
|
|
run: python3 scripts/ci/check_profile_archive_boundary.py
|
|
|
|
# Retry once on transient Docker Hub / buildkit pull failures
|
|
# (connection reset, auth token timeout, rate limiting). The action
|
|
# generates a unique builder name per invocation so the retry doesn't
|
|
# collide with the failed first attempt. A genuine persistent failure
|
|
# still fails the job — only the first attempt has continue-on-error.
|
|
# Refs: docker/setup-buildx-action#510
|
|
- name: Set up Docker Buildx
|
|
id: buildx
|
|
continue-on-error: true
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
- name: Set up Docker Buildx (retry)
|
|
if: steps.buildx.outcome == 'failure'
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
# Build once, load into the local daemon for testing. Cached
|
|
# per-arch; the push step below reuses every layer from this build.
|
|
- name: Build image (${{ matrix.arch }})
|
|
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
|
with:
|
|
context: .
|
|
file: Dockerfile
|
|
load: true
|
|
platforms: ${{ matrix.platform }}
|
|
tags: ${{ env.IMAGE_NAME }}:test
|
|
build-args: |
|
|
HERMES_GIT_SHA=${{ github.sha }}
|
|
HERMES_BOT_DESKTOP=${{ matrix.variant == 'desktop' && '1' || '0' }}
|
|
# Slim owns the scope; desktop is slim plus two RUN steps, so it reads
|
|
# and writes nothing — a 5.5 GB mode=max scope would blow the 10 GB cap.
|
|
cache-from: type=gha,scope=docker-${{ matrix.arch }}
|
|
cache-to: ${{ (github.event_name != 'pull_request' && matrix.variant == 'slim') && format('type=gha,mode=max,scope=docker-{0}', matrix.arch) || '' }}
|
|
|
|
|
|
# Run the docker-integration test suite against the freshly-built
|
|
# image already loaded into the local daemon (`:test`).
|
|
#
|
|
# Piggybacking here avoids a second image build: the build step
|
|
# already loaded the image into the daemon under
|
|
# `${IMAGE_NAME}:test`, so we just point ``HERMES_TEST_IMAGE`` at
|
|
# that. The fixture's ``HERMES_TEST_IMAGE`` branch (see
|
|
# tests/docker/conftest.py:62-63) short-circuits the rebuild.
|
|
#
|
|
# Why this job and not a standalone one: the image is 5GB+; passing
|
|
# it between jobs via ``docker save``/``upload-artifact`` is slower
|
|
# than the build itself. Reusing the existing daemon state is the
|
|
# cheapest path to coverage on every PR that touches docker code.
|
|
# ---------------------------------------------------------------------
|
|
- name: Install uv (for docker tests)
|
|
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # 8.2.0
|
|
with:
|
|
# Pinned: unpinned setup-uv fetches a 'latest' manifest from
|
|
# raw.githubusercontent.com every job; transient fetch failures
|
|
# fail the job (2026-07-28 incident). Keep in sync with tests.yml.
|
|
version: "0.9.28"
|
|
|
|
- name: Set up Python 3.11 (for docker tests)
|
|
uses: ./.github/actions/retry
|
|
with:
|
|
command: uv python install 3.11
|
|
|
|
- name: Install Python dependencies (for docker tests)
|
|
# ``dev`` extra pulls in pytest, pytest-asyncio —
|
|
# everything tests/docker/ needs. We deliberately avoid ``all``
|
|
# here because the docker tests only drive the container via
|
|
# subprocess and don't import hermes_agent's optional deps.
|
|
uses: ./.github/actions/retry
|
|
with:
|
|
command: uv sync --locked --python 3.11 --extra dev
|
|
|
|
- name: Run docker integration tests
|
|
env:
|
|
# Skip rebuild; use the image already loaded by the build step.
|
|
HERMES_TEST_IMAGE: ${{ env.IMAGE_NAME }}:test
|
|
# Match the policy in tests.yml :: test job — no accidental
|
|
# real-API calls from inside the harness.
|
|
OPENROUTER_API_KEY: ""
|
|
OPENAI_API_KEY: ""
|
|
NOUS_API_KEY: ""
|
|
run: |
|
|
# Each of these tests drives a container, so the docker daemon sets
|
|
# the limit and not the processor. This caps the workers. The
|
|
# default from run_tests.sh is cpu_count*2, which starts 64
|
|
# containers together on the 32-core amd64 runner.
|
|
HERMES_TEST_WORKERS=$(nproc) scripts/run_tests.sh tests/docker/ --file-timeout 600
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Rebuild and push each architecture only after the unprivileged build/test
|
|
# matrix passes. This job is the sole Docker Hub credential boundary.
|
|
# ---------------------------------------------------------------------------
|
|
publish:
|
|
if: github.repository == 'NousResearch/hermes-agent' && (github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release')
|
|
needs: [build]
|
|
environment: container-publish
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
arch: [amd64, arm64]
|
|
variant: [slim, desktop]
|
|
include:
|
|
- arch: amd64
|
|
runner: ubuntu-latest-32-core
|
|
platform: linux/amd64
|
|
# Native arm64 for the same reason as the build matrix above.
|
|
- arch: arm64
|
|
runner: ubuntu-latest-32-arm-core
|
|
platform: linux/arm64
|
|
runs-on: ${{ matrix.runner }}
|
|
timeout-minutes: 30
|
|
steps:
|
|
- name: Checkout trusted source
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Reject profile exports in the build context
|
|
run: python3 scripts/ci/check_profile_archive_boundary.py
|
|
|
|
# Retry once on transient Docker Hub / buildkit pull failures.
|
|
# See build job for rationale; same pattern.
|
|
- name: Set up Docker Buildx
|
|
id: buildx
|
|
continue-on-error: true
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
- name: Set up Docker Buildx (retry)
|
|
if: steps.buildx.outcome == 'failure'
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
- name: Log in to Docker Hub
|
|
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
|
with:
|
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
|
|
# Push by digest only (no tag). The merge job assembles the tagged
|
|
# manifest list after both architecture publishers complete.
|
|
- name: Push ${{ matrix.arch }} by digest
|
|
id: push
|
|
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
|
with:
|
|
context: .
|
|
file: Dockerfile
|
|
platforms: ${{ matrix.platform }}
|
|
labels: |
|
|
org.opencontainers.image.revision=${{ github.sha }}
|
|
build-args: |
|
|
HERMES_GIT_SHA=${{ github.sha }}
|
|
HERMES_BOT_DESKTOP=${{ matrix.variant == 'desktop' && '1' || '0' }}
|
|
outputs: type=image,name=${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
|
|
cache-from: type=gha,scope=docker-${{ matrix.arch }}
|
|
cache-to: ${{ matrix.variant == 'slim' && format('type=gha,mode=max,scope=docker-{0}', matrix.arch) || '' }}
|
|
|
|
- name: Export digest
|
|
run: |
|
|
mkdir -p /tmp/digests
|
|
digest="${{ steps.push.outputs.digest }}"
|
|
touch "/tmp/digests/${digest#sha256:}"
|
|
|
|
- name: Upload digest artifact
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: digest-${{ matrix.variant }}-${{ matrix.arch }}
|
|
path: /tmp/digests/*
|
|
if-no-files-found: error
|
|
retention-days: 1
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Stitch both per-arch digests into a single tagged multi-arch manifest.
|
|
# This is a registry-side operation — no building, no layer re-push —
|
|
# so it runs in ~30 seconds.
|
|
#
|
|
# On main pushes: tags both :main and :latest.
|
|
# On releases: tags :<release_tag_name>.
|
|
# ---------------------------------------------------------------------------
|
|
merge:
|
|
# `needs` is the whole 4-leg matrix and a failed need skips the job, so a
|
|
# transient desktop push would take slim's :latest with it. Guard below.
|
|
if: ${{ !cancelled() && github.repository == 'NousResearch/hermes-agent' && (github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release') }}
|
|
runs-on: ubuntu-latest
|
|
needs: [publish]
|
|
timeout-minutes: 10
|
|
environment: container-publish
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
# One manifest list per variant. `slim` keeps the unsuffixed tags it has
|
|
# always had, so nothing that pulls :latest today changes; `desktop`
|
|
# publishes the same digests under a -desktop suffix.
|
|
include:
|
|
- variant: slim
|
|
suffix: ""
|
|
- variant: desktop
|
|
suffix: "-desktop"
|
|
steps:
|
|
- name: Download digests
|
|
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
|
with:
|
|
path: /tmp/digests
|
|
pattern: digest-${{ matrix.variant }}-*
|
|
merge-multiple: true
|
|
|
|
# Retry once on transient Docker Hub / buildkit pull failures.
|
|
# See build job for rationale; same pattern.
|
|
- name: Set up Docker Buildx
|
|
id: buildx
|
|
continue-on-error: true
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
- name: Set up Docker Buildx (retry)
|
|
if: steps.buildx.outcome == 'failure'
|
|
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
|
|
- name: Log in to Docker Hub
|
|
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
|
with:
|
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
|
|
- name: Create manifest list and push
|
|
working-directory: /tmp/digests
|
|
env:
|
|
IMAGE_NAME: ${{ env.IMAGE_NAME }}
|
|
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
|
SUFFIX: ${{ matrix.suffix }}
|
|
run: |
|
|
set -euo pipefail
|
|
# Without nullglob an empty dir yields the literal `*`: one bogus entry.
|
|
shopt -s nullglob
|
|
args=()
|
|
for digest_file in *; do
|
|
args+=("${IMAGE_NAME}@sha256:${digest_file}")
|
|
done
|
|
# One per arch in the publish matrix; a short set means a leg failed, and
|
|
# stitching it would publish a single-arch :latest. Keep in step with it.
|
|
if [ "${#args[@]}" -ne 2 ]; then
|
|
echo "::error::variant ${{ matrix.variant }}: want 2 digests, found ${#args[@]} (a publish leg failed)"
|
|
exit 1
|
|
fi
|
|
if [ "${{ github.event_name }}" = "release" ]; then
|
|
tags=(-t "${IMAGE_NAME}:${RELEASE_TAG}${SUFFIX}")
|
|
else
|
|
tags=(-t "${IMAGE_NAME}:main${SUFFIX}" -t "${IMAGE_NAME}:latest${SUFFIX}")
|
|
fi
|
|
# Retry: Docker Hub API + just-pushed digest eventual consistency
|
|
# can transiently fail the create; the operation is idempotent.
|
|
for i in 1 2 3; do
|
|
if docker buildx imagetools create "${tags[@]}" "${args[@]}"; then
|
|
break
|
|
fi
|
|
if [ "$i" = 3 ]; then
|
|
echo "::error::imagetools create failed after 3 attempts"
|
|
exit 1
|
|
fi
|
|
echo "::warning::imagetools create failed (attempt $i); retrying in 20s"
|
|
sleep 20
|
|
done
|
|
|
|
- name: Inspect image
|
|
env:
|
|
IMAGE_NAME: ${{ env.IMAGE_NAME }}
|
|
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
|
SUFFIX: ${{ matrix.suffix }}
|
|
run: |
|
|
if [ "${{ github.event_name }}" = "release" ]; then
|
|
docker buildx imagetools inspect "${IMAGE_NAME}:${RELEASE_TAG}${SUFFIX}"
|
|
else
|
|
docker buildx imagetools inspect "${IMAGE_NAME}:main${SUFFIX}"
|
|
fi
|