cdce56faf2
The admission gate ran the install scanner (dangerous fails) but nothing refused a compiled binary, committed bytecode, or an archive hiding either — the shapes the scanner cannot read. This adds that enforcement to pinned-source-validate. - scripts/scan_plugin_payloads.py (stdlib-only, so the no-install validator and third-party CI can run it): flags compiled binaries by magic header (ELF, Mach-O, PE/MZ, WebAssembly — not by extension, which a rename defeats), .pyc/.pyo bytecode (incl. sourceless/hash-forged caches that run instead of the .py beside them), and archives whose members are not reviewable source. Plain-text source bundles and inert assets (fonts, images) pass. - plugin-catalog-ci.yml: after `hermes plugins validate`, each changed entry is scanned; a finding fails it. An entry may list `attested_binaries:` paths, each verified with `gh attestation verify --repo <repo> --source-digest <sha>` against the pinned commit; verified paths are dropped before the scan, any other binary still fails. Repo-controlled filenames are newline-stripped in the log so none can forge a workflow command. - validate_plugin_catalog.py: `attested_binaries` is a known key, shape-checked as a list of relative paths. - Tests: unit tests for the detector and two new workflow-driven bypass cases (bundled .exe, committed .pyc) in the confinement harness, which runs the real `run:` block. Full-content re-sweep of all 543 pins: only tabro carries a native binary (it is delisted in the companion PR). Three entries (crawl4ai, hermes-vision-inline, mermail) commit __pycache__/*.pyc build litter and would fail rule 17 on their next pin bump — pre-existing, not touched here.
362 lines
18 KiB
YAML
362 lines
18 KiB
YAML
name: Plugin Catalog CI
|
|
|
|
# Admission gate for plugin-catalog entries. Fires ONLY on PRs touching
|
|
# plugin-catalog/** so it can never go red on unrelated PRs.
|
|
#
|
|
# Two gates:
|
|
# structural — cheap schema check, no hermes install needed
|
|
# pinned-source-validate — supply-chain gate: the pinned sha MUST be
|
|
# reachable in the entry's repo, and the plugin
|
|
# at that exact commit must pass
|
|
# `hermes plugins validate`.
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- "plugin-catalog/**"
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
structural:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7.0.1
|
|
with:
|
|
persist-credentials: false # this job is read-only
|
|
|
|
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
- name: Install ruamel.yaml
|
|
uses: ./.github/actions/retry
|
|
with:
|
|
command: pip install ruamel.yaml==0.18.17
|
|
|
|
- name: Validate catalog files (structural)
|
|
run: |
|
|
set -euo pipefail
|
|
# Validating the whole directory is simpler than diffing and keeps
|
|
# the invariant that EVERYTHING in plugin-catalog/ stays valid.
|
|
python3 scripts/validate_plugin_catalog.py plugin-catalog/
|
|
|
|
pinned-source-validate:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7.0.1
|
|
with:
|
|
# Full commit graph + tags, blobs on demand: the diff below needs the
|
|
# merge commit's first parent, and Hermes resolves its own version
|
|
# from the release tags merged into HEAD. A shallow checkout reports
|
|
# "unknown", which plugins checking the host version reject (omh).
|
|
fetch-depth: 0
|
|
filter: blob:none
|
|
# `hermes plugins validate` below runs pinned-repo code; keep the
|
|
# job's token out of .git/config where that code could read it.
|
|
persist-credentials: false
|
|
|
|
- name: Find changed catalog entries
|
|
id: changed
|
|
run: |
|
|
set -euo pipefail
|
|
# HEAD is the PR merge commit: diffing it against its first parent is
|
|
# exactly what the PR would land. Refuse anything else rather than
|
|
# silently diffing a single commit.
|
|
if ! git rev-parse -q --verify 'HEAD^2' >/dev/null; then
|
|
echo "::error::expected the pull_request merge commit as HEAD"
|
|
exit 1
|
|
fi
|
|
# Added + modified + renamed + typechanged entry files only; deletions
|
|
# and removed.yaml have nothing to clone. A rename or file->symlink
|
|
# change that slipped past this list would skip the supply-chain gate.
|
|
CHANGED=$(git diff --name-only --diff-filter=AMRT 'HEAD^1' HEAD \
|
|
-- 'plugin-catalog/*.yaml' 'plugin-catalog/*.yml')
|
|
# The grep can only shrink the list and runs on its own so a git diff
|
|
# failure still fails the step instead of looking like "no changes".
|
|
CHANGED=$(printf '%s' "$CHANGED" | grep -v '/removed\.yaml$' || true)
|
|
|
|
if [ -n "$CHANGED" ]; then
|
|
# Admission PRs must be data-only: the gate's own enforcement code
|
|
# (this workflow, the validator, the composite actions, the built
|
|
# hermes) resolves from the PR checkout, so an entry PR that also
|
|
# touches tooling could neuter the gate vetting it. Contributor
|
|
# email maps are data the gate never executes; README/policy/CI PRs
|
|
# with no entry change skip this check.
|
|
ALL=$(git diff --name-only 'HEAD^1' HEAD)
|
|
OUTSIDE=$(printf '%s' "$ALL" | grep -vE '^(plugin-catalog|contributors/emails)/' || true)
|
|
if [ -n "$OUTSIDE" ]; then
|
|
echo "::error::catalog entry PRs may only touch plugin-catalog/** and contributors/emails/**, found:"
|
|
# Filenames are PR-controlled; a newline in one could smuggle a
|
|
# workflow command into the log.
|
|
printf '%s' "$OUTSIDE" | tr '\0\n' ' '
|
|
echo
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
echo "Changed catalog entries:"
|
|
echo "${CHANGED:-<none>}"
|
|
{
|
|
echo 'files<<__EOF__'
|
|
echo "$CHANGED"
|
|
echo '__EOF__'
|
|
} >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Prepare Hermes from the PR's own checkout
|
|
if: steps.changed.outputs.files != ''
|
|
uses: ./.github/actions/setup-pm
|
|
with:
|
|
toolchain: python
|
|
extras: '[]'
|
|
|
|
- name: Clone each entry at its pinned sha and validate
|
|
if: steps.changed.outputs.files != ''
|
|
env:
|
|
CHANGED_FILES: ${{ steps.changed.outputs.files }}
|
|
# A hostile repo URL must never block the job on a credential prompt.
|
|
GIT_TERMINAL_PROMPT: "0"
|
|
# `gh attestation verify` (README rule 17) reads the attestation from
|
|
# GitHub and needs a token. Read-only default token; the step runs
|
|
# pinned-repo code, so nothing broader is exposed here.
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
FAILED=0
|
|
WORKROOT=$(realpath "$(mktemp -d)")
|
|
# Entries arrive on fd 3, not stdin: `hermes plugins validate` executes
|
|
# plugin-declared install code that could otherwise read stdin and
|
|
# silently drain the remaining entry list.
|
|
while IFS= read -r entry <&3; do
|
|
[ -z "$entry" ] && continue
|
|
echo "::group::validate $entry"
|
|
|
|
# A symlinked or missing entry file is not a catalog entry.
|
|
if [ -L "$entry" ] || [ ! -f "$entry" ]; then
|
|
echo "::error file=$entry::catalog entry must be a regular file"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
# Parse repo / sha / subdir from the entry yaml. These fields are
|
|
# PR-controlled, so the entry is rejected here before anything uses
|
|
# them -- the clone must be remote-only and the subdir a plain
|
|
# relative path, or the pinned sha ends up validating files outside
|
|
# the pinned commit. `python3 -I` keeps the PR checkout's own
|
|
# modules off sys.path (cwd is the PR tree) so it cannot shadow the
|
|
# parser's imports; ruamel.yaml comes from installed packages.
|
|
PARSED=$(python3 -I - "$entry" <<'PYEOF'
|
|
import re
|
|
import shlex
|
|
import sys
|
|
|
|
from ruamel.yaml import YAML
|
|
|
|
try:
|
|
with open(sys.argv[1], encoding="utf-8") as fh:
|
|
data = YAML(typ='safe').load(fh)
|
|
except Exception:
|
|
data = None
|
|
repo = str(data.get('repo', '')) if isinstance(data, dict) else ''
|
|
sha = data.get('sha', '') if isinstance(data, dict) else ''
|
|
subdir_raw = data.get('subdir', '') if isinstance(data, dict) else ''
|
|
if subdir_raw is None:
|
|
subdir_raw = '' # `subdir:` with no value means the repo root
|
|
subdir = subdir_raw if isinstance(subdir_raw, str) else ''
|
|
bad = ""
|
|
if data is None:
|
|
bad = "entry yaml is empty or unparseable"
|
|
elif not isinstance(data, dict):
|
|
bad = "entry yaml must be a mapping"
|
|
elif not re.fullmatch(r"https://\S+", repo):
|
|
# No whitespace either: the value is echoed into the log, where a
|
|
# newline would smuggle workflow commands like ::error::.
|
|
bad = "repo must be an https:// git URL"
|
|
elif not isinstance(sha, str):
|
|
bad = "sha must be a string"
|
|
elif not re.fullmatch(r"[0-9a-f]{40}", sha):
|
|
# A ref name would validate the branch tip, not the pin; a leading
|
|
# '-' would become a checkout option.
|
|
bad = "sha must be exactly 40 lowercase hex characters"
|
|
elif not isinstance(subdir_raw, str):
|
|
# Never coerce: a falsy non-string (0, false) would silently mean
|
|
# the clone root while the structural gate rejects the entry.
|
|
bad = "subdir must be a string"
|
|
elif subdir and (
|
|
not re.fullmatch(r"[A-Za-z0-9._/-]+", subdir)
|
|
or any(seg in ("", ".", "..") for seg in subdir.split("/"))
|
|
):
|
|
bad = "subdir must be a relative path without '.' or '..' segments"
|
|
print(f"REPO={shlex.quote(repo)}")
|
|
print(f"SHA={shlex.quote(str(sha))}")
|
|
print(f"SUBDIR={shlex.quote(subdir)}")
|
|
print(f"BAD={shlex.quote(bad)}")
|
|
PYEOF
|
|
) || PARSED="BAD='catalog entry parser failed'"
|
|
# Reset before eval: a killed or truncated parser must not let this
|
|
# entry inherit a previous entry's repo/sha and pass on its content.
|
|
REPO="" SHA="" SUBDIR="" BAD=""
|
|
eval "$PARSED"
|
|
# Check before echo: a rejected value must never reach the log, where
|
|
# a newline in it would smuggle workflow commands like ::error::.
|
|
if [ -n "$BAD" ]; then
|
|
echo "::error file=$entry::$BAD"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
echo "repo=$REPO sha=$SHA subdir=$SUBDIR"
|
|
|
|
CLONE_DIR=$(realpath "$(mktemp -d -p "$WORKROOT")")
|
|
# Full clone (no --depth 1): the pinned sha may not be the branch tip.
|
|
# 600s: listed repos reach >1 GB (hermes-jackal-verified 1.43 GB,
|
|
# hindsight 741 MB); 120s would false-fail a re-pin on a slow runner.
|
|
if ! timeout 600 git clone "$REPO" "$CLONE_DIR" </dev/null; then
|
|
echo "::error file=$entry::clone failed for $REPO"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
# The pin must name a commit object -- a tag-object sha would peel
|
|
# to a commit the reviewer never declared as the pin.
|
|
if [ "$(git -C "$CLONE_DIR" cat-file -t "$SHA" 2>/dev/null)" != "commit" ]; then
|
|
echo "::error file=$entry::pinned sha $SHA is not a commit object in $REPO"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
# SUPPLY-CHAIN GATE: the pinned sha must be reachable in the repo.
|
|
if ! git -C "$CLONE_DIR" checkout --detach "$SHA" </dev/null; then
|
|
echo "::error file=$entry::pinned sha $SHA is not reachable in $REPO"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
PLUGIN_DIR="$CLONE_DIR${SUBDIR:+/$SUBDIR}"
|
|
# A symlinked directory inside the clone could still point outside
|
|
# it, so confine the resolved path under the clone, not the lexical one.
|
|
RESOLVED_DIR=$(realpath -m -- "$PLUGIN_DIR")
|
|
if [ "$RESOLVED_DIR" != "$CLONE_DIR" ] && [ "${RESOLVED_DIR#"$CLONE_DIR"/}" = "$RESOLVED_DIR" ]; then
|
|
echo "::error file=$entry::subdir '$SUBDIR' resolves outside the pinned clone"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
# Confinement is per-symlink too, across the whole clone: find does
|
|
# not descend into a symlinked directory, so an allowed in-clone
|
|
# link like plugin/ext -> ../lib would otherwise hide lib/evil ->
|
|
# /outside from a PLUGIN_DIR-only scan while reads through it still
|
|
# escape the pin.
|
|
if ! find "$CLONE_DIR" -path "$CLONE_DIR/.git" -prune -o -type l -print0 | while IFS= read -r -d '' link; do
|
|
target=$(realpath -m -- "$link")
|
|
case "$target" in "$CLONE_DIR"|"$CLONE_DIR"/*) ;; *) exit 1 ;; esac
|
|
done; then
|
|
echo "::error file=$entry::a symlink in the repo resolves outside the pinned clone"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
# Native plugin.yaml OR portable Agent Plugins v1 plugin.json
|
|
# (#81196; native manifest wins when both exist).
|
|
if [ ! -f "$PLUGIN_DIR/plugin.yaml" ] && [ ! -f "$PLUGIN_DIR/plugin.yml" ] && [ ! -f "$PLUGIN_DIR/plugin.json" ]; then
|
|
echo "::error file=$entry::no plugin.yaml or plugin.json at subdir '$SUBDIR' of $REPO@$SHA"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
# Manifest schema + declared-vs-registered capability check.
|
|
if ! timeout 600 hermes plugins validate --install-deps "$PLUGIN_DIR" </dev/null; then
|
|
echo "::error file=$entry::hermes plugins validate failed"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
# UNSCANNABLE-PAYLOAD GATE (README rule 17). A reviewer and the install
|
|
# scanner must be able to read everything shipped at the pin as source:
|
|
# no compiled binaries (by header), committed bytecode, or archives
|
|
# hiding either. Entries that genuinely need a native component list
|
|
# each one under `attested_binaries:` and the binary must carry a
|
|
# GitHub build-provenance attestation tying it to this repo's own
|
|
# workflow at the pinned source digest; those paths are exempted here
|
|
# and verified below. No stdin: the parser must not drain the entry fd.
|
|
ATTESTED=$(python3 -I - "$entry" <<'PYEOF' </dev/null
|
|
import re, shlex, sys
|
|
from ruamel.yaml import YAML
|
|
try:
|
|
with open(sys.argv[1], encoding="utf-8") as fh:
|
|
data = YAML(typ="safe").load(fh)
|
|
except Exception:
|
|
data = None
|
|
paths = data.get("attested_binaries") if isinstance(data, dict) else None
|
|
if not isinstance(paths, list):
|
|
paths = []
|
|
ok = []
|
|
for p in paths:
|
|
# Relative path, no '.'/'..' segments, no absolute/newline: it is
|
|
# joined onto the clone and echoed into the log.
|
|
if (isinstance(p, str) and re.fullmatch(r"[A-Za-z0-9._/-]+", p)
|
|
and not any(seg in ("", ".", "..") for seg in p.split("/"))):
|
|
ok.append(p)
|
|
else:
|
|
print("BAD=" + shlex.quote(f"attested_binaries entry {p!r} is not a plain relative path"))
|
|
sys.exit(0)
|
|
print("PATHS=" + shlex.quote("\n".join(ok)))
|
|
PYEOF
|
|
) || ATTESTED="BAD='attested_binaries parser failed'"
|
|
ATT_BAD=""; ATT_PATHS=""
|
|
eval "${ATTESTED/PATHS=/ATT_PATHS=}"; eval "${ATTESTED/BAD=/ATT_BAD=}" 2>/dev/null || true
|
|
if [ -n "${ATT_BAD:-}" ]; then
|
|
echo "::error file=$entry::$ATT_BAD"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
# Verify each declared binary's provenance against THIS repo at the
|
|
# pinned commit; a verified path is dropped from the clone so the
|
|
# detector that follows ignores it, and any other binary still fails.
|
|
ATT_OK=1
|
|
if [ -n "${ATT_PATHS:-}" ]; then
|
|
OWNER_REPO=$(printf '%s' "$REPO" | sed -E 's#^https://github\.com/##; s#\.git$##; s#/$##')
|
|
while IFS= read -r bpath; do
|
|
[ -z "$bpath" ] && continue
|
|
BFILE="$PLUGIN_DIR/$bpath"
|
|
if [ ! -f "$BFILE" ] || [ -L "$BFILE" ]; then
|
|
echo "::error file=$entry::attested_binaries path is not a regular file: $bpath"
|
|
ATT_OK=0; break
|
|
fi
|
|
if ! timeout 120 gh attestation verify "$BFILE" \
|
|
--repo "$OWNER_REPO" --source-digest "$SHA" \
|
|
--predicate-type https://slsa.dev/provenance/v1 </dev/null; then
|
|
echo "::error file=$entry::no build-provenance attestation for $bpath at $OWNER_REPO@$SHA (README rule 17)"
|
|
ATT_OK=0; break
|
|
fi
|
|
rm -f -- "$BFILE"
|
|
done <<EOF_ATT
|
|
$ATT_PATHS
|
|
EOF_ATT
|
|
fi
|
|
if [ "$ATT_OK" -ne 1 ]; then
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
if ! PAYLOAD=$(python3 "$GITHUB_WORKSPACE/scripts/scan_plugin_payloads.py" "$PLUGIN_DIR" </dev/null); then
|
|
# Filenames are repo-controlled: strip newlines so none can forge a
|
|
# workflow command in the log.
|
|
echo "::error file=$entry::unscannable executable payload (README rule 17): $(printf '%s' "$PAYLOAD" | tr '\n' ';')"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
|
|
# SELF-UPDATER GATE (README rule 3): the pin is the only update path.
|
|
# A desktop bundle that both fetches from GitHub AND writes/renames
|
|
# plugin files is a self-updater; either half alone is fine (a
|
|
# plugin may read the API, or manage its own data files).
|
|
# NUL-delimited both ways: committed filenames with spaces or
|
|
# newlines must not split into grep options or forged log lines.
|
|
SELF_UPDATE=$(grep -rlZE 'releases/latest|releases/download|raw\.githubusercontent\.com|codeload\.github\.com' \
|
|
--include='*.js' --include='*.mjs' --include='*.cjs' --include='*.ts' \
|
|
--include='*.jsx' --include='*.tsx' --include='*.mts' --include='*.cts' \
|
|
"$PLUGIN_DIR" \
|
|
| xargs -r0 grep -lZE 'writeTextFile|renamePath|writeFile\(|writeFileSync|renameSync|copyFileSync|cpSync|createWriteStream' || true)
|
|
SELF_UPDATE=$(printf '%s' "$SELF_UPDATE" | tr '\0\n' ' ')
|
|
if [ -n "$SELF_UPDATE" ]; then
|
|
echo "::error file=$entry::self-updating code in catalog build (fetches GitHub AND writes plugin files): $SELF_UPDATE"
|
|
FAILED=1; echo "::endgroup::"; continue
|
|
fi
|
|
echo "✅ PASS: $entry"
|
|
echo "::endgroup::"
|
|
done 3<<< "$CHANGED_FILES"
|
|
rm -rf "$WORKROOT"
|
|
|
|
if [ "$FAILED" -ne 0 ]; then
|
|
echo "❌ FAIL: one or more catalog entries failed pinned-source validation"
|
|
exit 1
|
|
fi
|
|
echo "✅ PASS: all changed catalog entries validated at their pinned shas"
|