plugin-catalog CI: enforce rule 17 — scan pinned entries for unscannable payloads

The admission gate ran the install scanner (dangerous fails) but nothing refused
a compiled binary, committed bytecode, or an archive hiding either — the shapes
the scanner cannot read. This adds that enforcement to pinned-source-validate.

- scripts/scan_plugin_payloads.py (stdlib-only, so the no-install validator and
  third-party CI can run it): flags compiled binaries by magic header (ELF,
  Mach-O, PE/MZ, WebAssembly — not by extension, which a rename defeats),
  .pyc/.pyo bytecode (incl. sourceless/hash-forged caches that run instead of
  the .py beside them), and archives whose members are not reviewable source.
  Plain-text source bundles and inert assets (fonts, images) pass.
- plugin-catalog-ci.yml: after `hermes plugins validate`, each changed entry is
  scanned; a finding fails it. An entry may list `attested_binaries:` paths,
  each verified with `gh attestation verify --repo <repo> --source-digest <sha>`
  against the pinned commit; verified paths are dropped before the scan, any
  other binary still fails. Repo-controlled filenames are newline-stripped in
  the log so none can forge a workflow command.
- validate_plugin_catalog.py: `attested_binaries` is a known key, shape-checked
  as a list of relative paths.
- Tests: unit tests for the detector and two new workflow-driven bypass cases
  (bundled .exe, committed .pyc) in the confinement harness, which runs the real
  `run:` block.

Full-content re-sweep of all 543 pins: only tabro carries a native binary (it is
delisted in the companion PR). Three entries (crawl4ai, hermes-vision-inline,
mermail) commit __pycache__/*.pyc build litter and would fail rule 17 on their
next pin bump — pre-existing, not touched here.

.github/workflows/plugin-catalog-ci.yml

@@ -119,6 +119,10 @@ jobs:
           CHANGED_FILES: ${{ steps.changed.outputs.files }}
           # A hostile repo URL must never block the job on a credential prompt.
           GIT_TERMINAL_PROMPT: "0"
+          # `gh attestation verify` (README rule 17) reads the attestation from
+          # GitHub and needs a token. Read-only default token; the step runs
+          # pinned-repo code, so nothing broader is exposed here.
+          GH_TOKEN: ${{ github.token }}
         run: |
           set -euo pipefail
           FAILED=0
@@ -257,6 +261,78 @@ jobs:
               FAILED=1; echo "::endgroup::"; continue
             fi
 
+            # UNSCANNABLE-PAYLOAD GATE (README rule 17). A reviewer and the install
+            # scanner must be able to read everything shipped at the pin as source:
+            # no compiled binaries (by header), committed bytecode, or archives
+            # hiding either. Entries that genuinely need a native component list
+            # each one under `attested_binaries:` and the binary must carry a
+            # GitHub build-provenance attestation tying it to this repo's own
+            # workflow at the pinned source digest; those paths are exempted here
+            # and verified below. No stdin: the parser must not drain the entry fd.
+            ATTESTED=$(python3 -I - "$entry" <<'PYEOF' </dev/null
+          import re, shlex, sys
+          from ruamel.yaml import YAML
+          try:
+              with open(sys.argv[1], encoding="utf-8") as fh:
+                  data = YAML(typ="safe").load(fh)
+          except Exception:
+              data = None
+          paths = data.get("attested_binaries") if isinstance(data, dict) else None
+          if not isinstance(paths, list):
+              paths = []
+          ok = []
+          for p in paths:
+              # Relative path, no '.'/'..' segments, no absolute/newline: it is
+              # joined onto the clone and echoed into the log.
+              if (isinstance(p, str) and re.fullmatch(r"[A-Za-z0-9._/-]+", p)
+                      and not any(seg in ("", ".", "..") for seg in p.split("/"))):
+                  ok.append(p)
+              else:
+                  print("BAD=" + shlex.quote(f"attested_binaries entry {p!r} is not a plain relative path"))
+                  sys.exit(0)
+          print("PATHS=" + shlex.quote("\n".join(ok)))
+          PYEOF
+          ) || ATTESTED="BAD='attested_binaries parser failed'"
+            ATT_BAD=""; ATT_PATHS=""
+            eval "${ATTESTED/PATHS=/ATT_PATHS=}"; eval "${ATTESTED/BAD=/ATT_BAD=}" 2>/dev/null || true
+            if [ -n "${ATT_BAD:-}" ]; then
+              echo "::error file=$entry::$ATT_BAD"
+              FAILED=1; echo "::endgroup::"; continue
+            fi
+            # Verify each declared binary's provenance against THIS repo at the
+            # pinned commit; a verified path is dropped from the clone so the
+            # detector that follows ignores it, and any other binary still fails.
+            ATT_OK=1
+            if [ -n "${ATT_PATHS:-}" ]; then
+              OWNER_REPO=$(printf '%s' "$REPO" | sed -E 's#^https://github\.com/##; s#\.git$##; s#/$##')
+              while IFS= read -r bpath; do
+                [ -z "$bpath" ] && continue
+                BFILE="$PLUGIN_DIR/$bpath"
+                if [ ! -f "$BFILE" ] || [ -L "$BFILE" ]; then
+                  echo "::error file=$entry::attested_binaries path is not a regular file: $bpath"
+                  ATT_OK=0; break
+                fi
+                if ! timeout 120 gh attestation verify "$BFILE" \
+                    --repo "$OWNER_REPO" --source-digest "$SHA" \
+                    --predicate-type https://slsa.dev/provenance/v1 </dev/null; then
+                  echo "::error file=$entry::no build-provenance attestation for $bpath at $OWNER_REPO@$SHA (README rule 17)"
+                  ATT_OK=0; break
+                fi
+                rm -f -- "$BFILE"
+              done <<EOF_ATT
+          $ATT_PATHS
+          EOF_ATT
+            fi
+            if [ "$ATT_OK" -ne 1 ]; then
+              FAILED=1; echo "::endgroup::"; continue
+            fi
+            if ! PAYLOAD=$(python3 "$GITHUB_WORKSPACE/scripts/scan_plugin_payloads.py" "$PLUGIN_DIR" </dev/null); then
+              # Filenames are repo-controlled: strip newlines so none can forge a
+              # workflow command in the log.
+              echo "::error file=$entry::unscannable executable payload (README rule 17): $(printf '%s' "$PAYLOAD" | tr '\n' ';')"
+              FAILED=1; echo "::endgroup::"; continue
+            fi
+
             # SELF-UPDATER GATE (README rule 3): the pin is the only update path.
             # A desktop bundle that both fetches from GitHub AND writes/renames
             # plugin files is a self-updater; either half alone is fine (a

scripts/scan_plugin_payloads.py

@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+"""Scan a pinned plugin tree for unscannable executable payloads (catalog rule 17).
+
+A reviewer and the install scanner must be able to read everything shipped at the
+pinned commit as source. This refuses:
+
+  * compiled binaries, by magic header (ELF, Mach-O incl. universal, PE/``MZ``,
+    WebAssembly) — never by extension, which a rename defeats;
+  * committed Python bytecode (``.pyc``/``.pyo`` and sourceless caches), which the
+    interpreter runs in place of the ``.py`` a reviewer read;
+  * archives (zip/jar/whl, tar, gzip/xz/bzip2/zstd) whose members are not all
+    themselves reviewable source — a tarball of ``.py``/``.md`` is fine, one hiding
+    a binary is not.
+
+stdlib only, so the no-install catalog validator and third-party CI can run it.
+A native component is allowed only with a GitHub build attestation; that check
+needs a token and lives in the workflow, not here.
+
+Usage:
+    scan_plugin_payloads.py <plugin-dir>
+Exit 0 = clean, 1 = a payload was found (lines on stdout), 2 = bad usage.
+"""
+from __future__ import annotations
+
+import io
+import sys
+import tarfile
+import zipfile
+from pathlib import Path
+
+# Source extensions whose archives stay reviewable. Anything else inside an
+# archive (a nested archive, a binary, an unknown blob) fails the whole archive.
+_ARCHIVE_SOURCE_SUFFIXES = {
+    ".py", ".pyi", ".ts", ".tsx", ".js", ".mjs", ".cjs", ".jsx", ".json", ".jsonc", ".jsonl",
+    ".md", ".txt", ".rst", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".csv", ".tsv",
+    ".html", ".htm", ".css", ".scss", ".svg", ".sh", ".ps1", ".bat", ".sql", ".proto",
+    ".c", ".h", ".cpp", ".hpp", ".cc", ".rs", ".go", ".java", ".kt", ".rb", ".lua",
+    ".patch", ".diff", ".lock", ".gitignore", ".gitattributes", ".env", ".service",
+    ".xml", ".properties", ".gradle", ".cmake", ".mk", ".conf", ".d.ts", ".map",
+}
+# Inert, non-executable asset types that may ride along inside a source bundle:
+# fonts and images carry no code the scanner needs to read. An unknown binary
+# blob is NOT inert and still fails the archive.
+_ARCHIVE_ASSET_SUFFIXES = {
+    ".ttf", ".otf", ".woff", ".woff2", ".eot",
+    ".png", ".jpg", ".jpeg", ".gif", ".webp", ".ico", ".bmp", ".avif",
+}
+# Extensionless filenames that are conventionally plain-text source/config.
+_SOURCE_BASENAMES = {
+    "dockerfile", "makefile", "license", "licence", "notice", "readme", "copying",
+    "authors", "contributors", "changelog", "codeowners", "procfile", ".gitignore",
+    ".gitattributes", ".dockerignore", ".npmignore", ".editorconfig",
+}
+_ARCHIVE_SUFFIXES = {".zip", ".jar", ".whl", ".egg", ".apk", ".war",
+                     ".tar", ".tgz", ".gz", ".bz2", ".xz", ".zst", ".7z", ".rar"}
+_BYTECODE_SUFFIXES = {".pyc", ".pyo"}
+
+
+def _magic_kind(head: bytes) -> str | None:
+    """Name the compiled-binary format a byte header indicates, else None."""
+    if head.startswith(b"\x7fELF"):
+        return "ELF executable/library"
+    if head.startswith((b"\xcf\xfa\xed\xfe", b"\xce\xfa\xed\xfe",
+                        b"\xca\xfe\xba\xbe", b"\xbe\xba\xfe\xca", b"\xfe\xed\xfa\xce", b"\xfe\xed\xfa\xcf")):
+        return "Mach-O binary"
+    if head.startswith(b"MZ"):
+        return "PE (Windows) executable/DLL"
+    if head.startswith(b"\x00asm"):
+        return "WebAssembly module"
+    return None
+
+
+def _is_source_member(name: str) -> bool:
+    low = name.lower().rstrip("/")
+    if not low or low.endswith("/"):
+        return True  # a directory entry
+    base = low.rsplit("/", 1)[-1]
+    if base in _SOURCE_BASENAMES:
+        return True
+    suffix = "." + base.rsplit(".", 1)[1] if "." in base else ""
+    return suffix in _ARCHIVE_SOURCE_SUFFIXES or suffix in _ARCHIVE_ASSET_SUFFIXES
+
+
+def _archive_problem(path: Path, data: bytes) -> str | None:
+    """Return a reason the archive is unreviewable, or None if every member is source."""
+    try:
+        if zipfile.is_zipfile(io.BytesIO(data)):
+            with zipfile.ZipFile(io.BytesIO(data)) as zf:
+                bad = [n for n in zf.namelist() if not _is_source_member(n)]
+                return f"archive member not reviewable source: {bad[0]}" if bad else None
+        bio = io.BytesIO(data)
+        with tarfile.open(fileobj=bio) as tf:  # autodetects gzip/bz2/xz
+            for member in tf.getmembers():
+                if member.isdir():
+                    continue
+                if not member.isfile():
+                    return f"archive has a non-regular member: {member.name}"
+                if not _is_source_member(member.name):
+                    return f"archive member not reviewable source: {member.name}"
+                # A member named like source/asset must not actually be a compiled
+                # binary (an .exe renamed .png rides in otherwise).
+                fobj = tf.extractfile(member)
+                if fobj is not None and _magic_kind(fobj.read(8)):
+                    return f"archive member is a compiled binary: {member.name}"
+            return None
+    except (tarfile.TarError, zipfile.BadZipFile, OSError, EOFError):
+        # Not a parseable archive we recognise. A file carrying an archive
+        # suffix we cannot open is itself unreviewable.
+        return "archive could not be opened for review"
+    return "archive could not be opened for review"
+
+
+def _file_problem(path: Path, rel: str) -> str | None:
+    """Return a rule-17 reason this single file is unreviewable, or None."""
+    if path.suffix.lower() in _BYTECODE_SUFFIXES:
+        return f"{rel}: committed Python bytecode (ships instead of readable source)"
+    try:
+        head = path.read_bytes()[:8] if path.stat().st_size else b""
+    except OSError:
+        return None
+    kind = _magic_kind(head)
+    if kind:
+        return f"{rel}: compiled binary — {kind} (not reviewable source)"
+    if path.suffix.lower() in _ARCHIVE_SUFFIXES:
+        try:
+            reason = _archive_problem(path, path.read_bytes())
+        except OSError:
+            reason = "archive could not be read"
+        if reason:
+            return f"{rel}: {reason}"
+    return None
+
+
+def scan(plugin_dir: Path) -> list[str]:
+    findings: list[str] = []
+    for path in sorted(plugin_dir.rglob("*")):
+        if path.is_symlink() or not path.is_file():
+            continue
+        if "/.git/" in "/" + path.as_posix():
+            continue
+        problem = _file_problem(path, path.relative_to(plugin_dir).as_posix())
+        if problem:
+            findings.append(problem)
+    return findings
+
+
+def main(argv: list[str]) -> int:
+    if len(argv) != 2:
+        print("usage: scan_plugin_payloads.py <plugin-dir>", file=sys.stderr)
+        return 2
+    root = Path(argv[1])
+    if not root.is_dir():
+        print(f"not a directory: {root}", file=sys.stderr)
+        return 2
+    findings = scan(root)
+    for f in findings:
+        print(f)
+    return 1 if findings else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))

scripts/validate_plugin_catalog.py

@@ -76,6 +76,7 @@ KNOWN_KEYS = {
     "title",
     "onboarding",
     "known_issues",
+    "attested_binaries",
 }
 # Cosmetic labels attached to the pin. ``version`` is never parsed; ``image`` and ``screenshots``
 # may only point at GitHub so the Desktop catalog browser and the docs site never fetch from
@@ -236,9 +237,30 @@ def validate_entry(data: object) -> tuple[list[str], list[str]]:
             ):
                 errors.append(f"capabilities.{key} must be a list of strings")
 
+    # attested_binaries: relative paths to compiled files that carry a GitHub
+    # build-provenance attestation (README rule 17). The CI gate verifies the
+    # attestation; here we only shape-check the paths.
+    _check_attested_binaries(data.get("attested_binaries"), errors)
+
     return errors, warnings
 
 
+def _check_attested_binaries(attested: object, errors: list[str]) -> None:
+    """Shape-check the optional ``attested_binaries`` list of relative paths."""
+    if attested is None:
+        return
+    if not isinstance(attested, list) or not all(isinstance(p, str) for p in attested):
+        errors.append("attested_binaries must be a list of strings")
+        return
+    for p in attested:
+        if not re.fullmatch(r"[A-Za-z0-9._/-]+", p) or any(
+            seg in ("", ".", "..") for seg in p.split("/")
+        ):
+            errors.append(
+                f"attested_binaries path {p!r} must be a relative path without '.' or '..' segments"
+            )
+
+
 def validate_removed(data: object) -> tuple[list[str], list[str]]:
     """Validate the removed.yaml document. Returns (errors, warnings)."""
     errors: list[str] = []

tests/ci/test_plugin_catalog_ci_confinement.py

@@ -102,6 +102,7 @@ def _gate(tmp_path: Path, origin: Path, entries: list[Path],
         # CI runs the step from the hermes-agent checkout; `-I` must ignore this.
         "PYTHONPATH": str(ROOT),
         "GIT_TERMINAL_PROMPT": "0",
+        "GITHUB_WORKSPACE": str(ROOT),
         "GIT_CONFIG_COUNT": "1",
         "GIT_CONFIG_KEY_0": f"url.{origin.as_uri()}.insteadOf",
         "GIT_CONFIG_VALUE_0": REPO_URL,
@@ -156,6 +157,18 @@ def _self_updater_in_spaced_dir(tp, origin, sha):
     return _entry(tp, "evil.yaml", _commit(origin)), "self-updating code"
 
 
+def _bundled_native_binary(tp, origin, sha):
+    # A PE binary committed in the plugin tree (README rule 17). Header, not ext.
+    (origin / "plugin" / "runtime").mkdir()
+    (origin / "plugin" / "runtime" / "host.exe").write_bytes(b"MZ\x90\x00" + b"\x00" * 64)
+    return _entry(tp, "evil.yaml", _commit(origin)), "unscannable executable payload"
+
+
+def _committed_bytecode(tp, origin, sha):
+    (origin / "plugin" / "mod.pyc").write_bytes(b"\x00" * 16)
+    return _entry(tp, "evil.yaml", _commit(origin)), "unscannable executable payload"
+
+
 def _log_forging_filename(tp, origin, sha):
     (origin / "plugin" / "evil\n::error::forged.js").write_text(
         "releases/latest writeFile(", encoding="utf-8")
@@ -176,6 +189,7 @@ def _shadowed_parser_import(tp, origin, sha):
 @pytest.mark.parametrize("case", [
     _traversal_subdir, _symlinked_subdir, _symlink_behind_in_clone_link, _tag_object_sha,
     _newline_sha, _log_forging_repo, _symlinked_entry_file, _self_updater_in_spaced_dir,
+    _bundled_native_binary, _committed_bytecode,
     _log_forging_filename, _shadowed_parser_import,
 ], ids=lambda f: f.__name__.lstrip("_"))
 def test_bypass_fails_the_gate(tmp_path, origin, case):

tests/scripts/test_scan_plugin_payloads.py

@@ -0,0 +1,111 @@
+"""Unit tests for the catalog payload scanner (README rule 17).
+
+``scripts/scan_plugin_payloads.py`` refuses compiled binaries, committed Python
+bytecode and archives hiding either, while leaving plain-text source bundles and
+inert assets (fonts, images) alone. Invariants, not line counts:
+one benign shape stays clean, one attack shape per bypass still fails.
+"""
+from __future__ import annotations
+
+import importlib.util
+import io
+import subprocess
+import sys
+import tarfile
+import zipfile
+from pathlib import Path
+
+import pytest
+
+SCRIPT = Path(__file__).resolve().parents[2] / "scripts" / "scan_plugin_payloads.py"
+
+_spec = importlib.util.spec_from_file_location("scan_plugin_payloads", SCRIPT)
+mod = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(mod)
+
+
+def _scan(d: Path) -> list[str]:
+    return mod.scan(d)
+
+
+def _run(d: Path) -> subprocess.CompletedProcess:
+    return subprocess.run([sys.executable, str(SCRIPT), str(d)], capture_output=True, text=True, check=False)
+
+
+def test_plain_source_tree_is_clean(tmp_path):
+    (tmp_path / "__init__.py").write_text("x = 1\n")
+    (tmp_path / "README.md").write_text("# hi\n")
+    (tmp_path / "data.json").write_text("{}\n")
+    assert _scan(tmp_path) == []
+    assert _run(tmp_path).returncode == 0
+
+
+@pytest.mark.parametrize("name,head", [
+    ("evil.exe", b"MZ\x90\x00"),
+    ("renamed.js", b"MZ\x90\x00"),       # PE header, source extension
+    ("lib.so", b"\x7fELF\x02\x01\x01"),
+    ("odd.dat", b"\x7fELF"),             # ELF, unknown extension
+    ("mod.wasm", b"\x00asm\x01\x00"),
+    ("bin.dylib", b"\xcf\xfa\xed\xfe"),
+])
+def test_compiled_binary_is_flagged_by_header(tmp_path, name, head):
+    (tmp_path / name).write_bytes(head + b"\x00" * 32)
+    findings = _scan(tmp_path)
+    assert any(name in f and "compiled binary" in f for f in findings), findings
+
+
+@pytest.mark.parametrize("name", ["cached.pyc", "sub/thing.pyo"])
+def test_committed_bytecode_is_flagged(tmp_path, name):
+    p = tmp_path / name
+    p.parent.mkdir(parents=True, exist_ok=True)
+    p.write_bytes(b"\x00" * 16)
+    findings = _scan(tmp_path)
+    assert any(name.split("/")[-1] in f and "bytecode" in f for f in findings), findings
+
+
+def test_source_tarball_is_clean(tmp_path):
+    arc = tmp_path / "bundle.tar.gz"
+    with tarfile.open(arc, "w:gz") as tf:
+        for member_name, body in [("a.py", b"x=1\n"), ("docs/readme.md", b"# hi\n"),
+                                  ("Dockerfile", b"FROM scratch\n"), ("logs.jsonl", b"{}\n"),
+                                  ("font.ttf", b"\x00\x01\x00\x00font")]:
+            ti = tarfile.TarInfo(member_name); ti.size = len(body)
+            tf.addfile(ti, io.BytesIO(body))
+    assert _scan(tmp_path) == []
+
+
+def test_tarball_hiding_a_binary_fails(tmp_path):
+    arc = tmp_path / "bundle.tar.gz"
+    with tarfile.open(arc, "w:gz") as tf:
+        body = b"\x7fELF" + b"\x00" * 40     # ELF under a .py member name
+        ti = tarfile.TarInfo("mod.py"); ti.size = len(body)
+        tf.addfile(ti, io.BytesIO(body))
+    findings = _scan(tmp_path)
+    assert any("compiled binary" in f for f in findings), findings
+
+
+def test_tarball_with_unknown_blob_member_fails(tmp_path):
+    arc = tmp_path / "bundle.tgz"
+    with tarfile.open(arc, "w:gz") as tf:
+        body = b"\x00\x01\x02opaque"
+        ti = tarfile.TarInfo("payload.bin"); ti.size = len(body)
+        tf.addfile(ti, io.BytesIO(body))
+    findings = _scan(tmp_path)
+    assert any("not reviewable source" in f for f in findings), findings
+
+
+def test_zip_hiding_a_binary_fails(tmp_path):
+    arc = tmp_path / "bundle.zip"
+    with zipfile.ZipFile(arc, "w") as zf:
+        zf.writestr("inner.exe", b"MZ\x90\x00")
+    findings = _scan(tmp_path)
+    assert any("not reviewable source" in f for f in findings), findings
+
+
+def test_symlinks_are_not_followed(tmp_path):
+    (tmp_path / "real.py").write_text("x=1\n")
+    try:
+        (tmp_path / "link").symlink_to("/bin/ls")
+    except OSError:
+        pytest.skip("symlinks unsupported")
+    assert _scan(tmp_path) == []