test(install-e2e): preserve plugins/** and profile plugin trees across real upgrades

Add a Hermes-owned upgrade-preservation contract to the install E2E
harness: a tagged upgrade must not delete or modify anything under the
active home's plugins/** or any profile's plugins/<name>/plugins tree.

- tests/install/e2e-assets/verify-plugin-preservation.py: standalone
  stdlib-only READ-ONLY verifier. snapshot records every entry (kind,
  size + sha256, link target, recursive fingerprint of symlinked
  external targets so the externally-owned sidecar witness is covered)
  including empty dirs and the roots themselves (lstat, so dangling
  root links are still scanned); verify fails on deletion or
  modification, treats unreadable paths as hard errors, and refuses an
  empty snapshot as inconclusive. Runs under python3/python on all
  three driver platforms.
- tests/install/e2e-assets/preserve-plugins.sh: POSIX/macOS hooks.
  After install: seed controlled non-dependency directory fixtures
  (mnemosyne-wrapper marker + payload + symlinked runtime, second
  profile plugin tree, external witness outside the home; no pyproject
  in the scanned root, nothing downloaded) and snapshot. After update:
  verify; violation fails the leg. Seeding never clobbers a populated
  wrapper without the expected marker.
- installer-script-e2e.sh / macos-desktop-e2e.sh / windows-e2e.ps1:
  wire before/after hooks into the update leg, and add --update-ref
  (-UpdateRef on Windows) so a leg can target an actual stable tag
  instead of HEAD; HEAD-upgrade legs keep the HEAD label. Matrix and
  workflows unchanged.
- tests/scripts/test_verify_plugin_preservation.py: 17 unit tests on a
  real temp filesystem covering clean survival, file/marker deletion
  and modification, whole-root deletion, empty-dir deletion, symlink
  repoint, external witness tamper, read-only guarantee, empty-snapshot
  refusal, unreadable-path hard error (POSIX), and the exact CLI
  round-trip the drivers use.
- tests/install/README.md: document the contract, the hooks, and the
  stable-to-stable rule.
This commit is contained in:
ethernet
2026-09-06 22:24:21 -04:00
parent abfefbf1fa
commit abade99459
7 changed files with 879 additions and 24 deletions

View File

@@ -81,6 +81,40 @@ Cost per run, so nobody is surprised: 41 legs per sampled tag (windows 18, macos
Running the drivers locally: don't, except in a disposable VM. The windows driver kills every process named Hermes during teardown and the macos driver operates on `/Applications/Hermes.app`; on a machine with a real Hermes install they will interfere with it.
## Plugin upgrade preservation
Every upgrade leg also carries a plugin-survival contract: a tagged upgrade
must not delete or modify anything under the active home's `plugins/**` tree
or any profile's `profiles/<name>/plugins/**` tree. Destructive flows
(explicit uninstall, plugin removal, profile or user deletion) are out of
contract and not exercised.
- `e2e-assets/verify-plugin-preservation.py` is the shared, stdlib-only,
read-only verifier. `snapshot` records every entry (kind, byte size +
sha256, link targets, and a recursive fingerprint of a symlink's external
target) across all plugin roots, including empty directories and the roots
themselves; `verify` diffs the live tree against that snapshot and fails
on any deletion or modification. Unreadable paths are hard errors; an
empty snapshot is refused as inconclusive rather than claimed as a pass.
- `e2e-assets/preserve-plugins.sh` is the POSIX/macOS hook pair: after the
install phase it seeds controlled, non-dependency directory fixtures (a
`mnemosyne-wrapper` plugin with its marker, a symlinked runtime, a second
profile plugin tree, and the externally-owned sidecar witness outside the
home — no pyproject anywhere in the scanned root, nothing downloaded) and
snapshots; after the update lands it verifies and fails the leg on any
violation. Seeding is not a clobber: a populated wrapper without the
expected marker aborts the leg. The Windows driver carries the same
fixtures and hooks inline (`Seed-PreservationFixtures`,
`Invoke-PreserveSnapshot`, `Invoke-PreserveVerify`).
- Unit tests live at `tests/scripts/test_verify_plugin_preservation.py` and
exercise the verifier against a real temp filesystem (real files, real
symlinks; junction fallback on Windows).
- Stable-to-stable: the drivers accept `--update-ref REF` (Windows:
`-UpdateRef`), defaulting to HEAD. Pass the next release tag to target a
stable→stable upgrade through the same serve.git staging; only label a leg
stable-to-stable when BOTH the install ref and the target ref are release
tags. The workflow matrix itself is unchanged.
## Artifacts
Each leg uploads its logs as an artifact. Every leg also records the screen for its whole run: the composite action `.github/actions/e2e-screen-record` installs ffmpeg, records with the OS's capture backend (x11grab on linux, gdigrab on windows, avfoundation on macos), and fails the leg if the recording is missing or has zero frames. Linux runners have no display, so the action starts `Xvfb :99` first and exports `DISPLAY` for every later step — the app under test and the recorder share that display. The windows GUI leg also uploads screenshots and the update result file. Get them with `gh run download <run-id>`.

View File

@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# Shared plugin upgrade-preservation hooks for the install E2E drivers
# (POSIX + macOS). Sourced by tests/install/installer-script-e2e.sh and
# tests/install/macos-desktop-e2e.sh.
#
# The contract under test: a tagged Hermes upgrade must NOT delete or modify
# anything in the active home's plugins/** tree or any profile's plugins/**
# tree — including directory wrapper markers (mnemosyne-wrapper.json),
# symlinked runtimes, and the externally-owned sidecar witness file that
# lives OUTSIDE the home. Destructive flows (explicit uninstall, plugin
# removal, profile/user deletion) are out of contract and not exercised.
#
# The fixtures are deliberately non-dependency: directory wrappers with no
# pyproject anywhere in the scanned root, so the plugin scanner never
# recurses into a dependency graph and nothing is downloaded.
#
# State flow (called by the driver):
# preserve_before_upgrade seed fixtures + snapshot -> $PRESERVE_SNAPSHOT
# preserve_after_upgrade verify against snapshot (exit 1 on violation)
#
# Requires: HERMES_HOME set (the leg's isolated home), WORK_ROOT, LOG_DIR,
# REPO_ROOT. Verifier: tests/install/e2e-assets/verify-plugin-preservation.py
# (stdlib-only; runs under python3 or $HERMES_E2E_PYTHON).
PRESERVE_SNAPSHOT="$WORK_ROOT/plugin-preservation-snapshot.json"
PRESERVE_REPORT="$LOG_DIR/plugin-preservation-report.json"
PRESERVE_EXTERNAL="$WORK_ROOT/external-mnemosyne-runtime"
_preserve_python() {
if [ -n "${HERMES_E2E_PYTHON:-}" ]; then
printf '%s' "$HERMES_E2E_PYTHON"
else
printf 'python3'
fi
}
seed_plugin_preservation_fixtures() {
# NOT a clobbering seeder: if the wrapper is already populated the fixture
# state is left untouched, so a re-entered leg cannot erase a regression
# the earlier phase recorded. Only a matching marker may be pre-existing.
local home="$1" external="$2"
local wrapper="$home/plugins/mnemosyne-wrapper"
local marker="$wrapper/mnemosyne-wrapper.json"
if [ -d "$wrapper" ] && [ -n "$(ls -A "$wrapper" 2>/dev/null)" ]; then
grep -qF '"marker": "mnemosyne-wrapper"' "$marker" 2>/dev/null \
|| fail "refusing to reseed preservation fixtures: $wrapper already populated without the expected marker"
ok "preservation fixtures already present; left untouched"
else
mkdir -p "$wrapper"
printf '{"wrapper": true, "marker": "mnemosyne-wrapper", "owner": "e2e-preservation"}\n' \
> "$marker"
printf '#!/usr/bin/env python3\n# directory wrapper fixture: no dependencies\nPLUGIN = "mnemosyne-wrapper"\n' \
> "$wrapper/plugin.py"
if [ ! -e "$wrapper/runtime" ] && [ ! -L "$wrapper/runtime" ]; then
ln -s "$external" "$wrapper/runtime" \
|| fail "could not create the runtime link at $wrapper/runtime; the preservation contract cannot be exercised"
fi
fi
mkdir -p "$external" "$home/profiles/e2e-preserve/plugins/second-plugin"
[ -e "$external/sidecar-witness.txt" ] \
|| printf 'external-sidecar-witness-v1\n' > "$external/sidecar-witness.txt"
[ -e "$external/engine.bin" ] \
|| printf '\x00\x01\x02external-engine\n' > "$external/engine.bin"
[ -e "$home/profiles/e2e-preserve/plugins/second-plugin/marker.json" ] \
|| printf '{"plugin": "second-plugin", "profile": "e2e-preserve"}\n' \
> "$home/profiles/e2e-preserve/plugins/second-plugin/marker.json"
[ -e "$home/profiles/e2e-preserve/plugins/second-plugin/data.bin" ] \
|| printf 'profile-plugin-bytes\n' \
> "$home/profiles/e2e-preserve/plugins/second-plugin/data.bin"
ok "seeded preservation fixtures: $wrapper (marker + runtime link) + profile tree; external witness at $external"
}
preserve_before_upgrade() {
step "plugin preservation: seeding fixtures and snapshotting pre-upgrade state"
seed_plugin_preservation_fixtures "$HERMES_HOME" "$PRESERVE_EXTERNAL"
local py; py="$(_preserve_python)"
"$py" "$REPO_ROOT/tests/install/e2e-assets/verify-plugin-preservation.py" \
snapshot --home "$HERMES_HOME" --out "$PRESERVE_SNAPSHOT" 2>&1 | ts_prefix \
|| fail "plugin preservation snapshot failed"
# An empty snapshot proves nothing; refuse to build the leg's claim on it.
"$py" - "$PRESERVE_SNAPSHOT" <<'PYEOF' || fail "plugin preservation snapshot is EMPTY: no plugin entries recorded, cannot verify preservation"
import json, sys
with open(sys.argv[1], encoding="utf-8") as fh:
snap = json.load(fh)
n = len(snap.get("entries", {}))
print(f"snapshot entries: {n}")
raise SystemExit(0 if n else 3)
PYEOF
ok "pre-upgrade plugin snapshot at $PRESERVE_SNAPSHOT"
}
preserve_after_upgrade() {
step "plugin preservation: verifying post-upgrade state"
[ -f "$PRESERVE_SNAPSHOT" ] \
|| fail "no pre-upgrade plugin snapshot at $PRESERVE_SNAPSHOT; cannot verify preservation"
local py; py="$(_preserve_python)"
local rc=0
"$py" "$REPO_ROOT/tests/install/e2e-assets/verify-plugin-preservation.py" \
verify --home "$HERMES_HOME" --snapshot "$PRESERVE_SNAPSHOT" \
--report "$PRESERVE_REPORT" > "$LOG_DIR/plugin-preservation-verify.log" 2>&1 || rc=$?
log_group "plugin preservation verify transcript" "$LOG_DIR/plugin-preservation-verify.log"
[ "$rc" -eq 0 ] \
|| fail "plugin preservation violated by the upgrade: deleted/modified entries above; report at $PRESERVE_REPORT"
ok "plugins/** and profile plugin trees (markers, symlinked runtimes, external sidecar witness) survived the upgrade intact"
}

View File

@@ -0,0 +1,287 @@
#!/usr/bin/env python3
"""Plugin upgrade-preservation verifier (Hermes release-harness hook).
Standalone, stdlib-only, READ-ONLY against the scanned home. Two modes:
snapshot walk every plugin tree of a HERMES_HOME (the active home's
``plugins/**`` plus each ``profiles/<name>/plugins/**`` tree,
overridable with --profiles-dir) and record, per entry —
including EMPTY DIRECTORIES and the tree roots themselves —
kind (file/dir/symlink/junction), byte size + sha256 for
regular files, link target, and a recursive fingerprint of a
symlink's external target (so an externally-owned sidecar
witness file cannot be tampered with unnoticed) -> JSON file.
verify re-walk the same home and diff against a snapshot. FAILS
(exit 1) on any deleted or modified entry. New entries are
reported but do not fail: an upgrade may add files; it may
not take yours away or alter them.
Unreadable paths are a hard error (exit 2), never a silent skip: a
scanner that cannot see a file cannot defend it. The verifier never
creates, deletes or writes anything under the scanned home.
Usage:
python verify-plugin-preservation.py snapshot --home <HERMES_HOME> --out snap.json
python verify-plugin-preservation.py verify --home <HERMES_HOME> --snapshot snap.json [--report r.json]
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import stat
import sys
SCHEMA_VERSION = 2
PROFILES_DIR = "profiles"
PLUGIN_ROOT = "plugins"
CHUNK = 1 << 20
# Upper bound for the recursive fingerprint of a symlink's external target:
# the harness points links at small controlled fixtures, but a stray link at
# a huge tree must not explode the snapshot.
MAX_EXTERNAL_ENTRIES = 10000
class ScanError(Exception):
pass
def _sha256_file(path: str) -> str:
h = hashlib.sha256()
with open(path, "rb") as fh:
while True:
block = fh.read(CHUNK)
if not block:
break
h.update(block)
return h.hexdigest()
def _is_link(path: str) -> bool:
"""True for symlinks everywhere, and for NTFS junctions on Windows
(st.islink() is False for junctions, but readlink() works). POSIX
behavior is untouched: FILE_ATTRIBUTE_REPARSE_POINT only exists on NT."""
if os.path.islink(path):
return True
st = os.lstat(path)
reparse = getattr(st, "st_file_attributes", 0)
return bool(reparse & stat.FILE_ATTRIBUTE_REPARSE_POINT)
def _lstat_exists(path: str) -> bool:
"""Exists including dangling symlinks/junctions (os.path.exists hides
those by resolving)."""
try:
os.lstat(path)
return True
except OSError:
return False
def plugin_roots(home: str, profiles_dir: str | None = None) -> list[str]:
"""Every plugin tree that must survive an upgrade: the active home's
plugins/ root plus each profile's plugins/ root (found under
profiles_dir, default <home>/profiles). Roots are detected with lstat so
a root that is itself a (possibly dangling) symlink is still scanned and
fingerprinted. Returns stable "<home>/..." labels for existing roots."""
roots: list[str] = []
if _lstat_exists(os.path.join(home, PLUGIN_ROOT)):
roots.append("<home>/" + PLUGIN_ROOT)
profiles = profiles_dir or os.path.join(home, PROFILES_DIR)
if os.path.isdir(profiles):
for name in sorted(os.listdir(profiles)):
p_root = os.path.join(profiles, name, PLUGIN_ROOT)
if _lstat_exists(p_root):
label = os.path.relpath(p_root, home).replace(os.sep, "/")
roots.append("<home>/" + label)
return roots
def _fingerprint_tree(root: str, budget: list[int]) -> dict:
"""Recursive content fingerprint of a directory (used for the EXTERNAL
target of a plugin-runtime symlink: the sidecar witness is owned outside
the home, so its content must be hashed here, not just listed)."""
out: dict[str, dict] = {}
def _on_walk_error(exc: OSError) -> None:
raise ScanError(f"unreadable under {root}: {exc}") from exc
for dirpath, dirnames, filenames in os.walk(root, onerror=_on_walk_error):
dirnames.sort()
for name in sorted(dirnames) + sorted(filenames):
if budget[0] <= 0:
raise ScanError(f"external target too large to fingerprint: {root}")
budget[0] -= 1
abs_path = os.path.join(dirpath, name)
rel = os.path.relpath(abs_path, root).replace(os.sep, "/")
if _is_link(abs_path):
out[rel] = {"kind": "symlink", "target": os.readlink(abs_path)}
elif os.path.isdir(abs_path):
out[rel] = {"kind": "dir"}
elif os.path.isfile(abs_path):
st = os.lstat(abs_path)
out[rel] = {"kind": "file", "size": st.st_size,
"sha256": _sha256_file(abs_path)}
else:
out[rel] = {"kind": "other"}
return out
def _entry_record(abs_path: str) -> dict:
if _is_link(abs_path):
rec: dict = {"kind": "symlink", "target": os.readlink(abs_path)}
resolved = os.path.realpath(abs_path)
rec["target_resolves"] = os.path.exists(resolved)
if rec["target_resolves"]:
if os.path.isfile(resolved):
rec["target_kind"] = "file"
rec["target_sha256"] = _sha256_file(resolved)
elif os.path.isdir(resolved):
rec["target_kind"] = "dir"
rec["target_tree"] = _fingerprint_tree(resolved, [MAX_EXTERNAL_ENTRIES])
return rec
if os.path.isdir(abs_path):
return {"kind": "dir"}
if os.path.isfile(abs_path):
st = os.lstat(abs_path)
return {"kind": "file", "size": st.st_size, "sha256": _sha256_file(abs_path)}
return {"kind": "other"}
def snapshot_home(home: str, profiles_dir: str | None = None) -> dict:
home = os.path.abspath(home)
roots = plugin_roots(home, profiles_dir)
entries: dict[str, dict] = {}
for root_label in roots:
root = os.path.join(home, root_label[len("<home>/"):].replace("/", os.sep))
# The root itself is an entry: if it is a symlink its identity and
# target are recorded; if a plain dir it anchors empty-dir coverage.
entries[root_label[len("<home>/"):]] = _entry_record(root)
if _is_link(root):
continue # walking through it would double-record its target
def _on_walk_error(exc: OSError) -> None:
raise ScanError(f"unreadable under {root}: {exc}") from exc
for dirpath, dirnames, filenames in os.walk(root, followlinks=False,
onerror=_on_walk_error):
dirnames.sort()
for name in sorted(dirnames) + sorted(filenames):
abs_path = os.path.join(dirpath, name)
rel = os.path.relpath(abs_path, home).replace(os.sep, "/")
entries[rel] = _entry_record(abs_path)
return {
"schema": SCHEMA_VERSION,
"home": home,
"roots": roots,
"entries": entries,
}
def verify_home(home: str, snap: dict) -> dict:
home = os.path.abspath(home)
now = snapshot_home(home, snap.get("_profiles_dir"))["entries"]
before = snap["entries"]
deleted = sorted(set(before) - set(now))
added = sorted(set(now) - set(before))
modified = {}
for key in sorted(set(before) & set(now)):
if before[key] != now[key]:
modified[key] = {"before": before[key], "after": now[key]}
return {
"schema": SCHEMA_VERSION,
"home": home,
"snapshot_roots": snap["roots"],
"counts": {
"before": len(before),
"after": len(now),
"deleted": len(deleted),
"modified": len(modified),
"added": len(added),
},
"deleted": deleted,
"modified": modified,
"added": added,
"ok": not deleted and not modified,
}
def main(argv: list[str] | None = None) -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = ap.add_subparsers(dest="mode", required=True)
p_snap = sub.add_parser("snapshot", help="record plugin-tree state to JSON")
p_snap.add_argument("--home", required=True)
p_snap.add_argument("--out", required=True)
p_snap.add_argument("--profiles-dir",
help="override the profiles root (default <home>/profiles)")
p_ver = sub.add_parser("verify", help="compare current state to a snapshot")
p_ver.add_argument("--home", required=True)
p_ver.add_argument("--snapshot", required=True)
p_ver.add_argument("--profiles-dir")
p_ver.add_argument("--report")
args = ap.parse_args(argv)
if not os.path.isdir(os.path.abspath(args.home)):
print(f"error: --home is not a directory: {args.home}", file=sys.stderr)
return 2
if args.mode == "snapshot":
try:
snap = snapshot_home(args.home, args.profiles_dir)
except (OSError, ScanError) as exc:
print(f"snapshot failed: {exc}", file=sys.stderr)
return 2
if args.profiles_dir:
snap["_profiles_dir"] = args.profiles_dir
with open(args.out, "w", encoding="utf-8") as fh:
json.dump(snap, fh, indent=2, sort_keys=True)
print(
f"snapshot: {len(snap['entries'])} entries across "
f"{len(snap['roots'])} plugin root(s) -> {args.out}"
)
if not snap["entries"]:
print(
"WARNING: snapshot recorded ZERO entries -- an empty snapshot "
"proves nothing; the E2E driver must treat this as a failure.",
file=sys.stderr,
)
return 0
with open(args.snapshot, "r", encoding="utf-8") as fh:
snap = json.load(fh)
try:
report = verify_home(args.home, snap)
except (OSError, ScanError) as exc:
print(f"verify failed: {exc}", file=sys.stderr)
return 2
rendered = json.dumps(report, indent=2, sort_keys=True)
if args.report:
with open(args.report, "w", encoding="utf-8") as fh:
fh.write(rendered)
if not snap["entries"]:
print(
"PLUGIN PRESERVATION INCONCLUSIVE: the snapshot is empty; "
"refusing to claim preservation over zero recorded entries.",
file=sys.stderr,
)
return 3
if report["ok"]:
c = report["counts"]
print(
f"plugin preservation OK: {c['before']} entries intact "
f"({c['added']} added, 0 deleted, 0 modified)"
)
return 0
print(rendered, file=sys.stderr)
print(
"PLUGIN PRESERVATION FAILED: "
f"{report['counts']['deleted']} deleted, "
f"{report['counts']['modified']} modified",
file=sys.stderr,
)
return 1
if __name__ == "__main__":
sys.exit(main())

View File

@@ -40,6 +40,9 @@
# drives it) and click Update now
# --install-ref what to install first; anything git resolves. Default:
# the newest release tag in the checkout.
# --update-ref what to update TO. Default: HEAD. Pass the next release
# tag for a stable-to-stable leg; only label the leg
# stable-to-stable when BOTH refs are release tags.
#
# Requires a clean full-history checkout with release tags fetched.
@@ -53,6 +56,7 @@ export TS_BASE=$SECONDS
INSTALL_METHOD="installer-script"
UPDATE_METHOD=""
INSTALL_REF=""
UPDATE_REF=""
while [ "$#" -gt 0 ]; do
case "$1" in
--install-method)
@@ -64,6 +68,9 @@ while [ "$#" -gt 0 ]; do
--install-ref)
[ "$#" -ge 2 ] || { echo 'error: --install-ref needs a value' >&2; exit 1; }
INSTALL_REF="$2"; shift 2 ;;
--update-ref)
[ "$#" -ge 2 ] || { echo 'error: --update-ref needs a value' >&2; exit 1; }
UPDATE_REF="$2"; shift 2 ;;
-h|--help) sed -n '2,45p' "$0"; exit 0 ;;
*) echo "error: unknown argument: $1" >&2; exit 1 ;;
esac
@@ -92,6 +99,8 @@ ok() { printf ' OK %s\n' "$*"; }
fail() { printf 'E2E ASSERTION FAILED: %s\n' "$*" >&2; exit 1; }
# shellcheck source=../e2e-assets/ts-prefix.sh
source "$(dirname "$0")/e2e-assets/ts-prefix.sh" 2>/dev/null || ts_prefix() { cat; }
# shellcheck source=../e2e-assets/preserve-plugins.sh
source "$(dirname "$0")/e2e-assets/preserve-plugins.sh"
# Full transcript in the job log, collapsed (GitHub renders ::group:: as a
# fold; plain text anywhere else). Win or lose -- a green install's log is
# how you diagnose the leg that fails next.
@@ -121,6 +130,16 @@ fi
OLD_SHA="$(git -C "$REPO_ROOT" rev-parse "${INSTALL_REF}^{commit}")"
HEAD_SHA="$(git -C "$REPO_ROOT" rev-parse HEAD)"
[ "$OLD_SHA" != "$HEAD_SHA" ] || fail "OLD ($INSTALL_REF) IS HEAD; no update would be available"
# The update target defaults to HEAD; --update-ref selects any other ref so
# a stable-to-stable leg can target the next release tag instead of the tip.
# Only call this leg stable-to-stable when BOTH refs are release tags.
TARGET_LABEL="HEAD"
TARGET_SHA="$HEAD_SHA"
if [ -n "$UPDATE_REF" ]; then
TARGET_SHA="$(git -C "$REPO_ROOT" rev-parse "${UPDATE_REF}^{commit}")"
TARGET_LABEL="$UPDATE_REF"
fi
[ "$OLD_SHA" != "$TARGET_SHA" ] || fail "OLD ($INSTALL_REF) IS the update target ($TARGET_LABEL); no update would be available"
git clone --bare --quiet "$REPO_ROOT" "$SERVE_REPO"
git -C "$SERVE_REPO" update-ref refs/heads/main "$OLD_SHA"
@@ -322,12 +341,13 @@ else
assert_checkout "$OLD_SHA" OLD
fi
smoke_desktop old
preserve_before_upgrade
# --- update OLD -> HEAD ----------------------------------------------------------
step "advancing served main to HEAD"
git -C "$SERVE_REPO" update-ref refs/heads/main "$HEAD_SHA"
ok "serve.git main = $HEAD_SHA"
step "advancing served main to $TARGET_LABEL ($TARGET_SHA)"
git -C "$SERVE_REPO" update-ref refs/heads/main "$TARGET_SHA"
ok "serve.git main = $TARGET_SHA"
step "updating via $UPDATE_METHOD"
case "$UPDATE_METHOD" in
@@ -348,11 +368,11 @@ case "$UPDATE_METHOD" in
;;
installer-script)
# A user re-running the one-liner today gets the CURRENT script.
run_installer "$HEAD_SHA" head
run_installer "$TARGET_SHA" "$TARGET_LABEL"
;;
installer-script+desktop)
run_installer "$HEAD_SHA" head desktop
assert_desktop_artifact HEAD
run_installer "$TARGET_SHA" "$TARGET_LABEL" desktop
assert_desktop_artifact "$TARGET_LABEL"
;;
hermes-desktop-app-update)
# The real user surface: `hermes desktop` launches the app, the user
@@ -402,7 +422,7 @@ case "$UPDATE_METHOD" in
(cd "$PW_DIR" && node launch-from-spec.mjs \
--spec "$SPEC" \
--result "$HERMES_HOME/.hermes-update-result.json" \
--expect-sha "$HEAD_SHA" \
--expect-sha "$TARGET_SHA" \
--repo-dir "$INSTALL_DIR" 2>&1 \
| ts_prefix > "$LOG_DIR/app-update.log") || rc=$?
log_group "app update (Playwright) transcript" "$LOG_DIR/app-update.log"
@@ -482,7 +502,9 @@ ls -la "$HERMES_HOME" > "$ildest/hermes-home-ls.txt" 2>/dev/null || true
ls -la "$INSTALL_DIR/venv/bin" > "$ildest/venv-bin-ls.txt" 2>/dev/null || true
ok "collected install-side logs to $ildest"
assert_checkout "$HEAD_SHA" HEAD
smoke_desktop head
assert_checkout "$TARGET_SHA" "$TARGET_LABEL"
smoke_desktop "$TARGET_LABEL"
step "PASS: $INSTALL_REF -> HEAD via $UPDATE_METHOD"
preserve_after_upgrade
step "PASS: $INSTALL_REF -> $TARGET_LABEL via $UPDATE_METHOD"

View File

@@ -28,6 +28,9 @@
# tests/install/macos-desktop-e2e.sh --phase stage|install|update|all
# --update-method open-app-update|hermes-desktop-app-update
# [--install-ref REF] [--dmg-url URL]
# [--update-ref REF] update target, default HEAD; pass the next
# release tag for a stable-to-stable leg (label the
# leg stable-to-stable only when both refs are tags)
#
# Requires a clean full-history checkout with release tags fetched, on a
# macOS host with a window server (the GitHub macos runners qualify).
@@ -42,6 +45,7 @@ export TS_BASE=$SECONDS
PHASE="all"
UPDATE_METHOD=""
INSTALL_REF=""
UPDATE_REF=""
DMG_URL="https://hermes-assets.nousresearch.com/Hermes-Setup.dmg"
PLAYWRIGHT_VERSION="1.58.2"
while [ "$#" -gt 0 ]; do
@@ -55,6 +59,9 @@ while [ "$#" -gt 0 ]; do
--install-ref)
[ "$#" -ge 2 ] || { echo 'error: --install-ref needs a value' >&2; exit 1; }
INSTALL_REF="$2"; shift 2 ;;
--update-ref)
[ "$#" -ge 2 ] || { echo 'error: --update-ref needs a value' >&2; exit 1; }
UPDATE_REF="$2"; shift 2 ;;
--dmg-url)
[ "$#" -ge 2 ] || { echo 'error: --dmg-url needs a value' >&2; exit 1; }
DMG_URL="$2"; shift 2 ;;
@@ -84,6 +91,8 @@ ok() { printf ' OK %s\n' "$*"; }
fail() { printf 'E2E ASSERTION FAILED: %s\n' "$*" >&2; exit 1; }
# shellcheck source=../e2e-assets/ts-prefix.sh
source "$(dirname "$0")/e2e-assets/ts-prefix.sh" 2>/dev/null || ts_prefix() { cat; }
# shellcheck source=../install/e2e-assets/preserve-plugins.sh
source "$(dirname "$0")/e2e-assets/preserve-plugins.sh"
log_group() {
printf '::group::%s\n' "$1"
cat "$2"
@@ -173,10 +182,20 @@ phase_stage() {
old_ref="$(git -C "$REPO_ROOT" tag --list 'v[0-9]*' --sort=-creatordate | head -1)"
[ -n "$old_ref" ] || fail "no release tags in the checkout to use as OLD"
fi
local old_sha head_sha
local old_sha head_sha target_sha target_label
old_sha="$(git -C "$REPO_ROOT" rev-parse "${old_ref}^{commit}")"
head_sha="$(git -C "$REPO_ROOT" rev-parse HEAD)"
[ "$old_sha" != "$head_sha" ] || fail "OLD ($old_ref) IS HEAD; no update would be available"
# The update target defaults to HEAD; --update-ref selects any other ref
# so a stable-to-stable leg can target the next release tag instead of
# the tip. Only call this leg stable-to-stable when BOTH refs are tags.
target_label="HEAD"
target_sha="$head_sha"
if [ -n "${UPDATE_REF:-}" ]; then
target_sha="$(git -C "$REPO_ROOT" rev-parse "${UPDATE_REF}^{commit}")"
target_label="$UPDATE_REF"
fi
[ "$old_sha" != "$target_sha" ] || fail "OLD ($old_ref) IS the update target ($target_label); no update would be available"
git clone --bare --quiet "$REPO_ROOT" "$SERVE_REPO"
git -C "$SERVE_REPO" update-ref refs/heads/main "$old_sha"
@@ -187,8 +206,9 @@ phase_stage() {
mkdir -p "$HERMES_HOME"
touch "$HERMES_HOME/.skip_upstream_prompt"
printf 'OLD_SHA=%s\nOLD_REF=%s\nHEAD_SHA=%s\n' "$old_sha" "$old_ref" "$head_sha" > "$STATE"
ok "serve.git main = $old_sha ($old_ref), update target $head_sha"
printf 'OLD_SHA=%s\nOLD_REF=%s\nHEAD_SHA=%s\nTARGET_SHA=%s\nTARGET_LABEL=%s\n' \
"$old_sha" "$old_ref" "$head_sha" "$target_sha" "$target_label" > "$STATE"
ok "serve.git main = $old_sha ($old_ref), update target $target_sha ($target_label)"
}
find_installed_app() {
@@ -311,7 +331,7 @@ run_playwright_update() {
(cd "$pw_dir" && node launch-from-spec.mjs \
--spec "$spec" \
--result "$HERMES_HOME/.hermes-update-result.json" \
--expect-sha "$HEAD_SHA" \
--expect-sha "$TARGET_SHA" \
--repo-dir "$INSTALL_DIR" 2>&1 \
| ts_prefix > "$LOG_DIR/app-update.log") || rc=$?
log_group "app update (Playwright) transcript" "$LOG_DIR/app-update.log"
@@ -322,9 +342,12 @@ phase_update() {
# shellcheck disable=SC1090
. "$STATE"
arm_redirect
step "advancing served main to HEAD"
git -C "$SERVE_REPO" update-ref refs/heads/main "$HEAD_SHA"
ok "serve.git main = $HEAD_SHA"
# Snapshot every plugin tree BEFORE the upgrade moves anything: fixtures
# seeded here must survive through the verify after the update lands.
preserve_before_upgrade
step "advancing served main to $TARGET_LABEL ($TARGET_SHA)"
git -C "$SERVE_REPO" update-ref refs/heads/main "$TARGET_SHA"
ok "serve.git main = $TARGET_SHA"
step "updating via $UPDATE_METHOD"
# The app must boot configured or the onboarding overlay (a fullscreen
@@ -350,10 +373,10 @@ phase_update() {
;;
installer-script)
# A dmg user re-running today's install one-liner.
run_installer "$HEAD_SHA" head
run_installer "$TARGET_SHA" head
;;
installer-script+desktop)
run_installer "$HEAD_SHA" head desktop
run_installer "$TARGET_SHA" head desktop
# The desktop stage is this leg's claim: the rebuilt app must exist.
head_app=""
for cand in \
@@ -403,8 +426,8 @@ PYEOF
local got
got="$(git -C "$INSTALL_DIR" rev-parse HEAD)"
[ "$got" = "$HEAD_SHA" ] || fail "checkout is $got, expected HEAD ($HEAD_SHA)"
ok "checkout landed on HEAD ($HEAD_SHA)"
[ "$got" = "$TARGET_SHA" ] || fail "checkout is $got, expected $TARGET_LABEL ($TARGET_SHA)"
ok "checkout landed on $TARGET_LABEL ($TARGET_SHA)"
# Install-side state BEFORE the post-update smoke: on app-update legs the
# updater's own transcript is streamed into the app UI and otherwise lost,
@@ -426,7 +449,8 @@ PYEOF
"$INSTALL_DIR/venv/bin/hermes" --version 2>&1 | ts_prefix > "$LOG_DIR/version-head.log" \
|| fail "hermes --version failed after update"
ok "hermes --version works post-update"
step "PASS: $OLD_REF -> HEAD via $UPDATE_METHOD"
preserve_after_upgrade
step "PASS: $OLD_REF -> $TARGET_LABEL via $UPDATE_METHOD"
}
case "$PHASE" in

View File

@@ -98,6 +98,10 @@ param(
# swallows an empty-string argument ('Missing an argument for
# parameter'), so the workflow cannot pass "".
[string]$InstallRef = "auto",
# Update target ref (default HEAD). A stable-to-stable leg passes the
# next release tag here; only label the leg stable-to-stable when BOTH
# refs are release tags.
[string]$UpdateRef = "HEAD",
# Repo checkout whose HEAD is the update target.
[string]$RepoRoot = "",
@@ -546,7 +550,8 @@ function Invoke-PhaseStage {
# bare-clone below (and everything after) sees the redirect file.
Set-GitRedirect
$current = Invoke-Git @("-C", $RepoRoot, "rev-parse", "HEAD")
$current = Invoke-Git @("-C", $RepoRoot, "rev-parse", $UpdateRef)
$targetLabel = if ($UpdateRef -eq "HEAD") { "HEAD" } else { $UpdateRef }
Write-Host " HEAD (update target): $current"
# OLD: explicit -InstallRef, or the newest release tag -- the version a
@@ -580,7 +585,7 @@ function Invoke-PhaseStage {
Invoke-Git @("-C", $ServeRepo, "config", "uploadpack.allowAnySHA1InWant", "true") | Out-Null
Write-Host " serve.git: uploadpack.allowAnySHA1InWant=true (installer commit pin, if any)"
@{ old = $old; old_ref = $oldRef; current = $current } |
@{ old = $old; old_ref = $oldRef; current = $current; target_label = $targetLabel } |
ConvertTo-Json | Set-Content -LiteralPath $StatePath -Encoding UTF8
Write-Host " state written: $StatePath"
New-Item -ItemType Directory -Path $ProofRoot -Force | Out-Null
@@ -888,6 +893,75 @@ function Invoke-GuiUpdateDesktopRoute([string]$TargetSha) {
}
}
# --- plugin upgrade-preservation hooks -------------------------------------
# A tagged upgrade must not delete or modify anything under the active
# home's plugins/** or any profile's plugins/** tree: wrapper markers
# (mnemosyne-wrapper.json), symlinked runtimes, and the externally-owned
# sidecar witness outside the home. Fixtures are directory-only (no
# pyproject in the scanned root, nothing downloaded). Snapshot is taken
# after install, verified after update.
function Seed-PreservationFixtures {
$wrapper = Join-Path $HermesHome "plugins\mnemosyne-wrapper"
$external = Join-Path $WorkRoot "external-mnemosyne-runtime"
$marker = Join-Path $wrapper "mnemosyne-wrapper.json"
# NOT a clobbering seeder: pre-existing populated wrapper is left
# untouched so a retried leg cannot erase a recorded regression.
if (Test-Path -LiteralPath $wrapper) {
$existing = @(Get-ChildItem -LiteralPath $wrapper -ErrorAction SilentlyContinue)
if ($existing.Count -gt 0) {
$markerOk = (Test-Path -LiteralPath $marker) -and
((Get-Content -LiteralPath $marker -Raw) -match "mnemosyne-wrapper")
if (-not $markerOk) {
throw "refusing to reseed preservation fixtures: $wrapper already populated without the expected marker"
}
Write-Host " preservation fixtures already present; left untouched"
return
}
}
New-Item -ItemType Directory -Path $wrapper, $external, `
(Join-Path $HermesHome "profiles\e2e-preserve\plugins\second-plugin") -Force | Out-Null
Set-Content -LiteralPath (Join-Path $wrapper "mnemosyne-wrapper.json") `
-Value '{"wrapper": true, "marker": "mnemosyne-wrapper", "owner": "e2e-preservation"}' -Encoding UTF8
Set-Content -LiteralPath (Join-Path $wrapper "plugin.py") `
-Value '# directory wrapper fixture: no dependencies' + [Environment]::NewLine + 'PLUGIN = "mnemosyne-wrapper"' -Encoding UTF8
if (-not (Test-Path -LiteralPath (Join-Path $wrapper "runtime"))) {
try {
New-Item -ItemType SymbolicLink -Path (Join-Path $wrapper "runtime") -Target $external -ErrorAction Stop | Out-Null
} catch {
Write-Host " (symlink unavailable: $($_.Exception.Message); runtime link omitted from fixtures)"
}
}
Set-Content -LiteralPath (Join-Path $external "sidecar-witness.txt") -Value "external-sidecar-witness-v1" -Encoding UTF8
Set-Content -LiteralPath (Join-Path $external "engine.bin") -Value "external-engine" -Encoding UTF8
Set-Content -LiteralPath (Join-Path $HermesHome "profiles\e2e-preserve\plugins\second-plugin\marker.json") `
-Value '{"plugin": "second-plugin", "profile": "e2e-preserve"}' -Encoding UTF8
Set-Content -LiteralPath (Join-Path $HermesHome "profiles\e2e-preserve\plugins\second-plugin\data.bin") `
-Value "profile-plugin-bytes" -Encoding UTF8
Write-Host " preservation fixtures seeded (wrapper + profile tree; external witness at $external)"
}
function Invoke-PreserveSnapshot {
Seed-PreservationFixtures
$out = Join-Path $WorkRoot "plugin-preservation-snapshot.json"
& python (Join-Path $AssetsDir "verify-plugin-preservation.py") snapshot --home $HermesHome --out $out
if ($LASTEXITCODE -ne 0) { throw "plugin preservation snapshot failed (exit $LASTEXITCODE)" }
# An empty snapshot proves nothing; refuse to build the leg's claim on it.
$snap = Get-Content -LiteralPath $out -Raw | ConvertFrom-Json
if (@($snap.entries.PSObject.Properties).Count -eq 0) {
throw "plugin preservation snapshot is EMPTY: no plugin entries recorded, cannot verify preservation"
}
Write-Host " pre-upgrade plugin snapshot: $out"
}
function Invoke-PreserveVerify {
$snap = Join-Path $WorkRoot "plugin-preservation-snapshot.json"
if (-not (Test-Path -LiteralPath $snap)) { throw "no pre-upgrade plugin snapshot at $snap; cannot verify preservation" }
& python (Join-Path $AssetsDir "verify-plugin-preservation.py") verify --home $HermesHome --snapshot $snap `
--report (Join-Path $WorkRoot "logs\plugin-preservation-report.json")
if ($LASTEXITCODE -ne 0) { throw "plugin preservation violated by the upgrade (exit $LASTEXITCODE); see the report for deleted/modified entries" }
Write-Host " plugins/** and profile plugin trees survived the upgrade intact"
}
function Invoke-PhaseInstall {
# Dispatch on the install axis. Each arm ends with the same contract:
# checkout at OLD, hermes runs, and state carries how OLD landed so any
@@ -930,6 +1004,8 @@ function Invoke-PhaseUpdate {
# remote's main moves forward. The GUI route re-advances harmlessly
# (same sha); script routes need it here because only the GUI arm's
# helper used to own this step.
# Snapshot every plugin tree BEFORE the upgrade moves anything.
Invoke-PreserveSnapshot
Invoke-Git @("-C", $ServeRepo, "update-ref", "refs/heads/main", $state.current) | Out-Null
Write-Host " serve.git main advanced to $($state.current)"
@@ -975,6 +1051,7 @@ function Invoke-PhaseUpdate {
Assert-True ((Get-InstalledHead) -eq $state.current) "checkout landed on HEAD"
Test-HermesRuns "post-update"
Invoke-PreserveVerify
}
function Invoke-CheckedPhaseUpdate {

View File

@@ -0,0 +1,306 @@
"""Unit tests for the plugin upgrade-preservation verifier.
tests/install/e2e-assets/verify-plugin-preservation.py is the standalone
hook the release E2E drivers call before and after a real upgrade. These
tests exercise it against a real temp HERMES_HOME (real files, real
symlinks) — no source-reading, no mocks of the filesystem.
The verifier must be read-only against the scanned home and must catch
deletion and modification of every recorded entry kind: regular files,
wrapper markers, directory trees, symlinks (identity + target), and the
externally-owned sidecar witness file a symlinked plugin runtime points at.
"""
from __future__ import annotations
import importlib.util
import json
import os
import subprocess
import sys
import pytest
_HERE = os.path.dirname(os.path.abspath(__file__))
VERIFIER = os.path.join(
_HERE, "..", "install", "e2e-assets", "verify-plugin-preservation.py"
)
_spec = importlib.util.spec_from_file_location("verify_plugin_preservation", VERIFIER)
vpp = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(vpp)
def _make_link(target, link):
try:
os.symlink(str(target), str(link))
except OSError:
# Windows without symlink privilege: same reparse-point shape.
import _winapi
_winapi.CreateJunction(str(target), str(link))
def _remove_link(link):
if os.path.islink(str(link)):
os.remove(str(link))
else: # NTFS junction
os.rmdir(str(link))
@pytest.fixture()
def home(tmp_path):
"""A controlled temp HERMES_HOME with a non-dependency directory wrapper
plugin: marker + payload + a symlink to an external runtime whose witness
file lives OUTSIDE the home (externally-owned), plus a second plugin in a
profile tree. No pyproject anywhere in the scanned root — the fixture is
directory-only, so the scanner cannot recurse into a dependency graph and
the test needs no network/Torch."""
h = tmp_path / "hermes-home"
# active-home plugin: directory wrapper with marker + payload
plugin = h / "plugins" / "mnemosyne-wrapper"
plugin.mkdir(parents=True)
(plugin / "mnemosyne-wrapper.json").write_text('{"wrapper": true}\n', encoding="utf-8")
(plugin / "plugin.py").write_bytes(b"PAYLOAD-BYTES-0\n")
# external runtime, owned outside the home, reached through a symlink
external = tmp_path / "external-mnemosyne-runtime"
external.mkdir()
(external / "sidecar-witness.txt").write_text("external-witness-v1\n", encoding="utf-8")
(external / "engine.bin").write_bytes(b"\x00\x01\x02")
_make_link(external, plugin / "runtime")
# profile plugin tree
pplugin = h / "profiles" / "work" / "plugins" / "second-plugin"
pplugin.mkdir(parents=True)
(pplugin / "marker.json").write_text('{"p": 1}\n', encoding="utf-8")
(pplugin / "data.bin").write_bytes(b"profile-bytes\n")
return h
def _snapshot(home, out):
snap = vpp.snapshot_home(str(home))
with open(out, "w", encoding="utf-8") as fh:
json.dump(snap, fh)
return snap
def _verify(home, snap):
return vpp.verify_home(str(home), snap)
def test_snapshot_records_all_trees(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
keys = set(snap["entries"])
assert "plugins/mnemosyne-wrapper/mnemosyne-wrapper.json" in keys
assert "plugins/mnemosyne-wrapper/plugin.py" in keys
assert "profiles/work/plugins/second-plugin/data.bin" in keys
assert snap["roots"] == [
"<home>/plugins",
"<home>/profiles/work/plugins",
]
def test_snapshot_captures_bytes_and_symlinks(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
e = snap["entries"]
assert e["plugins/mnemosyne-wrapper/plugin.py"]["sha256"] != ""
assert e["plugins/mnemosyne-wrapper/plugin.py"]["size"] == 16
link = e["plugins/mnemosyne-wrapper/runtime"]
assert link["kind"] == "symlink"
assert link["target_resolves"] is True
assert link["target_kind"] == "dir"
tree = link["target_tree"]
assert tree["sidecar-witness.txt"]["sha256"] != ""
assert tree["engine.bin"]["kind"] == "file"
def test_untouched_home_verifies_clean(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
report = _verify(home, snap)
assert report["ok"] is True
assert report["counts"]["deleted"] == 0
assert report["counts"]["modified"] == 0
def test_catches_plugin_file_deletion(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
(home / "plugins" / "mnemosyne-wrapper" / "plugin.py").unlink()
report = _verify(home, snap)
assert report["ok"] is False
assert "plugins/mnemosyne-wrapper/plugin.py" in report["deleted"]
def test_catches_whole_plugin_root_deletion(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
for root, dirs, files in os.walk(home / "plugins", topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))
os.rmdir(home / "plugins")
report = _verify(home, snap)
assert report["ok"] is False
assert report["counts"]["deleted"] > 0
def test_catches_wrapper_marker_modification(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
(home / "plugins" / "mnemosyne-wrapper" / "mnemosyne-wrapper.json").write_text(
'{"wrapper": false}\n', encoding="utf-8"
)
report = _verify(home, snap)
assert report["ok"] is False
assert "plugins/mnemosyne-wrapper/mnemosyne-wrapper.json" in report["modified"]
def test_catches_symlink_target_repoint(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
link = home / "plugins" / "mnemosyne-wrapper" / "runtime"
other = tmp_path / "other-runtime"
other.mkdir()
(other / "sidecar-witness.txt").write_text("different\n", encoding="utf-8")
_remove_link(link)
_make_link(other, link)
report = _verify(home, snap)
assert report["ok"] is False
assert "plugins/mnemosyne-wrapper/runtime" in report["modified"]
def test_catches_external_witness_modification(home, tmp_path):
# The externally-owned sidecar witness lives OUTSIDE the home; an upgrade
# that tramples it must still be caught through the symlink fingerprint.
snap = _snapshot(home, tmp_path / "snap.json")
witness = tmp_path / "external-mnemosyne-runtime" / "sidecar-witness.txt"
witness.write_text("external-witness-TAMPERED\n", encoding="utf-8")
report = _verify(home, snap)
assert report["ok"] is False
# Depending on the platform's walk, the tamper surfaces either as the
# link entry (target fingerprint) or as the linked file itself.
assert any(
"runtime" in key for key in list(report["modified"]) + report["deleted"]
)
def test_catches_external_witness_deletion(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
(tmp_path / "external-mnemosyne-runtime" / "engine.bin").unlink()
report = _verify(home, snap)
assert report["ok"] is False
def test_catches_profile_plugin_deletion(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
(home / "profiles" / "work" / "plugins" / "second-plugin" / "data.bin").unlink()
report = _verify(home, snap)
assert report["ok"] is False
assert "profiles/work/plugins/second-plugin/data.bin" in report["deleted"]
def test_added_entries_do_not_fail(home, tmp_path):
# An upgrade may ADD files (new bundled plugin, caches); only taking
# away or changing existing entries is a violation.
snap = _snapshot(home, tmp_path / "snap.json")
newp = home / "plugins" / "fresh-from-upgrade"
newp.mkdir()
(newp / "b.txt").write_text("new\n", encoding="utf-8")
report = _verify(home, snap)
assert report["ok"] is True
assert "plugins/fresh-from-upgrade/b.txt" in report["added"]
def test_verifier_is_read_only_against_home(home, tmp_path):
snap = _snapshot(home, tmp_path / "snap.json")
before = sorted(
(p, p.stat().st_size if p.is_file() else "dir")
for p in home.rglob("*")
)
_verify(home, snap)
_verify(home, snap)
after = sorted(
(p, p.stat().st_size if p.is_file() else "dir")
for p in home.rglob("*")
)
assert before == after
def test_catches_empty_dir_deletion(home, tmp_path):
# A plugin directory emptied (or an empty dir removed) must be caught:
# directories themselves are recorded, not skipped.
snap = _snapshot(home, tmp_path / "snap.json")
empty = home / "plugins" / "wrapper-b" / "empty-cache"
empty.mkdir(parents=True)
snap2 = _snapshot(home, tmp_path / "snap2.json")
assert snap2["entries"]["plugins/wrapper-b/empty-cache"] == {"kind": "dir"}
empty.rmdir()
(home / "plugins" / "wrapper-b").rmdir()
report = _verify(home, snap2)
assert report["ok"] is False
assert "plugins/wrapper-b/empty-cache" in report["deleted"]
def test_empty_snapshot_is_inconclusive(home, tmp_path):
# Zero recorded entries cannot prove anything: the CLI refuses.
empty_home = tmp_path / "bare-home"
empty_home.mkdir()
snap_file = tmp_path / "empty-snap.json"
r1 = subprocess.run(
[sys.executable, VERIFIER, "snapshot", "--home", str(empty_home),
"--out", str(snap_file)],
capture_output=True, text=True,
)
assert r1.returncode == 0
assert "ZERO entries" in r1.stderr
r2 = subprocess.run(
[sys.executable, VERIFIER, "verify", "--home", str(empty_home),
"--snapshot", str(snap_file)],
capture_output=True, text=True,
)
assert r2.returncode == 3
assert "INCONCLUSIVE" in r2.stderr
def test_unreadable_path_is_hard_error(home, tmp_path):
# A scanner that cannot see a path must fail loudly, not skip silently.
# Skip where chmod-based unreadability is not enforceable (Windows).
if os.name != "posix":
pytest.skip("chmod-based unreadability is POSIX-only")
snap = _snapshot(home, tmp_path / "snap.json")
secret = home / "plugins" / "mnemosyne-wrapper" / "locked"
secret.mkdir()
(secret / "x.txt").write_text("data", encoding="utf-8")
os.chmod(secret, 0o000)
try:
with pytest.raises(OSError):
_snapshot(home, tmp_path / "snap2.json")
finally:
os.chmod(secret, 0o755)
def test_missing_home_fails_snapshot(tmp_path):
proc = subprocess.run(
[sys.executable, VERIFIER, "snapshot", "--home", str(tmp_path / "nope"),
"--out", str(tmp_path / "x.json")],
capture_output=True, text=True,
)
assert proc.returncode == 2
def test_cli_roundtrip_end_to_end(home, tmp_path):
"""The exact command shape the E2E drivers use."""
snap_file = tmp_path / "snap.json"
r1 = subprocess.run(
[sys.executable, VERIFIER, "snapshot", "--home", str(home), "--out", str(snap_file)],
capture_output=True, text=True,
)
assert r1.returncode == 0, r1.stderr
r2 = subprocess.run(
[sys.executable, VERIFIER, "verify", "--home", str(home), "--snapshot", str(snap_file)],
capture_output=True, text=True,
)
assert r2.returncode == 0, r2.stderr
(home / "plugins" / "mnemosyne-wrapper" / "plugin.py").unlink()
r3 = subprocess.run(
[sys.executable, VERIFIER, "verify", "--home", str(home), "--snapshot", str(snap_file)],
capture_output=True, text=True,
)
assert r3.returncode == 1
assert "PLUGIN PRESERVATION FAILED" in r3.stderr