cache/generated holds provider-generated images/videos that are now
deliberately never swept (they are the only copy of the user's output).
_KEPT_CACHE_SUBDIRS did not list it, so `hermes backup` treated it as a
regenerable cache and dropped it — for the root home and profile homes.
Add GENERATED_SUBDIR (stdlib-only agent.provider_media) to the kept set
and pin both locations in the existing cache-pruning backup test.
`/snapshot prune -1` passed keep=-1 to prune_quick_snapshots, whose slice
then deletes the newest snapshots instead of the oldest - the same bug
class #128886 fixed for `hermes backup --keep`.
Use isdecimal() for both entry points. For `backup --keep` the argparse
type is now the single validation (cmd_backup is run_backup's only
caller), so the duplicate run_backup guard is removed; non-numeric input
still exits 2 with a clear message.
The test is folded into tests/hermes_cli/test_backup.py, replacing
test_backup_retention_validation.py.
A writer in the update's mutation window replaced every agent-job
prompt with the job's own name while the job count stayed identical,
so the count-based restore_cron_jobs_if_emptied net passed the loss
undetected (#82990). Add a field-level safety net mirroring the config
model-settings net: restore only the prompt field of a live agent job
whose id matches a snapshot job and whose live prompt is blank or
collapsed to the job name — a legitimate user edit that merely
differs is never stomped. Wired into the post-migration restore
sequence for the active profile and every sibling profile.
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule
The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:
- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
names, restored dead definitions) and the three re-export stub modules
(gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
`plugins.allow_deprecated_imports` escape hatch
An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.
hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).
In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.
* chore: retrigger CI (zero-job startup_failure phantom)
* test: drop resolution allowlist rows for the two deleted which() sites
hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
_write_full_zip_backup_locked chose clean/salvage/discard in _publish_path
and then re-derived the same choice with an inverse test after the with
block. If only one copy changed later, .stat() could hit a path that was
never published and raise out of a "never raises" helper. _publish_path now
records the destination and the stat/return reuse it.
The `destination is None` discard branch in _atomic_output_path had no
teeth: publishing the empty all-failed archive over out_path kept every
test green. The serialization test now asserts an all-failed automatic run
leaves the previous good archive's members unchanged. Also refresh a stale
comment that still described a renamed salvage archive.
Review cleanups on the incomplete-backup salvage path:
- The incomplete / nothing-salvaged warnings joined every per-entry
error into one log line; a broken tree can fail thousands of entries,
so log the first 10 plus "(+N more)".
- Drop the _entry_error helper: its per-entry logger.debug duplicated
the summary warning, so errors are now collected by a plain lambda.
- claw migrate: a None pre-migration backup can mean an incomplete run
whose salvage was kept, so point the user at the possible
pre-migration-*.incomplete.zip instead of claiming there is no
restore point at all.
- Wrap an overlong create_pre_update_backup docstring line.
No partial can land under the complete out_path name: publish is a
single os.replace from the hidden partial, and any failure (including
the replace itself) unlinks the partial in _atomic_output_path.
_write_full_zip_backup_locked published the partial archive to out_path via
_atomic_output_path and only then renamed it to the .incomplete.zip salvage
name, so an incomplete run destroyed a pre-existing good backup at out_path
(test_zip_captures_live_wal_and_cleans_failed_staging[True] regressed vs base).
_atomic_output_path now takes an optional publish_path callable evaluated at
publish time: the full-zip writer publishes the hidden partial straight to
out_path when clean, to the salvage path when some entries failed, and
discards it (returns None) when every entry failed, since an empty salvage
archive restores nothing. out_path is never touched on an incomplete run.
Incomplete automatic backups kept the normal <prefix><ts>.zip name, so they
counted toward retention: the next complete run pruned by count and deleted
the last complete backups, and repeated failing runs piled up. Rename them to
<stem>.incomplete.zip, exclude that suffix from _prune_prefixed_zips, and cap
salvage archives at one.
The failed member's bytes also stayed in the file behind a valid local header,
visible to streaming readers as a ghost entry (#124564). Truncate at the first
dropped header and rewind start_dir so later members overwrite it.
Also: name skipped paths in one merged warning, fix a stale comment, drop a
redundant str(), update None-return docstrings, remove an empty duplicate
section heading, and warn in claw migrate when no pre-migration backup was made.
_safe_restore_db now also returns False when the snapshot copy fails its
SQLite integrity check, but restore_quick_snapshot still logged every False
as "live-safe restore refused", pointing users at running processes when
the real cause was a corrupt snapshot. Make the log line and comment
neutral and defer to the preceding detailed log line from
_safe_restore_db; no second integrity check is added on this path.
The three percent-encoded mode=ro connects added earlier in this stack
re-implemented hermes_state_holders.read_only_db_uri, which already exists
for exactly this '#'/'?' truncation bug and is the canonical builder used by
hermes_state and doctor_state. Using it keeps a single definition of the
encoding so a future fix lands everywhere. Behaviour is unchanged
(timeout=1.0 kept at the backup.py site).
verify_sqlite_integrity() (now the source gate in _safe_restore_db) opens
the file through _query_ro_sqlite with a raw f"file:{path}?mode=ro" URI.
Under a path containing '#', SQLite treats the rest as a fragment: the
?mode=ro is dropped, the pre-'#' prefix is opened read-write (and created
empty), and the integrity check passes a corrupt source. Use
Path.as_uri(), matching the other restore-flow sites.
Salvaged from PR #123374.
Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
The integrity pre-flight caught only BadZipFile/zlib.error/EOFError, so a
bzip2 member with a bad stream (OSError "Invalid data stream"), an lzma
LZMAError or a media read error still escaped as a traceback instead of the
clean "archive is damaged" refusal. Name the archive-read errors once
(_ZIP_MEMBER_READ_ERRORS) and use the same tuple, plus OSError, in the
pre-flight and in the per-member catch of the restore (PermissionError is an
OSError, so it no longer needs listing).
The pre-flight also decompressed members the restore never writes
(gateway.pid and the other runtime files, archived SQLite sidecars), so a rot
in one of those refused an otherwise restorable backup. Move the skip rule
into _import_skipped() and use it in both places so they cannot drift.
run_import's docstring now states the real contract: 1 for a damaged archive
or an incomplete restore, None on success or a declined prompt.
`hermes import` only checked the central directory (`is_zipfile`,
`namelist`), so an archive with one member whose deflate stream or CRC is
rotten passed validation and blew up mid-restore with a zlib.error
traceback -- after config.yaml and everything before the bad member had
already been replaced, with later members never written (#121258).
Add a pre-flight pass in run_import that streams every member through
1 MiB reads (zipfile verifies the CRC at EOF) and collects every
BadZipFile / zlib.error / EOFError. If any member is damaged the command
prints a capped list and returns 1 with the home untouched. Stdlib
`ZipFile.testzip()` is deliberately not used: it lets zlib.error escape
and names at most the first bad member.
Widen the per-member catch from the previous commit with EOFError so a
member that rots between the two passes still becomes a "skipped" warning
+ `Import incomplete` / exit 1 instead of a traceback. Rework that
commit's test to drive the per-member path (the archive now never reaches
it with a corrupt member), and let `_break_member` serve the pre-flight
read before failing the restore's own read.
Co-authored-by: KoNit-K <konit.block@protonmail.com>
Co-authored-by: John Paul Soliva <soliva.johnpaul@icloud.com>
`_import_members` catches the PermissionError/OSError of each member it
cannot publish and records it in `errors`. That includes the state.db the
live-safe restore refuses because a gateway or dashboard still holds it
(#100960, #110179). `run_import` printed those under "Warnings (N files
skipped)", then printed "Done. Your Hermes configuration has been
restored." and returned None. `cmd_import` did not forward a return value,
so the shell status was 0. A script chained on `hermes import --force`
carried on over a partial restore, and the dashboard's import action
showed the green "done" badge. For the refused state.db case, the user's
sessions were never restored.
`run_import` now returns 1 when `errors` is non-empty. It words both the
summary header and the final line as "Import incomplete", and `cmd_import`
forwards the return code, the same contract `hermes backup` has for an
incomplete archive. Runtime files the import deliberately keeps
(gateway.pid, SQLite sidecars) and the older-backup session warning stay
warnings. The gateway revive still runs, so the files that did land come
up as before.
Measured on main with a fresh HERMES_HOME through `main()`: a 3-file
backup with one read-only target directory, and a refused state.db (live
DB left at 3 sessions / 12 messages). Both used to exit 0 after "Done ...
restored". Both now exit 1 after "Import incomplete: 1 file(s) were not
restored". A clean import still exits 0 and prints "Done".
(cherry picked from commit a0f808ec9f868128f4174e4816e095ac17f18d28)
`hermes_cli.config.atomic_config_write` is now THE config.yaml writer: it delegates to
`utils.atomic_roundtrip_yaml_save` (ruamel round-trip), which merges the new state onto the
on-disk document so user comments, key order, quoting and blank lines survive every write.
Why: config.yaml is hand-edited and commented, and every writer that re-serialised the parsed
dict through PyYAML (`save_config`, `config set/unset`, migrations, plugin bookkeeping, auth
provider reset, credential scrub, channel strip, backup restore, profile seed, telegram topic
persistence) destroyed those comments — and `save_config` re-appended the stock boilerplate on
top (#92554, #63039, #50698, #109611, #107511, #66752). The round-trip writer existed
(tui_gateway only) but nothing else used it, so each new writer regressed the class.
- save_config / _write_user_config / atomic_config_write -> round-trip merge; the commented
example blocks are appended only when the file is created.
- round-trip merge only reassigns nodes whose value changed (element-wise for lists), so an
untouched scalar/list keeps its inline comments; YAML 1.1-ambiguous strings (off/yes/no...)
are force-quoted at every depth; duplicate keys are tolerated like PyYAML.
- direct PyYAML writers in auth.py, credential_lifecycle.py, profile_channels.py, backup.py,
profiles.py, telegram adapter and tui_gateway/server.py now call atomic_config_write.
main extracted the launchd backend and the setup wizard out of hermes_cli/gateway.py. The
eight launchd functions pm-clean had changed are ported into gateway_launchd.py in its
_gw() style: runtime_command/installation_command for the gateway argv (no VIRTUAL_ENV in
the plist, XML-escaped args), _prepare_service_launcher before every plist write,
utf-8-sig plist reads, and launchd_restart's refresh-first + bounded bootstrap revival.
The systemd service-unit cluster stays in the facade (no gateway_service_unit sibling).
backup.py takes main's browser_profiles backup-only exclusion on our profile_root_entry
shape. main.ts takes main's attach-first backend block with our explicit types.
Merge fallout (my resolution errors, all caught by CI):
- hermes_cli/backup.py + gateway.py: `theirs` on those hunks re-imported clusters HEAD had
already moved to backup_restore.py / kept in the facade. backup.py loses the 349-line
duplicate (main's #110179 fix is ported into backup_restore._import_db_member); the
systemd service-unit cluster returns to gateway.py (PM's _prepare_service_launcher /
_pm_managed_node_dirs / _systemd_command have no home in main's extraction) with main's
utf-8-sig read. gateway_service_unit.py is dropped.
- gateway/run.py: main's plugin-update chore is not profile-scoped (the housekeeping
ordering test pins the scope/drain sequence).
- pyproject + 30 test files: `import yaml` -> `import hermes_yaml as yaml` (pm-clean has no
pyyaml); gateway/config._bundled_platform_manifest_name reads through hermes_yaml.
- tests re-seamed onto pm-clean's shape: residency admission (installed_engine),
supervisor child env (binary is a constructor argument), update import guard
(update_cmd_deps is gone; our probe already scrubs PYTHONPATH — both #115032 invariants
pass), shallow-count git responses (stash path asks `status --porcelain -z`); dropped
tests for retired code (_run_node_bootstrap/_ensure_tui_node, Windows resume demotion).
- tests/tools/test_local_env_blocklist.py: restore the two helpers the suite-reduction
commit dropped and the blocklist import.
Real fixes:
- pm: classify_uv_failure/ResolutionConflict move beside the uv runner (pm.environment,
stdlib-only). pm.workspace imports tomllib at module level and cannot load on the 3.10
bootstrap python that streams uv output in the Docker arm64 image.
- tools/browser_tool.warm_agent_browser_npx_cache: back as a permanent definition — it is on
the frozen old-updater surface, and the revert-scheduled compat pointer does not count.
- hermes_cli/memory_setup: the dashboard's pip row uses pm.environments.
running_from_selected_environment for installed vs restart_required.
- scripts/windows-build-deps.ps1: export DISTUTILS_USE_SDK/MSSdk so setuptools trusts the
primed MSVC environment instead of asking vswhere (`env -i` test runner on win32-arm64
compiling ruamel-yaml-clib); run_tests.sh forwards them.
- tests/pm/test_windows_build_deps.py: start the protocol test from a parent env without the
toolchain variables the runner job already exports.
- tests/conftest.py scrubs HERMES_BUNDLED_PLUGINS (Nix-wrapped hermes on the dev host);
tests/home_io_guard.py treats sys.path site-packages under the real home as the
interpreter's installation (PM-activated developer shell).
- tests-js: four `curly` lint errors from main's new scripts.
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).
Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.
uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
`_import_db_member` guards the replace path with `_safe_restore_db`, but its
"target is missing" branch treated absence as proof of no holders. A gateway or
dashboard that had the database open when it was unlinked keeps writing the
deleted inode, so an atomic publish there re-creates exactly the #90950 split
brain the replace path exists to prevent.
Consult `_foreign_db_holder_pids` before the publish and raise an OSError
naming the PIDs, which the caller already reports as a skipped file.
Credit: @ngpestelos (#110179)
- 15 `MERGE-CHECK:` conflict-resolution comments removed from prod code (two were
TODOs already done: the utf-8-sig sessions.json read lives in session_persistence,
the pm-aware cron script helpers in scheduler_script).
- 49 imports the branch left unused (ruff F401, none present at the merge base,
none inside PLUGIN-COMPAT blocks). update_cmd's frozen-surface re-exports are
trimmed to the names tests/compat/old_updater_surface.json actually lists under
hermes_cli.update_cmd; the rest resolve through hermes_cli.main.__getattr__.
- tools/environments/local_gitbash_probe.py: nothing imported it once _find_bash
delegated to pm.shell().
- Three try/except wrappers around calls that cannot raise (install_truststore,
get_hermes_home, and a duplicated except clause in supermemory).
Gate review: `--keep` pruning ran after the summary regardless of `errors`,
so a timer hitting the same unreadable file every run would exit 1 each time
and still rotate every complete `hermes-backup-*.zip` out after N runs,
leaving only incomplete archives. Prune only after a complete backup; the
test pins a pre-existing good archive surviving an incomplete run with
`--keep 1`. The summary no longer hard-codes the caller's exit code.
`hermes backup` recorded per-file failures, printed `Backup incomplete: <path>`
and still returned shell status 0, so a cron job or systemd timer would publish
"successful" archives missing state.db indefinitely.
`run_backup()` now returns whether the archive is complete and `cmd_backup()`
maps False to exit status 1. The zip is kept so the operator can still restore
the rest; hard failures keep their SystemExit(1)/(2). `--quick` is unchanged.
Slim redo of #101096 on current main (the branch predates the run_backup /
_run_backup_locked split and the backup lock); same policy, same exit codes.
Supersedes #68866 (@jbryce) which proposed the policy first.
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
backup.py imports hermes_cli.profiles only lazily and profiles.py never imports backup, so
there was no cycle to justify two literals. LOCAL_RUNTIME_ROOT_DIRS now feeds both
backup._EXCLUDED_ROOT_DIRS and the clone-all root gate; one invariant test pins the identity.
Squashed integration of the user-facing message audit for this surface set.
Full per-finding receipts: /tmp/ux-audit/lanes/*-receipt.md (campaign artifacts).
Every hand-rolled writer this PR folded into utils._atomic_write created a
NEW file with write_text()/open("w"), i.e. at 0o666 masked by the umask
(0644 under 022). The canonical helper publishes through mkstemp, whose
temp is 0600, and with no explicit mode and no existing target to copy bits
from it left that 0600 in place - so debug, model_catalog, profiles,
breadcrumbs, worktree_ops, web_result_cache, plugin_compat, write_approval,
rich_sent_store, active_sessions and the google_meet state files were
silently tightened to owner-only, the volume-mount hazard
_restore_file_metadata's own docstring warns about. Undeclared in the PR.
Fix at the canonical: when mode is None and the target does not exist,
apply default_new_file_mode() (0o666 masked by the umask, read via the
umask two-call trick with a transient 0o077 so a racing thread can only get
a tighter file). The helper is hermes_cli/backup._default_new_file_mode
moved into utils and reused. Secret writers (mode=0o600) are 0600 before,
during and after as before; an existing target keeps its bits; on non-POSIX
the helper returns None so nothing is chmod'd.
Preserve upstream fixes without restoring retired dependency installers.
Run configured-feature checks in the selected build interpreter. Reuse a
supported base Python during bootstrap, and preserve durable backup media.
Refresh the dependency lock through PM. Keep the frozen historical import
surface unchanged. Adapt incoming native tests to the platform markers.
Verification: the incoming 86-file pass found two fixture mismatches;
both passed after correction. Targeted PM/update/compatibility checks,
Electron and renderer typechecks, and desktop tests passed.
Native Windows/macOS update journeys and the full suite remain unrun.
Excluding cache/ wholesale at profile roots dropped media the gateway
delivered to or received from the user (cache/images, audio, videos,
documents, screenshots) and the grounded-citations evidence ledger
(cache/citations/ledger.json) — none of which can be regenerated.
Prune only the regenerable cache/<x> subtrees; keep those six.
