The plugin now registers a pre_tool_call hook, but plugin.yaml still listed
only post_tool_call/on_session_end, so `plugin doctor` warned "registration
adds hook 'pre_tool_call' not listed in provides_hooks" and catalog/capability
listings under-reported what the plugin hooks.
Also: README notes calls without a tool_call_id (execute_code RPC) are not
tracked; a single _owner() helper keys both _pre_call and
_recent_test_tracks so the two can't drift; drop the unused `result`
parameter from the post hook (still absorbed by **_); trim the _pre_call
comment.
The pre/post snapshot handshake had two ways to mark a user's existing
file as created, which queues it for deletion when the turn ends:
- Snapshots were keyed by tool_call_id alone. llama.cpp sends one
constant id for every call. When session B's pre hook ran between
session A's pre and post, A consumed B's snapshot. A's
already-existing test_user.py then looked "not in existing", and A
tracked it.
- Every pre_tool_call hook sees the original args. Another plugin's
`modify` directive is merged afterwards, and post_tool_call sees the
final args. A path swapped in by that hook was never snapshotted, so
"not in existing" held for it too. The file was tracked even when the
write was refused.
Snapshots are now keyed by (task_id or session_id, tool_call_id), the
identity both hooks receive from the dispatcher. They store the paths the
call named that did not exist yet, instead of the ones that did. The post
hook tracks a path only if its own snapshot saw it absent and it exists
now. Any path the snapshot did not cover fails closed.
I reproduced both end to end through model_tools.handle_function_call
with a second plugin's modify hook and two interleaved sessions. The
user's file was deleted before this fix and survives after it, and a
file the call really created is still cleaned up. The README hook
contract is updated to match.
(cherry picked from commit 8d0b853981f1ab0775549f4831a3d4f733a46106)
post_tool_call sent write_file/patch paths, every absolute path in a
terminal command AND every path in its output to dg.track(), gated only
on `p.exists()` plus the test_*/tmp_* name rule. Category 'test' is
deleted at age 0 by quick(), which on_session_end runs after every turn,
and directories are rmtree'd. So a user's cron script
$HERMES_HOME/scripts/test_uptime.py the agent merely patched, or a hook
dir $HERMES_HOME/hooks/test_notify it listed (`ls -la`, `find ... -name
'test_*'`), was irreversibly deleted when the turn ended — then the
emptied scripts/ and hooks/ dirs too. The plugin's contract is files
"created during Hermes sessions".
A pre_tool_call hook now snapshots which of the call's argument paths
already exist (keyed by tool_call_id; it never raises, since pre hooks
fail closed; entries expire after an hour if the post hook never
fires). post_tool_call tracks a path only when the call named it, it
did not exist before and exists now. Paths seen only in terminal output
carry no such proof and are no longer tracked; no snapshot means
nothing is tracked. Existing tests now drive the real hook order.
(cherry picked from commit e0e0e1f7450df0c75a8499fe57b63dc46c6d2af2)
_git_tracks hand-rolled `git ls-files` without windows_hide_flags, an
isolated git env or stdin=DEVNULL. It runs from the synchronous
post_tool_call hook, so on a windowless Windows host each test_*/tmp_*
candidate could flash a console (#54220/#56747 class), and an inherited
GIT_DIR/GIT_WORK_TREE would point it at the wrong repo. Reuse
hermes_cli.source_check._git_ok, which already does all three and returns
False on any failure. The timeout drops to 5s (ls-files needs no more).
git reads the argument as a pathspec, so an untracked `test_[1].py` or
`tmp_*` glob-matched a tracked sibling and was never cleaned. Prefix it
with `:(literal)`.
Only spawn git when a .git exists at or above HERMES_HOME (HERMES_HOME is
a checkout, or sits inside a dotfiles repo). Without one, no repo can
track the file, so a stock install now does a few stats and skips the
process spawn on every qualifying tool call.
Drop the docstring paragraph that repeated _git_tracks' rationale.
The tracked-file guard only ran when HERMES_HOME/.git existed, so a
HERMES_HOME nested in an enclosing repo (a ~/.git dotfiles repo tracking
~/.hermes/scripts/test_x.py) still had that committed file deleted by
quick() - the same bug class the guard was added for.
guess_category only reaches this check for test_*/tmp_* names, so a
per-candidate 'git -C <parent> ls-files --error-unmatch -- <name>' is
cheap. It finds whichever repo encloses the file, and it replaces the two
lru_caches plus the index-stat signature: with no cache there is nothing
that can outlive the index, and ls-files output no longer needs decoding.
An exact tracked check stays safe above HERMES_HOME, unlike the bare .git
probe that was dropped for being too broad.
Co-authored-by: David Crandall <david@convergentdesign.dev>
_git_tracks wrapped the lookup in a bare except Exception. The only real
failure it hid was UnicodeDecodeError from text=True on non-UTF-8 paths in
the ls-files output, which escapes the (OSError, SubprocessError) catch.
Decode with surrogateescape (matching how Python decodes filesystem paths)
and drop the catch-all so real bugs surface.
Co-authored-by: David Crandall <david@convergentdesign.dev>
`_git_tracked_index()` cached one `git ls-files` set per process, so a
long-lived process could keep classifying a file as untracked after git had
taken ownership of it. `guess_category()` runs on every post-tool-call, so the
gateway can cache the index while `test_scratch.py` is still untracked; an
auto-snapshot then stages and commits it, `quick()` re-validates the stored
"test" entry through `guess_category()`, the stale cache still reports the file
as untracked, and `_delete_item()` unlinks a file git now owns (`git status`
shows `D test_scratch.py`).
Key the cached set on the git index's stat signature as well as the home, so a
staged, committed or unstaged change is a new cache key and no invalidation
hook is needed anywhere. The index path comes from `git rev-parse --git-path
index`, cached per home — so the per-call cost stays a single `os.stat`, which
also covers a linked worktree (where `.git` is a pointer file) and an explicit
`GIT_INDEX_FILE`. Fail-open behaviour is unchanged: no index and no git mean the
empty set, and the guard never raises.
Regression test: classify an untracked `test_scratch.py`, then `git add` and
`git commit` it in the same process with no `cache_clear()`, and assert
`quick()` leaves it on disk while an untracked control beside it is still
cleaned.
(cherry picked from commit a3271750608ca4e02e5205311a09b906e16d33d5)
The `_inside_git_worktree()` guard sliced its parent chain at HERMES_HOME, so
`.git` entries strictly below the home were the only ones consulted. That misses
the case where HERMES_HOME IS a git checkout: on this install `~/.hermes` is the
userfiles repo, so `~/.hermes/scripts/` and any top-level `tmp_*` file sit inside
a worktree yet resolve to no `.git` below the home.
Observed live: the bundled disk-cleanup plugin classified two COMMITTED
regression tests in `~/.hermes/scripts/` as disposable session scratch and
unlinked them — `scripts/test_analyze_upstream_opportunities.py` and
`scripts/test_customization_protocol_v2.py`; the deletion was then committed by
the next auto-snapshot (ce8dc99), so the suite that would have caught a scanner
regression was silently gone. 13 tracked `test_*` files under `scripts/` plus 17
tracked top-level `tmp_*` files were in the same blast radius.
Fix: when HERMES_HOME itself carries `.git`, ask git whether it TRACKS the exact
path (`git ls-files`, one cached subprocess per process keyed on the home) rather
than treating the whole home as protected — a scratch file merely living beside
tracked ones must still be cleanable, or cleanup would be disabled entirely.
Tests: two regression tests — a git-tracked `test_*` file is never classified
disposable and a stale pre-fix tracked.json entry is dropped by quick()'s
re-validation instead of deleted; plus the control that an UNTRACKED `test_*`
file in the same repo is still cleaned. Existing suite 30 -> 32 passed.
(cherry picked from commit c5555940108286c4c1070e84d6d694ad9a35bb76)
(cherry picked from commit c7c39e60d4dc467bf9681c3dcb5781ac96557724)
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).
Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.
uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
The cherry-picked ownership check walked every ancestor up to `/`, so a
HERMES_HOME kept inside a dotfiles checkout (`~/.git`) turned every
root-level test_* scratch file into a protected "git-owned" file and
silently disabled the plugin's core contract. Cut the walk at
HERMES_HOME for in-home paths; out-of-home (/tmp/hermes-*) trees keep
the full walk since is_safe_path already bounds them.
Tests trimmed to the two invariants: quick() drops a stale tracked entry
for a committed test inside a linked worktree (.git pointer FILE) instead
of deleting it, and root-level scratch is still deleted even with a .git
above HERMES_HOME. Dropped the contributor's literal /tmp test (the repo
never writes /tmp) and the guess_category-only case the quick() test
already drives.
guess_category() matched test_*/tmp_* by basename alone, so a committed
regression test inside a git worktree under $HERMES_HOME/worktrees/ or a
/tmp/hermes-* checkout was tracked and auto-deleted by quick() at session
end (#115295; the protected-top-level-dir half landed in #114770).
Classify such files as non-disposable whenever a .git entry (directory or
linked-worktree pointer file) exists on the directory chain. quick() and
dry_run() already re-validate stored "test" entries through
guess_category(), so stale pre-fix tracked.json entries are dropped from
tracking instead of deleted — no separate migration needed. Scratch
test_* files outside git-owned trees keep aging out as before.
Fixes#115295
Resolved toward the branch: PM provisions uv/python (main's install.ps1 uv-shim
salvage + its test and workflow steps dropped), the shim re-exec stays retired,
package.json carries no electron-builder block (afterExtract identity stamp wired
into electron-builder.config.cjs instead; after-pack.mjs keeps signing only),
Desktop workspace-deps helpers stay retired. Main's scratch-dir bootstrap
(export_scratch_tmp_env) is taken and re-run after profile resolution.
The seventeen remaining literals are container-side paths, AF_UNIX socket-path-limit
candidates on darwin, detection needles, guard regexes and guidance text that tells the
model to avoid /tmp. Each carries an inline `no-tmp: ok — <why>` so the reason lives
next to the line; the baseline keeps only a fenced tree listing where a marker would render.
- 15 `MERGE-CHECK:` conflict-resolution comments removed from prod code (two were
TODOs already done: the utf-8-sig sessions.json read lives in session_persistence,
the pm-aware cron script helpers in scheduler_script).
- 49 imports the branch left unused (ruff F401, none present at the merge base,
none inside PLUGIN-COMPAT blocks). update_cmd's frozen-surface re-exports are
trimmed to the names tests/compat/old_updater_surface.json actually lists under
hermes_cli.update_cmd; the rest resolve through hermes_cli.main.__getattr__.
- tools/environments/local_gitbash_probe.py: nothing imported it once _find_bash
delegated to pm.shell().
- Three try/except wrappers around calls that cannot raise (install_truststore,
get_hermes_home, and a duplicated except clause in supermemory).
The tracked-item delete path in quick() called shutil.rmtree() on any tracked
directory without consulting the protection list, which only the empty-dir
sweep used. guess_category() files every path under cache/ as "temp", so a
terminal command that merely mentioned $HERMES_HOME/cache tracked the
directory itself, and 7 days later quick() removed cache/ wholesale — taking
cache/terminal (terminal snapshots) with it and breaking every later command
with a mktemp "No such file or directory".
Fix: _is_protected_dir() — a tracked DIRECTORY that is HERMES_HOME itself or
sits under an _EMPTY_DIR_PROTECTED_TOP_LEVEL tree is never tracked by
guess_category(), never listed by dry_run(), and skipped (logged SKIPPED,
entry dropped) by quick(). Files under cache/ still age out as before.
kanban/ (task attachments and workspaces have their own lifecycle) is added to
both _NEVER_TRACK_TOP_LEVEL and _EMPTY_DIR_PROTECTED_TOP_LEVEL; stale pre-fix
"test" entries under it are dropped by the existing re-validation instead of
deleted. The kanban row was first proposed in #80842 (@nicha16).
Fixes#114552
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
Extends the salvaged #112860 entry (`workspace`) to the whole class: `plans` and
`home` are bootstrapped alongside `workspace` by `hermes_cli/profiles.py::_PROFILE_DIRS`
and listed as user data in `profile_distribution.py::USER_OWNED_EXCLUDE`, so a
`test_*`/`tmp_*` file inside any of them is a user's file, never scratch.
Trims the contributor's five tests to two invariants: the end-to-end
post_tool_call -> on_session_end path (workspace file survives, a root-level
`tmp_scratch.py` control is still removed) and the empty-dir sweep leaving a
directory under `workspace/` alone. Documents the protected trees in the plugin
README and the built-in-plugins page.
Dropped: test_workspace_project_tree_never_tracked, test_quick_keeps_workspace_test_file,
test_root_level_test_file_still_auto_deleted (folded into the E2E test as its control).
Part of #112859.
``workspace/`` is a user-owned project tree: every profile bootstraps it
(``profiles.py::_PROFILE_DIRS``), ``profile_distribution.py`` excludes it as
user data, and bundled plugins keep durable state there (``google_meet``
writes auth/registry JSON under ``workspace/meetings/``).
It was missing from ``_NEVER_TRACK_TOP_LEVEL``, so ``guess_category()``
returned "test" for ``workspace/<project>/tests/test_*.py``;
``_is_auto_delete("test", age)`` accepts any age, so the ``on_session_end``
sweep unlinked those files seconds after pytest ran them green - 20 file
deletions and 476 empty-dir removals (chrome-profile data dirs among them)
over three days on one install. It was also missing from
``_EMPTY_DIR_PROTECTED_TOP_LEVEL``, so meaningful empty dirs inside a project
tree were swept too.
Add "workspace" to both sets, beside the sibling user project trees
(``projects``, ``patches``, ``skins``, ``themes``, ``contributors``) that
#75403 / #32164 / #37721 already protect. No ``tracked.json`` migration is
needed: ``quick()`` re-validates stored categories through
``guess_category()`` and drops mismatches.
Regression tests: 4 of the 5 new tests fail against the unpatched plugin,
including the end-to-end ``post_tool_call`` -> ``on_session_end`` path; the
fifth pins that genuine ephemeral test files (root-level ``test_*.py``) are
still cleaned.
Ramp Router efforts cache + warm/disk flags, xAI and OpenRouter image catalogs,
Hindsight append-capability verdict, memory-provider skill registry, OpenViking
atexit provider, Honcho loopback flow status, Langfuse client (os.environ-only
credentials) and disk-cleanup's protected cron paths held one profile's
credential- or home-derived value process-wide; YuanbaoAdapter._active_instance
was last-connected-wins across profiles.
Keyed by home key / credential fingerprint under an override, credentials read
through the secret scope, warm threads run under copy_context(); unscoped module
slots stay for the single-profile path and the existing monkeypatch tests.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
The scope-normalization sweep only fixed the read paths
(list_providers/get_provider/registry_generation) in most registries;
the write paths (register/snapshot/restore) still stored under the raw
scope string while reads resolved through hermes_home_key — so a
scoped registration was written under one key and read under another
(agent/terminal_env_registry.py test_scoped_registration_isolated
failed on both Linux and Windows CI).
- add hermes_constants.normalize_scope(scope): hermes_home_key for
non-None, None preserved for the process-global layer — one named
function instead of the expression repeated at ~20 call sites
- apply it on both sides of all nine scoped registries
(browser, terminal_env, image_gen, transcription, tts, video_gen,
web_search, secret_sources, dashboard_auth, tools.registry)
- agent/deadline: MAX_SAFE_TIMEOUT_S on win32 leaves 60s headroom under
the DWORD-ms ceiling so margin-adding consumers (human-wait +60s)
stay platform-safe; tools/approval fail-closed fallback mirrors it
- tools/approval: strip shell escapes everywhere except inside
drive-anchored paths (the Windows shell is bash, so r\m still spells
rm; only C:\... backslashes must survive deobfuscation)
- consolidate the duplicated _rmtree_force into hermes_cli/fs_utils
and repoint both callers
- hermes_cli/kanban + disk-cleanup: shlex.split(posix=False) keeps
wrapping quotes in tokens; strip them before dispatch/matching
- test gating: SIGKILL durability conformance cells are linux_only
(TerminateProcess is not SIGKILL), macOS TCC anchor symlink-tree
classes are linux_only, Linux chromium profile tests resolve the
fixture home via expanduser patch + posix-joined expecteds so they
run identically on every host
Gate the POSIX-only and symlink-only tests with the linux_only and
require_symlinks markers. Fix the real cross-platform bugs:
- file_operations: use the translate_path flag, send snippets as base64, and
use sys.executable (the MS Store python3 stub and the list2cmdline
backslash collapse both broke snippets)
- approval: treat backslash as a Windows path separator, not an escape
- registry, browser_registry, secret_sources, plugins: normalize scope-key case
- checkpoint_manager, plugins_cmd: clear read-only bits before delete
- deadline: make MAX_SAFE_TIMEOUT_S fit the Windows limit
- image_routing, acp, cua_backend, daytona: fix Windows and POSIX paths
- hermes_state: match backslash in the retag LIKE clause
- kanban, disk-cleanup: match drive paths and split command arguments
- scripts: emit host separators through as_posix
207 test files are gated or isolated.
Read text files with the encoding utf-8-sig so a BOM at the start of a
file does not cause a Unicode decode error (Windows editors add BOMs).
Reconstructed from ethie/pm commits 48a32b135b + 013219e814 onto the
current upstream/main base: only the utf-8 -> utf-8-sig transforms were
carried (370 exact line pairs across 205 files); pm-rename hunks that
rode in the original commit were left to the pm-store commit, and
utf8sig hunks entangled with content changes ride their owning commit.
Rebuilt on ethie/pm-clean off ac6c8028e0 (upstream/main).
Address review feedback from teknium1:
1. Re-validate stale 'test' category entries in quick() — existing
tracked.json entries under now-protected directories (patches/,
projects/, etc.) are re-classified via guess_category() and
dropped instead of deleted, mirroring the cron-output pattern.
2. Add patches, projects, skins, themes, contributors to
_EMPTY_DIR_PROTECTED_TOP_LEVEL so the empty-directory sweep
never traverses into these user-authored project trees.
guess_category() classified any file whose name starts with 'test_' or
'tmp_' as disposable, even when the file lived under user-authored
directories like patches/, projects/, skins/, or themes/. Files in
these trees were silently deleted on session end.
Added the missing user-project directories to the exclusion list so
that basename-based classification only applies to files in temporary
or scratch locations, not durable project trees.
AST-driven pass over every Path.read_text()/write_text() without an
explicit encoding= across non-test code: 71 sites in 34 files
(skills_hub, hermes_cli/main+profiles+service_manager+container_boot,
mem0/hindsight/honcho plugins, achievements dashboard, release/CI
scripts, productivity+comfyui skill helpers, agent/*). Verified zero
positional-encoding collisions before insertion; per-file compile()
check after.
Adds a check-windows-footguns rule flagging bare single-line
read_text/write_text (multi-line forms stay covered by the AST guard
test from #38985). Together with the salvaged contributor commits this
retires the ~169-site bare file-I/O class (#37423's long tail).
Review follow-up: the _is_protected_cron_path docstring listed output/ next
to jobs.json/.tick.lock as 'the directory itself', which is slightly
ambiguous. Spell out that the match is EXACT-path only and must not be
'simplified' into a blanket cron/output/* guard (children stay cleanable) —
prevents a future editor from re-introducing the wholesale-delete bug this
fix closes.
Only classify files below cron/output/ as disposable cron output.
The cron/output directory itself is a durable container for retained
job history and should not be tracked or deleted wholesale.
Add regression coverage for both category detection and cleanup of a
stale tracked entry pointing at the output root.
quick() and dry_run() previously trusted the stored category from
tracked.json without re-validating at delete time. Stale entries from
before #34840 could carry category="cron-output" for cron control-plane
paths (e.g. cron/jobs.json), causing quick() to delete the live
scheduler registry.
Fix:
- Fix guess_category() to only classify cron/output/** as cron-output
(was classifying ALL cron/* paths, missing the #34840 fix).
- Re-validate cron-output entries via guess_category() at delete time
in quick() and dry_run(); stale entries that are no longer classified
as cron-output are skipped and removed from tracked.json.
- Add _is_protected_cron_path() as a hard defense-in-depth guard that
blocks deletion of cron/cronjobs directories and known control-plane
files (jobs.json, .tick.lock) regardless of stored category.
- Update test_cron_subtree_categorised to match fixed guess_category
(only cron/output/* is cron-output, not all of cron/).
Tests: add 5 regression tests in TestStaleCronEntryMigration.
