Commit Graph

22 Commits

Author SHA1 Message Date
kshitijk4poor
6e7dad7608 test(tools): overwriting the database file itself is still refused as binary
The test trim dropped the headline case: text written over an EXISTING .db. Sidecar paths return at is_sqlite_sidecar before the overwrite branch, so a mutant that drops has_binary_extension from the overwrite condition survived every remaining test. Add a third parametrize case that targets the held-open WAL database file itself and asserts the binary refusal with bytes unchanged.

Same commit: the _check_binary_document_write docstring now names SQLite sidecars (always) and every BINARY_EXTENSIONS suffix (on overwrite), and the suffix is computed once at the top instead of three times.
2026-09-17 21:17:26 +05:30
kshitijk4poor
c6f3a3cf24 refactor(tools): one suffix helper for the binary-extension checks; drop the dead .db3 member
_has_extension_in and is_sqlite_sidecar each re-implemented the same
rfind('.') / -1 / .lower() suffix extraction, and the write guard used a
third spelling (filepath[filepath.rfind("."):]) for its refusal messages
next to os.path.splitext in the overwrite branch. Add _lower_suffix()
("" when no dot) and route both predicates through it; the write guard
now uses os.path.splitext for every message's displayed extension.

_SQLITE_EXTENSIONS was `{.db,.sqlite,.sqlite3,.db3} & BINARY_EXTENSIONS`,
which silently dropped .db3 (not a binary extension anywhere in the
codebase, so `x.db3-wal` was never a sidecar). Spell the set as the three
members that actually take effect and assert the subset invariant instead
of hiding it behind an intersection. Behaviour is unchanged.
2026-09-17 21:17:26 +05:30
kshitijk4poor
d219e5755a fix(tools): a SQLite sidecar path is refused even when no sidecar file exists yet
The refactor that moved sidecar detection into binary_extensions dropped the
contributor's unconditional refusal for the db family and made every binary
extension a plain overwrite guard. That is right for the MAIN .db/.sqlite
(text fixtures named *.db exist), but a -wal/-shm/-journal path is never a
legitimate text target: a checkpointed database has no sidecar on disk, so
write_file("kanban.db-wal", text) silently dropped a garbage WAL next to a
live database.

Add is_sqlite_sidecar(path) and refuse such paths in
_check_binary_document_write regardless of is_file(). Scope the marker
stripping in _has_extension_in to SQLite suffixes so report.docx-wal no
longer counts as an opaque document. Hoist the duplicated is_pdf_path call.
Parametrize the write_file WAL test over sidecar existing/absent.
2026-09-17 21:17:26 +05:30
kshitijk4poor
9b7e24bccf refactor(tools): recognise SQLite sidecars in binary_extensions, drop write-guard regex
Move the .db-wal/-shm/-journal detection from a write-guard-local regex
into tools/binary_extensions._has_extension_in so has_binary_extension
(read guard AND write guard) agree: a sidecar counts as its database's
extension. read_file now refuses sidecars instead of returning lossy
text, which also gives write_file the no-baseline overwrite refusal.

Behaviour change vs the contributor's commit: creating a NEW .db /
.sqlite file via write_file stays allowed (main allows it and text
fixtures named *.db exist) — only overwriting an existing binary is
refused. The generic has_binary_extension overwrite branch is kept and
merged with the PDF branch because patch has no full-read baseline
check: on main, patch on an existing .db with a matching old_string
rewrites the header in place.

Tests folded into the existing guard test classes (one write_file, one
patch refusal on a real WAL sidecar); the PR's 12-test file and its
private-regex assertions are dropped.
2026-09-17 21:17:26 +05:30
Martin Mogis
be79102ed8 fix(tools): refuse text writes into database files and binary overwrites
The text tools' read->modify->write round-trip re-encodes binary bytes
lossily: the terminal transport decodes stdout with errors=replace, so
a patch on a SQLite database reads mojibake and writes it back, silently
destroying the file (observed in the wild: a kanban board DB lost 550k
bytes / 26 days of history this way on 2026-09-15).

read_file already refuses to display binary files (extension + byte
sniffing), but write_file/patch had no write-side equivalent, and
patch_replace reads via plain cat so the read-side guard never fires
for it.

Guard rules (extends _check_binary_document_write, the existing
binary-document write guard):

- .db/.sqlite/.sqlite3 and their -wal/-shm/-journal sidecars: ALWAYS
  refused (even new-file creation) — plain text can never be a valid
  database, mirroring the opaque-document rule. The sidecars need a
  basename regex because os.path.splitext('x.db-wal') yields '.db-wal',
  which is not in BINARY_EXTENSIONS.
- Other BINARY_EXTENSIONS (.png, .zip, ...): refused when OVERWRITING an
  existing file, mirroring the existing .pdf-overwrite rule — the model
  can only have read mojibake, so writing back destroys it. Creating a
  NEW file with such an extension stays allowed.

Tests: new test_binary_db_write_guard.py covers the regex (main names,
sidecars, non-db names), the guard function, and the write_file/patch
tool paths against real SQLite files (integrity_check still ok, bytes
untouched). 29 new+existing guard tests pass; file-tools/patch suites
green (196 tests).
2026-09-17 21:17:26 +05:30
teknium1
2dfb795cb7 fix(approval): undelivered or unanswered CLI approval prompts are not user denials
When the CLI approval callback raises, when no callback is registered on the
thread while prompt_toolkit owns the terminal, or when the input() read is
interrupted, prompt_dangerous_approval returned "deny" and the command gate
rendered "BLOCKED: User denied this command" — attributing a refusal to a
user who was never asked (#22992). #112308 fixed the gateway half of the
class (withdrawn prompts -> outcome "cancelled" with a cause); this closes
the CLI residual on the same shape.

- tools/approval_prompt.py: those three paths return an Unanswered("cancelled")
  sentinel carrying the cause; MCP elicitation consent maps it to "cancel".
- tools/approval.py: the CLI gate renders "BLOCKED: <noun> was not approved: the
  approval prompt could not be delivered or was not answered (<cause>)" with
  outcome "cancelled" — still fail-closed, "Silence is not consent".
- tools/file_tools_write_guards.py: the protected-instruction write gate
  reports the undelivered prompt instead of "was denied by the user".
- Shared metrics: "cancelled" is a counted approval outcome (contract + v2
  schema) instead of falling into "unknown".
- Docs: hook `choice="cancelled"` now covers the CLI causes.

Fixes #22992
2026-09-15 21:46:37 -07:00
teknium1
6332216384 fix(approval): withdrawn gateway approval prompts no longer read as a user deny
When a gateway approval wait ends without anyone answering — the parent's
delegate_task finishing and tearing the child down, a /stop, or the turn's
notifier being unregistered at turn end — the tool result said
"BLOCKED: Command denied by user" (outcome="denied", user_summary "You denied
this command"). The user never saw or answered the prompt, so the parent agent
went on reasoning about a refusal that never happened (#112026, #22992).

The action stays fail-closed (the command does not run, the model still gets
the NOT-consented stop text), but the attribution is now truthful:

- tools/approval_gateway_wait.py: `_cancel_cause()` reads the existing
  per-thread interrupt-cause channel (`get_interrupt_reason()`, a trusted fixed
  category — no string matching) for the interrupted state and marks a
  notifier-unregister wake (event set, result None) as "the turn ended before
  the prompt was answered". Both the direct and the coalesced-follower wait
  return `cancelled=<cause>`; the post_approval_response hook fires
  choice="cancelled" instead of "deny"/"timeout".
- tools/approval.py: a cancelled decision renders
  "BLOCKED: Command approval was withdrawn before the user answered (<cause>)."
  with outcome="cancelled" and its own user_summary; an explicit /deny is
  untouched.
- tools/delegate_tool_child_run.py: `_signal_child_stop` publishes a fixed
  tool_reason ("parent delegation ended"; the late-child mirror forwards the
  parent's own category) so a child's pending approval can tell teardown from a
  user /stop — previously it rode the default "explicit stop requested".
- tools/file_tools_write_guards.py / tools/approval_prompt.py: the protected
  instruction-file gate and MCP elicitation consume the same key instead of
  reporting "denied by the user" / "decline".

Co-authored-by: zccyman <16263913+zccyman@users.noreply.github.com>
Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
2026-09-15 18:44:46 -07:00
Teknium
faf71eb4c1 Inspired by Claude Code: file tools reject Windows NT-namespace paths (NTLM leak hardening)
Claude Code v2.1.234 (Aug 17, 2026) hardened its pre-approval file
accesses to reject Windows NT-namespace (\??\) paths against the NTLM
credential-leak vector. Port the same guard into Hermes file safety:

- agent/file_safety.py: is_nt_namespace_path() / get_nt_namespace_error()
  raw-string check (never resolves — resolving IS the leak trigger).
  Wired as the first check in get_read_block_error() and the write
  denial classifier.
- tools/file_tools.py: raw-string guard at read_file_tool entry and in
  _check_sensitive_path (covers write_file_tool + patch_tool), before
  the task-base join can anchor the prefix under a POSIX base dir.
- Blocks \??\, \\.\, \\?\UNC\, \\?\GLOBALROOT. Extended-length
  local drive paths (\\?\C:\...) and plain UNC shares stay allowed.
- tests/agent/test_nt_namespace_guard.py: 10 blocked forms, 11 allowed
  forms, no-resolve proof, tool-layer chokepoint coverage.
- docs: protected-paths table in user-guide/security.md
2026-09-15 04:17:31 -07:00
teknium1
dddefaefae fix: paged, extracted and post-compaction reads count as a write_file baseline
The stale-overwrite refusal made write_file permanently unusable for any
existing file it could not show in one read_file page: every >2000-line
(or >100K-char) page was recorded as partial, no full baseline ever
existed, and the refusal told the model to "re-read the whole file", which
the tool cannot do. Track the line ranges each task pages through per path
at one mtime; contiguous pages from line 1 to total_lines are a full read
(a new mtime between pages restarts the coverage). The same gap hit two
siblings: the extracted-document branch (.ipynb, text-authorable) returned
before any read bookkeeping, so an existing notebook could never be
overwritten; and reset_file_dedup dropped every baseline on compaction
while keeping read_timestamps, so every write after compaction was refused
even for files unchanged on disk. Baselines now survive compaction exactly
like the dedup mtime map does — only while the recorded mtime still matches.

Refusal texts no longer embed the pre-PR "Warning: … Consider re-reading"
copy inside "Refusing to overwrite", and every refusal names a recovery the
model can perform: read the remaining pages, or use patch.
2026-09-15 03:57:24 -07:00
DanSpicyTaco
a8af57fcd2 fix: block stale write_file overwrites
Require an explicit full-file baseline before replacing existing host-visible files with write_file, and fail closed when that baseline is stale. This prevents stale conversation context from clobbering manual or external edits.\n\nRefs #65604
2026-09-15 03:57:24 -07:00
teknium1
818d781c0b test(tools): trim the salvaged #110632 / #110307 tests to two invariants each
The file-safety salvage carried five tests and the execute_code salvage four;
fold them into the two behaviour contracts per fix: (a) the named-profile scope
exempts the root's direct files and the write lands with no prompt; a child env
sees the ACTIVE profile's home per turn; (b) negatives hold: a checkout's
.hermes/config.yaml stays gated fail-closed, a lookalike profiles/ tree exempts
nothing; a dedicated process with no override is untouched. Both red on base.
Also compress the _hermes_exempt_homes docstring (WHY only; cite #110630).
2026-09-14 16:13:51 -07:00
moxian
ab78e71a9e fix(file-safety): exempt the Hermes ROOT, not just the profile home
Under a named profile (``hermes -p <name>``, i.e. ``HERMES_HOME=<root>/profiles/<name>``)
the protected-instruction gate exempted only the profile dir, so the ROOT's DIRECT
files (LEDGER.md / MEMORY.md / USER.md / SOUL.md / AGENTS.md / DECISIONS.md ...) fell
through to the ``.hermes`` component rule and were treated as project-local
``<repo>/.hermes/config.yaml``. That gate always-asks and fails closed without a human
channel, so every write there was refused headless — it blocked #54 (LEDGER.md edit
could not land). Subdirectories (scripts/, cron/, data/) were unaffected, which is why
#55 sailed through and #54 did not.

``_hermes_exempt_homes()`` now returns the active profile home plus the Hermes ROOT
when that home really is a named profile (``named_profile_home`` validates: ``.hermes``
name, home markers, tombstone, or the resolved default root), so a coincidental
``profiles/`` directory never exempts its parent. ``_get_real_hermes_home()`` keeps its
per-profile semantics — the #107327 multiplex regression test pins that — and the
exemption consumes the new helper instead.

Refs: https://gitlab.yx.netease.com/agent-projects/platforms/agent-mentor/-/issues/60
2026-09-14 16:13:51 -07:00
PRATHAMESH75
7af5006b24 fix(file-safety): bind the write-guard resolver fallback to the active profile
The per-call home/config getters fell back to `_expand_tilde("~/.hermes...")`
when the primary resolver raised. `_expand_tilde` follows the subprocess-HOME
contract, which under host `auto` mode can be the real/default user home rather
than the active multiplex `HERMES_HOME`. So on the exception path the guards
recreated the very cross-profile authority bug the happy path fixed: beta's
`config.yaml` was compared against the default/root config (hard-block fails
open), and the protected-instruction exemption resolved against the wrong home.

Re-derive both fallbacks from the same `get_hermes_home()` key the happy path
uses (`Path(home)/config.yaml`, `realpath(home)`), and substitute no unrelated
home if the active security path cannot be established — a `None` fails closed
at the protected-instruction consumer (exemption skipped, gate runs).

Adds opposite-side regressions: forcing the primary config resolver to raise
keeps beta's own config refused (and does not spuriously protect alpha's under
beta's scope); forcing the primary home resolver to raise keeps beta's
instruction-file exemption resolved against beta.

Addresses the fallback-authority review on #107335 (thanks @andrexibiza).

(cherry picked from commit 119d88b46f745ba081f12adf3f6ebba457d95d68)
2026-09-11 15:29:15 -07:00
PRATHAMESH75
1271622e4b fix(file-safety): resolve HERMES_HOME/config per call so multiplex profiles don't poison the write guards (#107327)
In a multiplexed gateway (`gateway.multiplex_profiles: true`) each profile turn
scopes `HERMES_HOME` through a per-turn contextvar. But
`tools/file_tools_write_guards.py` memoised the resolved home and config path in
process-global module state, filled once by whichever profile ran first. Both
the protected agent-instruction approval gate (`_get_real_hermes_home` →
exemption for a profile's own home) and the `config.yaml` hard-block
(`_get_hermes_config_resolved`) therefore became order-dependent: a later
profile's own `workspace/AGENTS.md` was gated against a *sibling* profile's home,
and — worse — its own `config.yaml` stopped matching the block, so a
prompt-injected agent could rewrite the very file the block exists to protect
(reproduced end-to-end in #107327).

Resolve both values per call instead. `get_hermes_home()` / `get_config_path()`
are contextvar-scoped, so the getters now track the active profile; the guard
already pays a `realpath` per call, so the extra cost is negligible. The two
module slots are kept purely as a test-override surface (set the slot + its
`_loaded` flag to pin a value); production leaves them unset and resolves live,
which also removes the cross-test poisoning the process memo could cause.

Adds regression coverage: both getters track the active profile after a prior
profile's scope, and the `config.yaml` hard-block fires for beta's own config
even after an alpha turn ran first.

(cherry picked from commit 36b257391da497ac31e6c440727dc57ecd584e71)
2026-09-11 15:29:15 -07:00
Teknium
707161e77b simplify(compat): file_tools/send_message_tool/cronjob_tools/process_registry — drop 34 re-exports, repoint 5 callers + 27 test files 2026-09-03 13:30:27 -07:00
Teknium
e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium
0a96fd9504 refactor(tools): unify path anchoring, cached lookups and resolve-or-none clones 2026-09-02 23:26:43 -07:00
Teknium
e4436ec9e5 refactor(tools): hand-compact docstrings across file_tools group (keep every WHY) 2026-09-02 23:17:52 -07:00
Teknium
48de33b77f refactor(tools): compact file_tools group docstrings; regex exact strategy; one approval tail 2026-09-02 23:10:54 -07:00
Teknium
40f2c7951b refactor(tools): AST-neutral layout compaction of file_tools group 2026-09-02 22:51:55 -07:00
Teknium
8d6232878e refactor(tools): compact file_tools companions; one eviction helper in file_state 2026-09-02 22:44:46 -07:00
Teknium
d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00