Commit Graph

196 Commits

Author SHA1 Message Date
teknium1
8d8836ddb1 feat(telemetry): harness-accuracy shared metrics for the agent loop
Five opt-in counters that tune the agent loop itself, recorded through
record_process_mark (disabled config => zero rows, zero files):

- hermes.file_edit.count: tool, mode, outcome, match_strategy. The fuzzy
  matcher reports the strategy that landed (or no_match / ambiguous) into a
  context-local probe opened only around the patch/write_file handlers, so we
  learn which strategies earn their keep without counting internal callers.
- hermes.loop_guard.count: provider, model, signal, detector. Hooked where
  the guardrail already warns/blocks/halts and at turn end for the iteration
  budget; latched once per turn per (signal, detector).
- hermes.tool_recovery.count: provider, model, tool, next_tool,
  next_outcome. One row per failed tool call, resolved against the model's
  next round (same tool first) or the turn end (no_tool_call / gave_up).
- hermes.terminal.outcome.count: backend, command_kind, outcome. The kind is
  a table lookup on the first program word, never the text. Hermes' own
  deadline/interrupt now set hermes_timed_out / hermes_interrupted on the
  backend result, so a command's own `exit 124` reads nonzero, not timeout.
  Disjoint dims from hermes.execution_backend.count (served vs not).
- hermes.model_reply_issue.count: provider, model, issue. Refusal and
  truncation only from the structured finish reason; empty/reasoning_only
  from the normalized message; issue=none per response as the denominator
  (model_route counts attempts and files empty replies as failures).

Background review / curator loops and _host_local / unmetered terminal
calls never count. Helpers live in shared_metrics_harness.py; call sites
are one or two lines each.
2026-09-28 12:43:03 -07:00
Teknium
a5bd246865 Old pre-decomposition import paths are gone: plugin compat layer removed on schedule (#126164)
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule

The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:

- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
  names, restored dead definitions) and the three re-export stub modules
  (gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
  the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
  `plugins.allow_deprecated_imports` escape hatch

An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.

hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).

In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.

* chore: retrigger CI (zero-job startup_failure phantom)

* test: drop resolution allowlist rows for the two deleted which() sites

hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
2026-09-28 10:21:41 -07:00
brooklyn!
b9d5e4d17f fix(docker): remap a mounted host session cwd to /workspace
A Desktop Docker session that registers an absolute host directory
such as /mnt/... or /srv/... as its cwd skipped the /Users|/home/
drive-letter heuristic, so the mount check never ran and commands
were wrapped with cd to that host path. Classify the mounted host
directory as unusable before that heuristic, and remap it to
/workspace in the live-env write and the per-command resolver.
2026-09-26 17:34:46 -05:00
John Paul Soliva
0a99750128 fix(patch): V4A Delete and Move act on a symlink itself, not the file it points to
patch_tool rewrites every V4A header to the path _resolve_path_for_task
returns, and on a host backend that is Path.resolve(), which follows a
symlink in the last component. For Update/Add that is harmless: the shell
layer reads and writes the target through the link either way. Delete and
Move act on the directory entry, so "*** Delete File: config/local.yaml"
(a link to base.yaml) deleted base.yaml and left the link dangling, and
"*** Move File: current.txt -> previous.txt" renamed the link's target,
both reported as success.

Delete headers and both Move endpoints now resolve their parent directory
only (_resolve_entry_for_task), keeping the final component, so the link
is removed or renamed. The same paths are locked and reported in
files_modified. Update/Add headers are unchanged.

(cherry picked from commit 7b1fe43d30db6015155349b67ca814b340df0e18)
2026-09-27 01:10:56 +05:30
xxxigm
666681cbf3 fix(terminal): do not cd an ssh session into the container home
The session cwd can already be the Hermes host's /opt/data/home. That
path is not on the SSH target, so the implicit cd exits 126 and the
command never runs.

(cherry picked from commit 4e99397bb4e18416b6a3e0b850d51604c0ad0d1d)
2026-09-26 06:20:21 +05:30
ethernet
c9bd7459c2 Merge remote-tracking branch 'origin/main' into ethie/pm-clean
# Conflicts:
#	scripts/run_tests_parallel.py
#	tui_gateway/model_switch.py
2026-09-22 00:28:30 -04:00
teknium1
602e76a88b fix: bind baseline reads to the task local path 2026-09-21 18:39:24 -07:00
teknium1
e636aedebf fix: preserve known file baselines across partial rereads 2026-09-21 18:39:24 -07:00
ethernet
e1576d06a6 Merge remote-tracking branch 'origin/main' into ethie/pm-clean
Resolved toward the branch: PM provisions uv/python (main's install.ps1 uv-shim
salvage + its test and workflow steps dropped), the shim re-exec stays retired,
package.json carries no electron-builder block (afterExtract identity stamp wired
into electron-builder.config.cjs instead; after-pack.mjs keeps signing only),
Desktop workspace-deps helpers stay retired. Main's scratch-dir bootstrap
(export_scratch_tmp_env) is taken and re-run after profile resolution.
2026-09-19 22:57:07 -04:00
teknium1
11d68e4652 feat(tools): read_file renders SQLite schemas and flags merge conflicts; web_extract refuses binary payloads
Three small gaps from the OMP comparison (#77367) that the existing tools
almost covered:

- read_file on .db/.sqlite/.sqlite3 was refused as binary. It now extracts a
  schema overview (CREATE per table, row count, first 5 rows, indexes/views)
  through the same document-extraction path as .docx/.xlsx, opened read-only
  and immutable so a live database is never locked. A .db whose magic bytes are
  not SQLite is refused with that reason. The Hermes read denylist now runs
  before extraction so protected stores cannot be reached through an extractor.
- read_file reports conflict_blocks: N (plus a hint) when the served range has
  balanced <<<<<<< / >>>>>>> marker lines, so the model resolves the conflict
  instead of editing around it. A lone marker in prose is not counted.
- web_extract returned 824K chars of raw SQLite bytes as page "content" for a
  .sqlite URL (live: Chinook_Sqlite.sqlite via the configured backend). Bodies
  starting with an unambiguous file signature (SQLite, ZIP, gzip, xz, 7z, ELF,
  Mach-O, PNG/JPEG/GIF/TIFF, FLAC/Ogg) become a typed error pointing at
  terminal + read_file. Backends drop NUL bytes, so signatures are compared
  NUL-stripped; two-letter signatures (BM, MZ, ID3) are excluded on purpose.
2026-09-19 03:07:47 -07:00
ethernet
a6ae6ace51 Merge remote-tracking branch 'origin/main' into ethie/pm-clean
# Conflicts:
#	.github/workflows/js-tests.yml
#	agent/model_metadata.py
#	apps/desktop/electron/main.ts
#	apps/desktop/scripts/bundle-electron-main.mjs
#	apps/desktop/src/app/settings/about-settings.tsx
#	apps/desktop/src/app/settings/gateway-settings.test.tsx
#	apps/desktop/src/app/settings/gateway-settings.tsx
#	apps/desktop/src/app/updates-overlay.tsx
#	gateway/shutdown_flush.py
#	hermes_bootstrap.py
#	hermes_cli/local_runtime/binaries.py
#	hermes_cli/main.py
#	hermes_cli/managed_uv.py
#	hermes_cli/update_cmd.py
#	hermes_cli/update_cmd_deps.py
#	hermes_cli/update_cmd_fleet.py
#	hermes_cli/update_cmd_maint.py
#	hermes_cli/update_receipt.py
#	hermes_cli/update_serve_obligations.py
#	hermes_constants.py
#	tests/hermes_cli/test_doctor.py
#	tests/hermes_cli/test_managed_uv.py
#	tests/hermes_cli/test_pending_supervisor_recovery.py
#	tests/hermes_cli/test_startup_fast_guards.py
#	tests/hermes_cli/test_update_desktop_stale_warning.py
#	tests/hermes_cli/test_update_fleet_restart_pending.py
#	tests/hermes_state/test_hermes_state.py
#	tests/tools/test_tirith_security.py
#	tools/bot_relay.py
#	tools/checkpoint_manager.py
#	tools/write_approval.py
#	website/docs/getting-started/updating.md
#	website/docs/reference/environment-variables.md
2026-09-18 17:26:10 -04:00
kshitijk4poor
027d1a8a60 fix(tools): read_file's dedup stub skips the review fork too
Sibling of the previous commit: read_file_tool returned the unchanged-stub
before _record_read, and only _record_read calls
mark_background_review_skill_read. A skill file the parent had already read
left the fork with a stub and a refused skill_manage write. Same rule as
skill_view: no dedup stub when is_background_review().

Also: is_background_review hoisted to module level in skills_tool (no cycle —
skill_provenance imports only contextvars), comment reworded to the two real
reasons, test renamed to the shipped design, fixture seeds a fresh read-mark
store so the mark cannot leak between tests.
2026-09-19 00:33:58 +05:30
teknium1
9814fb9286 fix(guardrails): mark every file-tool loop refusal and exempt it on the live failure seam
The salvaged commit adds the `guardrail_refusal` marker to the read-dedup block
and exempts it in `classify_tool_failure`. That classifier is only the fallback:
the executor (`agent/tool_executor.py::_commit_tool_result`) passes
`failed=_detect_tool_failure(...)` from `agent/display.py` into
`after_call`, so on the real path the refusal was still counted and the
escalation to `repeated_exact_failure_block` still fired (live probe: block at
call 8 of an identical read after 5 refusals, on the salvaged head as on main).

- `agent/tool_result_classification.py`: `GUARDRAIL_REFUSAL_KEY` +
  `is_guardrail_refusal()` so the marker has one definition and both
  classifiers share the predicate.
- `agent/display.py::_detect_tool_failure`: exempt refusals (the live seam).
- `tools/file_tools.py`: the two sibling loop refusals -- the consecutive-read
  BLOCK in `read_file_tool` and the repeated-search BLOCK in `search_tool` --
  carry the same marker; they are emitted via `tool_error` too and escalated
  the same way.
- Tests trimmed to two invariants covering all three refusals on both
  classifiers, plus the negative (a genuine error still counts; a non-boolean
  marker is ignored).

Approval-denied write blocks (`file_tools_write_guards`) deliberately keep
counting: a user's denial retried unchanged is exactly the loop the streak
exists to stop.
2026-09-18 10:03:04 -07:00
rlaope
c962c698f8 fix(guardrails): a harness refusal is not a tool failure
The read-dedup block in `_dedup_stub_or_block` returns its message through
`tool_error`, so the body carries `"error"` for the model's benefit. That is
exactly the substring `classify_tool_failure` keys on, so the guardrail counted
its own refusal as a failed call: refusing a repeated read raised the failure
streak that produces the next, harder refusal, and the resulting
`repeated_exact_failure_block` told the model N identical calls had failed when
none had.

Observed on a real session (55 patches, 73 reads, 13 searches, 14 terminals):
13 of the 14 results the classifier called failures were this block. One tool
genuinely failed -- a `gh` call with exit 8. The model was told to change
strategy over a count the harness had produced itself, and could not diagnose it
because the message did not describe anything that had happened.

The block now carries `guardrail_refusal`, and the classifier exempts a body
marked that way. The exemption is keyed on the marker rather than on wording, so
a body that genuinely failed still counts and the streak that stops a real loop
is unchanged.

Signed-off-by: rlaope <piyrw9754@gmail.com>
(cherry picked from commit b3b09822f3db5e70a181e7dd19725afe1b1583a7)
2026-09-18 10:03:04 -07:00
ethernet
b4a294fff9 Merge origin/main; keep PM as plugin dependency owner
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
2026-09-17 13:52:05 -04:00
teknium1
d77cf5ae51 fix(tools): search_files only defaults None/blank paths, not wrong-typed ones
The #112424 normalisation used `not isinstance(path, str) or not
path.strip()`, which also silently mapped a non-null wrong-typed path
(e.g. 5 or a list) to "." and searched the whole cwd instead of reporting
the bad argument — wider than the issue (blank/null) asked for. Restrict
the predicate to `path is None or (isinstance(path, str) and not
path.strip())` so any other type falls through to search_tool's existing
error path, exactly as on main.

Test: test_handler_preserves_nonblank_path now also covers 5 and ["src"];
those two cases fail on the previous commit (path became ".") and pass here.
2026-09-16 17:19:31 -07:00
teknium1
ba96e68d47 fix(tools): search_files treats a JSON-null path like a blank one
The model can send `"path": null` as easily as `"path": ""`; on base a
None path reached search_tool and surfaced as an os.PathLike TypeError
instead of the documented current-directory default. Widen the blank-path
normalisation in _handle_search_files to any non-string and pin it in the
existing parametrized test.

Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com>
2026-09-16 17:19:31 -07:00
KoNit-K
66cc3f165d fix(tools): normalize blank search paths 2026-09-16 17:19:31 -07:00
shehjaddev
d205cef418 fix(redact): mask assignments in secret-bearing file reads
read_file/search_files passed file_read=True, which folded into code_file=True and skipped
the ENV/JSON/YAML assignment passes, so an opaque prefix-less credential under a
credential-shaped key reached the model in cleartext from a secret-bearing file — the
file-read half of the #110228 gate (#110567).

Two defects on that path, both fixed here:

- The rendered line-number gutter ("5|      ADS_API_TOKEN: ..." from read_file,
  "6:      ADS_API_TOKEN: ..." from grep -n / cat -n) defeated the line-anchored patterns,
  so the real rendered read leaked exactly what the raw text masked. A gutter-free fixture
  cannot see this, which is why the tool-level tests carry the real render shape.
- _is_secret_file_arg() could not see the RESOLVED Hermes home: the default home's basename
  is an installation detail (".hermes" on POSIX, "hermes" under AppData/Local on Windows) and
  a resolved path never spells $HERMES_HOME, so the managed Windows home's config.yaml was
  classified as ordinary YAML on both the file-read and the terminal surface.

Changes:

- redact_sensitive_text(): secret_file= re-enables the assignment passes for content the
  caller classified with _is_secret_file_arg, keeping code_file behaviour everywhere else.
  It is authoritative over code_file, so a caller cannot be fail-open on the security flag
  by setting both.
- _redact_assignments(): mask_nonreusable selects the non-reusable sentinel for file reads,
  so the #35519 write-back hazard stays closed.
- _should_redact_assignment(): no longer re-masks an already-masked value, which was erasing
  the vendor label the sentinel deliberately keeps.
- _is_secret_file_arg(): consult the resolved Hermes home for the config.yaml arm.
- _CFG_ANCHORED_RE / _YAML_ASSIGN_RE: tolerate a rendered line-number gutter.
- file_tools.py: classify the resolved path at all three file-read call sites.

Closes #110567
2026-09-15 06:26:29 -07:00
Teknium
faf71eb4c1 Inspired by Claude Code: file tools reject Windows NT-namespace paths (NTLM leak hardening)
Claude Code v2.1.234 (Aug 17, 2026) hardened its pre-approval file
accesses to reject Windows NT-namespace (\??\) paths against the NTLM
credential-leak vector. Port the same guard into Hermes file safety:

- agent/file_safety.py: is_nt_namespace_path() / get_nt_namespace_error()
  raw-string check (never resolves — resolving IS the leak trigger).
  Wired as the first check in get_read_block_error() and the write
  denial classifier.
- tools/file_tools.py: raw-string guard at read_file_tool entry and in
  _check_sensitive_path (covers write_file_tool + patch_tool), before
  the task-base join can anchor the prefix under a POSIX base dir.
- Blocks \??\, \\.\, \\?\UNC\, \\?\GLOBALROOT. Extended-length
  local drive paths (\\?\C:\...) and plain UNC shares stay allowed.
- tests/agent/test_nt_namespace_guard.py: 10 blocked forms, 11 allowed
  forms, no-resolve proof, tool-layer chokepoint coverage.
- docs: protected-paths table in user-guide/security.md
2026-09-15 04:17:31 -07:00
teknium1
dddefaefae fix: paged, extracted and post-compaction reads count as a write_file baseline
The stale-overwrite refusal made write_file permanently unusable for any
existing file it could not show in one read_file page: every >2000-line
(or >100K-char) page was recorded as partial, no full baseline ever
existed, and the refusal told the model to "re-read the whole file", which
the tool cannot do. Track the line ranges each task pages through per path
at one mtime; contiguous pages from line 1 to total_lines are a full read
(a new mtime between pages restarts the coverage). The same gap hit two
siblings: the extracted-document branch (.ipynb, text-authorable) returned
before any read bookkeeping, so an existing notebook could never be
overwritten; and reset_file_dedup dropped every baseline on compaction
while keeping read_timestamps, so every write after compaction was refused
even for files unchanged on disk. Baselines now survive compaction exactly
like the dedup mtime map does — only while the recorded mtime still matches.

Refusal texts no longer embed the pre-PR "Warning: … Consider re-reading"
copy inside "Refusing to overwrite", and every refusal names a recovery the
model can perform: read the remaining pages, or use patch.
2026-09-15 03:57:24 -07:00
DanSpicyTaco
a8af57fcd2 fix: block stale write_file overwrites
Require an explicit full-file baseline before replacing existing host-visible files with write_file, and fail closed when that baseline is stale. This prevents stale conversation context from clobbering manual or external edits.\n\nRefs #65604
2026-09-15 03:57:24 -07:00
ethernet
cc01ae9d44 merge: connector UI e2e v2 into bundled onboarding (rebuilt)
# Conflicts:
#	apps/desktop/electron/main.ts
#	apps/desktop/src/app/settings/local-models-settings.test.tsx
#	hermes_wisdom/agent_led/share_flow.py
#	plugins/memory/hindsight/__init__.py
#	scripts/lib/wisdom-demo-env.sh
#	scripts/release.py
#	tests/skills/test_collective_wisdom_install_skill.py
#	tools/checkpoint_manager.py
2026-09-11 21:10:19 -04:00
Teknium
adf23550f5 fix(tools): profile-scoped checkpoint/snapshot paths, tool caches, TZ and schema paths under multiplex
Under `gateway.multiplex_profiles` one gateway process serves every profile
under ~/.hermes/profiles/NAME/; each routed turn runs with a context-local
HERMES_HOME override while `os.environ` still holds the DEFAULT profile's
values. Anything evaluated once at import, or memoised in a single unkeyed
module slot, therefore freezes the LAUNCH profile's value and leaks it into
every other profile's turns. This lands the tools-side half of that class:

- tools/process_registry.py, tools/environments/{modal,singularity}.py:
  `_checkpoint_path()` / `_snapshot_store()` resolve `get_hermes_home()` at
  call time (same seam as `tools/skills_tool._skills_dir`, so the existing
  `monkeypatch.setattr(CHECKPOINT_PATH)` test sites keep working). Completes
  the checkpoint_manager / sticker_cache half cherry-picked from #56315.
- plugins/platforms/feishu/feishu_comment_rules.py: `_MtimeCache` is now
  path-keyed (accepts a Path or a zero-arg resolver, one (mtime, data) slot
  per resolved path) with `invalidate()`; `_rules_file()` / `_pairing_file()`
  resolve the routed profile's files. Proposed in #63962.
- tools/tool_output_limits.py, tools/browser_tool.py, tools/browser_camofox.py:
  the process-lifetime config caches are dicts keyed by `hermes_home_key()`;
  the `_X_resolved` flags and the lifecycle reset keep their shape.
  tools/file_tools.py drops its private `file_read_max_chars` memo and reads
  the already mtime+path-cached `load_config_readonly()`.
- hermes_time.py: `get_timezone_name()`; when `is_multiplex_active()` the
  env `HERMES_TIMEZONE` (bridged from the default profile's config at gateway
  startup) is ignored in favour of the routed profile's config.yaml. Both
  sandbox TZ sites (code_execution_env/_tool) now use it.
- tools/cronjob_tools.py, tools/tts_tool.py, tools/skill_manager_tool.py:
  the static schema text is profile-neutral and `dynamic_schema_overrides=`
  rebuilds the `display_hermes_home()` / create-dir hint per
  `get_definitions()`, so a routed profile's model sees its own paths.

Refs #95685.

Co-authored-by: Nathan Shan <nathanielcrush51@gmail.com>
(cherry picked from commit 6d3fc6b07b3155c6196b1fd61a829283f1d7855c)
2026-09-11 15:44:00 -07:00
ethernet
ed568bd947 fix(runtime): route dependency hints through pm 2026-09-11 18:13:06 -04:00
Teknium
fdd140557b Merge pull request #103551 from NousResearch/fix/tool-output-caps
feat(file_tools): write_file flags a whole-file rewrite that mostly re-sends what is on disk (661 such writes, ≈$155, in one run)
2026-09-06 12:05:53 -07:00
liuhao1024
478e66475b fix(tools): keep truncated search_files output pure JSON
When results were truncated, search_files appended the pagination hint
as plain text after the serialized payload ("{...}\n\n[Hint: ...]"),
so the tool result was no longer parseable JSON — downstream tool-message
handling on providers strict about tool-content formatting could reject
or mishandle it, contributing to 400 upstream errors in sessions with
truncated search output (#90322).

Move the hint into the payload as a structured _hint field, matching the
existing _omitted/_warning side-channel convention in the same function.
The model-facing guidance (explicit next offset) is unchanged.

Fixes #90322
2026-09-07 00:20:11 +05:30
Teknium
3009efe75e fix(tools): read_file dispatch honors the advertised 2000-line default when the model omits limit
#76996 raised DEFAULT_READ_LIMIT, the read_file_tool signature and the schema
default from 500 to 2000, but the registry dispatch handler `_handle_read_file`
kept its own literal `args.get("limit", 500)`. Since models omit `limit` on most
reads, every dispatched read still stopped at 500 lines with `truncated: true`,
so the default flip never reached production traffic.

All three sites now read the single DEFAULT_READ_LIMIT constant, so the schema,
the Python default and the dispatch fallback cannot drift again.

Live probe (registry.dispatch on a 1500-line file, no limit): before 501 lines /
truncated=true; after 1501 lines / truncated=false.
2026-09-06 05:32:49 -07:00
Teknium
08a3c49812 fix(file_tools): the rewrite hint reads through the task's backend, skips non-regular and oversized files, and compares in linear time
Independent review: the hint read the HOST path even for remote/sandbox
backends (a different file), blocked forever on a host FIFO while holding
the write lock, and used SequenceMatcher, which is quadratic on repeated
lines (a 460 KB same-line file took ~22 s). It now reads the old content via
the task's file ops read_file_raw (regular files only, correct backend),
skips files above 400 K chars, and compares line multisets (Counter
intersection, linear).

Reviewer's own probe on this head: 460 KB repeated-line file 22.3 s -> 0.0 s
(skipped), 230 KB 46 ms, remote backend gets no host-derived hint, FIFO write
returns at once.
2026-09-05 06:32:30 -07:00
RelaxJonh
c851ef4d0f fix(tools): file-tools env creation uses _is_container_backend() for the cwd guard (#101013)
_create_terminal_env_for_file_ops tested the static _CONTAINER_BACKENDS set while the
terminal tool's sibling sites already go through _is_container_backend(), which also
honours plugin backends declaring is_container. A host cwd override on such a plugin
backend therefore reached the sandbox as its workdir from the file-tools path only.

Salvaged from #101389 by @RelaxJonh.
2026-09-05 18:03:05 +05:30
Teknium
a1ffb27ef8 feat(file_tools): write_file says when it just re-sent a large file that was already on disk
In the 1,393-agent refactor run write_file carried 92.8M chars of content
(23.2M output tokens, ~$580, 20% of all output tokens). 661 of those calls
were read -> whole-file rewrite of an existing file >20k chars (24.9M
chars, ~$155): 130k-char agent/model_metadata.py, 109k
model_setup_flows.py, 107k google_chat/adapter.py, re-sent in full to
change a few lines. Meanwhile patch succeeded 4,602/4,617 (99.7%) at 1.3k
chars per call. The tool result is the only place the model reads, so the
cost signal goes there.

When an overwrite targets an existing file and both old and new content
are >=20k chars, write_file diffs them by line (difflib, autojunk off)
and, if >=80% of lines are unchanged, adds a `hint` naming the unchanged
count, the changed count, the size, and "use patch". The write itself is
unchanged and still happens; new files, small files and genuine rewrites
get no hint. 300 ms on the 119k-char model_metadata.py.

Tests (2): a 40k-char file with 2 lines changed gets the hint and is
written; a new file, a small file and a wholly rewritten file get none.
2026-09-05 02:01:04 -07:00
Teknium
d63e380324 compat(plugins): warn once per name when a plugin resolves an old import path; lint step restored in CI
Every PLUGIN-COMPAT __getattr__ now calls hermes_cli.plugin_compat.warn_once(facade, name, target) before
resolving, emitting a HermesPluginCompatWarning (FutureWarning) once per process per name: old path, new
path, removal target. Importing a facade for its live API stays silent; only resolving a moved name warns.
COMPAT_MANIFEST.md documents the warning and how to silence it during migration.

Verified the runtime never routes through a pointer: every entry point (run_agent, cli, hermes_cli.main,
gateway.run, tui_gateway.server, web_server, model_tools + tool discovery, hermes_state, cron.scheduler,
browser_tool, mcp_tool, kanban, auth) imports clean and `hermes doctor` runs end to end with the warning
promoted to an error.

Also restores the check_compat_pointers CI step to .github/workflows/lint.yml, which a0be177aac dropped
when the compat layer was regenerated (the lint script itself was present; the workflow step was not).

hermes_cli/plugin_compat.py, tests/test_plugin_compat_warning.py and the two-line insert per facade are
part of the compat layer and go away with it.
2026-09-04 00:15:16 -07:00
Teknium
2776813df3 compat(plugins): temporary import-path shims for external plugins — ONE commit, revert on schedule
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:

    git revert <this sha>

removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.

What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
  so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
  tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
  relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)

Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
2026-09-03 17:13:22 -07:00
Teknium
eb8a30cc2f simplify(compat): skills_sync/skill_manager/kanban/computer_use — drop 37 re-exports/aliases, repoint 8 callers + 9 test files 2026-09-03 13:27:01 -07:00
Teknium
b818085298 simplify(compat): doctor/status — drop 13 re-exports + the doctor_* globals() facade (97 names), repoint 6 callers / 13 tests 2026-09-03 13:10:52 -07:00
Teknium
e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium
0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
Teknium
e4436ec9e5 refactor(tools): hand-compact docstrings across file_tools group (keep every WHY) 2026-09-02 23:17:52 -07:00
Teknium
48de33b77f refactor(tools): compact file_tools group docstrings; regex exact strategy; one approval tail 2026-09-02 23:10:54 -07:00
Teknium
af95ff9f9f refactor(tools): pack re-export import blocks in file_tools 2026-09-02 23:01:16 -07:00
Teknium
40f2c7951b refactor(tools): AST-neutral layout compaction of file_tools group 2026-09-02 22:51:55 -07:00
Teknium
b15ac86686 refactor(tools): file_tools shares write/patch plumbing and terminal env bring-up 2026-09-02 22:31:59 -07:00
Royalaid
413fd2a1fc feat(search): add fast discovery file ordering 2026-09-03 04:56:59 +05:30
RelaxJonh
6fdd93ab45 fix(tools): release per-task file registry state (#86514)
Clear file-operation trackers when a terminal task ends and remove per-path
locks after the last holder or waiter exits. This prevents long-lived gateway
processes from retaining file state indefinitely while preserving sibling-write
tracking and path-level serialization.

Fixes #86514
2026-09-03 04:34:45 +05:30
Teknium
aed2e156ad refactor(tools/files): finish file_operations wiring to extracted modules; restore WHY comments in file_tools; repoint test 2026-09-02 14:47:03 -07:00
Teknium
d4cec15b47 refactor(tools): first-wave simplification of tools/ (file ops split, lazy_deps, code_exec, approval, browser, delegate, mcp, skills, terminal, voice, media)
Behavior-neutral structural pass over tools/*: god-file extractions into
sibling modules (file_operations_common/lint/search, file_tools_paths/
read_tracking/write, code_execution_env/rpc, tool_search_catalog/names/
validation, tts_command_provider, ...), duplicate helper unification,
if/elif -> dispatch tables, dead-code removal, docstring compaction.
Tool schemas (get_tool_definitions) verified byte-identical to base.
2026-09-02 14:43:45 -07:00
Enzo Adami
02a6e95228 fix(files): allow one full read after compaction 2026-09-03 03:07:37 +05:30
webtecnica
4a0aae8aae fix(agent): preserve tool-output dedup state across context compaction (#84857) 2026-09-03 03:07:37 +05:30
muhifni
1cd736ff63 fix(terminal): scope terminal config per turn under profile multiplexing
A multiplexed Hermes process (gateway.multiplex_profiles, unified
dashboard/TUI, or cron) serves several profiles at once, but terminal.*
resolved through process-global TERMINAL_* env vars bridged ONCE at
startup from the launch profile (gateway/run.py ~2700-2760) plus the
one-shot _ensure_terminal_env_bridged() guard. Every routed profile
therefore inherited the launch profile's backend, cwd, docker volumes,
SSH target and shared-container key: a local profile ran inside another
profile's docker sandbox (or a docker profile escaped to the host), and a
container labeled profile A carried profile B's RW bind mounts.

Fix: an authoritative per-profile terminal policy seam, mirroring
agent/secret_scope.py:

- tools/terminal_scope.py: ContextVar holding the routed profile's
  COMPLETE effective TERMINAL_* policy (defined defaults <- profile .env
  TERMINAL_* <- config.yaml terminal:). While bound, terminal_env()
  resolves ONLY from it - an omitted key yields the defined default,
  never os.environ. Unreadable/malformed policy installs a refusal
  scope; terminal_tool / execute_code refuse instead of running under
  ambient launch-process policy (fail closed).
- Installed at every in-process profile boundary: gateway
  _profile_runtime_scope, tui_gateway session/build/turn scopes, cron
  per-job fire. The unscoped single-process path is byte-identical.
- Every terminal.* consumer reads through the scope: terminal_tool
  (_get_env_config, _resolve_container_task_id shared key, orphan
  reaper lifetime, degraded mode), gateway/platforms/base.py docker
  media translation (volumes, shared key, persistence), runtime_cwd /
  agent_init / skill_utils / code_execution_tool / file_tools cwd
  anchors, prompt_builder / browser_tool / env_probe backend checks,
  gateway footer, @-refs and slash-command cwd. env_probe resolves the
  backend in the caller's context, since the probe worker thread does
  not inherit the ContextVar.

Salvage of #99225 onto current main: adds the three ambient reads the PR
missed (tools/file_tools.py TERMINAL_CWD, tools/browser_tool.py and
tools/env_probe.py TERMINAL_ENV; shape from #79117) and trims the test
module to the leak matrix driven through the real gateway boundary,
omitted-key defaults, refusal, and boundary reset.

Fixes #68559
Fixes #94200
Fixes #101132
Fixes #95470

Co-authored-by: x7peeps <9640837+x7peeps@users.noreply.github.com>
Co-authored-by: Eva <239388517+100yenadmin@users.noreply.github.com>
Co-authored-by: ExitMaster <292490062+ExitMaster@users.noreply.github.com>
2026-09-02 05:34:28 -07:00
Teknium
1ee30352ca fix: background review can now read skills before patching — denial storm ended, cache parity intact (#61521, #39996)
The self-improvement review fork advertises the parent's full tool schema
(deliberate — tools[] must stay byte-identical for prompt-cache parity)
but denied everything except memory/skill tools at dispatch. Models
naturally reach for read_file to inspect a SKILL.md before patching, got
denied, then attempted a blind skill_manage patch which the
read-before-write guard correctly refused. One deployment logged ~142
denials + ~204 refusals over 2 days: the self-improvement loop ran
continuously but almost never landed a skill patch.

Fix is dispatch-side ONLY — zero request-body change, cache untouched:

- Whitelist read_file + search_files on the review fork (reads are
  side-effect-free). Write tools (write_file/patch/terminal) stay denied:
  autonomous maintenance must go through skill_manage's validation.
- read_file now registers full reads with the review fork's
  read-before-write guard (same as skill_view), so the natural
  read_file -> skill_manage(patch) sequence lands. Partial reads
  (offset>1 / truncated) don't count. No-op outside review forks.
- Self-correcting deny message: names skill_view/skill_manage/memory as
  substitutes so one denial redirects the model instead of a storm
  (the actionable half of #61521's proposal 2).

Rejects #39997's alternative (narrow the advertised schema on local
endpoints): local backends have KV/prefix caches too, and re-prefilling
a large snapshot is most expensive exactly there.

Live A/B (real dispatch path, isolated HERMES_HOME): on main,
read_file DENIED -> patch REFUSED (read-before-write); on this branch,
read_file OK -> patch LANDED. tools[] identical in both.
2026-08-29 18:33:41 -07:00