Desktop persisted a large paste under Electron's userData dir and attached
it as `@file:<abs path>`; the tui_gateway prompt path expands that reference
with `allowed_root=cwd`, so `_resolve_path` refused it with "path is outside
the allowed workspace" whenever the chat's cwd was not an ancestor of the
paste dir (always on Linux/macOS, and on Windows for any project cwd).
Electron now writes pastes under `<HERMES_HOME>/composer-pastes`, and
`agent/context_references.py::_resolve_path` admits exactly that anchored
directory (active profile home and global root, via file_safety._hermes_dirs)
as the one root besides `allowed_root`. A sibling directory that merely
contains the substring stays refused; the credential deny-list still runs
on the admitted path.
Supersedes the substring whitelist proposed in #117150.
Fixes#117149
@folder: targets resolve against allowed_root, which callers may widen
beyond cwd, but _build_folder_listing and _iter_visible_entries assumed
the resolved folder was under cwd: path.relative_to(cwd) raised
ValueError and the blanket except in _expand_reference surfaced it as a
confusing "not in the subpath of" warning instead of a listing.
The listing header now renders cwd-relative when possible, then
allowed_root-relative, else the absolute path. rg --files gets the
absolute folder path (rg echoes the arg as the output prefix, so the
lines parse correctly anywhere), the parent-dir walk drops only the
cwd stop-condition so in-cwd output is unchanged, and entry indentation
is computed relative to the target folder rather than cwd. The os.walk
fallback never assumed cwd.
Regression tests cover the widened-root target through the real
preprocess_context_references entry on both the rg and rg-blocked
paths, plus @file: parity and in-cwd display controls.
_run_quiet drained each pipe up to _MAX_QUIET_OUTPUT_BYTES and relied on
proc.kill() to make the returncode nonzero. A child that flushed past the
cap and exited 0 before the drain thread crossed it (a fast writer on a
loaded runner) was not killable, so the result came back returncode=0 with
truncated stdout: the caller's fallback path keys on the returncode and
treated the truncation as success. This is also why
test_run_quiet_caps_child_output failed on main's CI (`assert 0 != 0`)
while passing locally.
The drain now records that the cap was crossed and the result is forced to
returncode 137 (128 + SIGKILL) when the child exited 0, so the contract in
the docstring holds regardless of scheduling.
_next_line collected every readline(line_cap) piece until the newline, so a
one-line giant (minified JSON) was still fully materialized before the
total_chars > char_budget gate. Pass the remaining budget into the helper and
stop collecting as soon as it is exceeded; the caller returns the oversized
block and never needs the rest of the line.
@file:, @folder:, @diff, @staged, and @git: expansion materialized
unbounded amounts of data before any size gate ran:
- _is_binary_file called read_bytes() and sliced [:4096], loading the
whole file to sniff it.
- _expand_path_reference read_text()ed the whole file before the
max_inline_tokens check, so a refused file was fully loaded and
token-scanned anyway; ranged refs read the whole file to serve a
slice.
- _file_metadata read_text()ed every folder-listing entry (up to 200)
just to count lines.
- _run_quiet buffered the child's entire stdout, so git diff / rg
--files on a large tree loaded an unbounded stream.
- gather expanded every parsed reference in a message with no cap.
Remote-triggerable through the gateway: any inbound message can carry
@-references, so a single message could force GB-scale transient
allocations and full-file scans before the token gate refused them.
Bounds applied:
- Binary sniff reads a 4KB prefix via open().
- Whole-file refs stat() first; st_size > max_inline_tokens *
CHARS_PER_TOKEN is certainly oversized (the estimator is >= bytes/4
for every encoding mix) and refuses without reading.
- Ranged refs stream the requested window with readline() pieces
capped at the char budget; lines outside the window are skipped
without materializing, so single-line giants (minified JSON,
one-line logs) cannot expand a window read into a full read.
- Folder metadata streams the line count in 1MiB chunks and reports
byte size past 4MiB.
- _run_quiet drains pipes on threads up to a 4MiB ceiling and kills
the child on overflow; the nonzero returncode routes callers to
their existing fallback paths.
- At most 16 references expand per message; the rest get a warning.
Measured: a refused 21MB file cost 42.4MB peak traced allocation
before and ~3KB after; an 85MB mixed hostile message costs ~6KB.
Class sweep residual of the thread-boundary profile-scope fixes already on main (Nous account
fetches, reapers, periodic scheduler, async-delegation stale finalization): a sync wrapper
that detects a running loop and hops `asyncio.run(coro)` onto a fresh `ThreadPoolExecutor`
worker starts that worker with an EMPTY contextvars.Context, so the served profile's
HERMES_HOME override and secret scope are gone inside the coroutine.
- agent/context_references.py: the gateway/TUI turn path (running loop) ran the @file
credential guard against the LAUNCH profile's home, so a served profile's own
skills/.hub cache / credential stores were attachable. Security-relevant; covered by
an invariant test (red on base: the hub body was attached).
- tools/homeassistant_tool.py: HASS_URL / HASS_TOKEN are read via get_secret inside the
coroutine; under multiplexing the bare worker fails closed (UnscopedSecretError).
- tools/browser_cdp_tool.py, tui_gateway/methods_complete.py: same hop, same loss.
Co-authored-by: NanPan <111261006+poijygfdyy@users.noreply.github.com>
Three paths still resolved the raw model/remote-supplied string before the
guard could refuse it, so on Windows the NTLM-leak trigger (resolving the
path) ran anyway: the file-checkpoint helper stats write_file/patch targets
before the tool executes; the ACP file bridge resolves fs/read_text_file and
fs/write_text_file paths before its read/write denylists; and @file:/@folder:
references resolve their target before the reference allow-check. Each now
checks the raw string first and refuses. The GLOBALROOT form now requires
its path separator so a GLOBALROOT-prefixed local name is not misclassified.
The rationale comment names the vector instead of another product's
changelog, and the security docs say the row is enforced on reads as well
as writes, since it sits under the write-guard table.
- Check text tokens BEFORE building the fenced block: an oversized file no
longer allocates a second MB-scale string just to discard it, and the token
estimate runs once instead of twice.
- Drop the companion warning: the fallback block alone carries the message,
so the model no longer reads "too large to inline safely" twice (Context
Warnings + Attached Context).
- Extract _on_disk_reference_block + _agent_visible_path shared by the binary
and oversized blocks (stat/format_bytes/OSError fallback, container path
translation, nudge text were copy-pasted).
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
Hermes gathers workspace context by running git against the session
directory automatically — the coding-workspace snapshot, gateway
project-tree build, /diff, @diff|@staged context refs, goal-gate
fingerprint, and -w startup worktree add — before any prompt, tool call,
approval, or trust gate. Those probes ran the system git without
stripping the repository's own config, so a repo delivered as files with
its .git directory intact (a shared zip, sync folder, or USB stick;
git clone never transfers .git/config) could set an execution-sink git
setting and get arbitrary host code execution as the user with nothing
on screen.
- core.fsmonitor / core.hooksPath / pager / editor / credential helper:
neutralized by routing every automatic probe through
noninteractive_git_env(), which pins those keys to inert values via
GIT_CONFIG_* and ignores global/system config. bounded_git_probe (the
reported sink, coding_context._git + tui_gateway.git_probe) now defaults
to that env; worktree-add, working_diff, web_git, context_references,
goals, and subagent_worktree route through it too.
- Attribute-scoped [diff "x"] command=/textconv= drivers: the attacker
names the driver in .gitattributes, so GIT_CONFIG_KEY overrides can't
enumerate them. Added harden_git_argv(), which inserts
--no-ext-diff --no-textconv on diff-rendering subcommands (diff/show/
log/blame) only — status et al reject the flags. Both flags required
(verified empirically; each alone leaves the other live).
Builds on the noninteractive_git_env config-scrubbing from the
gemini-cli #28792 port. Real-git E2E regression suite arms a malicious
repo and asserts every automatic path neutralizes fsmonitor, hooks,
external-diff, and textconv; a baseline test proves the repo is armed.
Closes#26193
Adds ContextReferenceProvider ABC so plugins can register custom
@-prefixes (e.g. @issue:ENG-123) with autocomplete and expansion.
Plugin output flows through existing token-limit guards. Zero
breaking changes.
- Migrate the missed 6th inline formatter (update_cmd.py backup-size
display) to the shared helper.
- checkpoints._fmt_bytes: plain alias instead of a None-guard wrapper —
every caller feeds ints from checkpoint_manager (all size fields
initialize to 0), so the None path was dead defensive code.
- Drop the fallback= kwarg (zero production callers; '?' default is the
real inherited contract and stays).
- curator_backup + context_references: call format_bytes directly (single
internal call site each, zero external importers — alias was churn
avoidance with nothing to avoid). backup/_format_size and
doctor/_human_bytes keep their aliases (claw.py + tests pin the former;
three call sites use the latter).
- Reshape the loop so the trailing TB return is reachable (no dead line).
- Tests: replace alias-identity assertions (ossified the delegation
mechanism) with behavior-contract equality over a value sweep;
mutation-checked red-green.
- update_cmd parity: byte-identical B-GB vs the old inline loop; gains
the TB tier.
Five modules each carried a private near-identical human-readable byte
formatter (backup._format_size, checkpoints._fmt_bytes,
doctor._human_bytes, context_references._human_bytes,
curator_backup.format_size). Three of them silently topped out at GB and
rendered a 1 TiB value as '1024.0 GB'. All five now alias one shared
format_bytes in hermes_cli/sizefmt.py (sibling of timefmt.py, same
zero-dependency rationale), keeping each module's established local name
so no caller churns.
Deliberately NOT migrated (behavior differs on purpose):
- session_recovery._format_bytes: binary suffixes (KiB/MiB/GiB)
- qqbot chunked_upload.format_size: '100.0 B' one-decimal style, pinned
by its protocol tests
Net -33 production LOC before the new module; parity verified over a
16-value corpus against all five verbatim originals (only divergence:
the TB tier fix). Contract tests mutation-checked red-green.
Expanding an @file:/@folder: reference deleted the token from the message
it was typed in, leaving a hole in the sentence and no anchor for clients
to chip. The attached context block still names each ref, so nothing is
lost by leaving the token where the user put it.
The unquoted alternative in the directive pattern is `\S+`, so a ref built
by string interpolation truncates at the first space and strands the tail
as loose text next to a broken thumbnail. Composer images live in the app's
userData dir, which on macOS is `~/Library/Application Support/<App>/` — so
every pasted or dropped image hit this.
Adds format_reference_value next to REFERENCE_PATTERN, mirroring
formatRefValue in the desktop's directive-text.tsx, and covers the
round-trip through the parser.
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.
Fixes#53428 (master tracker for Windows GBK locale crash).
Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
`@file` / `@folder` context-reference expansion enforced its own narrow
deny-list (`_ensure_reference_path_allowed` in `agent/context_references.py`)
that only covered `~/.ssh` keys, a handful of shell dotfiles, `~/.hermes/.env`,
and `skills/.hub`. It never blocked the credential stores that the canonical
read guard (`agent/file_safety.get_read_block_error`) protects: provider API
keys (`~/.hermes/auth.json`), Anthropic OAuth tokens
(`~/.hermes/.anthropic_oauth.json`), MCP OAuth material (`~/.hermes/mcp-tokens/`),
webhook HMAC secrets, and project-local `.env` files.
This matters because the messaging gateway feeds **untrusted** remote text
straight into reference expansion: `gateway/run.py` calls
`preprocess_context_references_async(..., allowed_root=_msg_cwd)` where
`_msg_cwd` defaults to the operator's HOME when `TERMINAL_CWD` is unset. A chat
peer (Telegram/Discord/Slack/...) could send `@file:~/.hermes/auth.json`, pass
the `allowed_root` check (it resolves under HOME), slip past the narrow list,
and have the operator's live keys read into the agent's context — where the
model would typically echo or act on them.
Rather than duplicate and re-sync a second secret list, this routes the guard
through the existing single source of truth. A reviewer might ask "why not just
add `auth.json` to the local list?" — because the local list has already drifted
once (a prior commit had to add `.config/gh`); anchoring to
`get_read_block_error` means every future addition there protects this path too.
The narrow checks are kept as a fallback since they also cover dirs that guard
does not (`.aws`, `.gnupg`, `.kube`, etc.), and the canonical lookup is wrapped
so it can never crash reference expansion.
N/A
- [x] 🔒 Security fix
- `agent/context_references.py`: `_ensure_reference_path_allowed` now also
consults `agent.file_safety.get_read_block_error` after its existing checks
and refuses the reference when that canonical guard flags the resolved path.
The lookup is wrapped so guard-resolution failures fall back to the explicit
checks instead of breaking expansion.
- `tests/agent/test_context_references.py`: added
`test_blocks_canonical_read_denylist_credential_stores`, asserting that
`@file` attaches for `auth.json`, `.anthropic_oauth.json`, `mcp-tokens/*`, and
a project-local `.env` are all refused and their secret bodies never reach the
expanded message.
- `scripts/release.py`: added the contributor email to `AUTHOR_MAP` (release
gate).
1. `scripts/run_tests.sh tests/agent/test_context_references.py` — all 15 tests
pass, including the new credential-store case.
2. Regression proof: stash `agent/context_references.py`, run the suite with
`-- -k canonical`, and confirm the new test fails (secrets leak into the
message) without the fix; restore and confirm it passes.
3. `ruff check agent/context_references.py tests/agent/test_context_references.py`
and `python scripts/check-windows-footguns.py agent/context_references.py
tests/agent/test_context_references.py` both pass.
- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix (plus the AUTHOR_MAP release gate)
- [x] I've run the test suite for the touched area and all tests pass
- [x] I've added tests for my changes (required for bug fixes)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)
- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — or N/A
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
Multiple @-references in one message (esp. @url: refs, each a full
web_extract round-trip) were expanded in a serial `for ref in refs: await`
loop. Switch to asyncio.gather over the independent _expand_reference calls,
reassembling warnings/blocks in original positional order so output is
byte-identical to the serial path; the token-budget check is unchanged.
Generic + provider-agnostic: helps every web backend equally (exa/tavily/
firecrawl/parallel) since it's above the provider layer. RED/GREEN test:
3 url refs @ 0.2s each = 0.60s serial -> ~0.20s concurrent.
* feat(web_extract): truncate-and-store instead of LLM summarization
web_extract no longer runs an auxiliary LLM over scraped pages. The extract
backends (Firecrawl/Tavily/Exa/Parallel) already return clean, boilerplate-
stripped markdown, so we return it directly: pages within a char budget
(default 15000, web.extract_char_limit) come back whole; larger pages get a
head+tail window plus an explicit footer giving the stored full-text path and
the read_file call to page through the omitted middle. The full clean text is
written to cache/web (mounted read-only into remote backends like the other
cache dirs), so nothing is lost.
Inline base64 images are converted to [IMAGE: alt] placeholders (token bombs
dropped) while real http(s) image URLs are preserved as links so the agent can
still web_extract/vision_analyze them.
Removes process_content_with_llm + the chunked summarizer + check_auxiliary_model
+ _resolve_web_extract_auxiliary. context_references._default_url_fetcher is
updated to the truncate path and its stale data.documents shape read is fixed
to results (it was silently returning empty).
Live before/after eval (firecrawl, 4 URLs): 11.7x faster overall (176.6s ->
15.1s); 10-60x on large pages. Quality identical; findability 4/4 (answer
recoverable from stored full text on every truncated page). web_search is
unchanged.
No own scraper added; no changes to web_search.
* fix(web_extract): add char_limit to execute_code web_extract stub
The new web_extract char_limit param must appear in the code_execution_tool
_TOOL_STUBS signature (and doc line) or test_stubs_cover_all_schema_params
fails — the stub schema must cover every real schema param.
The Windows desktop GUI runs its backend headless via pythonw.exe. Several
auxiliary subprocess sites that run inside that windowless backend spawned
console-subsystem children (git, gh, wmic, powershell, bash, rg, taskkill)
WITHOUT CREATE_NO_WINDOW, so Windows allocated a fresh conhost per call and
flashed a black window on screen — sometimes continuously (the dashboard
Projects-tree git probe alone fired ~118 spawns in 60s on startup).
The terminal tool, cron, browser, code_execution, and gateway-spawn paths
already carry windows_hide_flags(); these auxiliary probe/scan/launcher legs
were missed. Wire the existing helper into them:
- tui_gateway/git_probe.py: run_git (+ encoding=utf-8/errors=replace, fixes the
cp950 UnicodeDecodeError on CJK paths from the same site)
- agent/coding_context.py: _git (per-turn git status/log/diff)
- agent/context_references.py: _run_git + _rg_files (@file/@ref resolution)
- hermes_cli/copilot_auth.py: gh auth token probe (auxiliary provider:auto)
- hermes_cli/gateway.py: wmic + PowerShell Get-CimInstance PID scan
- hermes_cli/main.py: wmic stale-dashboard PID scan
- gateway/status.py: taskkill /T /F force-kill
windows_hide_flags() returns 0 on POSIX, so every changed call is a no-op on
Linux/macOS (verified: real git/rg probes still work; Windows-simulated calls
all pass creationflags=CREATE_NO_WINDOW).
Scoped to the windowless-backend paths that cause the reported flashing. The
Electron updater-handoff leg (main.cjs windowsHide:false) and the
interactive-CLI banner probes (cli.py) are intentionally NOT touched here —
the former needs a Windows-tested change of its own, the latter runs in a
visible console anyway.
Tracking: #54220
Refs: #53178#53631#53781#53957#49602#52982#53424#53053#53016
Follow-up to #53791 addressing review feedback: the footgun checker treated
capture_output=/stdout=/stderr=/check_output as proof a subprocess can't pop a
Windows console. That invariant is false — stream redirection controls where a
child's output goes, not whether a console is allocated. From a console-less
parent (Desktop/Electron, pythonw.exe, detached gateway/cron) a console-subsystem
child still flashes a window even when fully captured.
- check-windows-footguns.py: capture/redirect/check_output is no longer a blanket
safe-pass. Added _WINDOWS_FLASHING_PROGRAMS (git/gh/npm/node/python/uv/ffmpeg/
docker/powershell/…); calls to those are flagged even when captured. Non-flashing
programs keep the capture exemption (no 271-site noise). _subprocess_compat.run/
popen calls are inherently safe (wrapper injects CREATE_NO_WINDOW).
- Routed the 35 genuine flashing git/gh/npm/uv/ffmpeg/docker spawns through the
_subprocess_compat.run/popen chokepoint (Brooklyn's wrapper from #53810) — the
durable fix, not per-site annotations. cmd.exe /c start stays # ok (intentional).
- Updated tests + CONTRIBUTING.md rule #17 to the corrected invariant.
A binary @file: ref (PDF, docx, spreadsheet, …) expanded to a bare
"binary files are not supported" warning with no content. The model saw a
failure and gave up — e.g. a dropped PDF came back as a text note claiming the
type was unsupported, even though the file was staged on disk right next to it.
Inject an actionable content block instead: the path, mime type, size, and a
nudge to use its tools to read/convert/view the file (and explicitly not to tell
the user the type is unsupported). General across every binary type — not
PDF-specific. The file already resolves where the agent's tools run (local cwd
or the staged copy in a remote session workspace), so it can act on it directly.
When Hermes runs in TUI mode, the gateway child process communicates with
the Node.js parent over a JSON-RPC protocol on stdin. Subprocess calls that
inherit this stdin fd can trigger a race condition where the child's stdin
read returns EOF, causing the gateway to exit cleanly (exit code 0) mid-tool-
execution.
This is the same root cause as issue #14036 (byterover plugin) and PR #39257
(SSH environment backend). This commit applies the fix — stdin=subprocess.DEVNULL
— to all 85 subprocess.run() and subprocess.Popen() calls that execute inside
the TUI gateway child process.
Scope: TUI-context code only (agent/, tools/, plugins/, tui_gateway/server.py).
CLI code (cli.py, hermes_cli/), tests, scripts, and gateway process management
are excluded — they don't run inside the TUI child and inherit the terminal's
stdin, not the JSON-RPC pipe.
85 call sites across 28 files. All files pass syntax check.
16 callsites across 14 files were re-deriving the hermes home path
via os.environ.get('HERMES_HOME', ...) instead of using the canonical
get_hermes_home() from hermes_constants. This breaks profiles — each
profile has its own HERMES_HOME, and the inline fallback defaults to
~/.hermes regardless.
Fixed by importing and calling get_hermes_home() at each site. For
files already inside the hermes process (agent/, hermes_cli/, tools/,
gateway/, plugins/), this is always safe. Files that run outside the
process context (mcp_serve.py, mcp_oauth.py) already had correct
try/except ImportError fallbacks and were left alone.
Skipped: hermes_constants.py (IS the implementation), env_loader.py
(bootstrap), profiles.py (intentionally manipulates the env var),
standalone scripts (optional-skills/, skills/), and tests.
Follow-up to PR #4305 — .config/gh was added to the write-deny list
but missed from _SENSITIVE_HOME_DIRS, leaving GitHub CLI OAuth tokens
exposed via @file:~/.config/gh/hosts.yml context injection.
_expand_git_reference() and _rg_files() called subprocess.run()
without a timeout. On a large repository, @diff, @staged, or
@git:N references could hang the agent indefinitely while git
or ripgrep processes slow output.
- Add timeout=30 to git subprocess in _expand_git_reference()
with a user-friendly error message on TimeoutExpired
- Add timeout=10 to rg subprocess in _rg_files() returning
None on timeout (falls back to os.walk folder listing)
Co-authored-by: memosr.eth <96793918+memosr@users.noreply.github.com>
Add @file:path, @folder:dir, @diff, @staged, @git:N, and @url:
references that expand inline before the message reaches the LLM.
Supports line ranges (@file:main.py:10-50), token budget enforcement
(soft warn at 25%, hard block at 50%), and path sandboxing for gateway.
Core module from PR #2090 by @kshitijk4poor. CLI and gateway wiring
rewritten against current main. Fixed asyncio.run() crash when called
from inside a running event loop (gateway).
Closes#682.
