Commit Graph

45 Commits

Author SHA1 Message Date
Benjamin PERRY
af595ff8a6 fix(files): preserve SQLite locks during previews
Co-Authored-By: Hermes Agent / OpenAI Codex / gpt-6-sol <noreply@agents.invalid>
(cherry picked from commit 60d879c125177cd4cacdbc840f16c2e048b4199c)
2026-09-27 20:45:56 +05:30
finn763
a164569429 fix(agent): admit and sync gateway-staged attachments on remote execution backends (#110174)
Desktop paste/file attachments land in Hermes-managed staging dirs on the
GATEWAY (composer-pastes/ for large text pastes, attachments/ for dropped
files), but on the Remote SSH topology the workspace root (TERMINAL_CWD) is a
path on the SSH HOST - the two filesystems are fully disjoint, as the issue
thread confirms. Two gaps combined to reject every staged attachment with
"path is outside the allowed workspace":

- _resolve_path admitted only allowed_root + composer-paste roots, so a
  gateway-staged attachments/ path was refused outright. Admit the
  _CACHE_DIRS staging roots (attachments/, images/, cache/*, composer-pastes/)
  via a helper that asks get_cache_directory_mounts - the gateway's OWN
  payload is never a workspace escape, and the path-traversal and
  credential-deny guards in _ensure_reference_path_allowed still run after.
  Anything else outside the workspace stays blocked.
- composer-pastes/ was missing from _CACHE_DIRS, so its bytes never reached
  the remote: ssh/daytona/vercel_sandbox sync via iter_sync_files ->
  iter_cache_files, and to_agent_visible_cache_path only translates mounted
  dirs - a paste attached on a fresh session dangled on the remote host.

Tests cover the disjoint-filesystem SSH topology end-to-end (text inlines,
binary renders the synced ~/.hermes path), the still-refused stranger path,
local-backend unchanged, and the composer-pastes mount+sync enumeration.

Consolidates PR #110387 by Finn763 (the _agent_staged_path guard widening and
the SSH-topology tests, adapted to the current _ensure_reference_path_allowed
ordering) with PR #103412 by ericmaddox (whose mapping insight is subsumed by
the _CACHE_DIRS entry, which fixes both the sync and the translation).

Co-authored-by: ericmaddox <ericmaddox@users.noreply.github.com>
2026-09-26 18:00:17 -05:00
ethernet
9f2ba1b74d merge origin/main (779 commits) into ethie/pm-clean
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).

Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.

uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
2026-09-21 00:58:39 -04:00
teknium1
60dd9778c6 fix(desktop): large text pastes attach from HERMES_HOME/composer-pastes when the chat cwd is elsewhere
Desktop persisted a large paste under Electron's userData dir and attached
it as `@file:<abs path>`; the tui_gateway prompt path expands that reference
with `allowed_root=cwd`, so `_resolve_path` refused it with "path is outside
the allowed workspace" whenever the chat's cwd was not an ancestor of the
paste dir (always on Linux/macOS, and on Windows for any project cwd).

Electron now writes pastes under `<HERMES_HOME>/composer-pastes`, and
`agent/context_references.py::_resolve_path` admits exactly that anchored
directory (active profile home and global root, via file_safety._hermes_dirs)
as the one root besides `allowed_root`. A sibling directory that merely
contains the substring stays refused; the credential deny-list still runs
on the admitted path.

Supersedes the substring whitelist proposed in #117150.

Fixes #117149
2026-09-20 19:45:30 -07:00
beardthelion
275002d7a4 fix(context_references): @folder: listing works outside cwd under a widened allowed_root
@folder: targets resolve against allowed_root, which callers may widen
beyond cwd, but _build_folder_listing and _iter_visible_entries assumed
the resolved folder was under cwd: path.relative_to(cwd) raised
ValueError and the blanket except in _expand_reference surfaced it as a
confusing "not in the subpath of" warning instead of a listing.

The listing header now renders cwd-relative when possible, then
allowed_root-relative, else the absolute path. rg --files gets the
absolute folder path (rg echoes the arg as the output prefix, so the
lines parse correctly anywhere), the parent-dir walk drops only the
cwd stop-condition so in-cwd output is unchanged, and entry indentation
is computed relative to the target folder rather than cwd. The os.walk
fallback never assumed cwd.

Regression tests cover the widened-root target through the real
preprocess_context_references entry on both the rg and rg-blocked
paths, plus @file: parity and in-cwd display controls.
2026-09-20 15:49:22 -07:00
teknium1
3e579ee7af fix: capped @-reference child output always reports a nonzero returncode
_run_quiet drained each pipe up to _MAX_QUIET_OUTPUT_BYTES and relied on
proc.kill() to make the returncode nonzero. A child that flushed past the
cap and exited 0 before the drain thread crossed it (a fast writer on a
loaded runner) was not killable, so the result came back returncode=0 with
truncated stdout: the caller's fallback path keys on the returncode and
treated the truncation as success. This is also why
test_run_quiet_caps_child_output failed on main's CI (`assert 0 != 0`)
while passing locally.

The drain now records that the cap was crossed and the result is forced to
returncode 137 (128 + SIGKILL) when the child exited 0, so the contract in
the docstring holds regardless of scheduling.
2026-09-20 13:57:43 -07:00
teknium1
6d1341a13b fix: ranged @file read bails at the char budget mid-line (review follow-up)
_next_line collected every readline(line_cap) piece until the newline, so a
one-line giant (minified JSON) was still fully materialized before the
total_chars > char_budget gate. Pass the remaining budget into the helper and
stop collecting as soon as it is exceeded; the caller returns the oversized
block and never needs the rest of the line.
2026-09-20 11:54:43 -07:00
beardthelion
0accda1f76 fix(agent): bound file/process reads in @ context reference expansion
@file:, @folder:, @diff, @staged, and @git: expansion materialized
unbounded amounts of data before any size gate ran:

- _is_binary_file called read_bytes() and sliced [:4096], loading the
  whole file to sniff it.
- _expand_path_reference read_text()ed the whole file before the
  max_inline_tokens check, so a refused file was fully loaded and
  token-scanned anyway; ranged refs read the whole file to serve a
  slice.
- _file_metadata read_text()ed every folder-listing entry (up to 200)
  just to count lines.
- _run_quiet buffered the child's entire stdout, so git diff / rg
  --files on a large tree loaded an unbounded stream.
- gather expanded every parsed reference in a message with no cap.

Remote-triggerable through the gateway: any inbound message can carry
@-references, so a single message could force GB-scale transient
allocations and full-file scans before the token gate refused them.

Bounds applied:

- Binary sniff reads a 4KB prefix via open().
- Whole-file refs stat() first; st_size > max_inline_tokens *
  CHARS_PER_TOKEN is certainly oversized (the estimator is >= bytes/4
  for every encoding mix) and refuses without reading.
- Ranged refs stream the requested window with readline() pieces
  capped at the char budget; lines outside the window are skipped
  without materializing, so single-line giants (minified JSON,
  one-line logs) cannot expand a window read into a full read.
- Folder metadata streams the line count in 1MiB chunks and reports
  byte size past 4MiB.
- _run_quiet drains pipes on threads up to a 4MiB ceiling and kills
  the child on overflow; the nonzero returncode routes callers to
  their existing fallback paths.
- At most 16 references expand per message; the rest get a warning.

Measured: a refused 21MB file cost 42.4MB peak traced allocation
before and ~3KB after; an 85MB mixed hostile message costs ~6KB.
2026-09-20 11:54:43 -07:00
ethernet
b4a294fff9 Merge origin/main; keep PM as plugin dependency owner
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
2026-09-17 13:52:05 -04:00
teknium1
3baf4c2cb3 fix(profiles): side-thread asyncio.run workers keep the caller's profile scope
Class sweep residual of the thread-boundary profile-scope fixes already on main (Nous account
fetches, reapers, periodic scheduler, async-delegation stale finalization): a sync wrapper
that detects a running loop and hops `asyncio.run(coro)` onto a fresh `ThreadPoolExecutor`
worker starts that worker with an EMPTY contextvars.Context, so the served profile's
HERMES_HOME override and secret scope are gone inside the coroutine.

- agent/context_references.py: the gateway/TUI turn path (running loop) ran the @file
  credential guard against the LAUNCH profile's home, so a served profile's own
  skills/.hub cache / credential stores were attachable. Security-relevant; covered by
  an invariant test (red on base: the hub body was attached).
- tools/homeassistant_tool.py: HASS_URL / HASS_TOKEN are read via get_secret inside the
  coroutine; under multiplexing the bare worker fails closed (UnscopedSecretError).
- tools/browser_cdp_tool.py, tui_gateway/methods_complete.py: same hop, same loss.

Co-authored-by: NanPan <111261006+poijygfdyy@users.noreply.github.com>
2026-09-16 16:59:03 -07:00
teknium1
bc3df8a4d5 fix: NT-namespace guard fires before every sibling resolve (checkpoint, ACP bridge, @file:)
Three paths still resolved the raw model/remote-supplied string before the
guard could refuse it, so on Windows the NTLM-leak trigger (resolving the
path) ran anyway: the file-checkpoint helper stats write_file/patch targets
before the tool executes; the ACP file bridge resolves fs/read_text_file and
fs/write_text_file paths before its read/write denylists; and @file:/@folder:
references resolve their target before the reference allow-check. Each now
checks the raw string first and refuses. The GLOBALROOT form now requires
its path separator so a GLOBALROOT-prefixed local name is not misclassified.

The rationale comment names the vector instead of another product's
changelog, and the security docs say the row is enforced on reads as well
as writes, since it sits under the write-guard table.
2026-09-15 04:17:31 -07:00
ethernet
ce49cdbc59 merge: reconcile upstream main with pm audit closeout
Merge upstream 5e645791ac.

Retain the PM feature-flag owner and add upstream connection options.
Use the deny-only window-open policy while trusted external links keep
the existing IPC path. Keep both session-import and external-link copy.
Preserve captured timeout output when adding terminal yield handoff.
Quickstart tests patch the explicit upstream model-assignment owner.
Migrate incoming legacy OS markers to the branch's platforms gate.

Desktop renderer and Electron typechecks passed. Targeted Electron tests
passed (42 tests), Python conflict checks passed (26 tests, 3 skips),
and the plugin-compat import checker passed. CI owns the broad merge gate.
2026-09-06 13:17:07 -04:00
kshitijk4poor
07b161e241 refactor(context): fold simplify findings into oversized-file fallback
- Check text tokens BEFORE building the fenced block: an oversized file no
  longer allocates a second MB-scale string just to discard it, and the token
  estimate runs once instead of twice.
- Drop the companion warning: the fallback block alone carries the message,
  so the model no longer reads "too large to inline safely" twice (Context
  Warnings + Attached Context).
- Extract _on_disk_reference_block + _agent_visible_path shared by the binary
  and oversized blocks (stat/format_bytes/OSError fallback, container path
  translation, nudge text were copy-pasted).
2026-09-05 16:44:23 +05:30
Gille
cfa1a90b8c fix(context): fall back for oversized file refs 2026-09-05 16:44:23 +05:30
ethernet
e8fcb007b9 Merge remote-tracking branch 'upstream/main' into ethie/pm-clean
# Conflicts:
#	AGENTS.md
#	acp_adapter/edit_approval.py
#	acp_adapter/server.py
#	agent/agent_init.py
#	agent/anthropic_adapter.py
#	agent/anthropic_credentials.py
#	agent/auxiliary_client.py
#	agent/azure_identity_adapter.py
#	agent/bedrock_adapter.py
#	agent/browser_registry.py
#	agent/chat_completion_helpers.py
#	agent/coding_context.py
#	agent/context_references.py
#	agent/conversation_loop.py
#	agent/copilot_acp_client.py
#	agent/credits_tracker.py
#	agent/curator.py
#	agent/curator_backup.py
#	agent/deadline.py
#	agent/display.py
#	agent/errors.py
#	agent/estop.py
#	agent/i18n.py
#	agent/image_gen_registry.py
#	agent/image_routing.py
#	agent/learning_graph.py
#	agent/learning_mutations.py
#	agent/lsp/servers.py
#	agent/model_metadata.py
#	agent/models_dev.py
#	agent/monitoring/gateway_health_export.py
#	agent/monitoring/otlp_exporter.py
#	agent/pet/store.py
#	agent/process_bootstrap.py
#	agent/prompt_builder.py
#	agent/proxy_sources/iron_proxy.py
#	agent/secret_sources/_cache.py
#	agent/secret_sources/bitwarden.py
#	agent/secret_sources/registry.py
#	agent/shell_hooks.py
#	agent/skill_bundles.py
#	agent/skill_commands.py
#	agent/skill_utils.py
#	agent/ssl_guard.py
#	agent/ssl_verify.py
#	agent/system_prompt.py
#	agent/terminal_env_registry.py
#	agent/trace_upload.py
#	agent/transcription_registry.py
#	agent/tts_registry.py
#	agent/verify/environment.py
#	agent/vertex_adapter.py
#	agent/video_gen_registry.py
#	agent/web_search_registry.py
#	cli.py
#	cron/jobs.py
#	cron/scheduler.py
#	gateway/agent_cache_pressure.py
#	gateway/cgroup_cleanup.py
#	gateway/channel_directory.py
#	gateway/config.py
#	gateway/control_socket.py
#	gateway/dead_targets.py
#	gateway/drain_control.py
#	gateway/hooks.py
#	gateway/kanban_watchers.py
#	gateway/lifecycle_ledger.py
#	gateway/mirror.py
#	gateway/pairing.py
#	gateway/platform_registry.py
#	gateway/platforms/helpers.py
#	gateway/platforms/weixin.py
#	gateway/readiness.py
#	gateway/restart_loop_guard.py
#	gateway/rich_sent_store.py
#	gateway/run.py
#	gateway/session.py
#	gateway/shutdown_flush.py
#	gateway/shutdown_forensics.py
#	gateway/slash_commands.py
#	gateway/status.py
#	gateway/sticker_cache.py
#	gateway/whatsapp_identity.py
#	hermes_bootstrap.py
#	hermes_cli/_early_recovery.py
#	hermes_cli/_install_repair.py
#	hermes_cli/_startup_fast.py
#	hermes_cli/_subprocess_compat.py
#	hermes_cli/agent_plugins.py
#	hermes_cli/auth.py
#	hermes_cli/backup.py
#	hermes_cli/banner.py
#	hermes_cli/browser_connect.py
#	hermes_cli/build_info.py
#	hermes_cli/cli_agent_setup_mixin.py
#	hermes_cli/cli_commands_mixin.py
#	hermes_cli/codex_models.py
#	hermes_cli/config.py
#	hermes_cli/config_defaults.py
#	hermes_cli/config_migrations.py
#	hermes_cli/container_boot.py
#	hermes_cli/dashboard_auth/registry.py
#	hermes_cli/debug.py
#	hermes_cli/dep_ensure.py
#	hermes_cli/doctor.py
#	hermes_cli/doctor_live.py
#	hermes_cli/dump.py
#	hermes_cli/env_loader.py
#	hermes_cli/foreign_sessions.py
#	hermes_cli/gateway.py
#	hermes_cli/gateway_windows.py
#	hermes_cli/gui_uninstall.py
#	hermes_cli/image_provenance.py
#	hermes_cli/install_identity.py
#	hermes_cli/kanban.py
#	hermes_cli/kanban_db.py
#	hermes_cli/linux_desktop_entry.py
#	hermes_cli/local_runtime/binaries.py
#	hermes_cli/local_runtime/endpoint.py
#	hermes_cli/local_runtime/growth.py
#	hermes_cli/local_runtime/supervisor.py
#	hermes_cli/logs.py
#	hermes_cli/macos_tcc_anchor.py
#	hermes_cli/main.py
#	hermes_cli/memory_setup.py
#	hermes_cli/model_catalog.py
#	hermes_cli/models.py
#	hermes_cli/nous_subscription.py
#	hermes_cli/npm_engine.py
#	hermes_cli/plugin_index.py
#	hermes_cli/plugins.py
#	hermes_cli/plugins_cmd.py
#	hermes_cli/profile_distribution.py
#	hermes_cli/profiles.py
#	hermes_cli/prompt_size.py
#	hermes_cli/psutil_android.py
#	hermes_cli/runtime_repair.py
#	hermes_cli/security_advisories.py
#	hermes_cli/security_audit.py
#	hermes_cli/security_audit_startup.py
#	hermes_cli/service_manager.py
#	hermes_cli/session_export_md.py
#	hermes_cli/setup.py
#	hermes_cli/skills_hub.py
#	hermes_cli/slack_cli.py
#	hermes_cli/status.py
#	hermes_cli/subcommands/gateway.py
#	hermes_cli/subcommands/uninstall.py
#	hermes_cli/tools_config.py
#	hermes_cli/uninstall.py
#	hermes_cli/update_cmd.py
#	hermes_cli/update_contract.py
#	hermes_cli/update_inventory.py
#	hermes_cli/update_lock.py
#	hermes_cli/update_receipt.py
#	hermes_cli/urllib_security.py
#	hermes_cli/web_routers/local_models.py
#	hermes_cli/web_routers/profiles.py
#	hermes_cli/web_routers/skills.py
#	hermes_cli/web_server.py
#	hermes_constants.py
#	hermes_state.py
#	plugins/disk-cleanup/__init__.py
#	plugins/disk-cleanup/disk_cleanup.py
#	plugins/google_meet/node/registry.py
#	plugins/google_meet/node/server.py
#	plugins/google_meet/process_manager.py
#	plugins/google_meet/realtime/openai_client.py
#	plugins/hermes-achievements/dashboard/plugin_api.py
#	plugins/memory/hindsight/__init__.py
#	plugins/memory/honcho/__init__.py
#	plugins/memory/honcho/cli.py
#	plugins/memory/honcho/client.py
#	plugins/memory/honcho/oauth.py
#	plugins/memory/honcho/session.py
#	plugins/memory/mem0/__init__.py
#	plugins/memory/mem0/_setup.py
#	plugins/memory/openviking/__init__.py
#	plugins/memory/retaindb/__init__.py
#	plugins/memory/supermemory/__init__.py
#	plugins/platforms/a2a/protocol.py
#	plugins/platforms/dingtalk/adapter.py
#	plugins/platforms/discord/adapter.py
#	plugins/platforms/feishu/adapter.py
#	plugins/platforms/google_chat/adapter.py
#	plugins/platforms/matrix/adapter.py
#	plugins/platforms/photon/adapter.py
#	plugins/platforms/photon/auth.py
#	plugins/platforms/photon/cli.py
#	plugins/platforms/slack/adapter.py
#	plugins/platforms/teams/adapter.py
#	plugins/platforms/telegram/adapter.py
#	plugins/platforms/wecom/callback_adapter.py
#	plugins/platforms/whatsapp/adapter.py
#	plugins/teams_pipeline/store.py
#	plugins/video_gen/fal/__init__.py
#	plugins/web/ddgs/provider.py
#	plugins/web/exa/provider.py
#	plugins/web/firecrawl/provider.py
#	plugins/web/parallel/provider.py
#	tests/agent/test_ssl_ca_guard.py
#	tests/hermes_cli/test_certifi_repair.py
#	tests/hermes_cli/test_cmd_update.py
#	tests/hermes_cli/test_cmd_update_apt.py
#	tests/hermes_cli/test_dashboard_unified_launch.py
#	tests/hermes_cli/test_dep_ensure.py
#	tests/hermes_cli/test_doctor.py
#	tests/hermes_cli/test_doctor_live.py
#	tests/hermes_cli/test_gui_command.py
#	tests/hermes_cli/test_kanban_boards.py
#	tests/hermes_cli/test_kanban_db.py
#	tests/hermes_cli/test_lazy_refresh_venv_repair.py
#	tests/hermes_cli/test_memory_setup_provider_arg.py
#	tests/hermes_cli/test_nous_subscription.py
#	tests/hermes_cli/test_pip_install_detection.py
#	tests/hermes_cli/test_profile_export_credentials.py
#	tests/hermes_cli/test_psutil_android_extract.py
#	tests/hermes_cli/test_status.py
#	tests/hermes_cli/test_tui_npm_install.py
#	tests/hermes_cli/test_update_fleet_restart_pending.py
#	tests/hermes_cli/test_update_head_moved_gate.py
#	tests/hermes_cli/test_update_interrupted_recovery.py
#	tests/hermes_cli/test_web_server.py
#	tests/hermes_cli/test_web_ui_build.py
#	tests/test_hermes_logging.py
#	tests/test_managed_runtime_resolution.py
#	tests/tools/test_browser_chromium_autoinstall.py
#	tests/tools/test_browser_chromium_check.py
#	tests/tools/test_browser_homebrew_paths.py
#	tests/tools/test_browser_lightpanda.py
#	tests/tools/test_browser_npx_warmup.py
#	tests/tools/test_browser_open_timeout.py
#	tests/tools/test_browser_orphan_reaper.py
#	tests/tools/test_browser_real_profile.py
#	tests/tools/test_browser_suspect_recycle.py
#	tests/tools/test_find_shell.py
#	tests/tools/test_local_env_blocklist.py
#	tests/tools/test_macos_protected_search.py
#	tests/tui_gateway/test_compute_host.py
#	tools/approval.py
#	tools/blueprints.py
#	tools/bot_mode_dm.py
#	tools/bot_mode_probe.py
#	tools/bot_relay.py
#	tools/browser_tool.py
#	tools/browser_use_cli.py
#	tools/checkpoint_manager.py
#	tools/code_execution_tool.py
#	tools/code_kernel.py
#	tools/computer_use/cua_backend.py
#	tools/cronjob_tools.py
#	tools/discord_tool.py
#	tools/environments/base.py
#	tools/environments/daytona.py
#	tools/environments/local.py
#	tools/environments/modal.py
#	tools/environments/vercel_sandbox.py
#	tools/fal_common.py
#	tools/file_operations.py
#	tools/lazy_deps.py
#	tools/mcp_tool.py
#	tools/neutts_synth.py
#	tools/process_registry.py
#	tools/read_extract.py
#	tools/registry.py
#	tools/skill_ledger.py
#	tools/skill_linter.py
#	tools/skill_manager_tool.py
#	tools/skill_usage.py
#	tools/skills_ast_audit.py
#	tools/skills_guard.py
#	tools/skills_hub.py
#	tools/skills_sync.py
#	tools/skills_sync_client.py
#	tools/skills_tool.py
#	tools/terminal_scope.py
#	tools/terminal_tool.py
#	tools/tirith_security.py
#	tools/transcription_tools.py
#	tools/tts_tool.py
#	tools/vision_tools.py
#	tools/voice_mode.py
#	tools/wake_word.py
#	tools/web_result_cache.py
#	tools/website_policy.py
#	tools/working_diff.py
#	tools/write_approval.py
#	tui_gateway/entry.py
#	tui_gateway/methods_tools.py
#	tui_gateway/server.py
2026-09-04 13:03:39 -04:00
ethernet
642579db60 Merge remote-tracking branch 'upstream/main' into ethie/pm-clean
# Conflicts:
#	.github/actions/detect-changes/action.yml
#	.github/workflows/ci.yaml
#	.github/workflows/tests-os.yml
#	agent/prompt_builder.py
#	agent/ssl_verify.py
#	agent/subdirectory_hints.py
#	apps/desktop/electron/main.ts
#	apps/desktop/electron/preload.ts
#	apps/desktop/src/app/settings/about-settings.tsx
#	apps/desktop/src/global.d.ts
#	apps/desktop/src/i18n/ar.ts
#	apps/desktop/src/store/updates.ts
#	cron/suggestions.py
#	gateway/channel_directory.py
#	hermes_cli/config.py
#	hermes_cli/doctor.py
#	hermes_cli/linux_desktop_entry.py
#	hermes_cli/main.py
#	hermes_cli/web_routers/profiles.py
#	hermes_constants.py
#	plugins/platforms/photon/adapter.py
#	scripts/ci/classify_changes.py
#	scripts/install.ps1
#	tests/agent/test_relay_runtime_plugins.py
#	tests/ci/test_classify_changes.py
#	tests/hermes_cli/test_gui_command.py
#	tests/hermes_cli/test_linux_desktop_entry.py
#	tests/hermes_cli/test_update_fleet_restart_pending.py
#	tests/state/test_fts_runtime_rebuild.py
#	tests/tools/test_lazy_deps.py
#	tests/tools/test_macos_protected_search.py
#	tools/browser_tool.py
#	tools/file_operations.py
#	tools/lazy_deps.py
#	tools/mcp_tool.py
#	tools/working_diff.py
#	uv.lock
2026-09-04 03:18:16 -04:00
Teknium
e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium
315e291609 refactor(agent/context_references,billing_*): unify path/binary checks and _parse_* helpers (1492->1118 LOC) 2026-09-02 18:49:57 -07:00
Teknium
44982309b8 refactor(agent/prompt): dispatch tables and helper extraction in display, context refs, breakdown, compaction
build_tool_preview -> _PREVIEW_BUILDERS per-tool table; git @refs -> _GIT_REFERENCE_ARGS; context_breakdown
_skills_block/_append_overflow dedupe; prune_pre_checkpoint_items summary retention folded into one closure;
build_skill_invocation_message reuses _render_skill_block; ruff SIM collapses; restored two compacted
cache-policy invariant comments.
2026-09-02 13:53:58 -07:00
Teknium
be5c6a2fd8 refactor(agent/prompt): remove dead code, unify duplicated helpers, compact docstrings across prompt/skill/redaction modules
Dead (zero refs): coding_system_blocks, get_friendly_tool_labels, get_scan_ordered_skills_dirs,
_project_quarantine_cache_clear, clear_stable_prefixes, _redact_http_request_target_query_params,
_has_http_method_substring, PromptCachePlan.marker_count, display _diff_* colour thunks (-> _diff_ansi),
pass-through RedactingFormatter.__init__.
Unified: _slugify -> slugify_skill_name; reload diff -> diff_command_snapshots; _is_summary_item ->
is_compaction_summary_message alias; sanitizer walkers -> _sanitize_messages/_sanitize_structure;
assignment redaction passes -> _redact_assignments/_should_redact_assignment; quiet-mode tool lines -> _CUTE_LINES table.
2026-09-02 13:53:57 -07:00
Teknium
f6234d00c5 fix(security): close GitSpawn RCE class — malicious repo .git/config no longer executes on context gathering (GHSA-7x36-8jrh-v4pw)
Hermes gathers workspace context by running git against the session
directory automatically — the coding-workspace snapshot, gateway
project-tree build, /diff, @diff|@staged context refs, goal-gate
fingerprint, and -w startup worktree add — before any prompt, tool call,
approval, or trust gate. Those probes ran the system git without
stripping the repository's own config, so a repo delivered as files with
its .git directory intact (a shared zip, sync folder, or USB stick;
git clone never transfers .git/config) could set an execution-sink git
setting and get arbitrary host code execution as the user with nothing
on screen.

- core.fsmonitor / core.hooksPath / pager / editor / credential helper:
  neutralized by routing every automatic probe through
  noninteractive_git_env(), which pins those keys to inert values via
  GIT_CONFIG_* and ignores global/system config. bounded_git_probe (the
  reported sink, coding_context._git + tui_gateway.git_probe) now defaults
  to that env; worktree-add, working_diff, web_git, context_references,
  goals, and subagent_worktree route through it too.
- Attribute-scoped [diff "x"] command=/textconv= drivers: the attacker
  names the driver in .gitattributes, so GIT_CONFIG_KEY overrides can't
  enumerate them. Added harden_git_argv(), which inserts
  --no-ext-diff --no-textconv on diff-rendering subcommands (diff/show/
  log/blame) only — status et al reject the flags. Both flags required
  (verified empirically; each alone leaves the other live).

Builds on the noninteractive_git_env config-scrubbing from the
gemini-cli #28792 port. Real-git E2E regression suite arms a malicious
repo and asserts every automatic path neutralizes fsmonitor, hooks,
external-diff, and textconv; a baseline test proves the repo is armed.
2026-09-02 10:33:43 -07:00
ethernet
6e294f543d fix: read source files with encoding=utf-8-sig
Read text files with the encoding utf-8-sig so a BOM at the start of a
file does not cause a Unicode decode error (Windows editors add BOMs).

Reconstructed from ethie/pm commits 48a32b135b + 013219e814 onto the
current upstream/main base: only the utf-8 -> utf-8-sig transforms were
carried (370 exact line pairs across 205 files); pm-rename hunks that
rode in the original commit were left to the pm-store commit, and
utf8sig hunks entangled with content changes ride their owning commit.

Rebuilt on ethie/pm-clean off ac6c8028e0 (upstream/main).
2026-08-29 21:23:00 -04:00
zccyman
b85e5bb4ba feat(plugins): allow plugins to register custom @-prefix context references
Closes #26193

Adds ContextReferenceProvider ABC so plugins can register custom
@-prefixes (e.g. @issue:ENG-123) with autocomplete and expansion.
Plugin output flows through existing token-limit guards. Zero
breaking changes.
2026-08-12 18:41:59 -07:00
webtecnica
464e7e4e5f fix(docker): read attached binary files in backend (#76577) 2026-08-08 05:44:18 -07:00
kshitij
973c14b57c refactor: fold simplify findings — 6th copy in update_cmd, drop dead wrapper + speculative kwarg, behavior-contract tests
- Migrate the missed 6th inline formatter (update_cmd.py backup-size
  display) to the shared helper.
- checkpoints._fmt_bytes: plain alias instead of a None-guard wrapper —
  every caller feeds ints from checkpoint_manager (all size fields
  initialize to 0), so the None path was dead defensive code.
- Drop the fallback= kwarg (zero production callers; '?' default is the
  real inherited contract and stays).
- curator_backup + context_references: call format_bytes directly (single
  internal call site each, zero external importers — alias was churn
  avoidance with nothing to avoid). backup/_format_size and
  doctor/_human_bytes keep their aliases (claw.py + tests pin the former;
  three call sites use the latter).
- Reshape the loop so the trailing TB return is reachable (no dead line).
- Tests: replace alias-identity assertions (ossified the delegation
  mechanism) with behavior-contract equality over a value sweep;
  mutation-checked red-green.
- update_cmd parity: byte-identical B-GB vs the old inline loop; gains
  the TB tier.
2026-08-08 15:10:35 +05:30
kshitij
7289898494 refactor: consolidate five duplicate byte formatters into hermes_cli.sizefmt
Five modules each carried a private near-identical human-readable byte
formatter (backup._format_size, checkpoints._fmt_bytes,
doctor._human_bytes, context_references._human_bytes,
curator_backup.format_size). Three of them silently topped out at GB and
rendered a 1 TiB value as '1024.0 GB'. All five now alias one shared
format_bytes in hermes_cli/sizefmt.py (sibling of timefmt.py, same
zero-dependency rationale), keeping each module's established local name
so no caller churns.

Deliberately NOT migrated (behavior differs on purpose):
- session_recovery._format_bytes: binary suffixes (KiB/MiB/GiB)
- qqbot chunked_upload.format_size: '100.0 B' one-decimal style, pinned
  by its protocol tests

Net -33 production LOC before the new module; parity verified over a
16-value corpus against all five verbatim originals (only divergence:
the TB tier fix). Contract tests mutation-checked red-green.
2026-08-08 15:10:35 +05:30
Brooklyn Nicholson
045811f5be fix(context): keep @ref tokens inline instead of stripping them from the prompt
Expanding an @file:/@folder: reference deleted the token from the message
it was typed in, leaving a hole in the sentence and no anchor for clients
to chip. The attached context block still names each ref, so nothing is
lost by leaving the token where the user put it.
2026-07-27 22:55:18 -05:00
Brooklyn Nicholson
6811a79b62 fix(desktop): quote persisted @image: paths so spaced paths render
The unquoted alternative in the directive pattern is `\S+`, so a ref built
by string interpolation truncates at the first space and strands the tail
as loose text next to a broken thumbnail. Composer images live in the app's
userData dir, which on macOS is `~/Library/Application Support/<App>/` — so
every pasted or dropped image hit this.

Adds format_reference_value next to REFERENCE_PATTERN, mirroring
formatRefValue in the desktop's directive-text.tsx, and covers the
round-trip through the parser.
2026-07-24 21:20:52 -05:00
Stoltemberg
c89481db5e fix: add explicit UTF-8 encoding to all subprocess text=True calls (#53428)
On Windows with Chinese locale (GBK), subprocess.run(text=True) without
explicit encoding causes UnicodeDecodeError crashes. This fix adds
encoding='utf-8', errors='replace' to all subprocess.run() and
subprocess.Popen() calls that use text=True across 76 non-test Python files.

Fixes #53428 (master tracker for Windows GBK locale crash).

Note: credential_pool.py and electron changes excluded per reviewer request —
those will be submitted as separate focused PRs.
2026-07-24 11:45:57 -07:00
mrparker0980
10a54ccc2c fix(security): anchor @file context refs to canonical read deny-list
`@file` / `@folder` context-reference expansion enforced its own narrow
deny-list (`_ensure_reference_path_allowed` in `agent/context_references.py`)
that only covered `~/.ssh` keys, a handful of shell dotfiles, `~/.hermes/.env`,
and `skills/.hub`. It never blocked the credential stores that the canonical
read guard (`agent/file_safety.get_read_block_error`) protects: provider API
keys (`~/.hermes/auth.json`), Anthropic OAuth tokens
(`~/.hermes/.anthropic_oauth.json`), MCP OAuth material (`~/.hermes/mcp-tokens/`),
webhook HMAC secrets, and project-local `.env` files.

This matters because the messaging gateway feeds **untrusted** remote text
straight into reference expansion: `gateway/run.py` calls
`preprocess_context_references_async(..., allowed_root=_msg_cwd)` where
`_msg_cwd` defaults to the operator's HOME when `TERMINAL_CWD` is unset. A chat
peer (Telegram/Discord/Slack/...) could send `@file:~/.hermes/auth.json`, pass
the `allowed_root` check (it resolves under HOME), slip past the narrow list,
and have the operator's live keys read into the agent's context — where the
model would typically echo or act on them.

Rather than duplicate and re-sync a second secret list, this routes the guard
through the existing single source of truth. A reviewer might ask "why not just
add `auth.json` to the local list?" — because the local list has already drifted
once (a prior commit had to add `.config/gh`); anchoring to
`get_read_block_error` means every future addition there protects this path too.
The narrow checks are kept as a fallback since they also cover dirs that guard
does not (`.aws`, `.gnupg`, `.kube`, etc.), and the canonical lookup is wrapped
so it can never crash reference expansion.

N/A

- [x] 🔒 Security fix

- `agent/context_references.py`: `_ensure_reference_path_allowed` now also
  consults `agent.file_safety.get_read_block_error` after its existing checks
  and refuses the reference when that canonical guard flags the resolved path.
  The lookup is wrapped so guard-resolution failures fall back to the explicit
  checks instead of breaking expansion.
- `tests/agent/test_context_references.py`: added
  `test_blocks_canonical_read_denylist_credential_stores`, asserting that
  `@file` attaches for `auth.json`, `.anthropic_oauth.json`, `mcp-tokens/*`, and
  a project-local `.env` are all refused and their secret bodies never reach the
  expanded message.
- `scripts/release.py`: added the contributor email to `AUTHOR_MAP` (release
  gate).

1. `scripts/run_tests.sh tests/agent/test_context_references.py` — all 15 tests
   pass, including the new credential-store case.
2. Regression proof: stash `agent/context_references.py`, run the suite with
   `-- -k canonical`, and confirm the new test fails (secrets leak into the
   message) without the fix; restore and confirm it passes.
3. `ruff check agent/context_references.py tests/agent/test_context_references.py`
   and `python scripts/check-windows-footguns.py agent/context_references.py
   tests/agent/test_context_references.py` both pass.

- [x] I've read the Contributing Guide
- [x] My commit messages follow Conventional Commits (`fix(scope):`, etc.)
- [x] I searched for existing PRs to make sure this isn't a duplicate
- [x] My PR contains **only** changes related to this fix (plus the AUTHOR_MAP release gate)
- [x] I've run the test suite for the touched area and all tests pass
- [x] I've added tests for my changes (required for bug fixes)
- [x] I've tested on my platform: macOS 15 (Darwin 25.5)

- [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A
- [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A
- [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A
- [x] I've considered cross-platform impact (Windows, macOS) — or N/A
- [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A
2026-07-01 02:43:49 -07:00
kshitijk4poor
c1b9de73f5 perf(context-refs): expand @-references concurrently
Multiple @-references in one message (esp. @url: refs, each a full
web_extract round-trip) were expanded in a serial `for ref in refs: await`
loop. Switch to asyncio.gather over the independent _expand_reference calls,
reassembling warnings/blocks in original positional order so output is
byte-identical to the serial path; the token-budget check is unchanged.

Generic + provider-agnostic: helps every web backend equally (exa/tavily/
firecrawl/parallel) since it's above the provider layer. RED/GREEN test:
3 url refs @ 0.2s each = 0.60s serial -> ~0.20s concurrent.
2026-06-30 00:19:49 -07:00
Teknium
ee8cbfdc03 feat(web_extract): truncate-and-store instead of LLM summarization (#54843)
* feat(web_extract): truncate-and-store instead of LLM summarization

web_extract no longer runs an auxiliary LLM over scraped pages. The extract
backends (Firecrawl/Tavily/Exa/Parallel) already return clean, boilerplate-
stripped markdown, so we return it directly: pages within a char budget
(default 15000, web.extract_char_limit) come back whole; larger pages get a
head+tail window plus an explicit footer giving the stored full-text path and
the read_file call to page through the omitted middle. The full clean text is
written to cache/web (mounted read-only into remote backends like the other
cache dirs), so nothing is lost.

Inline base64 images are converted to [IMAGE: alt] placeholders (token bombs
dropped) while real http(s) image URLs are preserved as links so the agent can
still web_extract/vision_analyze them.

Removes process_content_with_llm + the chunked summarizer + check_auxiliary_model
+ _resolve_web_extract_auxiliary. context_references._default_url_fetcher is
updated to the truncate path and its stale data.documents shape read is fixed
to results (it was silently returning empty).

Live before/after eval (firecrawl, 4 URLs): 11.7x faster overall (176.6s ->
15.1s); 10-60x on large pages. Quality identical; findability 4/4 (answer
recoverable from stored full text on every truncated page). web_search is
unchanged.

No own scraper added; no changes to web_search.

* fix(web_extract): add char_limit to execute_code web_extract stub

The new web_extract char_limit param must appear in the code_execution_tool
_TOOL_STUBS signature (and doc line) or test_stubs_cover_all_schema_params
fails — the stub schema must cover every real schema param.
2026-06-29 10:00:49 -07:00
Teknium
cb982ad997 fix(windows): hide console-window flash on backend git/gh/wmic/bash subprocess spawns
The Windows desktop GUI runs its backend headless via pythonw.exe. Several
auxiliary subprocess sites that run inside that windowless backend spawned
console-subsystem children (git, gh, wmic, powershell, bash, rg, taskkill)
WITHOUT CREATE_NO_WINDOW, so Windows allocated a fresh conhost per call and
flashed a black window on screen — sometimes continuously (the dashboard
Projects-tree git probe alone fired ~118 spawns in 60s on startup).

The terminal tool, cron, browser, code_execution, and gateway-spawn paths
already carry windows_hide_flags(); these auxiliary probe/scan/launcher legs
were missed. Wire the existing helper into them:

- tui_gateway/git_probe.py: run_git (+ encoding=utf-8/errors=replace, fixes the
  cp950 UnicodeDecodeError on CJK paths from the same site)
- agent/coding_context.py: _git (per-turn git status/log/diff)
- agent/context_references.py: _run_git + _rg_files (@file/@ref resolution)
- hermes_cli/copilot_auth.py: gh auth token probe (auxiliary provider:auto)
- hermes_cli/gateway.py: wmic + PowerShell Get-CimInstance PID scan
- hermes_cli/main.py: wmic stale-dashboard PID scan
- gateway/status.py: taskkill /T /F force-kill

windows_hide_flags() returns 0 on POSIX, so every changed call is a no-op on
Linux/macOS (verified: real git/rg probes still work; Windows-simulated calls
all pass creationflags=CREATE_NO_WINDOW).

Scoped to the windowless-backend paths that cause the reported flashing. The
Electron updater-handoff leg (main.cjs windowsHide:false) and the
interactive-CLI banner probes (cli.py) are intentionally NOT touched here —
the former needs a Windows-tested change of its own, the latter runs in a
visible console anyway.

Tracking: #54220
Refs: #53178 #53631 #53781 #53957 #49602 #52982 #53424 #53053 #53016
2026-06-28 05:28:45 -07:00
Teknium
d3d621f7c3 revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853)
* Revert "fix(windows): capture is not a no-window boundary; route flashing spawns through chokepoint (#53829)"

This reverts commit 2ecca1e7d3.

* Revert "fix(windows): stop terminal-window popups from background spawns (#53810)"

This reverts commit 5db1430af9.

* Revert "fix(windows): stop subprocess console-window popups + add CI guard (#53791)"

This reverts commit ef17cd204d.
2026-06-27 15:59:00 -07:00
Teknium
2ecca1e7d3 fix(windows): capture is not a no-window boundary; route flashing spawns through chokepoint (#53829)
Follow-up to #53791 addressing review feedback: the footgun checker treated
capture_output=/stdout=/stderr=/check_output as proof a subprocess can't pop a
Windows console. That invariant is false — stream redirection controls where a
child's output goes, not whether a console is allocated. From a console-less
parent (Desktop/Electron, pythonw.exe, detached gateway/cron) a console-subsystem
child still flashes a window even when fully captured.

- check-windows-footguns.py: capture/redirect/check_output is no longer a blanket
  safe-pass. Added _WINDOWS_FLASHING_PROGRAMS (git/gh/npm/node/python/uv/ffmpeg/
  docker/powershell/…); calls to those are flagged even when captured. Non-flashing
  programs keep the capture exemption (no 271-site noise). _subprocess_compat.run/
  popen calls are inherently safe (wrapper injects CREATE_NO_WINDOW).
- Routed the 35 genuine flashing git/gh/npm/uv/ffmpeg/docker spawns through the
  _subprocess_compat.run/popen chokepoint (Brooklyn's wrapper from #53810) — the
  durable fix, not per-site annotations. cmd.exe /c start stays # ok (intentional).
- Updated tests + CONTRIBUTING.md rule #17 to the corrected invariant.
2026-06-27 14:49:41 -07:00
Brooklyn Nicholson
7ffc216bc0 fix(agent): make a binary @file: reference actionable instead of a dead end
A binary @file: ref (PDF, docx, spreadsheet, …) expanded to a bare
"binary files are not supported" warning with no content. The model saw a
failure and gave up — e.g. a dropped PDF came back as a text note claiming the
type was unsupported, even though the file was staged on disk right next to it.

Inject an actionable content block instead: the path, mime type, size, and a
nudge to use its tools to read/convert/view the file (and explicitly not to tell
the user the type is unsupported). General across every binary type — not
PDF-specific. The file already resolves where the agent's tools run (local cwd
or the staged copy in a remote session workspace), so it can act on it directly.
2026-06-09 19:16:46 -05:00
m4dni5
d1f23bb2d5 fix: prevent TUI gateway stdin EOF crash across all TUI-context subprocess calls
When Hermes runs in TUI mode, the gateway child process communicates with
the Node.js parent over a JSON-RPC protocol on stdin. Subprocess calls that
inherit this stdin fd can trigger a race condition where the child's stdin
read returns EOF, causing the gateway to exit cleanly (exit code 0) mid-tool-
execution.

This is the same root cause as issue #14036 (byterover plugin) and PR #39257
(SSH environment backend). This commit applies the fix — stdin=subprocess.DEVNULL
— to all 85 subprocess.run() and subprocess.Popen() calls that execute inside
the TUI gateway child process.

Scope: TUI-context code only (agent/, tools/, plugins/, tui_gateway/server.py).
CLI code (cli.py, hermes_cli/), tests, scripts, and gateway process management
are excluded — they don't run inside the TUI child and inherit the terminal's
stdin, not the JSON-RPC pipe.

85 call sites across 28 files. All files pass syntax check.
2026-06-08 22:46:57 -07:00
Ruzzgar
60236862ee fix(agent): fall back when rg is blocked for @folder references 2026-04-20 01:56:41 -07:00
WAXLYY
c6e1add6f1 fix(agent): preserve quoted @file references with spaces 2026-04-10 13:05:01 -07:00
Teknium
187e90e425 refactor: replace inline HERMES_HOME re-implementations with get_hermes_home()
16 callsites across 14 files were re-deriving the hermes home path
via os.environ.get('HERMES_HOME', ...) instead of using the canonical
get_hermes_home() from hermes_constants. This breaks profiles — each
profile has its own HERMES_HOME, and the inline fallback defaults to
~/.hermes regardless.

Fixed by importing and calling get_hermes_home() at each site. For
files already inside the hermes process (agent/, hermes_cli/, tools/,
gateway/, plugins/), this is always safe. Files that run outside the
process context (mcp_serve.py, mcp_oauth.py) already had correct
try/except ImportError fallbacks and were left alone.

Skipped: hermes_constants.py (IS the implementation), env_loader.py
(bootstrap), profiles.py (intentionally manipulates the env var),
standalone scripts (optional-skills/, skills/), and tests.
2026-04-07 10:40:34 -07:00
Teknium
d3f1987a05 fix(security): add .config/gh to read protection for @file references (#4327)
Follow-up to PR #4305 — .config/gh was added to the write-deny list
but missed from _SENSITIVE_HOME_DIRS, leaving GitHub CLI OAuth tokens
exposed via @file:~/.config/gh/hosts.yml context injection.
2026-03-31 12:48:30 -07:00
maymuneth
655eea2db8 fix(security): protect .docker, .azure, and .config/gh from read and write 2026-03-31 12:47:10 -07:00
Teknium
83043e9aa8 fix: add timeout to subprocess calls in context_references (#3469)
_expand_git_reference() and _rg_files() called subprocess.run()
without a timeout. On a large repository, @diff, @staged, or
@git:N references could hang the agent indefinitely while git
or ripgrep processes slow output.

- Add timeout=30 to git subprocess in _expand_git_reference()
  with a user-friendly error message on TimeoutExpired
- Add timeout=10 to rg subprocess in _rg_files() returning
  None on timeout (falls back to os.walk folder listing)

Co-authored-by: memosr.eth <96793918+memosr@users.noreply.github.com>
2026-03-27 17:51:14 -07:00
Guts
2d8fad8230 fix(context): restrict @ references to safe workspace paths (#2601)
fix(context): block @ references from reading secrets outside the workspace. Defaults allowed_root to cwd, adds sensitive file blocklist.
2026-03-23 06:40:05 -07:00
Teknium
da44c196b6 feat: @ context references — inline file, folder, diff, git, and URL injection
Add @file:path, @folder:dir, @diff, @staged, @git:N, and @url:
references that expand inline before the message reaches the LLM.
Supports line ranges (@file:main.py:10-50), token budget enforcement
(soft warn at 25%, hard block at 50%), and path sandboxing for gateway.

Core module from PR #2090 by @kshitijk4poor. CLI and gateway wiring
rewritten against current main. Fixed asyncio.run() crash when called
from inside a running event loop (gateway).

Closes #682.
2026-03-21 15:57:13 -07:00