Commit Graph

10 Commits

Author SHA1 Message Date
teknium1
2dcebe6471 feat: Hermes-owned scratch dir replaces the system temp dir for every process and child
Hermes and everything it launches (browser profiles, PTY probes, skill scripts,
tempfile defaults in delegated code) wrote to the system temp dir, which is a
RAM-backed tmpfs on most Linux hosts and containers and fills under agent load.

- hermes_constants.get_scratch_dir(): HERMES_HOME/cache/scratch (0700), entries
  older than 72h pruned once per process / once per hour across processes.
- apply_scratch_tmp_env(env) / export_scratch_tmp_env(): TMPDIR/TMP/TEMP point at
  the scratch dir when the user or OS has not set them; a value Hermes itself
  exported (== HERMES_SCRATCH_DIR) is re-derived for a re-homed process or a
  child served under another profile, so profiles never share scratch.
- hermes_bootstrap runs the export on import (every entry point); hermes_cli.main
  re-runs it after --profile resolution; the subprocess HOME contract
  (apply_subprocess_home_env) and the routed-home rewrites in code_execution_env
  and served_profile_child_env apply it to child envs.
- The runtime-environment prompt block names the scratch dir so the model stops
  reaching for the system temp dir by reflex; hermes doctor reports the dir, its
  size and whether a user TMPDIR overrides it.
2026-09-19 10:44:26 -07:00
teknium1
d029cddfd7 fix(bootstrap): a racer bug surfaces instead of silently falling back to the serial connect
Both installed racers caught any non-OSError from the Happy Eyeballs core and
re-ran the stock serial ``create_connection``. That branch was untested and
its only effect was to hide a bug in the racer by silently reintroducing the
exact IPv6-first stall the racer exists to remove. OSError (every candidate
failed) still propagates unchanged, identical to the serial original; anything
else now raises. Invariant test drives both racers with a raising core.
2026-09-18 09:56:28 -07:00
teknium1
c350d3e44f fix(bootstrap): race IPv6/IPv4 for every sync connect from the process bootstrap, not per client
Wire the racer #114277 added once, at the seam every Hermes process already
crosses first: ``hermes_bootstrap`` (imported before anything else by
``hermes``, ``hermes-agent``, ``hermes-acp``, ``gateway.run``, ``batch_runner``,
``tui_gateway.entry`` and the slash worker). Installing it from
``init_agent`` was too late for the startup path the report is about — the TUI
gateway starts MCP discovery and the model-catalog prewarm before any AIAgent
exists, and ``hermes model`` / picker prewarm never build one.

- Move the stdlib-only racer core (``_happy_eyeballs_create_connection``,
  ``_interleave_addrinfos``) and the installer into ``hermes_bootstrap`` and
  apply it on import; ``agent.process_bootstrap`` keeps only the httpcore
  backend and imports the racer from there. The bootstrap must stay stdlib-only
  because entry points call ``harden_import_path()`` after importing it.
- Patch urllib3's own serial connect walker lazily through a one-shot
  ``sys.meta_path`` hook instead of importing urllib3 eagerly: ``hermes`` and
  the TUI gateway never load urllib3 at start, and importing it costs ~50 ms.
- Idempotence by a marker on the installed function, so a re-import of the
  bootstrap (tests, ``importlib.reload``) never wraps the racer twice.
- Drop the ``init_agent`` call site (redundant: ``process_bootstrap`` imports
  the bootstrap) and trim the five added tests to two invariants in the
  mirroring ``tests/test_hermes_bootstrap.py``; the racer unit test moves its
  monkeypatch seams to the new module.
- Docs: ``network.force_ipv4`` now describes the default racing behaviour.

Live A/B (stub resolver: blackholed 100::1 first, local IPv4 second, driving
the real clients after ``import hermes_cli.main``): catalog fetch via requests
5.05s -> 0.39s, shared keepalive httpx client 15.24s -> 0.32s, inline
httpx.Client 5.01s -> 0.26s; IPv4-only control 0.01s both sides. Refused ports
still fail instantly with the same exception types; a blackhole-only host still
raises ConnectTimeout at the configured connect timeout.

Fixes #114265
2026-09-18 09:56:28 -07:00
Teknium
857b43ef92 refactor(hermes_bootstrap,hermes_startup_watchdog,hermes_logging): collapse duplicated dump/handle plumbing, compact docs 2026-09-02 18:14:53 -07:00
teknium1
0fb0ba475d fix(windows): platform._syscmd_ver stub in bootstrap + PYTHONUTF8 in desktop backend env
Two gaps found auditing the decode-crash cluster:

1. suppress_platform_ver_console() only ran in hermes_cli.main processes;
   slash workers, tui_gateway/entry, run_agent, batch_runner, and cli.py
   import only hermes_bootstrap and were exposed to both the console
   flash and (on Python 3.11.0/3.11.1, which lack CPython's
   encoding='locale' fix) a UnicodeDecodeError inside platform.win32_ver()
   under PEP 540 — the crash #69413 reported. Move the stub into
   hermes_bootstrap so every entry point gets it; the _subprocess_compat
   copy stays for non-bootstrap callers.

2. The desktop Electron spawn built the backend env without PYTHONUTF8,
   so anything the Python child emitted before hermes_bootstrap ran
   (interpreter startup errors, pre-bootstrap tracebacks) decoded with
   the locale default. Re-port of PR #56499's env half (echoriver89) to
   backend-env.ts (original targeted the deleted backend-env.cjs);
   explicit user setting wins.
2026-07-24 15:47:12 -07:00
Teknium
d3d621f7c3 revert(windows): roll back terminal-popup PRs #53791 #53810 #53829 (#53853)
* Revert "fix(windows): capture is not a no-window boundary; route flashing spawns through chokepoint (#53829)"

This reverts commit 2ecca1e7d3.

* Revert "fix(windows): stop terminal-window popups from background spawns (#53810)"

This reverts commit 5db1430af9.

* Revert "fix(windows): stop subprocess console-window popups + add CI guard (#53791)"

This reverts commit ef17cd204d.
2026-06-27 15:59:00 -07:00
brooklyn!
5db1430af9 fix(windows): stop terminal-window popups from background spawns (#53810)
* fix(windows): stop terminal-window popups from background spawns

Native-Windows desktop/gateway users saw cmd/conhost windows flash on
gateway restart, image paste, the dashboard Projects tree, voice notes,
and ~5 min after closing the app (detached cron). Two root causes:

- Console-subsystem exes (taskkill, schtasks, wmic, netstat, tasklist,
  agent-browser, git, ffmpeg, powershell, git-bash) spawned via raw
  subprocess allocate a fresh console when the launching process has
  none (pythonw desktop backend / detached gateway) - even with output
  captured.
- uv venv pythonw shims re-exec console python.exe, so Python children
  get a console regardless of how they're launched.

Fixes:
- Single hidden-spawn primitive (_subprocess_compat.run/.popen) that ORs
  CREATE_NO_WINDOW on Windows, no-op on POSIX. Route every Hermes-owned
  console-exe spawn through it.
- FreeConsole() catch-all in hermes_bootstrap: any Python child that
  exclusively owns an auto-allocated console detaches it at startup
  (GetConsoleProcessList()==1 gate leaves shared interactive consoles
  untouched).
- Replace PowerShell/wmic gateway PID scans with in-process psutil.
- Skip schtasks queries on non-interactive desktop restarts.
- Prefer native agent-browser .exe over .cmd shims.
- Guard test bans raw subprocess spawns of the Windows-only console
  tools repo-wide so the popup class can't regress.

* fix(windows): scope FreeConsole to background entry points; fix merge fallout

Console detach review (per #53810 feedback): GetConsoleProcessList()==1 can't
tell a uv pythonw->python phantom console apart from a user opening the
interactive CLI/TUI in its own fresh console (double-click, shortcut, ConPTY) —
both report a single attached process with a tty. Running FreeConsole() in the
import-time bootstrap therefore risked detaching a legitimately-interactive
terminal.

- Extract FreeConsole into explicit hermes_bootstrap.detach_orphan_console();
  remove it from apply_windows_utf8_bootstrap() (import side effect).
- Call it only from known background mains: gateway run, dashboard backend
  (start_server, what the desktop spawns), cron standalone, tui_gateway entry,
  slash worker. Interactive CLI/TUI never calls it.
- Behavior-contract tests: frees only when solo owner, leaves shared console,
  no-op without console / on POSIX, and asserts it's not an import side effect.

Merge fallout from origin/main (#53791):
- local.py: 3-way merge left a dangling **_popen_kwargs (NameError crashing
  every terminal init). _subprocess_compat.popen already hides the window, so
  drop it.
- discord adapter: merge stacked an undefined windows_hide_flags() onto the
  primitive call; drop the redundant arg.
- test_gateway: scan now goes psutil-first (zero spawn); rewrite the
  case-variant test to drive that production path.

* test(claw): mock _subprocess_compat.run seam for Windows process scan

claw.py's Windows tasklist/powershell scan routes through the hidden-spawn
primitive; the tests still patched claw_mod.subprocess, so on win32 the mock
was never hit and real spawns returned nothing. Patch the actual seam.
2026-06-27 14:02:24 -07:00
Ben
cbd6ba1bdd fix(docker): redirect lazy installs to a durable target so opt-in backends work in the immutable image (#51136)
The published Docker image seals the agent venv (root-owned, read-only
/opt/hermes) and sets HERMES_DISABLE_LAZY_INSTALLS=1 so a runtime install
can't mutate and brick the core. But opt-in backends (Firecrawl web search,
Exa, Feishu, ...) deliberately keep their SDKs in tools/lazy_deps.py and out
of [all] (pyproject policy 2026-05-12: one quarantined release must not break
every install). The two policies collided: the SDK isn't baked in AND can't
lazy-install, so the default Firecrawl web_search/web_extract fail out of the
box in Docker (#51136), as do Exa (#49445) and Feishu (#50205).

Fix the whole class instead of baking in one backend: when
HERMES_LAZY_INSTALL_TARGET is set, lazy installs are redirected to a writable
dir on the durable /opt/data volume via `pip/uv install --target`, and that
dir is APPENDED to the end of sys.path. Because the core venv always wins
name collisions, a package installed this way can only ADD new modules — it
can never shadow, downgrade, or break a module the core ships. The worst a
bad/incompatible backend package can do is fail to import and report itself
unavailable; the agent core stays healthy. That structural guarantee is what
made it safe to seal the venv, and it is preserved here even with installs
re-enabled.

- tools/lazy_deps.py: durable-target mode — `--target` install + core-pinned
  `--constraint` file (shared deps resolve to core's versions, conflicts fail
  loudly at install time), append-only sys.path activation, ABI/Python-version
  stamp that wipes the store if an image rebuild bumps the interpreter, and a
  reworked gate so HERMES_DISABLE_LAZY_INSTALLS=1 redirects (rather than hard-
  blocks) when a target is set. security.allow_lazy_installs=false still
  disables installs in every mode.
- hermes_bootstrap.py: activate the durable target on sys.path at first import
  (before any backend imports its SDK) so packages installed on a previous run
  are importable on this run.
- Dockerfile: set HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages.
- docker/stage2-hook.sh: seed + chown the dir on the data volume.
- tests: real-install E2E proving installs land in the target, import cleanly,
  don't leak into the sealed venv, and that a core package is never shadowed;
  ABI-stamp wipe/preserve; gate matrix; Dockerfile/stage2 contract test.

Fixes #51136
2026-06-25 09:20:13 +10:00
Teknium
c39b2b50ee fix(tui): stop a cwd package named utils/proxy/ui from crashing the gateway child (#51693)
Launching Hermes from a directory that ships its own top-level package with a
Hermes-internal name (utils/, proxy/, ui/) crashed the gateway/TUI child with
an ImportError (exit 1, crash loop): from utils import atomic_replace resolved
to the user's package.

tui_gateway/entry.py already stripped the relative cwd forms ('' / '.'), but
the launch dir also reaches sys.path as its own ABSOLUTE path (venv activation
or a project that adds itself to PYTHONPATH), which the strip missed and which
sat ahead of the Hermes root.

Centralize a hardened guard in hermes_bootstrap.harden_import_path(): drop the
relative forms AND force the Hermes source root to the front even when an
absolute cwd entry is present. Wire it into tui_gateway/entry.py and
acp_adapter/entry.py (both spawn into arbitrary cwds); hermes_cli/main.py and
gateway/run.py already insert the root at front. gatewayClient.ts now also
exports HERMES_PYTHON_SRC_ROOT for defense in depth.
2026-06-23 23:29:45 -07:00
Teknium
d94fb47717 hermes_bootstrap: Windows-only UTF-8 stdio shim for all entry points
Codebase-wide fix for Python-on-Windows UTF-8 footguns, complementing
the earlier execute_code sandbox fixes (which remain load-bearing for
when the sandbox explicitly scrubs child env).

Problem: Python on Windows has two long-standing text-encoding pitfalls:

  1. sys.stdout/stderr are bound to the console code page (cp1252 on
     US-locale installs) — print('café') crashes with UnicodeEncodeError.
  2. Subprocess children don't know to use UTF-8 unless PYTHONUTF8 and/or
     PYTHONIOENCODING are set in their env — so any Python we spawn
     (linters, sandbox children, delegation workers) hits the same bug.

Solution: A tiny bootstrap module (hermes_bootstrap.py) imported as the
first statement of every Hermes entry point:

  - hermes_cli/main.py   (hermes / hermes-agent console_script)
  - run_agent.py         (hermes-agent direct)
  - acp_adapter/entry.py (hermes-acp)
  - gateway/run.py       (messaging gateway)
  - batch_runner.py      (parallel batch mode)
  - cli.py               (legacy direct-launch CLI)

On Windows, the bootstrap:
  - os.environ.setdefault('PYTHONUTF8', '1')       (PEP 540 UTF-8 mode)
  - os.environ.setdefault('PYTHONIOENCODING', 'utf-8')
  - sys.stdout/stderr/stdin.reconfigure(encoding='utf-8', errors='replace')

Children inherit the env vars → they run in UTF-8 mode.
Current process's stdio is reconfigured → print('café') works now.

On POSIX (Linux/macOS), the bootstrap is a complete no-op.  We don't
touch LANG, LC_*, or anything else — users who have intentionally
configured a non-UTF-8 locale aren't affected.  POSIX systems are
already UTF-8 by default in 99% of modern setups, so there's nothing
to fix.

setdefault() (not overwrite) means users who explicitly set PYTHONUTF8=0
or PYTHONIOENCODING=cp1252 in their environment are respected.

What this does NOT fix: bare open(path, 'w') calls in the *parent*
process still default to locale encoding because PYTHONUTF8 is only
read at interpreter init.  A ruff PLW1514 sweep (separate follow-up)
will add explicit encoding='utf-8' at those ~219 call sites for
belt-and-suspenders.

Tests (17): 16 passed, 1 skipped on Windows.
  - Windows: env vars set, stdio reconfigured, child inherits UTF-8 mode
  - POSIX: complete no-op (verified on fake POSIX + skipped on real
    POSIX since we don't have a Linux box in this session)
  - Idempotence: multiple calls safe
  - Graceful degradation: non-reconfigurable streams don't crash
  - User opt-out: explicit PYTHONUTF8=0 is respected
  - Load order: every entry point's FIRST top-level import is
    hermes_bootstrap, enforced by an AST-level parametrized test

pyproject.toml: added hermes_bootstrap to py-modules so it ships with
pip installs.
2026-05-08 14:27:40 -07:00