worktree_gc archived untracked files under ~/.hermes regardless of the active
profile or HERMES_HOME (and the Windows LOCALAPPDATA default); dashboard_procs'
remote-lock dir, both bot_mode `_default_home` copies, methods_bot_relay's
`_relay_root` (a hand copy of get_default_hermes_root's profiles/ strip) and
load_hermes_dotenv's default home all re-derived env-or-`~/.hermes` by hand and
so diverged from the platform default. Each now calls the canonical getter
with the intent it already had: get_hermes_home() where the active profile
matters (archive), get_process_hermes_home() where the process asset must stay
visible under a routed-profile override (locks, bot mode, startup .env),
get_default_hermes_root() for install-wide relay state.
Six independent line-parsers with three different quoting/comment semantics
read the same .env files: tools/skills_tool.load_env (strip("\"'"), no inline
comments), hermes_cli/managed_scope._parse_env (same, no export, no BOM),
web_server_cron._profile_env_value (plain utf-8, no BOM), profile_cmd
._env_file_has_key, env_loader._env_keys_defined_in_dotenv (utf-8, so a BOM'd
first key stayed "\ufeffKEY" and the dashboard profile scrub missed line 1),
mem0/_setup._prompt_api_key (startswith scan, no quote strip). The boundary
parsers (scrub key set, skill secret capture) therefore disagreed with the
parser that installs the profile scope.
Now every one is a 1-3 line forwarder onto load_env_file, and
hermes_cli.config.load_env is memo over it (public signature unchanged).
_parse_env_value moves next to its only caller in secret_scope.
load_env_file gains the same latin-1 fallback env_loader uses to install
into os.environ, so a mis-encoded file yields the same key set on both sides.
Managed .env keeps its fail-LOUD contract (decode error logs and ignores the
file) instead of load_env_file's fail-soft {}.
Behavior change: managed .env, skills_tool and mem0 setup now honour
`export`, quoted-value escapes and inline comments the way the profile scope
does; web_server_cron and the dashboard scrub tolerate a BOM.
Invariant test: a BOM'd/export/quoted/commented .env yields the same key set
via load_hermes_dotenv (installer), load_env_file (scope) and
_env_keys_defined_in_dotenv (scrub); fails with the old scrub parser.
Symptom (#102041): under a multiplex gateway the default profile's vault/1Password/
Bitwarden/plugin-sourced credentials vanished for the rest of the process after the first
cron fire or the post-discovery plugin refresh; with the key already in the process env
(systemd EnvironmentFile=) the scope was empty from boot. Every get_secret() read then
failed closed ("No usable credentials", every Telegram sender rejected).
Why: _apply_external_secret_sources marked the home applied after any real fetch, but only
snapshotted names in report.provenance — the NEWLY applied ones. On a re-apply the previous
apply's own write-back makes every key `skipped_existing`, so the snapshot latched to {} and
_hydrate_profile_secret_sources returned that empty snapshot forever. Separately,
reset_secret_source_cache() was process-wide, so one profile's cron re-pull dropped every
sibling's hydrated snapshot (1aa62ceb45 isolated the routed reload but not the reset).
Change:
- env_loader: snapshot every name a source supplied (provenance + skipped_existing) from
the home's effective environment, so a shadowed re-apply keeps the values it had.
- reset_secret_source_cache(hermes_home=None): optional per-home reset; global clear kept
for tests/config edits.
- cron per-fire re-pull and plugins._refresh_secret_sources_after_discovery reset + reload
only the home they resolve to.
- Two invariant tests (red on base) in tests/test_env_loader_secret_sources.py; docs note in
secret-source-plugin.md.
Reported-by: luochen1990
Addresses #102041
Under a multiplexed gateway, `_build_safe_env` forwarded `os.environ[name]` for
every name tagged in the process-global `_SECRET_SOURCES` map. That map is
filled by EVERY served profile's secret-source hydration, while `os.environ`
only ever holds the LAUNCH (default) profile's values — so once any profile's
1Password/Bitwarden source supplied e.g. GITHUB_TOKEN, every profile's stdio
MCP server was started with the default profile's token.
Resolve those names through the active profile's secret scope (`get_secret`)
instead: the routed profile's value, or omitted when that profile has none.
Under multiplex `get_secret` never falls through to environ; single-profile
runs keep the .env overlay + environ behaviour, so the existing "vault vars
reach MCP subprocesses" contract still holds there. `secret_source_names()`
exposes the tagged NAMES only — values are never read from the shared map.
Docs: the multi-profile guide's "MCP subprocesses only see their own profile's
secrets" claim is now true for source-injected names too; say so explicitly.
In a multiplex dashboard (one process serving every profile on the host),
a secondary profile's terminal.backend could leak into the shared
os.environ and silently override the launch profile's terminal backend.
A profile configured for terminal.backend: ssh would find itself running
every command locally because a sibling profile with backend: local had
polluted the shared environment.
The tell is an asymmetric env state on the launch session:
TERMINAL_ENV=local # leaked from a sibling profile
TERMINAL_SSH_HOST=10.10.0.103 # still the launch profile's, untouched
A restart does not help: as soon as the sibling profile is touched again
(a turn, a cron tick), the shared env is re-poisoned.
Root cause: env_loader._reapply_terminal_config_bridge() guards "only
re-bridge config into the shared os.environ when this load is for the
process's own profile" via _process_hermes_home(). That helper delegated
to get_hermes_home(), which follows the context-local
set_hermes_home_override a per-request task installs. When a per-turn
handler is scoped to a secondary profile and triggers a dotenv reload,
both sides of the comparison resolve to that secondary profile, the guard
passes, and the bridge writes the wrong profile's terminal.backend into
the shared environment. Because the secondary profile's config carries no
TERMINAL_SSH_* keys, only TERMINAL_ENV is overwritten, producing the
asymmetric state above.
The helper was introduced for the config-cache key (where following the
active home is correct) and later reused for the terminal bridge guard,
where the override-following semantics are wrong.
Fix: delegate _process_hermes_home() to get_process_hermes_home(), the
override-immune resolver built for exactly this. It reflects the scope the
process was launched under, ignoring per-task overrides. Both call sites
(the bridge guard and the secrets-cache reuse check) want the true process
home. Single-profile / CLI behaviour is unchanged: with no active
override the two resolvers are identical.
Complements the terminal_tool._active_environments session-key fix: that
scopes the per-session environment cache; this keeps the shared os.environ
the cache reads TERMINAL_ENV from uncontaminated in the first place.
Adds tests/hermes_cli/test_terminal_bridge_profile_scope.py covering both
the override-immune resolver and the no-leak reload behaviour.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
Follow-up to the #77592 salvage: emit a once-per-home debug line where the
multiplex guard skips the process-global dotenv load (requested on #77562),
and port the single-profile control test from #77970 so the guard is pinned
to the multiplex flag rather than the home override alone.
Co-authored-by: DonShelly <25538402+DonShelly@users.noreply.github.com>
Two upstream tests failed with the whitelist approach:
- test_real_plugin_source_discovery_applies_dotenv (plugin source named
HERMES_TEST_PLUGIN_BOOTSTRAP not in whitelist)
- test_external_secret_values_are_isolated_between_homes (test source
named test-source not in whitelist)
Fix: use 'v.get("enabled") is True' instead of 'v.get("enabled", True)'
on any dict value. This is stricter — only keys with an explicit
enabled: true pass — while remaining name-agnostic, so plugin and test
sources flow through without hardcoding a whitelist.
Refs #86781, #86782
Address review feedback from trevorgordon981 on PR #86782:
1. **Pre-register parsers at parse-time, lazy-import backends only**
- secrets_cli.register_cli() and onepassword_secrets_cli.register_cli()
now run eager at parser-build time (no deferral past parse_args)
- Only the agent.secret_sources.bitwarden/onepassword imports are lazy
(inside each cmd_* handler via _load_bitwarden()/_load_onepassword())
- This eliminates the 'invalid choice' and infinite-recursion risks
2. **Known-source-names gate for env_loader registry**
- Only keys in {bitwarden, onepassword, op, 1password, bw} trigger the
registry import; a generic dict entry no longer forces crypto load
- Prevents unrelated config dicts from paying crypto cost
3. **End-to-end tests for the real dispatch paths**
- test_bitwarden_setup_help: runs real CLI subprocess with --help
- test_bitwarden_status/disable/onepassword_status: run real handlers
- test_update_check_clean/no_self_lock: run real update --check
- test_update_check_no_cryptography: sys.modules inspection (backup)
4. **Fix flaky test_turn_lease.py** (unrelated pre-existing failure)
The architecture guarantees:
- parse-time: zero cryptography load (all backends lazy)
- dispatch-time: crypto loads exactly once per secrets command
- update path: completely clean of cryptography._rust mapping
Refs #86781, #86782
The env_loader eagerly imported agent.secret_sources.registry on every
startup, which loads agent/secret_sources/bitwarden.py and its
cryptography.* dependencies. On Windows, this causes the updater
process itself to map cryptography._rust.pyd before the self-lock
preflight runs, triggering a defer loop that blocks updates entirely.
Add an 'any_enabled' gate: scan the parsed config for actually-enabled
sources before paying for the registry import. A config with no enabled
sources costs one dict scan; a config with enabled sources pays the
crypto load exactly once, on demand.
Refs #86781, #83569, #83590
Test: 3 scenarios verified — main() clean, env_loader clean (no
enabled sources), env_loader loads crypto (enabled sources).
utf-8-sig only covers the primary decode. BOM + invalid UTF-8 (e.g.
PowerShell BOM + cp1252 body) forced latin-1, which kept EF BB BF as
part of the first key name and dropped the canonical name. Strip the
BOM before latin-1 decode and load via stream so override= is preserved.
PowerShell 5.1 Set-Content -Encoding UTF8 and Windows Notepad write a
UTF-8 BOM. load_dotenv(encoding="utf-8") kept U+FEFF on the first key
name, so the canonical name was absent from os.environ and Hermes looked
unconfigured with no error. utf-8-sig strips the BOM and is a no-op for
BOM-less UTF-8; latin-1 fallback unchanged.
A leftover TERMINAL_ENV in ~/.hermes/.env (written by `hermes setup` or
shell exports) was silently overriding terminal.backend in config.yaml,
so users switching from docker to local saw `hermes config show` agree
with their change while the gateway / cron / batch_runner still ran
against the old backend.
load_hermes_dotenv now re-applies config.yaml's terminal.* values on top
of whatever the .env files set, so the documented source of truth wins
for every entrypoint that goes through the loader.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
The salvaged hydrate_profile_secret_sources (#74549) seeded its
profile-local env from <home>/.env only, but the documented 1Password
bootstrap flow puts OP_SERVICE_ACCOUNT_TOKEN in the gitignored
<home>/.op.env (mirrored from load_hermes_dotenv). A cold profile using
that flow still failed 1Password hydration — the one unaddressed item
from the sweeper review on #74549. Seed .op.env via setdefault so .env
values win; never touches os.environ. Two regression tests.
The salvaged cleanup (#75197) scrubbed every known Hermes key absent from
the profile .env — deleting user-shell-exported credentials
(export OPENAI_API_KEY=...) on every hermes invocation, a documented flow
the author's own failing test_dump_flags_shell_only_key_not_in_dotenv
confirmed. A child process cannot distinguish shell exports from
parent-process leakage, so the scrub now covers ONLY
_PROFILE_MANAGED_ENV_KEYS (ACP routing keys: HERMES_ACP_*,
HERMES_COPILOT_ACP_*, COPILOT_CLI_PATH, COPILOT_ACP_BASE_URL) —
the vector from #75141. Cross-profile credential isolation is owned at
read time by agent.secret_scope.get_secret.
Adds shell-export survival regression + a scope-invariant test that fails
if the scrub set is ever widened toward credential-shaped keys.
Align load_hermes_dotenv() with reload_env() so known Hermes env vars
absent from the active profile .env are removed from os.environ instead
of leaking from a parent process / other profile.
Register ACP-related keys (HERMES_ACP_AUTH_METHOD, HERMES_COPILOT_ACP_*,
COPILOT_CLI_PATH, COPILOT_ACP_BASE_URL) in _EXTRA_ENV_KEYS so they
participate in known-key cleanup.
This is the same isolation gap class as #68367 / #66930, but:
- Not Desktop-only spawn scrub — CLI/gateway restart inheritance
- Not Matrix/messaging auto-enable only — copilot-acp provider/ACP config
- Startup dotenv clear so *any* inheritance path is covered
Example: HERMES_ACP_AUTH_METHOD=cursor_login leaking into a Claude Code
ACP profile caused authenticate -> Internal error -> Discord
'model provider failed after retries'.
Counted with an open()/read_text audit hook on a real 'hermes --version'
run: config.yaml was parsed 3x before load_config() even ran — once by
env_loader._load_secrets_config, once by main.py's early redact/ipv4
bridge (bespoke yaml.load), once by hermes_logging._read_logging_config.
Each raw parse is 1-3 ms with libyaml plus an open/stat — pure
duplication since all three want the same raw dict.
All three now route through read_raw_config()'s existing (mtime_ns,
size)-keyed shared cache:
- env_loader._load_secrets_config: uses the shared reader when reading
the process HERMES_HOME (the cache key's home); other homes (profile
seeding) keep the isolated direct parse. Parse-error isolation is
preserved — the shared reader also swallows errors and returns {}.
- main.py early bridge: drops the bespoke yaml.load for read_raw_config
(managed-scope overlay unchanged).
- hermes_logging._read_logging_config: prefers the shared reader,
falls back to the direct fast_safe_load parse when hermes_cli.config
isn't importable.
Measured: config.yaml opens per 'hermes --version' 3 -> 1.
348 targeted tests green (env_loader + secret sources + applied-homes +
bitwarden + hermes_logging + config).
The .env sanitizer inferred missing newlines from known KEY= substrings
inside existing values. Plain secrets containing those bytes could therefore
be split into synthetic assignments and rewritten to disk.
Treat each physical line as the only assignment boundary and keep bytes after
the first equals sign opaque for boundary discovery. Preserve safe formatting,
null-byte removal, BOM handling, and normal one-assignment-per-line parsing.
Cover direct loading, dotenv loading, sanitization, writers, and migration
with behavioral regressions.
Fixes#29155
Snapshot values applied by external secret sources per resolved HERMES_HOME so a later profile cannot replace an earlier profile scope through shared os.environ.
Keep provider and credential-pool fallback reads on the active secret scope, and fail closed on unscoped multiplex reads.
Tests: scripts/run_tests.sh tests/test_env_loader_secret_sources.py tests/test_env_loader_op_bootstrap.py tests/agent/test_secret_scope.py tests/agent/test_credential_pool.py tests/tools/test_credential_pool_env_fallback.py tests/hermes_cli/test_xiaomi_provider.py tests/cron/test_run_one_job.py tests/hermes_cli/test_api_key_providers.py tests/gateway/test_multiplex_credential_isolation.py -q (395 passed)
_apply_external_secret_sources() added the home to _APPLIED_HOMES before
loading config, so a malformed config.yaml, a missing secrets section, or
all-sources-disabled permanently disabled secret loading for the process
— even after the user fixed the config. Long-lived processes (gateway)
never recovered without a restart.
Now the home is marked only after apply_all() actually ran with at least
one enabled source. Fetch errors still mark the home (so import-time
load_hermes_dotenv() calls don't re-fetch and re-print the same failure
3-5x per startup); the cheap early-exit paths stay retryable.
Fixes#40597.
* feat(secrets): one-command token rotation + actionable startup errors for all secret sources
When a Bitwarden machine-account token expired, users saw a raw Rust
error dump (invalid_client + Location: + backtrace hints) and the only
fix was manually editing .env or re-running the whole setup wizard.
- New `hermes secrets bitwarden token` / `hermes secrets onepassword
token`: paste a new token (masked prompt or flag), the command probes
the backend BEFORE persisting — a rejected token changes nothing; a
good one is written to .env and the fetch caches are cleared.
- New optional SecretSource.remediation(kind, cfg) hook: startup
warnings now print a '→ Run `hermes secrets <name> token`…' fix-it
line after any fetch error, for bundled AND plugin sources (generic
per-ErrorKind defaults in the ABC).
- bws stderr is summarized to its cause line (Location:/backtrace noise
dropped) and invalid_client/invalid_grant/400 identity rejects are
now classified AUTH_FAILED (was INTERNAL) with a plain-English
explanation naming the token env var.
- op whoami probe accepts a candidate token so rotation validates the
NEW credential, not the ambient one.
Additive hook with defaults — no SECRET_SOURCE_API_VERSION bump.
* docs: fix MDX parse error in secret-source-plugin hook table
Escaped backticks around a <name> placeholder made MDX parse it as an
unclosed JSX tag, breaking the docs-site build. Use a plain code span
instead.
Hot-reload and multi-entry load_hermes_dotenv can hit the same UTF-32
file repeatedly; gate the refuse-to-mangle warning on a module-level
seen-set (house style: _WARNED_KEYS sibling) so logs are not spammed.
Notepad "Unicode" saves write UTF-16 with a BOM. The sanitizer decoded
those bytes as utf-8-sig with errors=replace, glued U+FFFD onto the first
key name, stripped NULs, and rewrote the mangled content permanently.
Sniff leading BOMs before any text decode (UTF-32 before UTF-16, because
UTF-32-LE's BOM starts with UTF-16-LE's FF FE). Decode UTF-16 correctly
and rewrite as clean UTF-8. Refuse UTF-32 (leave untouched + warning).
After errors=replace, do not persist a first line that starts with U+FFFD.
Does not touch _load_dotenv_with_fallback (#65124's surface).
The 1Password secret source resolves op:// references using
OP_SERVICE_ACCOUNT_TOKEN read from os.environ. Under systemd the gateway
gets that token via EnvironmentFile, but cron jobs, subprocesses, CLI
runs, macOS launchd, and Docker containers spawn fresh interpreters with
no inherited shell state — so they silently failed to resolve any
reference and fell back to empty strings.
Two patches close the gap, matching Bitwarden's reliability guarantees:
1. env_loader: auto-load ~/.hermes/.op.env after .env so the gitignored
bootstrap token is available everywhere. override=False plus an
explicit guard ensure it never clobbers a token already in env (e.g.
from a systemd EnvironmentFile, which keeps precedence).
2. credential_pool: _get_env_prefer_dotenv() now prefers the resolved
value in os.environ when .env still holds a raw op:// reference,
instead of handing a URL to provider auth. Non-op:// values keep the
existing .env-takes-precedence behaviour.
Also gitignore .op.env, document the three bootstrap-token options, and
add tests covering auto-load, no-override, and the resolved-vs-raw
precedence (plus regression guards).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The startup config/manifest reads used PyYAML's pure-Python SafeLoader,
which is ~8x slower than the libyaml-backed CSafeLoader C extension.
config.yaml is parsed several times during launch (cli config, raw
config, early interface/redaction bridge, logging config) and every
plugin manifest is parsed once — all on the slow path.
Add utils.fast_safe_load (CSafeLoader-preferring, pure-Python fallback,
true drop-in for safe_load) and route the hot startup parse sites
through it: hermes_cli/config.py (config + manifest reads),
hermes_cli/plugins.py (manifest parse), env_loader, cli.load_cli_config,
hermes_logging, and the two pre-config early YAML bridges in main.py.
Behavior is identical (same restricted safe tag set); only speed changes.
safe_load calls on the startup path drop from ~79 to ~0, cutting the
YAML parse cost from ~0.9s to ~0.15s under profiling.
Adds tests/test_fast_safe_load.py asserting equivalence with safe_load
across input shapes, empty-doc falsiness, C-loader preference, and that
python/object tags are still rejected (safe, not full loader).
load_hermes_dotenv now loads the managed-scope .env after user/project .env
and external secret sources, with override=True, so managed env values beat
the user .env and any pre-existing shell export. Reuses the existing dotenv
fallback + credential-sanitization path. Fail-open: no managed dir/.env is a
no-op and any error is swallowed so managed scope never blocks startup.
`load_hermes_dotenv()` is called at module-import time from cli.py,
hermes_cli/main.py, run_agent.py, trajectory_compressor.py, gateway/run.py,
tui_gateway/server.py, acp_adapter/entry.py, and a few others. Each call
triggered `_apply_external_secret_sources()`, which re-parsed config,
re-fetched from Bitwarden Secrets Manager (its own 300s cache mostly absorbed
this), re-ran the ASCII sanitization sweep, and reprinted
Bitwarden Secrets Manager: applied N secret(s) (...)
to stderr. Users saw the status line 3-5x per CLI startup.
Guard the function with a process-level set of HERMES_HOME paths that have
already had external secrets applied. Subsequent calls for the same home_path
are no-ops. `reset_secret_source_cache()` lets tests (and any future
long-running consumer that wants to refresh after a config change) force a
re-pull.
* perf(bitwarden): persist secret-fetch cache across CLI invocations
Every `hermes` invocation paid a ~380ms tax for `bws secret list` to
Bitwarden Secrets Manager because the existing cache was in-process only.
Back-to-back `hermes chat -q`, gateway-spawned agents, and cron-launched
runs all re-fetched.
Adds a disk-persisted L2 cache at `<hermes_home>/cache/bws_cache.json`
(mode 0600, never contains the access token — only the SHA-256
fingerprint prefix). Same TTL as the in-process cache. Read on miss,
write on bws success, ignored on key mismatch / corruption / expiry.
Measured on a startup profile:
load_hermes_dotenv() cold: 372ms → warm (disk cache hit): 20ms
End-to-end `hermes --version` cold→warm: 666ms → ~295ms.
In a hermes-vs-codex benchmark across 11 single- and multi-turn tasks
(framework overhead = wall − llm − tool_exec, median over 3 trials):
cohort before after saved
single-turn (median) 2.96s 2.31s -0.65s
multi-turn (5-turn) 9.40s 8.95s -0.45s (≈0.3s/turn)
Hermes now wins head-to-head on 6/11 tasks vs codex (was 4/11 before).
The remaining ~0.6s single-turn delta is mostly Python's own import
cost in hermes_cli.main, which is a separate optimization.
* perf(cli): lazy-load model catalog + dedupe config.yaml reads at startup
Two import-time wins on top of the bws disk-cache fix:
1. Lazy-load `hermes_cli.models._PROVIDER_MODELS` via PEP 562
module-level `__getattr__`. The catalog is ~55ms of work that was
eagerly imported on every CLI invocation (line 4557 `if not
_is_termux_startup_environment(): from hermes_cli.models import
_PROVIDER_MODELS`). Audit showed every internal call site already
does its own function-local import; only test code reads
`hermes_cli.main._PROVIDER_MODELS` as a module attribute, and
__getattr__ keeps that working transparently. First access triggers
the import once and caches the result on the module via
`globals()[name] = ...`, so subsequent reads are dict lookups.
2. Dedupe the double config.yaml read in the top-of-module bootstrap.
Previously: one raw yaml.safe_load for the `security.redact_secrets`
bridge, then a separate full `load_config()` (with deep-merge) for
`network.force_ipv4`. Both keys come from the same file. Merged
into one raw yaml load.
Combined with the bws cache fix in the previous commit:
hermes --version wall time:
original (cold): 666 ms
after bws fix (warm): 295 ms
after lazy-load + dedupe: 228 ms (-67 ms additional, -66% from original)
Tests:
- tests/hermes_cli/test_api_key_providers.py: 173/173 pass
(lazy __getattr__ correctly handles
`from hermes_cli.main import _PROVIDER_MODELS`)
- tests/test_ipv4_preference.py + tests/hermes_cli/test_redact_config_bridge.py +
tests/agent/test_redact.py: 93/93 pass (dedupe preserves both bridges)
- tests/test_bitwarden_secrets.py + env_loader tests: 49/49 pass
Closes#31370.
bws defaults to the US identity endpoint, so EU Cloud and self-hosted
machine-account tokens fail with [400 Bad Request] {"error":"invalid_client"}
during 'hermes secrets bitwarden setup'. The token is valid — it's just
being checked against the wrong region.
Add a Bitwarden region step to the wizard between the access-token and
project-list steps:
Step 1 Install bws
Step 2 Provide access token
Step 3 Pick region <-- new (US / EU / self-hosted-custom-URL)
Step 4 Pick project (now talks to the right endpoint)
Step 5 Test fetch
Region is stored in config.yaml as secrets.bitwarden.server_url and
plumbed into every bws subprocess as BWS_SERVER_URL (project list,
secret list, test fetch, and the env_loader startup pull).
Also:
- Non-interactive: 'hermes secrets bitwarden setup --server-url ...'
- Pre-existing BWS_SERVER_URL in the shell is detected and reused
- Cache key includes server_url so EU/US fetches don't collide
- 'hermes secrets bitwarden status' shows the configured region
- 'invalid_client' / '400 Bad Request' from bws now triggers a hint
pointing at the region setting instead of looking like a bad token
Null bytes in API key values (introduced by copy-paste) crash
os.environ[k] = v with ValueError: embedded null byte, preventing
hermes from starting at all.
When Bitwarden Secrets Manager supplies a provider key, 'hermes model'
and the setup wizard show 'credentials ✓' with no hint of where the
key came from — identical to the .env case. Users assume the integration
isn't wired up and re-enter the key (or hit Enter and cancel).
env_loader now tracks which env vars were injected by an external secret
source and exposes get_secret_source() / format_secret_source_suffix() so
the provider flows can render 'Anthropic credentials: sk-ant-... ✓
(from Bitwarden)' instead of an unlabeled checkmark.
Wired into _prompt_api_key (kimi, z.ai, minimax, opencode, ...), the
Anthropic provider flow, the Bedrock flow, and the GitHub Copilot token
display.
Future secret sources (Vault, 1Password, etc.) drop in by setting their
own label in _SECRET_SOURCES; format_secret_source_suffix() has a generic
fallback so no call sites need updating.
* feat(secrets): Bitwarden Secrets Manager integration with lazy bws install
Pull API keys from Bitwarden Secrets Manager at process startup
instead of storing them all in plaintext in ~/.hermes/.env. One
bootstrap token (BWS_ACCESS_TOKEN) replaces N per-provider keys, and
rotating a credential becomes a single change in the Bitwarden web
app.
Bitwarden defaults to source of truth: secrets pulled from BSM
overwrite any matching env vars on startup so rotations actually
take effect. Set secrets.bitwarden.override_existing: false in
config.yaml to invert.
The bws binary is auto-downloaded into ~/.hermes/bin/bws on first
use (pinned to v2.0.0, SHA-256 verified against the GitHub release
checksum file). No apt, brew, or sudo required.
New surfaces:
hermes secrets bitwarden setup — interactive wizard
hermes secrets bitwarden status — config + binary + token state
hermes secrets bitwarden sync — dry-run fetch / --apply exports
hermes secrets bitwarden disable — flip enabled: false
hermes secrets bitwarden install — just download the binary
Failures (missing binary, bad token, no network) never block Hermes
startup — they emit a one-line warning to stderr and continue with
whatever credentials .env already had.
Docs: website/docs/user-guide/secrets/{index,bitwarden}.md
Tests: tests/test_bitwarden_secrets.py (26 tests, hermetic — bws
subprocess and HTTP downloads fully mocked)
* chore(infographic): add bitwarden-secrets-manager bento-grid retro-pop-grid
Generated for PR #30035 — Bitwarden Secrets Manager integration.
Style picked via pick_pr_infographic_style.py rotation:
layout: bento-grid
style: retro-pop-grid
aspect: 1:1 square
Saved at infographic/bitwarden-secrets-manager/infographic.png
## Why
Hermes supports Linux, macOS, and native Windows, but the codebase grew up
POSIX-first and has accumulated patterns that silently break (or worse,
silently kill!) on Windows:
- `os.kill(pid, 0)` as a liveness probe — on Windows this maps to
CTRL_C_EVENT and broadcasts Ctrl+C to the target's entire console
process group (bpo-14484, open since 2012).
- `os.killpg` — doesn't exist on Windows at all (AttributeError).
- `os.setsid` / `os.getuid` / `os.geteuid` — same.
- `signal.SIGKILL` / `signal.SIGHUP` / `signal.SIGUSR1` — module-attr
errors at runtime on Windows.
- `open(path)` / `open(path, "r")` without explicit encoding= — inherits
the platform default, which is cp1252/mbcs on Windows (UTF-8 on POSIX),
causing mojibake round-tripping between hosts.
- `wmic` — removed from Windows 10 21H1+.
This commit does three things:
1. Makes `psutil` a core dependency and migrates critical callsites to it.
2. Adds a grep-based CI gate (`scripts/check-windows-footguns.py`) that
blocks new instances of any of the above patterns.
3. Fixes every existing instance in the codebase so the baseline is clean.
## What changed
### 1. psutil as a core dependency (pyproject.toml)
Added `psutil>=5.9.0,<8` to core deps. psutil is the canonical
cross-platform answer for "is this PID alive" and "kill this process
tree" — its `pid_exists()` uses `OpenProcess + GetExitCodeProcess` on
Windows (NOT a signal call), and its `Process.children(recursive=True)`
+ `.kill()` combo replaces `os.killpg()` portably.
### 2. `gateway/status.py::_pid_exists`
Rewrote to call `psutil.pid_exists()` first, falling back to the
hand-rolled ctypes `OpenProcess + WaitForSingleObject` dance on Windows
(and `os.kill(pid, 0)` on POSIX) only if psutil is somehow missing —
e.g. during the scaffold phase of a fresh install before pip finishes.
### 3. `os.killpg` migration to psutil (7 callsites, 5 files)
- `tools/code_execution_tool.py`
- `tools/process_registry.py`
- `tools/tts_tool.py`
- `tools/environments/local.py` (3 sites kept as-is, suppressed with
`# windows-footgun: ok` — the pgid semantics psutil can't replicate,
and the calls are already Windows-guarded at the outer branch)
- `gateway/platforms/whatsapp.py`
### 4. `scripts/check-windows-footguns.py` (NEW, 500 lines)
Grep-based checker with 11 rules covering every Windows cross-platform
footgun we've hit so far:
1. `os.kill(pid, 0)` — the silent killer
2. `os.setsid` without guard
3. `os.killpg` (recommends psutil)
4. `os.getuid` / `os.geteuid` / `os.getgid`
5. `os.fork`
6. `signal.SIGKILL`
7. `signal.SIGHUP/SIGUSR1/SIGUSR2/SIGALRM/SIGCHLD/SIGPIPE/SIGQUIT`
8. `subprocess` shebang script invocation
9. `wmic` without `shutil.which` guard
10. Hardcoded `~/Desktop` (OneDrive trap)
11. `asyncio.add_signal_handler` without try/except
12. `open()` without `encoding=` on text mode
Features:
- Triple-quoted-docstring aware (won't flag prose inside docstrings)
- Trailing-comment aware (won't flag mentions in `# os.kill(pid, 0)` comments)
- Guard-hint aware (skips lines with `hasattr(os, ...)`,
`shutil.which(...)`, `if platform.system() != 'Windows'`, etc.)
- Inline suppression with `# windows-footgun: ok — <reason>`
- `--list` to print all rules with fixes
- `--all` / `--diff <ref>` / staged-files (default) modes
- Scans 380 files in under 2 seconds
### 5. CI integration
A GitHub Actions workflow that runs the checker on every PR and push is
staged at `/tmp/hermes-stash/windows-footguns.yml` — not included in this
commit because the GH token on the push machine lacks `workflow` scope.
A maintainer with `workflow` permissions should add it as
`.github/workflows/windows-footguns.yml` in a follow-up. Content:
```yaml
name: Windows footgun check
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: {python-version: "3.11"}
- run: python scripts/check-windows-footguns.py --all
```
### 6. CONTRIBUTING.md — "Cross-Platform Compatibility" expansion
Expanded from 5 to 16 rules, each with message, example, and fix.
Recommends psutil as the preferred API for PID / process-tree operations.
### 7. Baseline cleanup (91 → 0 findings)
- 14 `open()` sites → added `encoding='utf-8'` (internal logs/caches) or
`encoding='utf-8-sig'` (user-editable files that Notepad may BOM)
- 23 POSIX-only callsites in systemd helpers, pty_bridge, and plugin
tool subprocess management → annotated with
`# windows-footgun: ok — <reason>`
- 7 `os.killpg` sites → migrated to psutil (see §3 above)
## Verification
```
$ python scripts/check-windows-footguns.py --all
✓ No Windows footguns found (380 file(s) scanned).
$ python -c "from gateway.status import _pid_exists; import os
> print('self:', _pid_exists(os.getpid())); print('bogus:', _pid_exists(999999))"
self: True
bogus: False
```
Proof-of-repro that `os.kill(pid, 0)` was actually killing processes
before this fix — see commit `1cbe39914` and bpo-14484. This commit
removes the last hand-rolled ctypes path from the hot liveness-check
path and defers to the best-maintained cross-platform answer.
Extract the islink/realpath guard from the 16743 fix into a single
atomic_replace() helper in utils.py, then migrate every os.replace()
call site in the codebase to use it.
The original PR #16777 correctly identified and fixed the bug, but
only patched 9 of ~24 call sites. The same bug class (managed
deployments that symlink state files silently losing the link on
every write) still existed at auth.json, sessions file, gateway
config, env_loader, webhook subscriptions, debug store, model
catalog, pairing, google OAuth, nous rate guard, and more.
Rather than add another 10+ copies of the same three-line guard,
consolidate into atomic_replace(tmp, target) which:
- resolves symlinks via os.path.realpath before os.replace
- returns the resolved real path so callers can re-apply permissions
- is a drop-in replacement for os.replace at the use sites
Changes:
- utils.py: new atomic_replace() helper + atomic_json_write /
atomic_yaml_write now call it instead of inlining the guard
- 16 files: all os.replace() call sites migrated to atomic_replace()
- agent/{google_oauth, nous_rate_guard, shell_hooks}.py
- cron/jobs.py
- gateway/{pairing, session, platforms/telegram}.py
- hermes_cli/{auth, config, debug, env_loader, model_catalog, webhook}.py
- tools/{memory_tool, skill_manager_tool, skills_sync}.py
Tests: tests/test_atomic_replace_symlinks.py pins the invariant for
atomic_replace + atomic_json_write + atomic_yaml_write, covers plain
files, first-time creates, broken symlinks, and permission preservation.
Refs #16743
Builds on #16777 by @vominh1919.
Load-time sanitizer silently removed non-ASCII codepoints from any
env var ending in _API_KEY / _TOKEN / _SECRET / _KEY, turning
copy-paste artifacts (Unicode lookalikes, ZWSP, NBSP) into opaque
provider-side API_KEY_INVALID errors.
Warn once per key to stderr with the offending codepoints (U+XXXX)
and guidance to re-copy from the provider dashboard.
API keys containing Unicode lookalike characters (e.g. ʋ U+028B instead
of v) cause UnicodeEncodeError when httpx encodes the Authorization
header as ASCII. This commonly happens when users copy-paste keys from
PDFs, rich-text editors, or web pages with decorative fonts.
Three layers of defense:
1. **Save-time validation** (hermes_cli/config.py):
_check_non_ascii_credential() strips non-ASCII from credential values
when saving to .env, with a clear warning explaining the issue.
2. **Load-time sanitization** (hermes_cli/env_loader.py):
_sanitize_loaded_credentials() strips non-ASCII from credential env
vars (those ending in _API_KEY, _TOKEN, _SECRET, _KEY) after dotenv
loads them, so the rest of the codebase never sees non-ASCII keys.
3. **Runtime recovery** (run_agent.py):
The UnicodeEncodeError recovery block now also sanitizes self.api_key
and self._client_kwargs['api_key'], fixing the gap where message/tool
sanitization succeeded but the API key still caused httpx to fail on
the Authorization header.
Also: hermes_logging.py RotatingFileHandler now explicitly sets
encoding='utf-8' instead of relying on locale default (defensive
hardening for ASCII-locale systems).
