--- sidebar_position: 15 title: "CLI Internals" description: "How hermes_cli is shaped: slash dispatch, config loaders, the skin engine, the transactional update pipeline, and process-identity rules" --- # CLI Internals Companion to `hermes_cli/AGENTS.md` (the rules) — this page holds the longer explanations. ## Update pipeline User-facing behaviour (receipts, `--plan`, snapshot modes) is in [Updating](../getting-started/updating.md); `hermes_cli/AGENTS.md` carries the short form of the rules. Fleet-update campaign #91277 (Aug 2026). A PR that weakens a stage must answer for the failure class it guards. `plan → snapshot → apply → restart-per-kind → verify → report` - **Plan** (`update_inventory.py`, `hermes update --plan`): read-only inventory — install kind, all profiles, every live gateway with supervisor + running code version. Deployment kinds are first-class: `git` updates in place; `docker`/`nix`/`apt` are NOT in-place-updatable and the updater reports the correct external command instead of fighting the deployment model. - **Snapshot** (`backup.py`): pre-update quick snapshot for EVERY profile (the code swap + fleet restart touch all of them), each into its own `state-snapshots/`, identical file set, 1 GiB per-file cap, keep=1. **Never add a partial/tiered snapshot set** — mixed coverage creates torn-restore states across schema generations. Quick snapshots are FILE-LOSS RECOVERY (the per-profile cron-jobs safety net restores from them), NOT code-rollback insurance; `--backup` full mode owns rollback. - **Apply**: git pull, or the Windows ZIP fallback — which fires ONLY when git itself failed (`_should_zip_fallback_on_update_error`, argv-classified; a dependency-install failure must never trigger a tree-clobbering re-download), REFUSES a dirty working tree (`-uall` + a pre-swap TOCTOU re-check — but classifies a `!!` line by whether the swap would destroy it: an ignored path under a root entry the ZIP does not ship (`.bytecode-fingerprint`, `.hermes-bootstrap-complete`, `hermes_agent.egg-info/`; tracked root entries stand in for the ZIP set before the download, the re-check gets the real one), a nested `__pycache__`/`node_modules`, or a `_ZIP_PRESERVED_NESTED` output is admitted; other ignored files under shipped dirs still block), and grafts the live nested build outputs (`_ZIP_PRESERVED_NESTED`: `apps/desktop/{release,dist,node_modules,build}`, `hermes_cli/web_dist`, `ui-tui/{dist,node_modules,packages/hermes-ink/dist}`, `web/node_modules`, `scripts/whatsapp-bridge/node_modules`) into the staged swap by hardlink (the GitHub source ZIP has none of them; without the graft the swap deletes them). Post-swap, the Desktop rebuild decision also trusts the build stamp under HERMES_HOME, so an install that already lost its artifacts in an earlier update is rebuilt instead of "forgotten" (#90495). - **Restart-per-kind**: systemd and launchd restarts are FLEET-WIDE within the updating install (every `hermes-gateway*` unit / `ai.hermes.gateway*` LaunchAgent whose home is the updating root or one of its `profiles/`), drain-first (SIGUSR1), with per-unit/per-label failure isolation. Restarting only the invoking profile's service leaves siblings on stale `sys.modules` until they crash — the largest dupe-PR cluster in the repo's history came from that bug. The fleet is bounded by HOME, not by namespace: `hermes_cli/update_fleet_scope.py` judges every unit/label/process by the home it actually runs on (live environ, unit `Environment=`, plist `HERMES_HOME`), and a runtime of another `HERMES_HOME` on the same account — a sibling install, the real `hermes-gateway.service` seen from a scratch home — is named and left alone, never restarted (#93349). - **Verify**: gateways stamp `code_sha`/`code_version` into `gateway_state.json` on every runtime-status write (`gateway/status.py`); the updater compares each live gateway against the fresh checkout and prints a fleet version matrix. After the commit point (the tree moved) nothing fails `hermes update` (exit 0 unless the code was rolled back): a provably-stale or stopped gateway, a failed build/launcher/config migration/bytecode sweep/channel adoption/Windows resume each print `⚠`, land in the receipt as `outcome: "success"` with `followups: [{step, reason}]`, and keep their own obligation armed (`source-completion-pending` for the tail, the fleet-restart obligation for gateways) so the next launch or `hermes update` — including "Already up to date" — retries them. The owed-fleet-restart CLI startup warning is what keeps a mixed-version fleet from ever looking healthy. A completion process that could not be started (temporary directory, request file, spawn) or died without a correlated result (OOM, SIGKILL) is the same kind of debt: an owed `completion` follow-up with the source-update tail re-armed, exit 0. Exit 2 (refused/concurrent) and exit 1 (nothing committed / rolled back) keep their meaning; the one committed run that exits 1 is an autostash whose restore conflicted and stays parked: the receipt is `partial` with a `user_action`, since nothing retries what only the user can re-apply (#122557). - **Report**: every run writes a machine-readable receipt to the ROOT home's `logs/update_receipts/` (never a sticky profile's; `update.log` likewise) — `latest.json` pointer; steps, skips WITH reasons, restart outcome, plan, fleet snapshot, `followups`. The receipt is on disk as `outcome: "running"` from the start and refreshed at each stage boundary; the next update marks a `running` record whose processes are gone `interrupted` and says so. Readers (`hermes logs update`, debug bundle, dashboard status, pm receipts) use the root home too. A dependency-sync failure in the completion bootstrap is a `dependencies` follow-up (A6), and Ctrl-C after the commit point finalizes `interrupted` (exit 130), never `failed`. Before a source swap, the parent captures plan/snapshots/receipt and its Windows pause token. `update_completion.py` runs new-code PM preparation with site initialization disabled, then selected-Python builds, maintenance, scans/restarts and verification. Git/current/ZIP share this owner; never reload or purge modules to continue in the old interpreter. The parent keeps the lock, waits, and accepts only a correlated terminal result. `cmd_update` still finalizes early failures and missing/killed-child outcomes; PM refusal data survives the handoff. See `website/docs/developer-guide/source-update-completion.md`. A begun-but-unwritten receipt is a bug. - **Nothing runs pulled code in the pre-pull interpreter.** This tree finishes updates through `update_completion.run_completion` / `_update_takeover.py`. `hermes_cli/update_handoff.py` and `hermes_cli/update_serve_obligations.py` are the FROZEN COMPAT SURFACE for releases that lazily import those module names from the NEW tree after the checkout swap. Keep their public names importable and behavior-preserving. They call into `_old_updater.stop_for_relaunch` → `_run_child`. Removing a name bricks every release mid-update. see `tests/compat/old_updater_surface.json`. Process-scan coordination between updater, serve/dashboard, and gateway is being replaced by a gateway-owned control socket (#92091); scans are the fallback layer for old/crashed processes — read #92091 before adding any heuristic. Process identity rules (never argv substrings; canonical matchers; parser-derived flag sets; never blanket-exclude gateway ancestors, #87594): root `AGENTS.md` and `website/docs/developer-guide/cli-internals.md`. The systemd blunt-restart fallback waits for the unit's `TimeoutStopUSec` plus `TimeoutStartUSec`, with 15 seconds of client-side slack. It reads the target unit in the same manager scope as the restart; both the initial attempt and retry use this budget, including the catch-up restart after an interrupted update. A start after a graceful drain uses only the start budget plus slack. A missing, unparseable, or infinite phase limit falls back to 90 seconds for that phase, keeping unattended updates bounded. Timing out the `systemctl` client does **not** cancel the manager's transaction. Custom multi-command stop chains or `EXTEND_TIMEOUT_USEC` can still outlast this estimate; a real timeout remains an incomplete restart, and successful commands still require the existing service-health and fleet-version verification. Raw numeric `*USec` values are microseconds, while formatted values use systemd's fixed units, including days, weeks, months and years. The combined timeout is capped below the native signed 32-bit millisecond poll limit (with rounding headroom), so exceptionally long unit limits cannot overflow subprocess polling. Zero/unknown/infinite phase limits use the bounded fallback. This does not change active-turn drain settings. ## Nous free tier sign-in Sign-in completion is one function, `settle_after_upgrade`, called by every caller that persists an account over a free-tier identity (CLI `upgrade_guest`, the desktop poller): it moves a config on the welcome route to the account's host and the tier's recommended default (`models.recommended_nous_default_model`, shared with `GET /api/model/recommended-default`). The shared flow, states, and copy live in `anon_sign_in.py`; CLI rendering lives in `anon_sign_in_cli.py`. `anon_auth.py` keeps identity, promotion polling, and settlement, and re-exports the existing sign-in API. The flow resolves identity and persistence collaborators through `anon_auth` at call time to preserve module-attribute monkeypatch seams. The sign-in itself is one composition: `anon_auth.run_sign_in()` yields `SignInState`s (`Code`, `Waiting`, `Completed`, `Declined`, `Superseded`, `TimedOut`, `Retired`, `Failed`, `AlreadySignedIn`, `Unavailable`). It reads the current state itself, holds one absolute deadline across both waits, persists only after a completed promotion **and** a token grant, runs `settle_after_upgrade` exactly once per completion, and never lets a persist or settle failure escape as an exception — it becomes `Failed`. Every state carries its own `.copy` (the chat form, which never contains a raw exception, a URL or a `hermes` verb) and `.copy_terminal`, so no caller maps a reason to a string. `cancelled()` stops an attempt; `cancel_wins_after_promotion` decides what happens when the server had already completed the transfer — the desktop keeps `True` (a DELETE means "not on this machine"), the gateway passes `False` (a supersede must not discard a transfer the user actually approved). `scope` is entered only around the precondition and persist blocks, never across a `yield` or a network wait, because `run_in_executor` does not carry contextvars. `upgrade_guest` (`hermes auth upgrade`), the CLI `/login` handler and the desktop promotion poller are renderers over it; a surface that needs the cancel check and the save to be atomic passes `persist_guard`. The desktop's plain "connect another Nous account" device-code login is a separate path (`_nous_plain_poller`) and must stay one. ## Process identity: never infer it from argv substrings The bug class behind ~10 fleet-update issues (#90778, #87594, #78089, #76129, #91964, ...): classifying a process by `"serve" in cmdline` or similar. `kanban --preserve-cache` contains "serve"; a flag VALUE can equal a subcommand (`-m dashboard serve`); truncated cmdlines hide the real subcommand. Rules: - Use the canonical matchers: `gateway.status.looks_like_gateway_command_line` (gateway run), `hermes_cli.update_cmd._hermes_holder_subcommand` (top-level subcommand of any Hermes argv). Never hand-roll token scans. - Flag sets must be DERIVED from the parser (`_holder_value_flags()` introspects `build_top_level_parser()`), never hand-written lists — they drift. - Never blanket-exclude ancestors from process scans: when `/update` runs as the gateway's child, a gateway ancestor must stay visible to the pause machinery (#87594). Exclude interactive ancestry, carve out gateway-shaped ancestors. - Match on FULL cmdlines; truncate only at display time (#78089). - Before adding any new scan heuristic, read #92091 — the gateway control socket replaces scans as the primary coordination mechanism; scans are the fallback layer for old/crashed processes. ## Skin engine — what skins customize | Element | Skin key | Used by | |---|---|---| | Banner panel border / title / section headers / dim / body | `colors.banner_border`, `banner_title`, `banner_accent`, `banner_dim`, `banner_text` | `banner.py` | | Response box border | `colors.response_border` | `cli.py` | | Spinner faces (waiting / thinking) | `spinner.waiting_faces`, `spinner.thinking_faces` | `display.py` | | Spinner verbs / wings (optional) | `spinner.thinking_verbs`, `spinner.wings` | `display.py` | | Tool output prefix / per-tool emojis | `tool_prefix`, `tool_emojis` | `display.py` → `get_tool_emoji()` | | Agent name / welcome / response label / prompt symbol | `branding.agent_name`, `welcome`, `response_label`, `prompt_symbol` | `banner.py`, `cli.py` | Built-in skins (`_BUILTIN_SKINS` in `hermes_cli/skin_engine.py`): `default` (classic gold/kawaii), `ares` (crimson/bronze with custom spinner wings), `mono` (grayscale), `slate` (cool blue). Add a built-in as a dict entry `{"name", "description", "colors", "spinner", "branding", "tool_prefix"}`. User skins are `~/.hermes/skins/.yaml` with the same keys, activated with `/skin ` or `display.skin: `; the full YAML template is in the [Skins & Themes](../user-guide/features/skins.md) user guide. ## Profiles: multi-instance support Hermes supports profiles — fully isolated instances, each with its own `HERMES_HOME` (config, API keys, memory, sessions, skills, gateway). For single-profile commands (`hermes -p x `), `_apply_profile_override()` in `hermes_cli/main.py` sets `HERMES_HOME` before any module imports, so every `get_hermes_home()` reference scopes to the active profile. The multiplex gateway and the Desktop/dashboard `serve` backend serve several profiles from one process instead: the active profile is a contextvar override bound per activity, `os.environ["HERMES_HOME"]` stays the launch profile's, and a module-level constant derived from the home freezes to that launch profile (see [Gateway Internals § Multiplexed profiles](./gateway-internals.md#multiplexed-profiles)). Profile operations are HOME-anchored (`_get_profiles_root()` returns `Path.home() / ".hermes" / "profiles"`, not `get_hermes_home() / "profiles"`) so `hermes -p coder profile list` sees all profiles regardless of which one is active — intentional. Profile-safe coding rules are in the root `AGENTS.md`; multiplex secret-scope rules in `gateway/AGENTS.md`.