The PM rework dropped the resolver's "existing hermes on PATH" rung, so
Desktop only accepts HERMES_DESKTOP_HERMES or $HERMES_HOME/hermes-agent. A
healthy install elsewhere (install.sh --dir, a setup-hermes.sh clone) now
falls to bootstrap-needed and the first-run choice, offering to install a
second copy. The old rung also read raw process.env.PATH, which a
Finder/Dock launch inherits without ~/.local/bin.
Add a rung before bootstrap-needed: probe the launcher published at its
fixed user-bin location (~/.local/bin or $HERMES_HOME/bin; Windows:
%LOCALAPPDATA%\hermes\bin), read the install directory its --version
reports, and resolve that root through resolveSourceInstallationBackend,
the same validated path ACTIVE_HERMES_ROOT uses. HERMES_DESKTOP_IGNORE_EXISTING
and repair requests still skip it, and a miss is now logged instead of
silent. The launcher lookup is shared with resolveInstallationLauncher.
Fixes#73014
Co-authored-by: JonthanaHanh <92574114+JonthanaHanh@users.noreply.github.com>
All ssh spawns (the -G config probe, the ssh-config:resolve IPC, SSH
terminals, and SshConnection) now go through one pure resolver:
desktop.ssh_path in config.yaml, then System32 OpenSSH, then Git for
Windows' usr\bin\ssh.exe, then bare ssh. Non-Windows keeps bare ssh.
A failed local ssh -G probe is now tagged terminal: boot latches on the
failure overlay (naming the client and the ssh_path setting) instead of
re-driving the same probe every ~2s.
Fixes#103288
Review round 3 (Gemini 3.8 Flash): counting every planned gateway row per profile
made a dirty baseline exit 1 even when the fleet was fully accounted for. With an
orphan row the restart phase killed (killed_pids) plus an active row that was
restarted, the killed row's own verdict ("stopped") was already in hand, yet its
count still marked the profile ambiguous and blocked credit for the row that did
restart.
The baseline count now skips rows bookkeeping already resolves (killed /
relaunched / name-matched) and counts only rows that still need a verdict, so the
ambiguity guard fires on genuine ambiguity instead of on any multi-row plan. The
debug message no longer claims a successor exists when the probe returned none.
Review round 3 (@GPT-OSS 120B): `kind not in _SERVE_KINDS` was read as
"excludes gateways" — the set is ("serve", "dashboard"), so the behaviour was
already gateway-only (a profile with a gateway plus a serve and a dashboard row
still credits the gateway), but the intent is now explicit: `kind == "gateway"`.
The served-kind test covers both non-gateway kinds for one profile to pin that.
Also documents what this fallback deliberately does NOT do, so the next reader
does not have to re-derive it:
- ambiguity resolves conservatively (pairing a successor to the planned process
needs a start-time identity neither the plan record nor the fleet row carries);
- credit is not gated on restart bookkeeping (the reported failure already had
bookkeeping; a self-respawned gateway has none, while the fleet row still
proves the successor runs the new code).
Review (@ehz0ah): one live successor cannot prove that two planned same-profile
gateways both restarted. `_collect_gateway_runtimes()` de-duplicates by PID, not
by profile, so duplicate/orphan states can put two `coder` rows in the plan —
and while the post-restart fleet probe publishes at most one row per profile,
both planned rows were credited from that single successor, letting an untouched
sibling vanish behind it and suppressing the unaccounted tripwire.
Successor credit now requires the profile to hold exactly ONE planned gateway
runtime (baseline identity); an ambiguous profile keeps the name-path verdict and
stays armed. Logged at debug so the reason is visible.
Reported repro, before -> after:
plan=[coder 76508, coder 76795], live_gateway_pids={"coder": {76796}},
empty bookkeeping
before: ['restarted', 'restarted'] tripwire: False
after: ['unaccounted', 'unaccounted'] tripwire: True
Review feedback (@kvnloo): when the post-restart fleet probe has no row for a
planned gateway's profile, the incarnation fallback silently does nothing and
the tripwire output reads identically to a genuinely missed restart. Log the
missing evidence at debug so the next report is diagnosable:
No post-restart gateway evidence for profile 'coder' (planned pid 76508 via
launchd) — reconciling on restart bookkeeping alone
Also make the call-site comment name the snapshot dependency the evidence
rests on (the same note #110343 carries at its wiring point).
`match_runtime_outcomes()` can only bridge a planned gateway runtime and the
restart bookkeeping by SERVICE NAME, and the two disagree whenever a service
serves a profile its name does not encode:
- macOS: the root-home LaunchAgent `ai.hermes.gateway` honours the install's
sticky active profile (`hermes profile use <name>`), so the process declares
`profile: coder` on its control socket while its label encodes the install
root. `_gateway_service_matches_profile("coder", "ai.hermes.gateway")` is
False for every naming rule, so the planned runtime is reported
`unaccounted` → `restart.incomplete` → exit 1 after a restart that verified
clean (fleet row: new PID, new SHA). The leftover `fleet_restart_pending`
marker then warns on every later invocation.
- Linux: a hash-suffixed unit for a custom HERMES_HOME has the same shape of
problem (#110238), which the open name-matching PRs only paper over for the
default profile.
Reconcile gateways on incarnation evidence too — the rule serve/dashboard
runtimes already use via `stale_serve_pids`: the post-restart fleet snapshot
(control-socket identity per profile) is passed in as `live_gateway_pids`, and
a planned PID that is gone while a gateway answers for the same profile counts
as `restarted`. Requiring a successor keeps the tripwire honest: no successor,
or the planned PID still answering, stays `unaccounted` and still exits 1.
`down` rows are excluded from the evidence — they report the pre-restart PID.
Fixes#110333
- _TS_RE now accepts the T-separated stamps update.log and
desktop-update-handoff.log actually write (posix.sh date +%Y-%m-%dT%H:%M:%S%z,
windows.ps1 yyyy-MM-ddTHH:mm:ssK, the update banner), so --since filtering
works for hermes logs update/handoff instead of silently dropping everything.
- test_logs' LOG_FILES stamp contract gains real writer samples for update/handoff.
- Replace the PR's source-shape (inspect.getsource) upload tests with behavioral
ones (monkeypatched uploader, --local capture); the upload/--local loops stay
data-driven over bundle.items() as on main.
When a Desktop-driven update fails at the Electron rebuild (updater
exit 6, 'Code and dependencies updated, but the Desktop app REBUILD
FAILED'), the only artifacts holding the root cause are
~/.hermes/logs/update.log (full stdout/stderr mirror of hermes update,
written by _UpdateOutputStream) and ~/.hermes/logs/desktop-update-handoff.log
(hand-off stages including the 'desktop --force-build --build-only' retry).
Both the updater's failure messages and the Desktop error box direct
users to 'hermes debug share', but the share captured neither file:
LOG_FILES and the debug.py capture chain covered only the classic
five logs, so update-failure issues (e.g. #100874) arrived with the
diagnostics that matter most missing.
Additionally, 'hermes logs list' (a directory scan) already listed
both files while LOG_FILES had no readable key for them, so
'hermes logs update' failed with 'Unknown log' against a file the
CLI itself displays.
Wire both files through the whole chain:
- hermes_cli/logs.py: add 'update' and 'handoff' LOG_FILES keys
(fixes list-vs-read inconsistency; tail/follow/filters come free).
- hermes_cli/debug.py: _capture_default_log_snapshots,
collect_debug_report (512 KB snapshot cap and the 100-line tail
budget both apply; a hand-built snapshot dict without the new keys
degrades to the old five-section report instead of KeyError),
collect_share_bundle full logs, the paste-upload label list in
build_debug_share, and the --local print list. The --nous bundle
and privacy notice inherit the new keys; force-redaction applies
via _capture_log_snapshot as for every other log.
- hermes_cli/subcommands/logs.py + website docs: name the new logs.
Tests (tests/hermes_cli/test_debug_share_update_logs.py): 12 new
tests — snapshot keys present with real content, missing files
degrade to the honest note without raising, report sections appear,
bundle keys omitted when absent, upload/--local label lists include
the new logs, LOG_FILES keys resolve, tail_log end-to-end. Sabotage
pass: 10 of 12 fail without the fix.
The windows_ssh_runtime `write-lock` op takes the ownership ID as its one
argv element and reads the lock JSON on stdin, but the atomic Windows
spawn command appended $lock as a second positional argument, so the
runtime's arity check rejected the spawn with `invalid operation` and a
Windows SSH backend could never claim ownership of its process
(#119111).
Pipe $lock into the helper's stdin instead, keeping the ownership ID
argument, non-zero-exit propagation and $spawnLines|Write-Output intact.
Fixes#119111
Co-authored-by: KoNit-K <konit.block@protonmail.com>
The font wait resolves null when host.isConnected is false at an await
boundary; the mount handler returned silently, so a pane rendered before
its host connected stayed blank forever — no term.open, no PTY spawn, no
log line, no retry (#118004). Poll frames until the host connects, then
retry the wait+mount once; cancel the watch on teardown. Same fix for the
agent mirror terminal, which shares the single-shot mount shape.
Two bugs that together made an SSH-primary Desktop show "This device" and
spawn a phantom local backend even though it was connected to the remote.
1. The Gateways form serialised a cleared Identity file (and User) as
`undefined`. Main merges the SSH settings with `??`, where `undefined`
means "inherit the saved value" and only '' clears, so a stale keyPath
could never be removed: the form refilled it on every Save and reconnect.
2. reconcileRegistryDrift treated a v1 SSH route as "already registered" on
host/user/port alone, while matchingConnectionId tags the live window by
the full route identity (keyPath, remoteHermesPath, remoteProfile too).
A v1 keyPath the registered entry never carried therefore resolved to no
connectionId; the renderer treats an untagged window as the unscoped
local backend, labels it "This device", and the roster force-spawns a
local child the user never asked for. The heal now aligns the registered
entry's identity fields with the route the app actually dials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Ai-assisted: true
The renderer's 60s keepalive (touchPoolBackend) refreshes lastActiveAt
for every OPEN chat — the only clock both the idle reaper and LRU
eviction read — so the pinned tier (backends whose profile chat was ever
opened) was exempt from both the maxBackends cap and the idle reaper
forever: N profile chats held N ~120 MB serve children until app quit
(126 processes / 7.5 GB, all parented to the live Hermes.exe) (#105239).
A keepalive proves the chat is open, not that anything streamed.
- touchPoolBackend stamps entry.lastStreamedAt on activeTurn=true
touches (a prompt turn leasing the backend IS streamed activity).
- pool-retire: PoolRetireEntry gains lastStreamedAt; retireIdle accepts
an idle-eligibility override so the reaper can key on the streamed
clock. Every other safeguard is unchanged: activeTurn veto, identity
checks, admission permit, backend proof probe.
- startPoolIdleReaper retires a local child whose last streamed turn is
older than POOL_PINNED_IDLE_MS (default 60 min,
HERMES_DESKTOP_POOL_PINNED_IDLE_MS override) even while keepalive-fresh;
entries without the stamp keep the legacy lastActiveAt clock. Killing
is never user-observable: re-focusing the chat re-ensures idempotently
via ensureBackend/ensureRegistryBackend entry reuse (#105239 req 3).
Note: #118246 already routes local profiles through one host backend;
this TTL bounds what still pools per-profile (isolated mode, unscopable
mutating REST requests, remote profiles' local descriptors aside).
The WS-orphan reap interrupts the turn (deny-resolving the pending approval
queue) and teardown pops it via unregister_gateway_notify — both silently.
A reconnecting client's parked prompt card then answers resolved: 0 (or a
bare 4001 on approval.pending), so the prompt looked lost rather than
cancelled and the user had no signal the command would never run (#106678).
- session_lifecycle: _announce_cancelled_gateway_approvals broadcasts the
dropped entries (session ids, reason, request ids/count) before the
deny-resolve drains the queue, from both _interrupt_session_turn and
_teardown_session. Fail-open: a missing key, empty queue, or broadcast
failure never blocks the deny-resolve or teardown.
- _interrupt_session_turn gains orphan= so the reaper labels its cancels
ws_orphan_reap explicitly (never request_id prefix sniffing).
- contracts: ApprovalCancelledPayload event model + regenerated TS/OpenRPC.
- desktop: gateway-event lifecycle consumes approval.cancelled and clears
the parked prompt cards by request id (mismatch is a no-op, so a newer
prompt survives); the turn's BLOCKED tool result is the in-transcript
signal, same as the timeout path.
Supersedes #106693 (KoNit-K) and #126455 (Finn763); design credit to both.
Co-authored-by: KoNit-K <konit.block@protonmail.com>
Fixes#106678
An inline agent-generated image (FAL CDN URL) that the Desktop client
cannot reach renders as a collapsed frame with no usable fallback
(#74564): resolveMediaDisplaySrc passes inline https sources through
untouched, and a failed <img> load only records the broken-key.
- Gateway: GET /api/media/proxy fetches an allowlisted remote image URL
on the client's behalf (the gateway generated the image and holds a
working route to the CDN) and returns the same data_url shape as
/api/media. Host-allowlist (fal.media/fal.run/storage.googleapis.com
and subdomains), 20s timeout, image content-type check, 25MB cap.
- Renderer: gatewayImageProxyDataUrl wraps the new endpoint with
owner-connection/profile pinning; useMediaImage retries a failed
inline https load ONCE through the proxy and swaps in the returned
data URL. The retry is latched per source: a failing proxied data URL
or a remount never loops, and non-https sources never reach it.
Gated mode accepted only browser-minted ws-tickets and server-internal
credentials, but a token-mode Remote desktop connection bakes its
stored session token into ?token= (buildGatewayWsUrl) — so the WS
upgrade to /api/ws 403'd forever against a password-gated server even
though the same token authenticates REST fine. The REST bearer leg
already verifies session tokens via _verify_access_token; the WS leg
was the only inconsistent gate.
Gated ?token= is now verified against the dashboard auth session
providers (the same verify_session seam), stamps {user_id, provider}
identity, and audits success/failure (TOKEN_AUTH_SUCCESS/FAILURE). The
legacy in-process _SESSION_TOKEN stays rejected; a ProviderError (all
providers unreachable) rejects with token_unavailable instead of
crashing the upgrade; the bot-desktop display-ticket guard is kept.
Co-authored-by: salch-cred <141555468+salch-cred@users.noreply.github.com>
The fixture's _find_live_session_by_key/_child_run_active stubs predate the
profile_home parameter; the profile_name assertion compared against a bare
create_session (NULL) instead of the production stamp
(profile_name_for_home(...) or _current_profile_name()) that
_ensure_session_db_row applies on first-prompt persistence — the exact
contract materialization must match.
Co-authored-by: ruangraung <github@nadyahermes.anonaddy.com>
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
A Desktop/TUI session created from the client can become permanently
unresumable if the backend dies or is restarted before the session's
first prompt. session.create deliberately writes no session row until
the first prompt arrives (avoiding "Untitled" litter), so the minted
key exists only in the backend's in-memory live registry. The managed
update pipeline restarts hermes-gateway.service and
hermes-dashboard.service at the end of every update; any session
minted but not yet prompted in that window is orphaned: the key exists
client-side (pinned tile, stored session id) but has no row anywhere,
and after a restart session.resume looks up the live registry and the
DB, misses both, and returns 4007 permanently.
Fix: in _resume_locate, before returning 4007, when the target is a
well-formed server-minted key (^\d{8}_\d{6}_[0-9a-f]{6}$) with no
persisted row and no live-session claim (any profile), materialize the
row via db.create_session(...) and re-resolve, resuming into a fresh
empty session. Fail-closed: garbage ids, 8-hex runtime ids, titles,
lazy watch windows, and keys claimed by any live session under another
profile still 4007 — an unscoped resume must never mint a phantom row
in the launch store.
Rebased onto current main by the maintainer landing crew: the resume
handler is split into _Resume helpers (methods_session.py), so the
materialize block lives in _resume_locate; the helpers stay in
server.py and resolve through the method_ctx rebinding. Test adapted
to the current handler shape (source parity with the env resolver,
profile_name stamped by the store).
Co-authored-by: ruangraung <github@nadyahermes.anonaddy.com>
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Hidden keep-alive pane layers stay mounted with visibility:hidden and
pointer-events:none so scroll position and state survive tab round-trips.
On some compositors these hidden layers can still paint for one frame during
a tab switch, causing accent-colored elements from the outgoing pane to flash
over the incoming pane (e.g. Notes ↔ Tokens).
Add opacity-0 so the compositor drops the hidden layer entirely while keeping
the DOM mounted. This fixes the flash without losing scroll/state preservation.
Closes#101476
On Windows, BrowserWindow.show() and focus() both seize the OS foreground.
focusWindow() did show() + unconditional focus(), and the
sendOpenUpdatesRequested / deep-link handlers did the same — so any ambient
IPC reaching those paths mid-stream dismissed other apps' native save and
confirm dialogs (the reported Notepad repro).
focusWindow() now reveals with showInactive() and guards focus() with
isFocused() so a redundant call never pumps SetForegroundWindow; the
updates-toast and deep-link call sites get the same guarded pattern. The
policy lives in electron/window-focus-policy.ts (pure, unit-tested).
User-intentional raises (pet overlay toggle/open clicks, HUD/Quick Entry
reveals) keep show()+focus() — a click is consent.
Supersedes #84026 (frizikk), which guarded only the renderer side and
explicitly left the main-process focus paths unchanged.
Fixes#83998
Co-authored-by: Jakub Wolniewicz <4850809+frizikk@users.noreply.github.com>
`_is_ancestor_pid` asked psutil for the whole parent chain at once:
any(parent.pid == pid for parent in psutil.Process().parents())
`Process.parents()` is eager: it walks all the way to the lowest pid
*before* returning a list, so the generator expression cannot
short-circuit when it reaches the matching ancestor. Its per-link
`parent()` tolerates only `NoSuchProcess`, so any process we may not
inspect anywhere above us raises `AccessDenied` out of the whole call,
and the `except Exception` guard discards every ancestor already found,
including the orchestrator one link down.
Under firejail with `ptrace_scope=1`, and in hardened containers,
`/proc/1` is exactly that unreadable process. The hand-off child then
failed to recognize its own parent's fresh marker, `UpdateLock.acquire`
treated a live orchestrator as a foreign concurrent updater, and the
in-app update exited 2 ("Another Hermes update is already running",
naming the hand-off script's own pid) on every attempt, indefinitely.
Walk the chain one link at a time and test each ancestor as it is
discovered, so a failure above a match can no longer erase it. A
failure encountered before any match still returns False, keeping the
conservative refusal for genuinely unprovable ancestry. The walk is
bounded by `_MAX_ANCESTRY_DEPTH` and skips a repeated pid so an
unexpected chain cannot spin.
Fixes#87514
A writer in the update's mutation window replaced every agent-job
prompt with the job's own name while the job count stayed identical,
so the count-based restore_cron_jobs_if_emptied net passed the loss
undetected (#82990). Add a field-level safety net mirroring the config
model-settings net: restore only the prompt field of a live agent job
whose id matches a snapshot job and whose live prompt is blank or
collapsed to the job name — a legitimate user edit that merely
differs is never stomped. Wired into the post-migration restore
sequence for the active profile and every sibling profile.
startAttachedBackendMonitor's dead/drifted-backend recovery called
invalidatePrimaryConnection(), which sets primaryRecoverySuppressed
before scheduleUnexpectedPrimaryRecovery() reads it back as an
intentional teardown and refuses to claim the respawn, leaving the
desktop app with no backend. Clear the slot via
backendConnectionState.invalidate() directly instead, since an
unexpected attach teardown is exactly what the supervisor's recovery
path exists for.
startAttachedBackendMonitor's 15s poll only checked that the attached
backend answers its readiness probe. /api/health and /api/status are
public routes, so a backend recycled by an external supervisor
(launchd KeepAlive) into a new process serving a brand-new session
token still answers 200 — the probe passes while every authenticated
call 401s forever, since child-exit recovery never fires for a
backend this app did not spawn.
Re-read the served token (already exposed via resolveServedDashboardToken)
on every tick and treat drift the same as "gone", routing into the
existing recovery path that re-adopts the token on respawn.
Fixes#121988
A providers.<slug> block with no base_url/url/api of its own is settings,
not a user-defined endpoint. Only remap to the custom: validation branch
when the user-config entry declares a base_url; otherwise openai-codex and
other listing-less built-ins keep their normal catalog path.
Closes#120020
providers.bedrock: {stale_timeout_seconds: 600} is the documented way to tune a
built-in provider's per-call timeout (agent/turn_recovery.py,
agent/thinking_timeout_guidance.py). resolve_user_provider() treated ANY dict
under providers.<name> as a full custom-endpoint definition, even with no
api/url/base_url — producing an empty openai_chat/api_key stub that shadowed
the real built-in overlay (bedrock_converse transport, aws_sdk auth).
For Bedrock this routed every /model switch through the generic
custom-endpoint /models probe against the Bedrock runtime host, which has no
such endpoint: the switch intermittently failed with "could not reach this
custom endpoint's model listing" depending on how fast the (irrelevant) probe
timed out.
resolve_user_provider() now requires a real endpoint (api/url/base_url) before
resolving an entry; a tuning-only block falls through to resolve_provider_full's
next rung, which picks up the built-in overlay correctly.
Three gaps stranded the app through a legitimately long or failed
source-update completion:
- update-gate: a fourth gate signal 'failed-receipt' (hasFailedReceipt on
UpdateGateDeps, optional) reports a FINISHED, failed update even when a
stale marker survives; waitForUpdateClearance gains abandonOn so the
boot path stops parking the full 20-minute budget on a receipt that
already says failed and boots the current build instead.
- backend-ready: while the backend prints venv_sync's source-completion
banners the 90s port-announce deadline is re-armed (5-min grace,
30-min total cap, real wall clock) so a healthy multi-minute repair is
not killed and re-run on every boot.
- venv_sync: relaunch-driven completion-tail retries are bounded — an
attempts record beside the pending marker keeps the first retry
immediate (the historical flaky-tail case), backs off after 2
consecutive failures (15-min window), and caps at 6 attempts, past
which the launch keeps the marker, skips the tail, and points at
'hermes update'; success clears both the marker and the record.
The skills security scanner had two blind spots (#63307 Part A):
1. destructive_home_rm only matched literal $HOME, not bare ~. A command
like 'rm -rf ~/Documents' scanned as safe for every trust tier.
2. No pattern detected the !`cmd` inline-shell DSL that
agent/skill_preprocessing.py expands via bash -c on skill view/load.
A skill carrying !`rm -rf ~/Documents` scanned safe.
Fix: extend destructive_home_rm to match ~/~ /~* (bare tilde forms),
add an inline_shell_exec pattern for the !`...` auto-exec snippet
(requires a non-space payload so a skill explaining the DSL is not
flagged), and bump SCANNER_VERSION to skills-guard-v7 so cached safe
verdicts are invalidated.
Also stop swallowing the scan verdict on the TUI-gateway skills.manage
install RPC (#63307 Part B): _skills_install returned installed:true on
every path, including a security-scan block, because its console was a
sink. It now keys the response off do_install's verdict and answers an
error envelope (5031) whose message/data carry the printed tail — the
scan verdict, the ambiguity candidate table, or the fetch failure —
instead of a silent success.
The Desktop one-click hub install path gets the same treatment from the
spawned CLI's exit code (hub-actions.ts already parses it).
Original scanner patterns by the PR author.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Busy remote hosts routinely take 60-150s to cold-boot a pooled
`hermes serve --isolated` before it prints HERMES_BACKEND_READY, but the
desktop gave up at the hardcoded 45s budget and surfaced a timeout toast
milliseconds before the healthy backend announced ready (issue #94642).
- remote-lifecycle.ts: raise DEFAULT_READY_TIMEOUT_MS 45s -> 120s and add
resolveReadyTimeoutMs(env) honoring HERMES_DESKTOP_REMOTE_READY_TIMEOUT_MS,
clamped to a 45s floor (mirrors the local analog in backend-ready.ts).
Wired into scrapeReadyPort and connect.
- windows-remote-lifecycle.ts: the Windows remote connect sibling used the
same hardcoded 45s budget; it now resolves through the same helper.
Regression tests cover the env override, below-floor clamp, fractional
rounding, and malformed / non-positive fallback of the resolver.
Pooled remote backends are probed from the renderer's reconnect IPC on a
minutes-long cadence once the primary connection is healthy. The shared
liveness tracker's 60s failure window — calibrated for the primary's
sub-minute retry loop — expired between pooled probes, so a dead pooled
descriptor's failure streak reset on every tick, the drop path was never
reached, and the pool served the dead descriptor (ECONNRESET on every
routed call) until app restart.
Give pooled revalidation its own tracker with a 15-minute failure window:
wide enough for the observed ~4-minute probe cadence to accumulate the
3-failure streak, narrow enough that genuinely stale failures still reset.
recordSuccess() clears streaks on recovery, so the wider window cannot
produce false drops.
Closes#94381
The code-skew watchdog only covered SSH-isolated backends; Desktop-owned
local serves share the same 'updater never restarts me' property, so they
served stale modules until the app happened to restart them (R1). Extend
start_code_skew_watchdog wiring to is_desktop_owned_backend() backends.
Centralize the skew refusal: POST /api/model/set and the tui_gateway
model.options / model.save_key RPCs now refuse with the same restart
message the picker and the gateway /model switch already return (R2).
Rust live_marker_owner now REMOVES a stale marker (dead pid, past ceiling,
unparseable) on read — previously stale bytes were only ignored, so a crashed
updater whose Drop never ran refused every later acquire until the 20-minute
ceiling (#77259). pid_is_alive gains pid-0 and zombie handling (Linux /proc
state, macOS ps stat), closing the kill(pid,0) false-positive that kept a
dead-but-unreaped updater 'live'.
Python _pid_is_running gains the same zombie awareness via a new
_process_state probe (Linux /proc, macOS ps), which read_live_update inherits
through update_lock._pid_alive — so the CLI gate self-heals a zombie-owned
marker in seconds instead of 20 minutes (#120635, #125932).
Electron readLiveUpdateMarker consults a new posixProcessState probe
(injectable, fail-open to the signal-0 verdict) so the desktop boot gate no
longer parks on a zombie-owned marker.
Rust fix cherry-picked from #77885 (authorship kept); Python and Electron
are new parity work.
The managed-update termination probe (buildOwnedTerminationCommand) signals
REFUSED, DARWIN_UNAVAILABLE, UNAVAILABLE, and TIMEOUT via a non-zero process
exit code, but SshConnection.exec() throws on any non-zero remote exit before
the caller can inspect stdout. terminateOwnedDashboardForUpdate's status
classification (result === 'REFUSED' / 'DARWIN_UNAVAILABLE' / ...) therefore
never ran for any of those outcomes; every one collapsed into the generic
"Could not terminate the Desktop-owned remote serve for update." On a Darwin
remote this fires on every managed SSH update, since the script unconditionally
refuses to signal there.
Align the probe with this file's own established convention (used by
buildOwnedStaleTerminationCommand, pidIsOurDashboard, remoteProcessCreationTime):
always exit 0 and let stdout carry the status.
A dispatched hermes update that dies after backup/fetch left a marker
whose PID was gone, but startup still treated it as live. Unlink a
dead or recycled non-update PID and log the detached run's start/exit.
* fix(desktop): a reused tool call id no longer claims a finished tool row
A tool.start whose id repeats a call that already completed in the same
message was merged into that finished row: the args and error of the new
call overwrote the successful one, so a loaded skill rendered as a failed
one. The stable-id match now claims only an unresolved part, as
toolCallOwnerMessageId and applyStoredToolResultToParts already do. A
completion with no unresolved same-id part still lands on the finished
part when it repeats that part's result or carries none, so replayed and
metadata-only completions add no phantom row.
Refs NS-1029 B3.
* fix(desktop): a tool row shows only its own diff
The desktop kept a side store of inline diffs keyed by the raw tool call
id, and a tool row preferred that diff over the one on its own part. With
a reused id, the newest patch was stored under the shared id, so every row
that still carried that id showed the newest diff. Now that a reused id
gets its own row, that wrong diff became visible on the first row.
Every live completion already writes inline_diff into the part's
toolResultMetadata, and cold hydration never filled the side store. The
row now reads the diff from its own part, and the side store is removed.
Refs NS-1029 B3.
window.hermes.send() from a ::preview frame silently sliced prompts to 500
chars and silently dropped a second intent within the 1s throttle, so a
widget sending a JSON payload reported success while the agent got garbage
or nothing.
The parent now rejects over-length and throttled intents explicitly and
posts an ack back to the frame; send() returns a Promise of
{ ok: true } or { ok: false, error: too_long | throttled | invalid |
undelivered } (with maxLength / retryAfterMs). The cap and throttle stay;
nothing is truncated. The gate is a pure evaluateIntent() under test.
Covers the truncation, throttle, and delivery-ack findings of #118973; the
side-preview-pane bridge (finding 1) is untouched.
A background stream can change the active runtime ref while session.create
is in flight. That mismatch aborted every first send: no prompt.submit,
no stored row, and an empty tab. Abort only when route and selection
moved to a chat whose runtime is not the one create returned.
The original 401-misclassification from #98647 (a rejected WS-ticket
mint reported as "Could not reach the remote Hermes gateway") was fixed
on main by #95701: the shared httpStatusError attaches err.statusCode
and gatewayTicketFailure routes 401/403 to the sign-in copy with
needsOauthLogin/isReauthRequired.
What remained live was the transport leg: resolveRemoteOauthTicket
wrapped EVERY non-auth mint failure - timeout, ERR_CONNECTION_REFUSED,
DNS failure, a reverse-proxy 502 - in one undifferentiated sentence
(remote-oauth-ticket.ts:38 passed a static transportMessage), so fleet
logs hid three different upstream causes behind identical copy
(~1,100 lines across 47 hours in the reporter's fleet, zero of which
named the actual failure).
- connection-config.ts: new exported gatewayTicketTransportMessage(error)
names the failure class: timeout/unreachable/DNS -> "timed out or the
host could not be resolved... check your network/VPN path";
ECONNREFUSED -> "connection was refused... the gateway process is
likely down"; anything else keeps the legacy one-liner.
- remote-oauth-ticket.ts: resolveRemoteOauthTicket derives its
transportMessage from the cause instead of a static string. Auth
classification is unchanged.
Refs #98647
The test passed on the author's macOS but exited 1 on CI: with
--skip-build the build lock is never acquired (the probe asserted a
release that never happened), and the unmocked platform launch fixups
ran against the fake empty executable — on Linux,
_packaged_desktop_launch_command's sandbox helper is missing and sudo
is unavailable, so cmd_gui exits 1 before the handoff.
- Run the real acquire path (no --skip-build) so the lock is genuinely
held, then probe its release at the Electron handoff.
- Pin HERMES_HOME to tmp so the checkout-keyed lock path is isolated
from the runner's home.
- Stub _desktop_linux_sandbox_fixup and _installed_desktop_launch_target:
environment-dependent platform fixups are not this test's subject.
Two concurrent `hermes desktop`/`hermes update` invocations mutate the same
checkout-scoped node_modules and apps/desktop/release with no serialization:
the check-then-build preflight (_desktop_build_needed → npm install → pack) runs
unguarded, so the loser trips over the winner's half-installed tree
(ENOTEMPTY) or packs against a missing tsc.
Lock the whole mutable window behind DesktopBuildLock, a checkout-keyed flock
under the profile-common Hermes root (kept out of the checkout so read-only
prebuilt launches still work, released by the OS if a builder crashes):
- `hermes desktop` acquires it before the freshness check and refuses a
contended build with an explicit message (exit 2) instead of racing.
- `hermes update`'s desktop rebuild (build_update_products) waits for an
in-flight build rather than failing the update.
- the lock is released before the Electron handoff (source, packaged, and
--build-only paths), so an open Desktop window never blocks a rebuild,
and after a build failure so the next run isn't deadlocked.
Supersedes #94101 (lock design; its main.py wiring no longer applies after the
cmd_gui extraction) and #96581 (wait/exit-2 UX; not checkout-keyed).
Co-authored-by: worlldz <cryptoworlldz@gmail.com>
Co-authored-by: lepetitprince716-prog <lepetitprince716-prog@users.noreply.github.com>
