A named-profile export dropped the stores themselves but shipped the copies Hermes writes of them: backups/ (pre-update zips of the whole home, config.yaml copies), state-snapshots/ (per-profile update snapshots with pairing stores), auth.json.corrupt, and the config migration's .env.bak-<stamp> / config.yaml.bak-<stamp>. None of their suffixes is one the text scrub edits. The default export's root allow-list already omits all of them.
Also lists feishu_comment_pairing.json beside the other pairing stores, and mirrors the credential-store wording in the zh-Hans distributions page.
(cherry picked from commit 0a24aeaaf55279bfa9af4cc9f5e80873fb611020)
Named-profile exports dropped only auth.json, .env and bot-desktop, so
other credential stores Hermes reads from a profile home were archived
as-is: .op.env (1Password service-account token), npmrc, OAuth token
files, mcp-tokens/, vault/, browser profiles and platform sessions.
.op.env and npmrc also escape the text scrub pass.
Add PROFILE_CREDENTIAL_PATHS (root-relative) as the one list, exclude it
from named exports, and fold its root entries into
DEFAULT_EXPORT_EXCLUDE_ROOT so distribution installs treat them as
user-owned.
(cherry picked from commit fb8ce7a9999049c1bc11aaf7a3d9b2190d8ed1aa)
The bind address never changes for an app's lifetime, so create_app now
canonicalises it and computes the wildcard flag and allowed Host set once
instead of on every request. _WILDCARD_HOSTS duplicated what
ip_address().is_unspecified already answers (only "" needs a special case),
the Sec-Fetch-Site sets become named module constants, and fetch_site is a
required argument so no caller can silently skip that check.
The 403 message now says what is actually refused (browser requests and
foreign Host names) rather than implying every non-local client is.
The parametrized proxy test asserted 200-vs-"some 4xx" via a tuple
comparison; it now pins status == 200/403 and the exact upstream request
count, so a regression to a different status or a leaked forward fails.
The duplicate #126202 covered three shapes the parametrized boundary test
did not: userinfo in Host, an Origin carrying a path, and a specific-IP
bind that must accept its own address and refuse a neighbour's. Add them
as rows instead of separate tests so the boundary stays in one table.
Co-authored-by: JoaoMarcos44 <joaomarcosdias444@gmail.com>
Browsers send no Origin on a GET, so a cross-site page (or a DNS-rebound
page on a wildcard bind) could still reach /v1/models with the operator's
credential. Sec-Fetch-Site, which a page cannot forge, now refuses those.
The Origin check compares the parsed host and port instead of the raw
string, so an Origin that omits the default :80 or spells an IPv6
address differently is matched the way a browser means it.
Co-authored-by: JoaoMarcos44 <joaomarcosdias444@gmail.com>
(cherry picked from commit 72bf4e9c34f6c6200658e42345f775291462697c)
A wildcard bind accepts any Host name, so a DNS-rebound page reaches it
with Origin equal to its own Host and the same-origin rule let it through
to the operator's credential. A wildcard listener has no origin of its
own, so an Origin-bearing request is refused there; API clients without
Origin keep working. Also treat the absolute form localhost. as localhost
and detect expanded IPv6 unspecified binds as wildcard.
(cherry picked from commit 5ce7aac4d7e4d160b2a6447f0442e747f489d72f)
hermes proxy attaches the operator's subscription credential to every
forwarded request but accepted any Host header and any Origin. A
loopback or specific-IP bind now serves only its own Host names, and a
request whose Origin is not the proxy's own is refused with 403.
Non-browser clients send no Origin and are unaffected; a wildcard bind
keeps accepting any Host.
(cherry picked from commit 3cf3870cfc2a795e439b1a8c0c53424f79c24505)
The unknown-scope branch printed a stray "(system unit)" suffix after the
sudo -S -p '' command, so copy-pasting it failed, and the three-way if/elif repeated
both command strings. Build the two commands once and pick by scope; the
fallback (scope unknown) prints both. The test now asserts the systemctl
line count equals the expected hints for every scope, not just the
single-hint ones.
The single-use _service_unit_owning_gateway helper split one decision
across two functions with two lazy imports of main_dashboard and two
docstrings; fold it into _refuse_restart_of_service_managed_gateway.
The hint previously fell back to `sudo systemctl restart` whenever the
cgroup scope was not "user", which is wrong when the scope is unknown
(no readable cgroup): print both the --user and system forms there, and
exactly one command for a known scope.
Stub _get_systemd_service_for_pid -> None in the restart_calls fixture
so the existing fallback tests stay hermetic on Linux CI, where the
runner's real /proc + systemd could otherwise make PID 4321 look
service-owned. Tighten the regression test to exact per-scope hints.
The wrapped callees cannot raise, so the `except Exception` blocks only hid
real bugs:
- main_dashboard._get_systemd_service_for_pid iterates
_pid_unified_cgroup_entries (which catches OSError/PermissionError on the
/proc read and yields nothing) and calls _unit_main_pid_is, whose systemctl
probe catches _SYSTEMCTL_ERRORS (FileNotFoundError, TimeoutExpired, OSError).
- _get_pid_cgroup_path is next(..., None) over the same swallowing generator.
- _extract_scope_from_cgroup is pure substring checks on a str (guarded by
`or ""`).
The local main_dashboard import is in-tree and stays lazy.
Pre-convention hermes.service installs are invisible to canonical-unit
checks, so the manual fallback SIGKILLed a live service-managed gateway
and spawned an unsupervised orphan in the caller's cgroup while the unit
flapped against the stolen lock. Reuse the dashboard MainPID-verified
lookup and refuse with the unit's systemctl command instead.
Closes#126474
(cherry picked from commit 1b3e022c7330a1ba823be0f62fba091c9259ec98)
_authored_request_text and _reappend_inflight_user_task each unwrapped the
in-flight replay header with identical code, so a later change to that
format could update one and miss the other. Both now call
_restated_request_text; the reappend path still keeps the reply pointer so
the restated task retains its disambiguation.
The reply-pointer test only exercised the split-turn gate, so reverting the
_latest_user_task_snapshot hunk (drop the gateway quote before eliding an
over-cap request) left it green. Assert the deterministic snapshot of the
long reply still contains the authored request, and reuse the parametrized
_oversized_active_turn helper instead of a near-copy _reply_turn.
Trim the salvaged tests to the two that pin the headline behaviour: a short
reply to a long answer still splits the oversized turn (reply, own-reply and
Discord-note pointers), and a restated reply keeps splitting on later
compactions. The snapshot and helper-table tests restate implementation
details already exercised by those two end-to-end paths.
The oversized-active-turn split (#80449) only fires when the turn-opening
request is at most _ACTIVE_TASK_MAX_CHARS. On messaging platforms the gateway
prepends `[Replying to: "<quoted message>"]` to a reply, so a short reply to a
long assistant answer measured as a long request: the split never fired, the
user anchor kept the whole in-progress turn in the protected tail, the
compress window collapsed to nothing, and every automatic pass became a
structural no-op (300 s backoff) while the turn kept growing past the
threshold. /compress hit the same empty window.
Measure the authored text instead: strip one leading gateway reply pointer
(optionally behind the Discord triggering note), and the in-flight replay
header first so a request restated by an earlier compaction keeps splitting
on later passes. The deterministic task snapshot uses the same text when a
row exceeds the cap, so head elision can no longer keep the quote and drop
the request. The token-size guard on the whole row is unchanged, and a quote
whose closing delimiter cannot be found keeps the conservative anchor.
(cherry picked from commit ccaf61b3b651a7fd68ffc0c69ab965881f8850b6)
launch_budget read free memory only from nvidia-smi, so a discrete AMD/Intel card launched at its
capacity window regardless of other programs' VRAM, the case that silently pages on Windows. It
now asks the engine's device probe afresh at launch (the polled view stays cached); the driver's
free count includes other processes (live: 16.2 GB held elsewhere read as 8.2 GB free of 26.0).
Docs: auto is a preference with the platform's pinned fallbacks, not an unconditional CUDA pick.
A Vulkan/HIP card's type and size come from the installed engine's own device probe, but
quickstart priced its pick before installing that engine, so a fresh AMD/Intel machine was sized
from system RAM and committed to the wrong download (live: an empty store picked the 35B-A3B
from a 50 GiB RAM budget; with the engine installed the same card picks the 27B).
The job now re-picks right after the engine install, before any bytes are committed, and updates
the job's target/model_id, which is what Desktop renders. The POST contract is unchanged.
The managed llama-server is one per process and serves every profile it hosts, but the budget
re-derived its engine from whichever profile's config the current thread saw. The on-demand boot
thread and the Local Models job workers were bare threads, so under multiplex a named profile on
Vulkan booted a server sized from the default profile's CPU config (262144-token window instead of
98304, reproduced live by an independent review).
- bootstrap records the engine it boots (cleared on shutdown and on a failed boot); the device
probes price that engine whenever one is up or booting, config only otherwise.
- The on-demand kick and the Local Models job/refresh threads carry the caller's contextvars
(spawn_context_thread), so config=None and the post-download refresh resolve the right profile.
_boot_in_flight gated the on-demand boot on installed_engine() with the
default "auto", not the configured local_runtime.backend. With no GPU
vendor detected, auto resolves to the CPU build only, so a user on
`backend: vulkan` with only the Vulkan engine installed never got a boot
from `hermes chat` ("The local model server isn't running"), while
ensure_local_runtime itself would have booted that exact engine. Found
during the live run for this PR.
Both engine probes called get_config_value("local_runtime.backend", "auto"),
but get_config_value is the `hermes config get` printer with signature
(key, *, as_json, raw): the call raised TypeError, suppress(Exception)
swallowed it, and the probe returned None every time. That made the
discrete Vulkan/HIP budget dead code in production, and the CUDA
--list-devices pool fallback has been dead since it landed (bdb52ce711).
Found by a live run with NVIDIA hidden at the OS level: the device probe
answered, yet probe_budget still returned the RAM-as-unified budget. One
helper now resolves the configured engine via load_config_readonly(), and
the Vulkan test reads a real config.yaml instead of patching the probe out.
The previous commit budgets any Vulkan/HIP device that `llama-server
--list-devices` prints as a discrete card. That output carries no
integrated-vs-discrete verdict, so an iGPU (Intel ADL-N reports 11.8 GB of
a 16 GB machine's shared memory) would be counted twice: once as VRAM and
again as system-RAM spill.
Ask the installed engine's ggml library instead, in a bounded child
process: ggml_backend_dev_type() is GPU for a discrete card and IGPU for
an integrated one (Vulkan reads it from VkPhysicalDeviceType). llama.cpp
places layers on discrete devices whenever one exists and falls back to an
integrated GPU only otherwise, so the budget follows the same rule:
- discrete device -> its own memory, system RAM as spill (uma=False)
- integrated GPU, no answer, CPU/Metal -> RAM-as-unified, unchanged
The probe runs only when no NVIDIA device answered and the configured
engine is Vulkan/HIP, and is cached per engine binary (the hardware
endpoint polls every few seconds). The CUDA --list-devices fallback for the
NVIDIA unified-pool quirk is left as it was on main.
Co-authored-by: Belaquaa <eee_pokkk@mail.ru>
Review of #131847: the retry ran for every rmtree failure on every OS.
On POSIX an unreadable directory hands the callback os.open, and
func(path) raised TypeError (missing flags), which escaped the OSError
set-aside path and failed an install that had already published. And
os.chmod followed a symlink inside an unwritable directory, rewriting
the mode of a file outside the tree to 0200.
The retry now only fires for a PermissionError from os.unlink/os.rmdir
on Windows, and clears the link's own attribute where chmod can follow
links; everything else raises as plain rmtree does.
The salvaged commit imported hermes_cli.fs_utils.rmtree_force inside
_remove_entry. PM installs run before the rest of the tree exists: the
Docker toolchain stage copies only pm/, so every image build died with
"No module named 'hermes_cli.fs_utils'" on its first fetch cleanup.
pm/filesystem.py is the stdlib-only home for exactly this kind of
primitive. hermes_cli/fs_utils.py stays as is; it is part of the frozen
old-updater surface.
A read-only file (PortableGit's etc/hosts) inside a replaced entry made
_remove_entry fail every attempt. Since the held-tree fix the install no
longer aborts, but the tree is set aside and gc can never reclaim it, so
every replaced git entry leaked. The live Windows case requires the
replaced tree to be gone outright after a reinstall. Also trims the
salvaged comment to its why.
`_remove_entry` used a bare `shutil.rmtree`. Git for Windows' PortableGit
archive ships `etc/hosts` read-only, and Windows refuses to unlink a
read-only file, so removing the rotated `.previous-git-*` entry raised
WinError 5 on every attempt. The retry loop cannot help here: a read-only
attribute is not a transient handle, so all 5 retries failed and the
whole update aborted with:
Source update preparation failed: git: install failed:
[WinError 5] Access is denied:
'...\.previous-git-2.53.0+3-win32-x64\etc\hosts'
The abort happened before the checkout was reset, so `hermes update` made
no progress at all and the install stayed pinned indefinitely.
Use the existing `hermes_cli.fs_utils.rmtree_force`, which clears the
read-only bit before deleting and whose docstring already describes this
exact scenario. It is imported lazily to match how the rest of `pm`
reaches into `hermes_cli`.
The #121865 regression test bounded ALL dials in one virtual second at 5,
but the reconnect ladder uses full jitter (delay in [0, ceiling)), so how
often the ladder itself redials depends on Math.random: a small draw gave
6 on CI, a 0.01 draw gives 12, a 0 draw 62, all with the throttle intact.
Timers never fire inside an awaited request under fake timers, so the
dials observed during each requestGatewayForAgent call are exactly the
poll's own. Assert those are zero (the #121865 bug was one per poll) and
that the ladder still redials. Green for draws 0, 0.01, 0.5, 0.999; red
when the background cooldown gate in openSecondaryForRequest is removed.
Review findings on the bot-identity fix:
- Edit Profile sent every seeded field on save, so a color change
reverted a rename made elsewhere while the dialog was open (a thin
row with no local record even cleared the backend title). The dialog
now diffs against the values it opened with (edit-profile-look.ts)
and sends only what the user changed.
- Rows retained from an earlier paint (source unreachable or
connect-on-demand) were reconciled as fresh backend truth, reverting
a newer local title during an outage. They are now tagged `retained`
and mergeServerMeta skips them and any unreachable-source row.
- A title-only answer minted an empty direct record that shadowed a
configured alias's appearance in botRosterMeta. A title now only
corrects an existing record; the row already carries it.
Two machines with a profile named `default` (local "Agent B", VPS "Agent A")
showed the VPS bot as "Agent B", told it so in group prompts, and the next
save on that row wrote `title: Agent B` into the VPS profile.yaml, from
where every relay roster republished it.
Root causes, all on the Desktop side:
- Write path: `saveBotMeta` sent the whole local cache entry as the
server's `hermes-bots` namespace. Any field the cache held and the server
did not (a restored or legacy name-keyed record) became that bot's server
truth. A save now reads the bot's namespace from its own backend, writes
that plus the save's own patch, and uses the existing per-key CAS
(`ui_meta_expected_revisions`), re-reading on conflict.
- Read path, thin rows: Electron's enumeration read `profile.title` from
`/api/profiles`, which has always named it `bot_title`, so bots on a
non-active connection never carried their backend's title, and
`mergeServerMeta` only ran over the active source's rows. Electron now
maps `bot_title` (empty string included), and reconciliation runs over
every live row: the name is whatever the bot's backend reports, and a
local title it does not carry is dropped.
- Read path, bare keys: `botFriendlyNames` read `meta[bot.name]` for any
non-remote row and `groupSpeakerLabel` fell back to `meta[profile]`, so a
scoped `default` row picked up the bare-key title another machine's
`default` wrote in single-source days. Both now read the row's own key;
the bare legacy rung applies only to unscoped rosters. Mention completion
goes through `botRosterMeta` too.
- A remote `default` with a real `display_name` no longer trades it for the
connection label.
Settings resolved the row only inside providers, so a legacy
custom_providers entry with key_env was saved as a keyless twin and
the list row stayed behind.
A connection apply made while a secondary scope was in front re-dialed the
primary socket to the new machine, but setPrimaryGatewayConnectionId
refused the id (#95628 hardening), leaving primaryConnectionId naming the
machine it had left. Owner routing then sent that machine's requests over
the primary socket to the other one. Live trace before:
primary-publish refused vps keeping=local active=conn:vps::alpha
conn:vps::default via=secondary primary=local/alpha dialed=vps
Every caller publishes the window's own primary socket (boot, reconnect,
connection apply, HMR survivor); #95692 already removed the secondary
write that #95628 was about. The id is now always recorded; the ambient
request connection still moves only while the primary is in front.
With a local and a remote backend that both run profile `default`, the
Bots pane showed the remote bot's title on "This device" (and the remote
profiles as "Status unknown" ghost rows) after a primary-connection
switch, and mergeServerMeta could carry that title into bot-meta-v2.
fetchRosterSnapshot binds activeConnectionId when it is called, but it
sent profiles.list over the ambient socket resolved later. A retry or late
run of a query keyed to the connection the window just left was answered
by the backend it switched to and cached under the old key. Live trace:
published connection=vps
[bot-roster] MISROUTED active=local answered=<vps install> expected=<local install>
Now the fetch refuses when its key is no longer the published connection
(stale retries never send), and asks the keyed connection by its exact
(connection, profile) route instead of the ambient socket. Without an
exact route (older Desktop host) it keeps the ambient request.
Change-only desktop.log / agent.log lines that name which writer moved a
bot identity when two connections share a profile name:
- [gateway-route] per (connection, profile): via=primary|secondary and the
window primary; `active` on every active-scope change (key, socket
connection, profile) and `published` on every published-connection
change, so a socket/label disagreement shows up side by side.
- [bot-roster] each Bots-pane snapshot: the install that ANSWERED
profiles.list vs the install expected for activeConnectionId;
MISROUTED when they differ.
- [bot-relay] rows per connection with the answering install; MISROUTED
when two connections answer from the same install.
- [bot-meta] every $botMeta title change per scoped key.
- backend: bot_relay.roster.sync and ui_meta['hermes-bots'].title writes.
profiles.list now echoes install_id (same value as /api/status) so a
client can prove which machine answered a routed list. Additive field;
older clients ignore it. Plugins reach the tracer through
host.traceIdentityChange?.() so wholesale SDK mocks stay valid.
#131910 refused to signal a process group whenever its leader's start time
no longer matched the spawn-time baseline, and a REAPED leader reads as None,
so "leader gone" was treated as "PGID recycled". POSIX never reuses a PGID
while any member of the group lives, so a gone leader still leaves the group
ours, and its reparented grandchildren (an MCP server's pipe holder, a
terminal command's daemon) leaked past host shutdown.
Only a LIVE leader with a different start time (drift-tolerant since
a5e7df27) proves recycling now; the recycled-PID skip is unchanged.
a5e7df27 kept "gone leader refuses" and asserted it in
test_pgid_starttime_drift.py; that assertion now expects the kill. Same fix in the terminal
environment's twin guard (tools/environments/local.py::_leader_is_ours).
Red on main: tests/e2e/core/parity/test_mcp_lifecycle.py::
test_mcp_server_death_mid_call_fails_fast_and_the_turn_completes and
test_entrypoint_parity[*] zero_mcp_orphans (e2e lane on eaecc99c).
A remote hop may percent-encode an /api/fs path on top of HTTP's own
query decoding, so a non-ASCII filename reaches the gateway as a literal
'%E5%8D%8A...' string; _fs_path never decodes it, the stat fails and the
dashboard preview returns 404 (#103425).
Resolve verbatim first and only fall back to an unquoted retry when that
path does not exist, so filenames that genuinely contain '%XX' keep
resolving as-is. The write-text endpoint opts out to keep creating
literal-percent filenames instead of silently overwriting a decoded
sibling.
A Nous 404 answering "This model is no longer free. To continue using
the paid variant, switch to '<slug>'." fell through _404_RULES to
_V_UNKNOWN, so the retry loop kept retrying a slug that can never
succeed again. The route retirement is a model-lifecycle change, not a
missing model body or an account balance problem, so it now maps to
model_not_found: non-retryable, fallback armed (#123180).
_real_profile_cdp acquires the process-global launch lock with a 30s bound
and fails closed with an actionable error when a previous (possibly
abandoned) resolver worker still holds it, instead of parking every later
real-profile call until the 420s outer tool deadline (#96731).
Conflict notes: rebased by hand from #96752 onto the extracted
browser_tool_real_profile module — the bounded acquire/finally-release
moves into _real_profile_cdp there; the regression test is the
synchronized version from the same PR.
Co-authored-by: fangliquanflq <280272527+fangliquanflq@users.noreply.github.com>
main replaced the linux_only/macos_only/windows_only marks with
platforms(...); the old marks now abort the whole test run with a
rewrite error. Follow up the rebase by switching the two capture-file
cleanup tests to platforms("linux"), matching test_spawn_site_child_env.
The timeout-path cleanup assertion failed on Windows: kill() returns
before Windows tears the killed CLI's handles down, so the best-effort
unlink hits a still-open file — the same Windows property as the
grandchild path, reached through the timeout branch instead.
Per AGENTS.md, express the platform split with @pytest.mark.linux_only
rather than inline sys.platform checks, so the cleanup contract stays
visible in both CI lanes (list_os_marked_tests.py selects the OS lanes
by the marker name).
The real-profile capture unlink is best-effort by design
(_unlink_command_output_files swallows OSError). On Windows the resident
daemon grandchild keeps its inherited handles open, so the temp files
survive the call until the daemon exits — the cleanup assertion must not
require unlink-while-open semantics there (#96731 follow-up, Windows
verification report).
