* feat(connectors): the backend serves a connector's tool list, cached for 24 hours
The Connectors page opens one app and shows every tool it has. The backend
had no way to read that list.
- `tools/connectors/portal/`: a client for the portal's tool-list route and a
JSON cache under the Hermes home, one file per portal origin and connector.
An entry is fresh for 24 hours. After that the read revalidates with the
stored ETag: 304 keeps the list, 404 deletes the entry, an upstream failure
serves the stored list marked stale, and a 401 never serves the cache.
- `connectors.tools {slug, refresh}`: account-level, routed by `profile`, no
chat session. Errors carry a fixed `reason` from one closed set on the rail.
- Every connector model that is not operation state moves into
`tui_gateway/contracts/connectors.py`. Handlers that no chat session owns
live in `tui_gateway/methods_connectors_account.py`.
The wire model is tolerant: an unknown facet reads as unclassified and one odd
tool never blanks a connector.
* feat(connectors): catalog, accounts and member tool rules by RPC
The Connectors page needs the app catalog, the connected account of one app,
a way to disconnect it, and the member's own on/off rules. None had an RPC.
- `connectors.catalog`: name, description, category and logo of each app.
- `connectors.accounts`, `connectors.accounts.remove`: read the accounts at
the tool gateway and remove one by id.
- `connectors.policy.get`: the rule layers that apply to the member, widest
first. The body is a union on `mode`, so a reader can name who turned a
tool off.
- `connectors.policy.set`: one change, a union on `type` (the tools of one
connector, or one connector on or off), with the revision the user saw. A
stale revision answers `POLICY_CONFLICT`. The backend composes the upstream
write in one pure function, so no renderer learns the upstream rules.
- Bundled MCP manifests can name their hosted twin with `connector:`, so the
page can show one card per app.
* feat(connectors): connect an app without a chat session
Every connector RPC took a `session_id`, and a connect that did not come from
the model's tool call minted a link with no watcher. The Connectors page has
no chat session, and its card must flip to connected by itself.
- `connectors.list`, `connectors.connect`, `connectors.operation.status`,
`connectors.operation.wake` and `connection.respond` take `owner`, a union
on `type`: `session` (today's behaviour and authorization) or `account`
(routed by `profile`, authorized by the live transport like `mcp.*`).
`session_id` is gone from these params; every desktop caller sends `owner`.
- An account connect runs the same operation lifecycle on a background
thread, under the profile's scope, so the watcher reads the account and
settles the operation. A second connect for an app that is already
connecting returns the open operation and mints nothing.
- `connection.update` carries `owner`. An account operation has no session to
address, so its updates go out on the session-less broadcast path.
* feat(mcp-catalog): eighteen more bundled entries name their hosted connector
A bundled MCP entry and a hosted connector for the same app are one card
on the Connectors page only when the manifest names its hosted twin.
Linear and Notion had the field. These entries get it too: airtable,
asana, attio, calendly, dropbox, figma, railway, supabase, todoist,
betterstack, canva, cloudflare, datadog, intercom, neon, sentry, stripe
and vercel. Atlassian maps to two hosted connectors and Prisma Postgres
is not clearly the same app, so both stay without one.
* refactor(connectors): the account handlers share one gate, one params model and one write table
The six account-level handlers each repeated the availability gate, the
auth catch and the catch-all reply. One decorator now owns that, and each
handler validates its params with its contract model instead of a ladder
of isinstance checks. The five connection RPCs share one guard for the
unexpected-failure reply.
The four write composers for the member rules were the same function
with a different list key and polarity. They are one table now.
The owner union lives in contracts/common.py, so the params side and the
event side stop declaring it twice and the import cycle is gone.
An account operation start carries one event and a flag, so the wait for
the sign-in link blocks instead of polling every 50 ms. run_operation
loses its two account-only parameters; drive_operation is the second
entry point.
Tests: four deleted (they exercised pydantic or the mock), three merged
into tables, two added (a client that still sends the old top-level
session_id is refused; all six account RPCs run off the server loop).
The shared reply helper and the HTTP and managed-client fakes move to
one place each. Comments are one line or gone.
* fix(connectors): a missing tool-list route reads as "unavailable", not "connector gone"
The tool-list read treated every 404 as the portal's "this connector is
not in the catalog" answer. It deleted the cache entry and answered
CONNECTOR_NOT_FOUND, so a page would offer to remove an app that is
connected and works. A portal that does not serve the route yet answers
a bare 404 for every app.
Only the portal's own {"error": "connector_not_found"} means the
connector is gone. Any other 404 is now a tool-list outage: the cached
list is served as stale, or the RPC answers TOOLS_UNAVAILABLE.
* fix(connectors): a connect from the page returns to the app after sign-in
The sign-in link carries a return target only when the session's surface
is the desktop. A chat session binds that surface. An account-owned call
has no chat session, so nothing bound it: the link was minted without a
return target and the browser ended on the portal's done page instead of
coming back to Hermes.
Every account-owned call now runs with the process's own surface bound,
next to its profile scope. The operation thread copies that context, so
the first link and every reissued link carry the return target and the
operation id.
* test(connectors): defer the new connector RPC coverage
The tests for the new account RPCs, the portal client, the tool-list cache
and the rule composer leave this PR and come back in one later change, after
the API is settled. The same was done for #111008.
Kept: the edits that existing tests need because the five connection RPCs
now take `owner` instead of `session_id`, and the rename of the managed
client seam.
Removed: six new test files, their two fakes and the gateway conftest, and
the new cases in test_mcp_catalog.py, test_connectors_gateway_client.py,
gateway-rpc.test.ts and notifications.test.ts. Reverting this commit restores
all of them.
* fix(cli): the connection panel hands the tool thread back at once
The classic CLI's connection callback waited on a queue for the user's first
decision. The operation's watcher starts only after the callback returns, and
the watcher is what polls a hosted account, runs the 300-second deadline and
sees Ctrl+C.
For a hosted connector the panel opens on the sign-in link, where the only
key that filled the queue was Cancel. The account was never polled: the user
signed in, the panel never changed, and Esc reported the app as skipped.
Ctrl+C set the interrupt flag but left the thread parked on the queue, so the
turn never ended.
The callback now opens the panel and returns, as the gateway's callback does
for the desktop and the Ink TUI. The panel's actions already reach the
operation through apply_answer on the UI thread, so the queue is removed. An
install with a form still waits for Connect, because the backend starts no
work for a pending row. Ctrl+C now settles the operation as `interrupt`, and
open rows become `not_connected`.
Checked on the e2e rig with the fake tool gateway: hosted connect completes on
the third status read; Ctrl+C ends the turn and the polling stops; an MCP
install with a plain and a secret field still saves config and both values.
* fix(connectors): "run it again" lives in the library, so the classic CLI can use it
Making a new sign-in link for a failed or expired hosted connector was
implemented only in the JSON-RPC layer (`_reissue`). The classic CLI does not
go through JSON-RPC: its Connect button on a failed row called apply_answer,
which does nothing for a hosted operation because it has no MCP runner. The
panel showed "Waiting…" until the deadline.
`tools.connectors.run.reissue(operation, names)` now holds the checks and the
per-kind action, and returns a refusal reason or None. The gateway maps each
reason to the same JSON-RPC error as before. The CLI calls it for a hosted
row; a refusal is shown on the row. MCP rows keep their path, because Connect
on a failed MCP row re-sends the form values.
Checked on the e2e rig: a scripted failed sign-in, then Connect: a second mint
with `reinitiate: true`, a new link with a new connection id, then connected.
* feat(connectors): the account list and disconnect go through the portal
`connectors.accounts` and `connectors.accounts.remove` called the tool
gateway. They now call the portal's account-management routes
(`GET /api/v1/connectors/accounts`, `DELETE /api/v1/connectors/accounts/{id}`),
which apply the organisation membership checks and write the disconnect audit
row. There is no fallback to the gateway when the portal is unavailable, and a
removal is never retried.
The read of ONE account stays on the gateway (`GET v1/connectors/accounts/{id}`):
the portal has no such route, and the operation watcher polls it once per second.
`ConnectorClient.list_accounts` and `delete_account` are removed. The removed
account's reply model carries `connector`, which both services send.
* fix(connectors): the account RPCs answer what the portal really sends
Checked against the portal source and against the staging and production
services.
- Errors are read from the upstream error code, not the HTTP status. A rule
write answered 409 for a stale revision and for a user with no organisation;
both read as "the policy changed". `org_required` is now `ORG_REQUIRED` and
403 `no_access` is `ORG_ACCESS_DENIED` on every account RPC; only a rejected
sign-in is `NEEDS_NOUS_AUTH`. `connectors.list` and `connectors.connect` with
the account owner map these too.
- `connectors.policy.get` and `connectors.policy.set` carry `effective`: the
portal's own result for this user, with its stamp and without provider or
subject ids. Nothing is recomputed locally.
- A rule write needs the revision the user saw: `expected_revision` is required
and must be a revision string; a bad one is refused before any HTTP call.
- A tool row carries `no_auth`; a list without the upstream flag is an invalid
answer, not `false`.
- `connectors.accounts.remove` returns the app of the removed account. An
invalid id is `INVALID_PARAMS`.
- The tool-list cache is per signed-in member (a hash of the token's `sub`),
so two Nous accounts on one profile do not share entries.
- A malformed slug is a local error, not a 404 from a server nobody called.
Live, staging: no revision and a malformed revision refused locally; a good
revision wrote one disabled Gmail tool and returned it in `effective`; the
same revision again answered `POLICY_CONFLICT`; the list row showed the tool;
the restore brought the member rules back to the start. Live, staging and
production, read-only: all 60 tool lists (5483 tools) parse.
* fix(connectors): the operation RPCs match their contract; a settled card cannot start a new link
Found by two adversarial reviews of the RPC layer and its types.
- `connectors.connect` from a chat session with no open operation is refused
(`UNKNOWN_OPERATION`). It used to call `manage_connections` through the tool
registry with no card: it made a link nobody watched, returned a reply
without the required `settled` field, and named an operation that was never
registered. There is one way into an operation: the agent's call, or the
account owner's `connectors.connect`. "Run it again" inside an open
operation is unchanged.
- `connection.update` for a session is routed by session key AND profile; two
profiles with the same key no longer cross-deliver a sign-in link. The event
payload gets the same redaction as the RPC replies.
- `connection.respond` runs on the long-handler pool: an approval can start MCP
OAuth discovery, which blocked every RPC of the gateway while it ran.
- `connectors.list` rows are a closed snake_case model: `connector`, `enabled`,
`connected`, `connection_status`, `status_reason`, `gateway_disabled_tools`.
The last one is display data: the gateway enforces the rules, the backend
only passes the list on. The phantom `name` and `description` are gone, and
the desktop uses the generated types instead of hand-written copies.
- `tools_listing` (model-only data) no longer rides on `connectors.operation.status`.
- `unavailable` is removed from the target states and settle reasons: nothing
produces it. The contract generator now fails when a contract enum and its
domain enum differ.
- `ConnectorErrorReason` is part of the generated TypeScript and OpenRPC.
- The desktop sends `connection.respond` on the socket that holds the session,
as wake and reissue already did.
- Contract violations are logged every time, at error level.
- An account connect whose prepare step is slow returns the live operation
instead of an error while the operation keeps running.
- The MCP-manifest `connector` field leaves this PR (it moves to a later one
on top of the catalog-reader change). `hermes_cli/mcp_catalog.py` and
`optional-mcps/` are untouched by this PR again.
anti-slop: no net-new findings (15 touched files).
* fix(connectors): the model gets no sign-in link wherever a card exists; side agents cannot connect
The flag that tells the model "a connection card exists" was the session
platform (`== "desktop"`). The Ink TUI and the classic CLI also draw a card,
so there a connector call on an unconnected app handed the model the raw
`connect_url` and told it to pass the link to the user.
- The agent turn now declares how a link can reach the user
(`tools/connectors/turn.py`): CARD when the agent was built with a
connection callback, SIDE for a subagent or a background turn, LINK for a
headless run (`-q`, cron, ACP, api_server, messaging). It is set once per
tool batch in the agent loop and read by the connector dispatch path, which
never sees the agent. The session platform decides return-to-app only.
- CARD: the result carries `connect_card_available` and our hint, never the
link and never the gateway's own hint.
- SIDE: subagents (`delegate_tool`), gateway background turns and the classic
CLI `/bg` are built with `side_agent=True`. They hold no `manage_connections`
tool on any path that derives the tool list, and a connector call on an
unconnected app gets no link, only "report this to the main agent".
- LINK is unchanged.
- The hosted path with no card builds a detached operation, as the MCP path
does, so no `connection.update` is emitted for an operation no client asked
for. Names and docstrings that said "off desktop" now say "no card".
- A settled card is dead on the desktop: `reissueConnectionTarget` and
`respondToConnectionRequest` share one guard and send nothing for a settled
or unknown operation.
- The model-facing settled result no longer carries `connection_id`; the model
repeated it to the user.
Shown on the real clients with a real model (rig, fake tool gateway): Ink TUI
and classic CLI get `connect_card_available` and no link, the model opens the
card, the account connects, the retried call succeeds; `-q` still gets the
link; a subagent and a background turn have no `manage_connections` and get
the no-link hint; on the desktop a card settled with Continue has no enabled
control and sends no RPC.
* feat(tools): every call made through tool_search + tool_call shows a real label on all three clients
A bridged call showed as a generic `tool_call` row in the Ink TUI and as
`⚡ tool_call` in the classic CLI, because the display looked the name up in
the tool registry and bridged names are made at run time. The desktop labelled
only batches that were all hosted connector calls, by parsing names itself.
- `tools/tool_labels.py` is the one place that turns a bridged call into a
label: kind, app, action, emoji and text. Hosted: `connectors__gmail__GMAIL_SEND_EMAIL`
→ "Gmail · send email". MCP: "Linear · list issues". A local deferred tool
keeps its own emoji, verb and primary-argument preview. A batch gets exactly
one label per entry, always; an entry with no name gets a generic label.
- Classic CLI: one row per inner call; the duration on the last row; the
failure text on the row of the call that failed. With friendly labels off
it prints what it printed before.
- Gateway: tool start, progress and complete events and stored transcript rows
carry a typed `labels` field. It does not depend on the classic CLI's
display setting. Clients no longer parse tool names.
- Ink TUI: rows from the labels; the verbose trail keeps Args and Result.
- Desktop: `ConnectorExecution` renders hosted, MCP and mixed turns from the
labels, one row per call. The labels reach the row under a key no tool
argument can use. The connect card it drew under a failed tool result is
gone: after `CONNECTION_REQUIRED` the one way in is the agent's own
`manage_connections` call.
- `tool_search` and `tool_describe` rows read "Searching tools · <query>" and
"Reading tool details · N tools".
Shown on the real desktop (video and screenshots), the Ink TUI and the classic
CLI with the rig: hosted rows, MCP rows, a two-entry batch, a failed entry, a
`CONNECTION_REQUIRED` row with no card under it, labels after a reload, and the
desktop rows with the classic CLI setting off.
* fix(connectors): the model can tell "hosted tools unavailable" from "no such tool"; manage_connections routes MCP names correctly
- A failed hosted search or describe used to return nothing, by design, so the
model saw only local tools and told the user that a connected app was
missing. The local results are unchanged; when the hosted leg failed, the
`tool_search` and `tool_describe` results carry
`connectors: {status: "unavailable", reason: "unreachable" | "sign_in_expired"}`
and one hint line. A rejected token is `sign_in_expired`; an entitlement
refusal or a shut gate adds nothing. `tool_describe` no longer lists those
names under `not_found` next to "search again".
- NS-932. The description now says which side a name belongs to: a bare name
is a hosted connector account; `mcp: true` only when the user asks for an MCP
server, a local server or an install, or when the name exists only in the
catalog; connect and reconnect are hosted verbs, install, enable and
authorize are MCP verbs. It names the three clients that draw a card.
- A misrouted target is refused with the call that works. Only when the
gateway does not know the connector (confirmed on that failure path) and the
name is a catalog entry does the target fail with "X is a local MCP server.
Call manage_connections with action install ...". It is a per-target
outcome: other targets of the same call keep their links and their card. A
vendor failure on a name both sides know stays an ordinary failed row. The
MCP side mirrors it, and never for an entry that is only not installed.
- "Do not re-ask after a skip or a timeout" no longer stops the model when the
USER asks for that app again; the description and the settled-result notes
say so. A builder saw the model refuse a direct user request.
Shown on the Ink TUI and the classic CLI with a real model: a dead gateway and
a 401; "connect fxmail" goes hosted; "install the fx-noauth MCP server" goes
MCP; "connect fx-noauth" reaches the MCP install card in one corrective round
with no hosted mint; a two-target call where one is misrouted still connects
the other with exactly one mint.
* fix(tui): the connection card answers every key, shows what is happening, and is dead once settled
Reproduced on the real Ink TUI with the rig, then fixed:
- The keyboard was dead during the sign-in wait: the card kept a `submitting`
flag that the normal OAuth path never cleared, and Esc went through the same
guard. The in-flight state now belongs to the answered row and clears when
that row moves, when any later frame of the operation arrives, or after
five seconds. Esc skips the row in every phase; Ctrl+C interrupts the turn
(the input handler had no branch for this overlay); Shift+arrows scroll the
transcript and the card ignores them; arrow keys no longer move the text
cursor and the field focus at once.
- The card was lost at turn idle: the overlay flag was cleared while the
operation stayed in the store, and a resume dropped the pending card. The
flag survives idle, a resume shows the pending card again, a session switch
clears it.
- States with no branch: `not_connected` and a row with no link fell into the
credential form; `expired` vanished with no note. The title and the row text
now name the action (connect, reconnect, install, enable, authorize); a
failed or expired row with no fields offers Try again / Skip; a failed row
WITH fields reopens the form over the typed draft, with the failure above it.
- A settled card is dead: at settle the overlay closes and one transcript line
per app states the outcome. A settled or dismissed operation id is
remembered, so no replay or resume can reopen its card. Esc in the last
"Finishing…" moment hides the card and still writes the outcome lines.
- A failed `connection.respond` and a browser that did not open are shown on
the card in one sentence.
Also: `tui_gateway/connector_payload.py` redacted the BOOLEAN `secret` flag of
a credential field to the string "[REDACTED]". On the desktop every credential
field therefore rendered as a password and lost its prefilled default. A
boolean is no longer redacted.
* chore(connectors): remove the comments and docstrings this branch added
Deletions only. Kept: tool directives (`# noqa`, `// eslint-disable`, ...),
`// SAFETY:` lines, and the docstrings of the contract models under
`tui_gateway/contracts/`, which become the descriptions in the generated
OpenRPC and TypeScript.
Checked that no code changed: every Python file has the same AST as before
once docstrings and `pass` are ignored (62 files), and every TypeScript file
prints the same with comments stripped by the TypeScript printer (32 files).
The generated contract files are unchanged.
* fix(connectors): a card restored after a reload answers again; every account RPC names auth and org failures
Found by the end-to-end runs on the pushed head.
- Desktop: after a window reload, Continue on the restored card sent nothing.
The answer looked up the backend that holds the session with the runtime
session id, the lookup wants the stored id, and a failed lookup returned
silently. When the lookup gives no owner the answer now goes out on the
window's active socket, which is what main does.
- `connectors.policy.get` answered `POLICY_UNAVAILABLE` for a rejected sign-in,
a refused scope, a non-member and a missing organisation alike: the handler
runs with the gateway's globals and did not import the reason enum, so its
own error mapping raised. `connectors.accounts.remove` caught auth failures
in its generic branch. `org_required` was mapped on `policy.set` only. All
six account RPCs now answer `NEEDS_NOUS_AUTH`, `FORBIDDEN_SCOPE`,
`ORG_ACCESS_DENIED` and `ORG_REQUIRED` for those four upstream answers.
gc_blobs carried two byte-identical `warning("malformed ledger line; blob
GC skipped"); return 0, 0` blocks — one under `except json.JSONDecodeError`,
one under `if not isinstance(row, dict)` (tools/skill_ledger.py:350-357;
simplify reuse #2, efficiency note, re-gate G2 S2). Funnel the decode
failure into the type guard (`row = None`) so the abort exists once.
Behaviour is unchanged for both the [malformed-json] and [non-dict-row]
parametrizations: any line that is not a JSON object still aborts the
sweep with the same warning.
compact_ledger, gc_blobs and list_entries each stamped the same prelude:
read the ledger, catch (OSError, UnicodeError), warn "skill_ledger: ledger
unreadable (%s); <what>", bail (tools/skill_ledger.py:305-310, :341-346,
:412-417 — simplify reuse #1). Fold them into a private sibling
_read_ledger(what, *, quiet_missing=False) -> Optional[bytes] that reads
bytes, validates the UTF-8 decode and warns once unless quiet_missing and
the ledger is merely absent (list_entries: a fresh install has no ledger).
Returns bytes rather than str so compact_ledger keeps reporting the on-disk
size in bytes_before instead of re-encoding. Warning texts ("compaction
skipped", "blob GC skipped", "listing empty") are unchanged, so the
existing invariant tests pass verbatim.
list_entries() (tools/skill_ledger.py:414) caught (OSError, UnicodeError) but
returned [] silently, so a ledger that EXISTS but cannot be decoded made
`hermes curator ledger` print "ledger is empty" and `hermes curator rollback`
report "no ledger entry with id" with no hint that the file is damaged — while
gc_blobs()/compact_ledger() already warn on the same condition (:309, :345).
Now warn for any read failure except FileNotFoundError (a missing ledger is the
normal empty state). `%s`-lazy logger call, same "ledger unreadable" prefix.
Also adds the regression test the UnicodeError widening (2213062284) lacked:
re-gate mutation M8 reverted the except clause to `except OSError:` and the
whole suite stayed green. New tests write b"\xff" to ledger_path() and assert
list_entries() == [], get_entry() is None and "listing empty" in caplog; a
second test pins that a merely missing ledger stays silent.
Red/green: `except OSError:` -> UnicodeDecodeError (1 red); warning dropped ->
caplog assert red; warning made unconditional -> missing-ledger test red.
The gc_blobs() docstring (tools/skill_ledger.py:334) only mentioned malformed lines as a
reason to abort the sweep; this branch added the unreadable/undecodable-ledger abort
without updating it. State both conditions and the invariant (blobs are kept).
list_entries() caught only OSError (tools/skill_ledger.py:409), so a ledger whose bytes
are not valid UTF-8 raised UnicodeDecodeError out of `hermes curator ledger`, get_entry()
and rollback_entry() — the very corrupt state gc_blobs() and compact_ledger() now
tolerate on this branch. Widen to (OSError, UnicodeError) and return [], matching the
existing "unreadable ledger == empty" contract; the read path is read-only, so nothing
is lost by treating the file as having no entries.
Manual probe (HERMES_HOME=<tmp>, ledger bytes b"\xff"):
before: list_entries() -> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff
after: list_entries() -> []; get_entry('abc') -> None;
rollback_entry('abc') -> (False, "no ledger entry with id 'abc'")
compact_ledger() returns (0, 0, 0) when the ledger cannot be read or decoded
(tools/skill_ledger.py:308-309) but did so silently, so `hermes curator ledger --compact`
reported a no-op compaction as if the ledger were simply empty. gc_blobs() already logs
"ledger unreadable (...); blob GC skipped" for the identical condition; emit the
symmetric warning here so the operator sees the ledger needs attention.
Manual probe (HERMES_HOME=<tmp>, ledger bytes b"\xff"):
WARNING skill_ledger: ledger unreadable ('utf-8' codec can't decode byte 0xff in
position 0: invalid start byte); compaction skipped
compact_ledger() -> (0, 0, 0); ledger bytes still b"\xff".
The regression test that pins this arrives in the following commit.
After json.loads() succeeds, gc_blobs() called row.get(...) unconditionally
(tools/skill_ledger.py:353). A syntactically valid but non-object line such as `[]`
or `"x"` raised AttributeError out of gc_blobs() and up through
`hermes curator ledger --compact`. list_entries() already tolerates such rows with an
isinstance(row, dict) check (~L414); mirror it here and abort the sweep with the same
"malformed ledger line; blob GC skipped" warning — a row we cannot interpret may still
hold blob references, so nothing may be deleted.
Test: `non-dict-row` param on the kept invariant test (ledger + b"[]\n"). RED with the
guard removed (AttributeError: 'list' object has no attribute 'get'), GREEN at head.
gc_blobs() aborts the sweep on a json.JSONDecodeError (tools/skill_ledger.py:351-352)
but did so silently, while the sibling read-failure abort a few lines above logs
"ledger unreadable (...); blob GC skipped". `hermes curator ledger --compact` therefore
printed "0 blobs removed" as if the sweep had run and found nothing. Emit the same
warning family so the operator learns the ledger needs repair before blobs can be GC'd.
Test: `malformed-json` param on the kept invariant test — (0, 0), blobs intact, and
"blob GC skipped" in caplog. RED with the warning removed
(assert 'blob GC skipped' in ''), GREEN at head.
compact_ledger() caught OSError on the read but decoded the bytes outside
the guard, so a ledger with invalid UTF-8 raised UnicodeDecodeError out of
`hermes curator ledger --compact`. Same escape class the gc_blobs() fix
closes; treat it identically (no-op, ledger left untouched).
The new read-failure branch in gc_blobs() returned (0, 0) silently, so
`hermes curator ledger --compact` printed "0 unreferenced blob(s) removed"
as if the sweep had run. Log a warning with the underlying error so an
operator can tell "nothing to reap" from "could not look".
The kept invariant test now also asserts the warning via caplog.
Pass ignore_cleanup_errors=True to the TemporaryDirectory that holds the
converted CAF->WAV file (tools/transcription_tools.py:420).
WHY: the sibling trim-dir cleanup a few lines below already swallows
errors (shutil.rmtree(..., ignore_errors=True)), but the CAF work dir did
not. On Windows an AV scanner or the search indexer briefly holding the
freshly written WAV makes rmtree raise out of TemporaryDirectory.__exit__,
turning an otherwise successful transcription into an exception. The
kwarg is available since 3.10; pyproject requires-python >= 3.11, and
plugins/teams_pipeline/meetings.py:199 already uses it.
No new test: the behaviour is stdlib.
skill_view loads SKILL.md whole and the content then rides in context for
every later call of the session, so body size is paid per turn. The only
size signal was the 100k hard cap in skill_manage, and agent-authored skills
grew by small patches until they sat right under it (31 of 432 local skills
over 40k chars, 7 at 100-115k; skill_view results averaging 32k chars).
- skill_linter: advisory `oversized-body` past _BODY_SOFT_BUDGET_CHARS (24k,
~3x the ~200-line standard; bundled skills average ~20k) naming the size,
the token estimate and the references/ split.
- skill_manage patch: attach lint findings the write INTRODUCED (diff of
rules before/after), so the crossing patch reports it once and a clean
patch on an already-large skill stays quiet. Create keeps reporting all.
- curator prompt: a body over the budget is itself a consolidation target.
- docs: skills.md linter paragraph.
Since operations[] became the only call shape (#97295) the batch's success
path rebuilt each row as {name, action, file_path, success} and dropped every
advisory key the per-op handler attached — lint_warnings/lint_hint from the
create linter and the org_sharing note. The model never saw a finding.
Carry those keys onto the row, mirroring what the failure path already does
for teaching payloads.
The #70760 commit widened _redact_process_result(result, *, task_id=) and passed the
caller's task_id from _handle_process. tests/gateway/test_completion_delivery.py
monkeypatches the function with a one-arg stub (CI red on #118845), and the caller's id
is the wrong identity anyway: a poll/log/wait result belongs to the process OWNER, whose
task_id the session record already carries. Resolve it inside the function from
result["task_id"] or the session looked up by result["session_id"]; the test pins the
owner's id ("t1") instead of the caller's.
A plugin registered on transform_terminal_output only ever saw foreground
`terminal` output: tools/terminal_tool_result.py::_apply_output_transform_hook
runs from finalize_foreground_result and nowhere else. Background output
reached the model through a different seam — process_manage poll/wait/log/kill
results, `list` previews and the completion/heartbeat/watch notifications all
pass through tools/process_registry.py::_redact_process_result — which redacted
but never transformed, so a fleet redaction or summarising plugin silently did
nothing for backgrounded commands.
Apply the same hook helper at that shared seam (one new
transform_process_output wrapper) and at the two gateway agent-notify sites
that read session.output_buffer directly. The order matches the foreground
path and teknium1's review note on #71401: hook first, redaction after, so a
replacement the plugin returns is still masked. returncode is None while the
process runs; env_type is not recorded per process and is passed empty.
Not changed: the spawn acknowledgement ("Background process started") carries
no command output, and the persistent local shell already goes through
_run_foreground and was transformed — the issue's reading of that branch was
wrong; the real gap was the process_manage/notification seam.
Fixes#70760
Slim redo of #71401 (Christopher-Schulze): same seam and ordering, without
the ANSI-stripping relocation and render helper.
Co-authored-by: Christopher <210261288+Christopher-Schulze@users.noreply.github.com>
(cherry picked from commit 84661de52f78ccb84054e1ded92b07166e40546d)
ToolRegistry.dispatch spread every injected keyword (task_id, session_id,
user_task, parent_agent, ...) into the handler, so a third-party plugin tool
written as `def handle(args)` raised TypeError on every call. The plugin
contract (plugins/AGENTS.md) says optional kwargs are signature-inspected, not
forwarded unconditionally; hooks already do this in plugins_dispatch. Handlers
taking **kwargs still receive the full payload. Docs updated: the narrow
signature is supported, **kwargs opts into the whole context.
Fixes#68318
credit: @vveerrgg #22146 (slim redo; #68636 is the later duplicate)
- hindsight-client==0.6.1 / mem0ai==2.0.10 exact pins made _is_satisfied() reject every newer
compatible release, so hermes update kept downgrading a working client and broke embedded
daemons whose DB a newer client had migrated (#86992, #39424, #98407, #99317). The lazy entries
now mirror the plugin manifests (>=0.6.1,<1 and >=2.0.10,<3); pyproject extras stay the floor
install. Slim redo of #99557 (nateEc) / #98416 / #98527 (ttomiczek) / #39754.
- A list-typed plugin.yaml is refused with "top level must be a mapping" instead of an
AttributeError swallowed as "Failed to parse" (#14066, discovery side).
- ``hooks:`` (the spelling bundled manifests carried) still populates provides_hooks (#108371).
- requires_hermes compared against stale editable-install dist metadata (0.21.0) while the
checkout ran 0.21.4, skipping plugins that required the release in use; hermes_cli.__version__
is now the source of truth, dist metadata only a fallback.
- PEP 440 pre/post suffixes glued to a segment (99.0.0rc1) made a clause permissive and an rc
running version disabled every gate; the segment parser drops the suffix.
- A plugin calling sys.exit() at import or in register() propagated SystemExit out of
discovery: the whole registry emptied and `hermes chat` exited 3 with no output. Load
isolation now covers SystemExit (KeyboardInterrupt still propagates).
- uv reads [tool.uv] exclude-newer from the cwd project only, so lazy/plugin dep installs
launched from $HOME, a gateway service or the Desktop backend were never quarantined; the uv
tier now runs from the checkout root when one exists.
- A flat user/project manifest naming a bundled key from a differently named directory no longer
displaces the bundled plugin (warn + skip); a same-named override is logged at INFO.
List reconciliation could race the live stdout reader, publish a completion before buffered descendant output was ingested, then close the pipe underneath that reader. For selectable POSIX pipes, mark the direct child exited but ask the reader to perform the final drain and remain the sole completion publisher. Keep the existing fallback for readers that cannot be coordinated.
7fe77f29a7 renamed cleanup_terminal_temp_cache's kwarg to max_idle_hours;
gateway/run.py::_run_media_cache_cleanup calls every registered cleanup
as fn(max_age_hours=24), so the terminal sweep raised TypeError inside
housekeeping on every tick. The kwarg is the shared cleanup_*_cache
signature (bot_relay notes the same parity); the idle semantics stay.
A test binds the loop's call shape against every registered cleanup.
Deleting an idle scratch entry left two things behind. Headless browsers
started by a lane's e2e run kept running for days with a "(deleted)" cwd
(about 20 on one host; the process registry never knew them because they
were grandchildren of a shell that had exited). Repos whose linked
worktree lived in the entry kept a dangling registration until someone
ran `git worktree prune` by hand (10 in one repo).
The prune now lives in hermes_constants_scratch (hermes_constants keeps the
entry point). Before an idle entry is removed, same-user processes whose
cwd is inside it, or inside any scratch path that no longer exists, are
TERMed then KILLed; the deleted-cwd sweep runs on every pass so orphans
from earlier deletions are caught too. `.git` files found in the entry
name their repo, which gets `git worktree prune` after the rmtree.
cache/terminal used its own 72h fixed-age sweep; it now shares the 24h
idle rule and the subtree check. `hermes doctor` warns about cache-root
directories over 1 GiB that neither pruner covers, since finished campaign
trees parked there sat for weeks (95 GB on one host). System-prompt
scratch line updated to match.
Twenty-five call sites told users to run
`python -c "from pm import sync_venv; sync_venv(['x'], explicit=True)"`
because `hermes pm install` only took package names. Add `--extra`
(repeatable; syncs the venv with the named extras and nothing else) and
pm.install_hint(extra), the single builder every site now uses, so the
advice stays correct when the command changes.
A cold PM runtime under allow_lazy_installs:false now reports the extra
the caller wanted and the command that provisions both, instead of a
bare "pm-runtime: not installed".
pm.security_packages calls it from the Tirith package; a leading
underscore hides a cross-module contract. The consumer still owns the
signature semantics (pm/security_packages._SignedBinary), so the lazy
pm→tools→pm reference stays; only the name changes.
pm.shell.bash() returned shutil.which("bash") first on Windows. On most
machines that is C:\Windows\System32\bash.exe, the WSL launcher stub
(exits 1, "no installed distributions"), so a source install without the
staged git package lost its shell — the #116818 regression main fixed in
0abc3b04. The System32 guard survived only in local._windows_bash_candidates,
which had no callers after _find_bash collapsed onto pm.shell.
Move the ladder into pm.shell as a pure function (Program Files Git first,
then PATH minus the System32/WindowsApps stubs) so it is testable on any
host, and delete the dead copy in tools.environments.local.
Every tool schema rides every API call; the first draft read like docs (111 est. tokens).
Kept the two facts the model needs — when to use it and what it delivers — the rest lives in
tools-reference.md.
The execute_code sandbox refuses background/notify modifiers on terminal(); heartbeat implies
notify and rides the same delivery path, so it joins the blocked set (and the stub-drift test's
mirror of it).
`terminal(background=true, heartbeat=N)` emits a `heartbeat` event every N seconds
(floor 60) carrying only the output produced since the previous one, plus the usual
completion notice. The agent stays current on a long bounded job (merge train, full
test suite, deploy) and reacts to a failure within N seconds instead of at exit.
Why: `notify=true` fires once at exit, so multi-hour merge trains ran silently — the
parent's prompt cache went cold and every "what's up" cost 5–10 rediscovery calls;
30 days of batch sessions show 4,930 hand status polls (14% of all tool calls).
`notify=[pattern]` cannot serve this: its lifetime cap (8) exists precisely to stop
periodic output from flooding the agent.
Mechanism: one daemon timer thread for every heartbeat session (reader threads block
on the pipe and cannot keep time); delta output via a total-ingested counter over the
rolling buffer; heartbeats only where the completion notice can be delivered (same
async-support/subagent gates), never after exit. `heartbeat_seconds` is checkpointed.
Rendered on CLI, gateway and TUI through the existing process-notification paths.
Conflicts, all inside the local-runtime / local-models subsystem:
- hermes_cli/local_runtime/binaries.py, hermes_cli/web_routers/local_models.py:
main's own transfer machinery (urllib ranged download, shutil.move publish) is
retired here — pm/downloader owns every fetch. Took ours.
- tests/hermes_cli/test_local_runtime_downloads.py: deleted here (its subject,
binaries._download, is gone); kept deleted.
- tests/hermes_cli/test_local_models_routes.py: ours (the pause/resume suite) plus
main's held-finished-file contract, re-seamed onto the pm downloader.
main's fix 515412f415 ("wait out a held finished download instead of copying it and
failing") is re-homed so it fits the PM model: the Windows rename that fails while an
antivirus or indexing scan still holds a just-closed multi-gigabyte file is now handled
once, in pm/downloader.replace_when_released, which is the single publish path for PM
archives and model files alike. A cleanup that cannot remove its leftover can no longer
mask the error that left it.
Tests: helper retries a transient refusal and lands the file; gives up with the
plain-language message chained to the OS error; a download publishes through two refused
renames with nothing left behind; a stuck hold reports the rename error, not the failed
cleanup. Route-level: the job lands the file and reports done.
_repair_object_shape() treated every dict carrying `required` as an object
declaration. A constraint fragment — {"required": ["chain"]} inside an
allOf/oneOf/anyOf/if branch, with no properties and no type — is not one.
Repairing it synthesised `properties: {}` and then pruned every name out of
`required`, so sibling branches collapsed into identical always-true schemas
and the enclosing oneOf had two matches: the tool appeared in the catalog and
every call failed validation.
Only the parameters-schema root still receives the dangling-`required` repair,
which is the single node handed to providers as the argument object.
`_submit_fal_request` (image) and `_submit_fal_video_request` (video plugin)
translated every managed-gateway 4xx into "This model may not yet be enabled
on the Nous Portal's FAL proxy — set FAL_KEY or pick a different model". For
HTTP 429 that remediation is wrong: the gateway body is RATE_LIMIT_EXCEEDED
with a retryAfter, the model is enabled, and agents reading the message
switched models or gave up. On one real install this fired 260 times in a
week (17% of image_generate calls).
Both surfaces now submit through one shared helper
(tools/fal_common.py::submit_managed_fal_with_rate_limit_retry): a 429 whose
Retry-After (header, else body error.retryAfter) fits a 30s cap is waited out
in interrupt-aware 0.5s slices and resubmitted once under a fresh
x-idempotency-key; a second 429, or an unknown/too-long Retry-After, raises
a ValueError that names the rate limit and tells the agent to retry later
rather than switch models. 429 can no longer reach the "may not yet be
enabled" text.
Reverts 93889b770d ("named profiles no longer inherit the root
profile's auth.json"). After the Desktop update every bot profile that had
relied on the root OpenAI Codex login failed with "No Codex credentials
stored. Run `hermes -p <bot> auth add openai-codex --type oauth`", and users
had to re-run the device-code flow once per bot (5-6 times in the field
report). Sharing one grant across profiles is the intended design: OAuth
refresh tokens are single-use, so ONE grant lives at the root, profiles
resolve it read-only, and a refresh under a profile writes the rotated chain
back to root (Codex / xAI write-through, borrowed-row pool bookkeeping,
forked-grant heal) — never a per-profile copy.
Restored: `_global_auth_file_path` / `_load_global_auth_store` fallback in
`_load_provider_state*` / `read_credential_pool` / `_provider_state_transaction`,
Codex + xAI root write-through, `credential_pool` borrowed-root persistence,
`heal_forked_single_use_oauth_grants`, `share_auth` on profile creation
(Desktop create dialog checkbox), and the docs. `profile_credential_audit.py`
(the `hermes update` "profiles without a provider" notice) is removed with it.
Kept from after #111724: `_save_codex_tokens(set_active=...)` for image gen,
the plugin-auth `status` dispatch and the external-login notice in
`hermes auth list`, and the registry-derived env-var hint in agent_init.
The connection-card backend (tools/connectors/mcp.py, Desktop/TUI/CLI
setup card) is a second install path that grew after this PR was opened
and wrote every declared env value to .env. Apply the same split as
install_entry: secret specs to .env, non-secret values inlined into the
server block. One invariant test, red without the backend change.
shutil.rmtree stops at the first entry it cannot unlink, and Git leaves loose object files read-only on Windows (r--r--r-- trees on POSIX package installs behave the same). Checkpoint clear, MCP catalog uninstall/reinstall and git-installed plugin removal all deleted clones with a bare rmtree, so each aborted mid-tree and left partial state.
Add utils.rmtree_readonly: clear the write bit on the failing entry and its parent, retry that one operation, and keep rmtree semantics for every other failure (both the 3.11 onerror and 3.12+ onexc callback shapes).
The first cut appended the recovery only in `_tcc_row`, which is built by the
0.10 fallback probes. Drivers that serve health_report (0.22+, the ones a
stale row actually bites) render their own tcc_* rows untouched, so `doctor`
never showed it. Apply the hint at the report seam like the display-count
guard so both paths get it.
Also: one CUA_DRIVER_BUNDLE_ID (permissions.py is the leaf; the daemon
imports it), the CLI derives the field set from the same table, and the
unverified "--upgrade repairs stale rows" claim is dropped from hint + docs
(the user hitting this is already on a current driver).
`hermes computer-use permissions status` and the doctor's fallback `tcc_*`
rows told a user whose System Settings toggle already showed CuaDriver ON to
"grant it in System Settings" — the one step that cannot help. macOS keys the
TCC row to the app's code-signing requirement; a row written for an earlier
CuaDriver build stops matching after a driver update and flipping the toggle
does not rewrite it (trycua/cua#3170, repaired by the cua-driver >= 0.22
installer on update). The daemon then reports Accessibility / Screen Recording
false while the pane shows ON, and every click silently no-ops (#99732).
`permissions.stale_tcc_grant_hint(*missing)` renders the reset for exactly
the missing services (`tccutil reset Accessibility|ScreenCapture
com.trycua.driver`, then `hermes computer-use permissions grant`); both
surfaces append it only when a grant is actually reported False. Docs: the
computer-use page still said bounded/unrestricted daemons run "under the
Hermes host identity" — they launch through CuaDriver.app since #95381 — and
gains the stale-row troubleshooting entry.
main folded the auto-archive housekeeping into per-profile state.db maintenance; the plugin
update-check chore stays. The consolidated-away TestWorkdirParallelPool stays removed. Two new
utf-8 reads in gateway/host_rendezvous.py read utf-8-sig.
Each per-profile pass could spend 15s against a 5s caller budget, so on a host with
several served profiles the trailing wildcard pass — the only one that stops the shared
MCP loop and its connections — might never run. shutdown_mcp_servers takes a timeout and
the caller gives each pass budget/(N+1).
The worker also ran under copy_context(), carrying a caller's HERMES_HOME override into
a pass that documents it has none: from inside profiles/poison's scope the wildcard pass
resolved live_home to that profile. It now starts from a fresh Context().
Both call sites replace `with suppress(Exception)` with a logged WARNING: the suppress
swallowed a real TypeError from this signature change, which surfaced only as a missing
teardown in an unrelated test.
