* feat(platform): resolver core with locate/inspect/probe tiers and ordered candidates
Every resource lookup needs one result shape and one cost contract. `locate` reads
metadata only, `inspect` may open files and call OS APIs in-process, `probe` is fresh
and the only tier that may spawn or connect. `Resolution.candidates` keeps probe order
so fan-out consumers can try every present binary.
Linear NS-921.
* feat(platform): AppResolver over AppDef with plist, PE, registry, and server.json sources
Desktop apps need presence, version, and liveness as separate observations. The runtime
file's bearer token is parsed, used for one request, and discarded inside the probe;
no public type carries it. Endpoints are accepted only when loopback with a numeric port.
* refactor(copilot): gh candidates through locate_command and the Homebrew table
First consumer of the resolver. The gh token probe still tries every present binary in
order; the allowlist loses its two copilot_auth rows.
* feat(platform): availability() over an application declaration
locate() + inspect() only, never probes; the fail-closed _version in
app.py treats a vendor's plist/PE/registry entry as untrusted input.
Salvaged from PR #118122; reads any object with requires_app,
min_version, app_for(os) — nothing here imports the MCP catalog.
* feat(platform): application declarations parsed into AppDef per OS
The parser slice of PR #118122's catalog manifest, re-homed as a
catalog-free module: whoever owns an MCP server declares the app it
fronts per OS and what it needs, and registers it here. Stdlib +
hermes_platform.resolver only. register/lookup/clear are the one seam
the MCP check_fn and the skill gate both read.
* feat(mcp): check_fn honours a registered application declaration
_make_check_fn ANDs the declared app's availability into the
connection-alive check; with nothing registered for the server the
behaviour is the pre-PR3 connection check. Provenance is explicit
registration, not endpoint matching. Returns a plain bool: the registry
caches bool(fn()).
* feat(skills): requires_apps gate through registered declarations
Offer-time filter beside environments:; names resolve through
hermes_platform.declaration, an unknown name hides the skill (fail
closed). The disk snapshot carries requires_apps and the fast path
re-evaluates it (snapshot version bumped to 3): app presence is a host
fact that changes without SKILL.md changing.
* docs: application declarations page
The plugin-facing schema reference: app: and requires: blocks,
availability() states, and the two gates that read the registry.
Registered under Extending > Plugins in the docs sidebar.
* test(platform): declaration parser, availability, gates
The PR3 app-block tests re-homed off the catalog: fixtures are dicts
passed to parse_declaration, the check_fn gate keys on explicit
registration (not endpoint matching), and the import-hygiene probe now
covers hermes_platform.declaration and resolver.availability.
_fire_reasoning_delta called stream_reasoning_deltas_enabled() on every reasoning
token; each call took _CONFIG_LOCK and paid load_config()'s full deepcopy on a
cache hit, so every concurrently streaming thread in a WebUI/gateway process
serialized behind one lock on the token path.
Cache the opt-in on the agent for the life of one stream (reset in
_reset_stream_delivery_tracking, so the next request re-reads config) and use
load_config_readonly() for the lookup itself.
(cherry picked from commit 20fab007e5c3e399e3923bb6d66649918292825c)
Only column-0 bullets participate in `_drop_repeated_recall_lines`.
An indented line is a continuation of the bullet above it (nested
child, provenance, wrapped prose) and is kept verbatim, so identical
children under two different parents both survive. The docstring
already promised this; the code now matches.
PROOF: with the old code, `- Project A\n - status: active\n- Project B\n
- status: active\n` lost B's child (probe s4_eff_nested.py) and the
new assertion in test_a_bullet_with_continuation_lines_is_never_touched
fails (AssertionError at :49); after the change the probe returns the
input unchanged and tests/agent/test_memory_context_dedupe.py passes.
`_RECALL_BULLET_RE.match(stripped)` was evaluated twice per line and the inline
comment restated the docstring's per-section scoping paragraph. Compute
`is_bullet` once and keep a one-line comment. Behaviour-neutral.
PROOF: mutation `is_bullet = True` → tests/agent/test_memory_context_dedupe.py
1 failed (repeated-bullet), restored → all 44 tests across the PR's four test
files pass; ruff clean; footguns clean; real import OK.
The per-section reset only fired on `#…`, `**…` and `---` lines. The
in-tree RetainDB provider writes prose headings (`Profile:`,
`Relevant memories:`, `Instructions:`), so its second section still
collapsed into the first: `Profile:\n- None\nRelevant memories:\n- None`
lost the second `- None` and left a heading claiming nothing — the
exact failure 931d240e14 set out to fix, for markdown headings only.
Now any non-empty, non-bullet line at column 0 (a heading of any style,
a `---`/`***`/`___` rule, prose) clears `seen`; bullets and indented
continuation lines stay inside the current section.
PROOF: tests/agent/test_memory_context_dedupe.py::
test_identical_placeholder_bullets_under_different_headings_both_survive
(RetainDB prose-heading shape + markdown shape). Red with
agent/memory_manager.py at the previous commit (prose case drops the
second `- None`), green after; the continuation-line carve-out test
stays green.
The `seen` set from #117399 was block-global, so identical placeholder
bullets under different headings ("- (none recorded)" under two stores)
collapsed into the first section and left the second heading claiming
nothing. Reset the set at every heading (`## …` / `**…**`, the PR's own
heading detection) and at `---`, so only a repeat inside the same
section is dropped.
Review follow-up on two real defects in the first cut, both reproduced before fixing.
A dropped duplicate left its continuation lines behind, and they re-parented under the surviving
bullet: `- prefers draft PRs` / ` (logged 12 Jan, supermemory)` followed by the same headline
logged elsewhere collapsed into ONE entry carrying BOTH provenance lines — inventing a record
neither provider reported, then stamping it into `api_content` to be replayed every turn. That is
worse than the duplicate the change set out to remove. A bullet that carries continuation lines is
now never dropped and never suppresses a later one, so two entries that share a headline and differ
underneath it both survive.
And `stripped[:1] in ("-", "*")` treated `**Preferences**` as a list item, so a repeated bold
section heading was silently deleted — contradicting the docstring's promise that headings survive.
The marker test now requires a real bullet: marker, whitespace, content (`[-*+]\s+\S`), which also
keeps `*emphasis*` out. Numbered items stay out of scope, and the docstring says so rather than
claiming "list items" generally.
Measured again on the same 166 real blocks: still 219,260 of 1,302,220 body bytes — 16.8%,
unchanged. Every duplicate in that sample is a self-contained bullet, so correctness cost nothing.
Tests: a bullet with continuation lines is untouched; a nested bullet is a child, not a repeat;
same-depth indented duplicates still collapse; bold headings, emphasis and numbered items are left
as written. Removing the continuation guard fails two. The tautological assertion the review flagged
is replaced by an exact whole-body comparison.
(cherry picked from commit 37a36f23658b8ca1e112f46ff60a705a66f80a5b)
Every user turn injects a `<memory-context>` block built from the providers' prefetch, and nothing
dedupes it: a provider merges several stores and `prefetch_all` merges several providers, so one
prefetch routinely surfaces the same fact two or three times.
It is not paid once. The composed block is stamped into the user row's `api_content` sidecar and
replayed verbatim on every later request for as long as that row is in context — deliberately, so
the prompt-cache prefix stays byte-stable. Compaction cannot reclaim it either: `_demote_tool_result_at`
bails on anything that is not `role == "tool"`, so a user row is never shrunk and drags its memory
dump along while counting against the tail budget. A duplicated bullet therefore costs its tokens
once per turn, for the life of the row, while telling the model nothing that same block has not
already said.
Measured on a real operator state.db (166 rows carrying a block, 46 sessions): 219,260 of 1,302,220
body bytes — 16.8%, ~54,815 tokens on first send alone — are list items byte-identical to an earlier
line of the SAME block. Median block ~2,034 tokens per user turn.
Only list items are considered, and only when identical after stripping: headings, prose, blank
lines and `---` rules are left exactly as the provider wrote them, so sections still read as
written and a line repeated deliberately as prose is untouched. Within one block only — deduping
against earlier turns would save far more (73% of bullet bytes in that sample were already sent
earlier in the session) but changes what the model sees on a turn, so it is not done here.
The "provider returned pre-wrapped context" warning stays keyed on sanitization alone: a deduped
bullet is routine, not a provider fault.
Fixes#117397
(cherry picked from commit e3b08ca9b1708a7cddc2f1a2191b51d60d194624)
* feat(connectors): the backend serves a connector's tool list, cached for 24 hours
The Connectors page opens one app and shows every tool it has. The backend
had no way to read that list.
- `tools/connectors/portal/`: a client for the portal's tool-list route and a
JSON cache under the Hermes home, one file per portal origin and connector.
An entry is fresh for 24 hours. After that the read revalidates with the
stored ETag: 304 keeps the list, 404 deletes the entry, an upstream failure
serves the stored list marked stale, and a 401 never serves the cache.
- `connectors.tools {slug, refresh}`: account-level, routed by `profile`, no
chat session. Errors carry a fixed `reason` from one closed set on the rail.
- Every connector model that is not operation state moves into
`tui_gateway/contracts/connectors.py`. Handlers that no chat session owns
live in `tui_gateway/methods_connectors_account.py`.
The wire model is tolerant: an unknown facet reads as unclassified and one odd
tool never blanks a connector.
* feat(connectors): catalog, accounts and member tool rules by RPC
The Connectors page needs the app catalog, the connected account of one app,
a way to disconnect it, and the member's own on/off rules. None had an RPC.
- `connectors.catalog`: name, description, category and logo of each app.
- `connectors.accounts`, `connectors.accounts.remove`: read the accounts at
the tool gateway and remove one by id.
- `connectors.policy.get`: the rule layers that apply to the member, widest
first. The body is a union on `mode`, so a reader can name who turned a
tool off.
- `connectors.policy.set`: one change, a union on `type` (the tools of one
connector, or one connector on or off), with the revision the user saw. A
stale revision answers `POLICY_CONFLICT`. The backend composes the upstream
write in one pure function, so no renderer learns the upstream rules.
- Bundled MCP manifests can name their hosted twin with `connector:`, so the
page can show one card per app.
* feat(connectors): connect an app without a chat session
Every connector RPC took a `session_id`, and a connect that did not come from
the model's tool call minted a link with no watcher. The Connectors page has
no chat session, and its card must flip to connected by itself.
- `connectors.list`, `connectors.connect`, `connectors.operation.status`,
`connectors.operation.wake` and `connection.respond` take `owner`, a union
on `type`: `session` (today's behaviour and authorization) or `account`
(routed by `profile`, authorized by the live transport like `mcp.*`).
`session_id` is gone from these params; every desktop caller sends `owner`.
- An account connect runs the same operation lifecycle on a background
thread, under the profile's scope, so the watcher reads the account and
settles the operation. A second connect for an app that is already
connecting returns the open operation and mints nothing.
- `connection.update` carries `owner`. An account operation has no session to
address, so its updates go out on the session-less broadcast path.
* feat(mcp-catalog): eighteen more bundled entries name their hosted connector
A bundled MCP entry and a hosted connector for the same app are one card
on the Connectors page only when the manifest names its hosted twin.
Linear and Notion had the field. These entries get it too: airtable,
asana, attio, calendly, dropbox, figma, railway, supabase, todoist,
betterstack, canva, cloudflare, datadog, intercom, neon, sentry, stripe
and vercel. Atlassian maps to two hosted connectors and Prisma Postgres
is not clearly the same app, so both stay without one.
* refactor(connectors): the account handlers share one gate, one params model and one write table
The six account-level handlers each repeated the availability gate, the
auth catch and the catch-all reply. One decorator now owns that, and each
handler validates its params with its contract model instead of a ladder
of isinstance checks. The five connection RPCs share one guard for the
unexpected-failure reply.
The four write composers for the member rules were the same function
with a different list key and polarity. They are one table now.
The owner union lives in contracts/common.py, so the params side and the
event side stop declaring it twice and the import cycle is gone.
An account operation start carries one event and a flag, so the wait for
the sign-in link blocks instead of polling every 50 ms. run_operation
loses its two account-only parameters; drive_operation is the second
entry point.
Tests: four deleted (they exercised pydantic or the mock), three merged
into tables, two added (a client that still sends the old top-level
session_id is refused; all six account RPCs run off the server loop).
The shared reply helper and the HTTP and managed-client fakes move to
one place each. Comments are one line or gone.
* fix(connectors): a missing tool-list route reads as "unavailable", not "connector gone"
The tool-list read treated every 404 as the portal's "this connector is
not in the catalog" answer. It deleted the cache entry and answered
CONNECTOR_NOT_FOUND, so a page would offer to remove an app that is
connected and works. A portal that does not serve the route yet answers
a bare 404 for every app.
Only the portal's own {"error": "connector_not_found"} means the
connector is gone. Any other 404 is now a tool-list outage: the cached
list is served as stale, or the RPC answers TOOLS_UNAVAILABLE.
* fix(connectors): a connect from the page returns to the app after sign-in
The sign-in link carries a return target only when the session's surface
is the desktop. A chat session binds that surface. An account-owned call
has no chat session, so nothing bound it: the link was minted without a
return target and the browser ended on the portal's done page instead of
coming back to Hermes.
Every account-owned call now runs with the process's own surface bound,
next to its profile scope. The operation thread copies that context, so
the first link and every reissued link carry the return target and the
operation id.
* test(connectors): defer the new connector RPC coverage
The tests for the new account RPCs, the portal client, the tool-list cache
and the rule composer leave this PR and come back in one later change, after
the API is settled. The same was done for #111008.
Kept: the edits that existing tests need because the five connection RPCs
now take `owner` instead of `session_id`, and the rename of the managed
client seam.
Removed: six new test files, their two fakes and the gateway conftest, and
the new cases in test_mcp_catalog.py, test_connectors_gateway_client.py,
gateway-rpc.test.ts and notifications.test.ts. Reverting this commit restores
all of them.
* fix(cli): the connection panel hands the tool thread back at once
The classic CLI's connection callback waited on a queue for the user's first
decision. The operation's watcher starts only after the callback returns, and
the watcher is what polls a hosted account, runs the 300-second deadline and
sees Ctrl+C.
For a hosted connector the panel opens on the sign-in link, where the only
key that filled the queue was Cancel. The account was never polled: the user
signed in, the panel never changed, and Esc reported the app as skipped.
Ctrl+C set the interrupt flag but left the thread parked on the queue, so the
turn never ended.
The callback now opens the panel and returns, as the gateway's callback does
for the desktop and the Ink TUI. The panel's actions already reach the
operation through apply_answer on the UI thread, so the queue is removed. An
install with a form still waits for Connect, because the backend starts no
work for a pending row. Ctrl+C now settles the operation as `interrupt`, and
open rows become `not_connected`.
Checked on the e2e rig with the fake tool gateway: hosted connect completes on
the third status read; Ctrl+C ends the turn and the polling stops; an MCP
install with a plain and a secret field still saves config and both values.
* fix(connectors): "run it again" lives in the library, so the classic CLI can use it
Making a new sign-in link for a failed or expired hosted connector was
implemented only in the JSON-RPC layer (`_reissue`). The classic CLI does not
go through JSON-RPC: its Connect button on a failed row called apply_answer,
which does nothing for a hosted operation because it has no MCP runner. The
panel showed "Waiting…" until the deadline.
`tools.connectors.run.reissue(operation, names)` now holds the checks and the
per-kind action, and returns a refusal reason or None. The gateway maps each
reason to the same JSON-RPC error as before. The CLI calls it for a hosted
row; a refusal is shown on the row. MCP rows keep their path, because Connect
on a failed MCP row re-sends the form values.
Checked on the e2e rig: a scripted failed sign-in, then Connect: a second mint
with `reinitiate: true`, a new link with a new connection id, then connected.
* feat(connectors): the account list and disconnect go through the portal
`connectors.accounts` and `connectors.accounts.remove` called the tool
gateway. They now call the portal's account-management routes
(`GET /api/v1/connectors/accounts`, `DELETE /api/v1/connectors/accounts/{id}`),
which apply the organisation membership checks and write the disconnect audit
row. There is no fallback to the gateway when the portal is unavailable, and a
removal is never retried.
The read of ONE account stays on the gateway (`GET v1/connectors/accounts/{id}`):
the portal has no such route, and the operation watcher polls it once per second.
`ConnectorClient.list_accounts` and `delete_account` are removed. The removed
account's reply model carries `connector`, which both services send.
* fix(connectors): the account RPCs answer what the portal really sends
Checked against the portal source and against the staging and production
services.
- Errors are read from the upstream error code, not the HTTP status. A rule
write answered 409 for a stale revision and for a user with no organisation;
both read as "the policy changed". `org_required` is now `ORG_REQUIRED` and
403 `no_access` is `ORG_ACCESS_DENIED` on every account RPC; only a rejected
sign-in is `NEEDS_NOUS_AUTH`. `connectors.list` and `connectors.connect` with
the account owner map these too.
- `connectors.policy.get` and `connectors.policy.set` carry `effective`: the
portal's own result for this user, with its stamp and without provider or
subject ids. Nothing is recomputed locally.
- A rule write needs the revision the user saw: `expected_revision` is required
and must be a revision string; a bad one is refused before any HTTP call.
- A tool row carries `no_auth`; a list without the upstream flag is an invalid
answer, not `false`.
- `connectors.accounts.remove` returns the app of the removed account. An
invalid id is `INVALID_PARAMS`.
- The tool-list cache is per signed-in member (a hash of the token's `sub`),
so two Nous accounts on one profile do not share entries.
- A malformed slug is a local error, not a 404 from a server nobody called.
Live, staging: no revision and a malformed revision refused locally; a good
revision wrote one disabled Gmail tool and returned it in `effective`; the
same revision again answered `POLICY_CONFLICT`; the list row showed the tool;
the restore brought the member rules back to the start. Live, staging and
production, read-only: all 60 tool lists (5483 tools) parse.
* fix(connectors): the operation RPCs match their contract; a settled card cannot start a new link
Found by two adversarial reviews of the RPC layer and its types.
- `connectors.connect` from a chat session with no open operation is refused
(`UNKNOWN_OPERATION`). It used to call `manage_connections` through the tool
registry with no card: it made a link nobody watched, returned a reply
without the required `settled` field, and named an operation that was never
registered. There is one way into an operation: the agent's call, or the
account owner's `connectors.connect`. "Run it again" inside an open
operation is unchanged.
- `connection.update` for a session is routed by session key AND profile; two
profiles with the same key no longer cross-deliver a sign-in link. The event
payload gets the same redaction as the RPC replies.
- `connection.respond` runs on the long-handler pool: an approval can start MCP
OAuth discovery, which blocked every RPC of the gateway while it ran.
- `connectors.list` rows are a closed snake_case model: `connector`, `enabled`,
`connected`, `connection_status`, `status_reason`, `gateway_disabled_tools`.
The last one is display data: the gateway enforces the rules, the backend
only passes the list on. The phantom `name` and `description` are gone, and
the desktop uses the generated types instead of hand-written copies.
- `tools_listing` (model-only data) no longer rides on `connectors.operation.status`.
- `unavailable` is removed from the target states and settle reasons: nothing
produces it. The contract generator now fails when a contract enum and its
domain enum differ.
- `ConnectorErrorReason` is part of the generated TypeScript and OpenRPC.
- The desktop sends `connection.respond` on the socket that holds the session,
as wake and reissue already did.
- Contract violations are logged every time, at error level.
- An account connect whose prepare step is slow returns the live operation
instead of an error while the operation keeps running.
- The MCP-manifest `connector` field leaves this PR (it moves to a later one
on top of the catalog-reader change). `hermes_cli/mcp_catalog.py` and
`optional-mcps/` are untouched by this PR again.
anti-slop: no net-new findings (15 touched files).
* fix(connectors): the model gets no sign-in link wherever a card exists; side agents cannot connect
The flag that tells the model "a connection card exists" was the session
platform (`== "desktop"`). The Ink TUI and the classic CLI also draw a card,
so there a connector call on an unconnected app handed the model the raw
`connect_url` and told it to pass the link to the user.
- The agent turn now declares how a link can reach the user
(`tools/connectors/turn.py`): CARD when the agent was built with a
connection callback, SIDE for a subagent or a background turn, LINK for a
headless run (`-q`, cron, ACP, api_server, messaging). It is set once per
tool batch in the agent loop and read by the connector dispatch path, which
never sees the agent. The session platform decides return-to-app only.
- CARD: the result carries `connect_card_available` and our hint, never the
link and never the gateway's own hint.
- SIDE: subagents (`delegate_tool`), gateway background turns and the classic
CLI `/bg` are built with `side_agent=True`. They hold no `manage_connections`
tool on any path that derives the tool list, and a connector call on an
unconnected app gets no link, only "report this to the main agent".
- LINK is unchanged.
- The hosted path with no card builds a detached operation, as the MCP path
does, so no `connection.update` is emitted for an operation no client asked
for. Names and docstrings that said "off desktop" now say "no card".
- A settled card is dead on the desktop: `reissueConnectionTarget` and
`respondToConnectionRequest` share one guard and send nothing for a settled
or unknown operation.
- The model-facing settled result no longer carries `connection_id`; the model
repeated it to the user.
Shown on the real clients with a real model (rig, fake tool gateway): Ink TUI
and classic CLI get `connect_card_available` and no link, the model opens the
card, the account connects, the retried call succeeds; `-q` still gets the
link; a subagent and a background turn have no `manage_connections` and get
the no-link hint; on the desktop a card settled with Continue has no enabled
control and sends no RPC.
* feat(tools): every call made through tool_search + tool_call shows a real label on all three clients
A bridged call showed as a generic `tool_call` row in the Ink TUI and as
`⚡ tool_call` in the classic CLI, because the display looked the name up in
the tool registry and bridged names are made at run time. The desktop labelled
only batches that were all hosted connector calls, by parsing names itself.
- `tools/tool_labels.py` is the one place that turns a bridged call into a
label: kind, app, action, emoji and text. Hosted: `connectors__gmail__GMAIL_SEND_EMAIL`
→ "Gmail · send email". MCP: "Linear · list issues". A local deferred tool
keeps its own emoji, verb and primary-argument preview. A batch gets exactly
one label per entry, always; an entry with no name gets a generic label.
- Classic CLI: one row per inner call; the duration on the last row; the
failure text on the row of the call that failed. With friendly labels off
it prints what it printed before.
- Gateway: tool start, progress and complete events and stored transcript rows
carry a typed `labels` field. It does not depend on the classic CLI's
display setting. Clients no longer parse tool names.
- Ink TUI: rows from the labels; the verbose trail keeps Args and Result.
- Desktop: `ConnectorExecution` renders hosted, MCP and mixed turns from the
labels, one row per call. The labels reach the row under a key no tool
argument can use. The connect card it drew under a failed tool result is
gone: after `CONNECTION_REQUIRED` the one way in is the agent's own
`manage_connections` call.
- `tool_search` and `tool_describe` rows read "Searching tools · <query>" and
"Reading tool details · N tools".
Shown on the real desktop (video and screenshots), the Ink TUI and the classic
CLI with the rig: hosted rows, MCP rows, a two-entry batch, a failed entry, a
`CONNECTION_REQUIRED` row with no card under it, labels after a reload, and the
desktop rows with the classic CLI setting off.
* fix(connectors): the model can tell "hosted tools unavailable" from "no such tool"; manage_connections routes MCP names correctly
- A failed hosted search or describe used to return nothing, by design, so the
model saw only local tools and told the user that a connected app was
missing. The local results are unchanged; when the hosted leg failed, the
`tool_search` and `tool_describe` results carry
`connectors: {status: "unavailable", reason: "unreachable" | "sign_in_expired"}`
and one hint line. A rejected token is `sign_in_expired`; an entitlement
refusal or a shut gate adds nothing. `tool_describe` no longer lists those
names under `not_found` next to "search again".
- NS-932. The description now says which side a name belongs to: a bare name
is a hosted connector account; `mcp: true` only when the user asks for an MCP
server, a local server or an install, or when the name exists only in the
catalog; connect and reconnect are hosted verbs, install, enable and
authorize are MCP verbs. It names the three clients that draw a card.
- A misrouted target is refused with the call that works. Only when the
gateway does not know the connector (confirmed on that failure path) and the
name is a catalog entry does the target fail with "X is a local MCP server.
Call manage_connections with action install ...". It is a per-target
outcome: other targets of the same call keep their links and their card. A
vendor failure on a name both sides know stays an ordinary failed row. The
MCP side mirrors it, and never for an entry that is only not installed.
- "Do not re-ask after a skip or a timeout" no longer stops the model when the
USER asks for that app again; the description and the settled-result notes
say so. A builder saw the model refuse a direct user request.
Shown on the Ink TUI and the classic CLI with a real model: a dead gateway and
a 401; "connect fxmail" goes hosted; "install the fx-noauth MCP server" goes
MCP; "connect fx-noauth" reaches the MCP install card in one corrective round
with no hosted mint; a two-target call where one is misrouted still connects
the other with exactly one mint.
* fix(tui): the connection card answers every key, shows what is happening, and is dead once settled
Reproduced on the real Ink TUI with the rig, then fixed:
- The keyboard was dead during the sign-in wait: the card kept a `submitting`
flag that the normal OAuth path never cleared, and Esc went through the same
guard. The in-flight state now belongs to the answered row and clears when
that row moves, when any later frame of the operation arrives, or after
five seconds. Esc skips the row in every phase; Ctrl+C interrupts the turn
(the input handler had no branch for this overlay); Shift+arrows scroll the
transcript and the card ignores them; arrow keys no longer move the text
cursor and the field focus at once.
- The card was lost at turn idle: the overlay flag was cleared while the
operation stayed in the store, and a resume dropped the pending card. The
flag survives idle, a resume shows the pending card again, a session switch
clears it.
- States with no branch: `not_connected` and a row with no link fell into the
credential form; `expired` vanished with no note. The title and the row text
now name the action (connect, reconnect, install, enable, authorize); a
failed or expired row with no fields offers Try again / Skip; a failed row
WITH fields reopens the form over the typed draft, with the failure above it.
- A settled card is dead: at settle the overlay closes and one transcript line
per app states the outcome. A settled or dismissed operation id is
remembered, so no replay or resume can reopen its card. Esc in the last
"Finishing…" moment hides the card and still writes the outcome lines.
- A failed `connection.respond` and a browser that did not open are shown on
the card in one sentence.
Also: `tui_gateway/connector_payload.py` redacted the BOOLEAN `secret` flag of
a credential field to the string "[REDACTED]". On the desktop every credential
field therefore rendered as a password and lost its prefilled default. A
boolean is no longer redacted.
* chore(connectors): remove the comments and docstrings this branch added
Deletions only. Kept: tool directives (`# noqa`, `// eslint-disable`, ...),
`// SAFETY:` lines, and the docstrings of the contract models under
`tui_gateway/contracts/`, which become the descriptions in the generated
OpenRPC and TypeScript.
Checked that no code changed: every Python file has the same AST as before
once docstrings and `pass` are ignored (62 files), and every TypeScript file
prints the same with comments stripped by the TypeScript printer (32 files).
The generated contract files are unchanged.
* fix(connectors): a card restored after a reload answers again; every account RPC names auth and org failures
Found by the end-to-end runs on the pushed head.
- Desktop: after a window reload, Continue on the restored card sent nothing.
The answer looked up the backend that holds the session with the runtime
session id, the lookup wants the stored id, and a failed lookup returned
silently. When the lookup gives no owner the answer now goes out on the
window's active socket, which is what main does.
- `connectors.policy.get` answered `POLICY_UNAVAILABLE` for a rejected sign-in,
a refused scope, a non-member and a missing organisation alike: the handler
runs with the gateway's globals and did not import the reason enum, so its
own error mapping raised. `connectors.accounts.remove` caught auth failures
in its generic branch. `org_required` was mapped on `policy.set` only. All
six account RPCs now answer `NEEDS_NOUS_AUTH`, `FORBIDDEN_SCOPE`,
`ORG_ACCESS_DENIED` and `ORG_REQUIRED` for those four upstream answers.
test_chat_sdk_transform_bypass stubs _relay_auxiliary_metadata with an empty
metadata dict; read aux_task/api_mode with .get so the seam keeps working.
Auxiliary LLM calls (titling, compression, MoA advisors/aggregator, vision,
approval, ...) never reached any plugin hook: hook-based observability and
cost plugins were structurally blind to them. Teknium's ruling on #79733:
NEW events rather than reusing the turn-scoped pre/post_api_request pair,
so existing subscribers keep their per-turn semantics.
- agent/auxiliary_hooks.py (new sibling): builds the pre_api_request /
post_api_request payload shape plus `aux_task`, `api_request_id`
(`aux-...`, shared by every attempt of one logical call), `retry_count`,
`streaming`, parent-turn `session_id`/`task_id`/`turn_id` when a main
turn is in flight; fail-open (a raising/hung subscriber is logged and
the aux task proceeds); post carries `error`/`error_type` on failure.
- agent/auxiliary_client.py: the three relay funnels every physical
attempt shares (_relay_sync_completion / _relay_async_completion /
_relay_sync_stream) run under the hook pair — retries and fallbacks
included. Main-loop *_api_request events do not fire for aux calls.
- Catalogue: VALID_HOOKS, bounded-timeout hook set, `hermes hooks test`
sample payloads, hooks.md / plugins index / observer-hooks / plugins.md
tables, agent + plugins AGENTS.md.
- tests/agent/test_auxiliary_hooks.py: 2 invariants (pair fires with
aux_task and no api_request events; raising subscriber never breaks
the call). First is red on origin/main.
Supersedes #32416 (@zrmnelson), #68060 (@JonZal), #77518 (@hsy5571615),
#79826 (@webtecnica) — their relay-boundary placement, usage
normalisation and fail-open policy shaped this implementation.
Co-authored-by: zrmnelson <zacharynelson1@gmail.com>
Co-authored-by: Jonas Zalys <jonas@tryholo.ai>
Co-authored-by: saitsuki <nukuom976228@gmail.com>
Co-authored-by: webtecnica <webtecnica@gmail.com>
`_finish` (agent/context_compressor.py:3586-3588) only wrapped
`_coverage` and re-derived `shown` from the final `selected`; it was
called once, at the return. With `_merged` hoisted above the slice loop
the classmethod now has two closures (`_merged`, `_render`) plus
`_coverage` instead of four. Same expressions, same evaluation order
(`_render` first, then the coverage counters), so the output and
coverage dict are byte-identical — verified on the 47-shape capture and
probe_C2 (0 violations, base identity x4 True).
`_bound_oversized_record` (agent/context_compressor.py:3485-3488) split
`remaining` with float arithmetic (`int(remaining * 0.5)`) and guarded
the tail slice with `if tail_len else ""`. `remaining = limit -
marker_reserve` is >= 1 because the line above already returned when
`limit <= marker_reserve`, so `head_len = remaining // 2 <= remaining - 1`
and `tail_len = remaining - head_len >= 1` always: the guard can never
fire. Use `remaining // 2` and drop the dead guard.
Byte-identical: `int(r * 0.5) == r // 2` holds for every non-negative
int (checked exhaustively for r in 0..10**6), and the guard was dead.
Capture of 800 bounded-record cases (4 record shapes x remaining 1..200)
plus the 47-shape sampling capture is identical before/after.
The `if cursor < len(records)` block at the end of `_render`
(agent/context_compressor.py:3580-3583) duplicated the gap-marker
arithmetic of the in-loop branch but had no path to it.
Invariant: `_render` is only reached with non-empty `records`
(empty returns early), and n >= 1, so the slice-build loop always runs
its last iteration, which anchors `end = len(records)` and
`start = end - 1 >= 0`, hence `end > start` and the final slice ends at
`len(records)`. `_merged` keeps `max(end)` for the last interval, and the
extension pass only grows the newest slice backward (`(s - 1, e)`) or
older slices forward but never past the next slice's start, so the last
slice's end stays `len(records)`. Therefore `cursor == len(records)`
after the loop for every input.
Byte-identical across the 47-shape capture (40 probe_C2 shapes + 7 edge
shapes incl. single-record and empty); probe_C2 -> 0 violations;
under-cap base identity x4 True.
The slice-build loop in _sample_summary_records hand-rolled the same
"overlapping/touching slice -> extend previous" fold that the `_merged`
closure 25 lines below implements (agent/context_compressor.py:3563-3568
vs :3590-3597). Hoist `_merged` above the loop, append raw (start, end)
pairs, and merge once before the extension pass; the first `_render`
no longer wraps `selected` in a redundant `_merged` since it is already
merged. One interval-merge implementation instead of two.
Output is byte-identical: the merge is applied once to the same ordered
slice list, so the extension pass starts from the same `selected`.
Verified with a 47-shape capture (40 probe_C2 shapes + 7 edge shapes)
diffed before/after: identical sampled text and coverage; probe_C2
40 shapes -> 0 violations; under-cap base identity x4 True.
The comment claimed "marker widths shrink as records leave a gap" — but
shrinkage can only make the render smaller, which the pre-check already
tolerates. The exact re-render exists for the opposite case: a gap's
first index can gain a digit or thousands separator (999 -> 1,000, +2
chars) when the added record moves a marker, and that growth can push a
render sitting at cap over it. Comment only; no behaviour change.
Finding: $D/regate/C.md suggestion 1 (agent/context_compressor.py:3619).
The budget-extension pass in _sample_summary_records grew the newest
slice until it hit the cap and only then moved to older slices, so on
mid-size records all headroom went to one region: 10K x 100 records ->
records/slice [1,1,1,1,1,1,1,8] (newest 4.3x the mean). That contradicts
the design comment above (regions must not consume each other's budget)
and the docs' "evenly sampled". Wrap the per-slice loop in a
`while grew` round so each slice adds at most ONE whole record per
round (newest grows backward, others forward); the cap pre-check and
exact re-render check are unchanged.
Before -> after (records per slice, fill unchanged):
10000x100 [1,1,1,1,1,1,1,8] -> [1,2,2,2,2,2,2,2] fill 0.944
8000x100 [2,2,2,2,2,2,2,5] -> [2,2,2,2,2,3,3,3] fill 0.957
4000x300 [4,4,4,4,4,4,4,11] -> [4,5,5,5,5,5,5,5] fill 0.987
2000x600 [9,9,9,9,9,9,9,15] -> [9,9,10,10,10,10,10,10] fill 0.995
500x2000 [37,...,37,39] -> [37,...,37,38,38] fill 0.998
Re-gate probe (40 random shapes): 0 violations, worst fill 0.892 -> 0.930,
worst char drift 4.27 -> 1.83; under-cap output byte-identical to base.
Test: test_lean_sampling_oversized_middle_record_does_not_evict_tail now
asserts no slice holds more than mean+1 records (RED with the previous
newest-first order: [16,16,16,16,16,16,18]; GREEN: [16,16,16,16,16,17,17]).
Finding: $D/regate/C.md W1 (agent/context_compressor.py:3599-3623).
The coverage docstring (agent/context_compressor.py:3510-3511) said the
counters count "record content only", but `sampled_chars` sums the
*display* records (post `_bound_oversized_record` truncation) while
`input_chars` sums the raw records. Spell that out so telemetry
consumers do not compute `omitted` two different ways (gate 2c
suggestion, L3512-3513/3524-3525).
`_begin_compression_telemetry` (agent/context_compressor.py:1947-1960)
did not seed the six `summary_input_*` counters that
`_record_summary_input_coverage` adds on the lean path, so the emitted
`compression_attempt` line (conversation_compression.py:1451-1470) had a
different key set for lean and legacy attempts. Seed them as None so the
schema is stable; probe: lean and legacy attempts now emit identical key
sets (gate 2c suggestion, L1947-1960).
The whole-record greedy fill in `_sample_summary_records`
(agent/context_compressor.py:3541,3555-3563) stops a slice as soon as the
next record would cross `target` (~20K), so each slice can be short by up
to one record. With mid-sized records that wastes a large share of the
160K cap: probe worst case (10K records) filled 90,660/160,000 (57%),
8K records 129,021 (81%); main filled 159,988 by splitting records.
Add a bounded extension pass after `selected` is built: newest slice
grows backward, older slices grow forward, one whole record at a time,
only while the exact rendered length stays <= cap and the slice does not
run into its neighbour (adjacent slices are merged so no separator is
lost). Records are never split and the cap is never exceeded.
After: worst random shape 150,581 (94%, remaining headroom < one 10K
record), 8K records 153,117 (96%), 1551-record region 159,996 with
968 whole records, 0 partial, 8 buckets within +-10%, telemetry
input_chars == sampled_chars + omitted_chars (probe: $D/fold/probe_C_fold.py).
The existing keeps_record_boundaries fixture (1.2K records) fills 99.1%
without this pass, so no fill assert was added there — it would be toothless.
Delete the post-render overflow-trim loop (agent/context_compressor.py
:3591-3607) and state the invariant in its place.
Why it is unreachable: every display record is pre-bounded to `target`
via _bound_oversized_record; each slice's greedy fill stops before
`size + sep + next > target`, so a slice renders to <= target chars and
the n slices to <= n*target (a merge of overlapping/adjacent slices is
their union: overlap only removes chars, adjacency adds one 2-char
separator but drops one marker of >= 2 chars). There are at most n-1
markers, each <= marker_len because marker_len is formatted with
first=last=len(records) and elided=total_len, the maxima. Since
target = (cap - (n-1)*marker_len) // n, the rendered length is
<= n*target + (n-1)*marker_len <= cap. Gate mutation M5 (loop disabled)
left all 7 sampler tests green; 30 random shapes never entered it.
Delete the `if len(res) > limit` re-trim at the end of
`_bound_oversized_record` (agent/context_compressor.py:3488-3489).
Why it can never fire: `marker_reserve` is the marker width formatted
with `elided=len(record)`, the largest value `elided` can take, so the
real marker is <= marker_reserve; `head` is `record[:head_len].rstrip`
(<= head_len) and `tail` is `record[-tail_len:].lstrip` (<= tail_len);
head_len + tail_len == limit - marker_reserve. Hence
len(head + marker + tail) <= limit always. The branch was untested and
unreachable (gate 2c W2).
Lean sampling is the documented budget mechanism, but nothing reported
how much of a region actually reached the summarizer, which is the gap
the reporter narrowed #118362 to. _sample_summary_records now returns
record-level coverage counters alongside the bounded transcript and
_generate_summary writes them into the active compression telemetry
(summary_input_chars / sampled_chars / omitted_chars / record_count /
sampled_record_count / elided_record_count), so the existing
compression_attempt JSON log line carries them with no transcript
content. Char counters cover record content only, not separators or
markers. Elision markers also name the one-based record range they
stand for, so a reader can tell how many turns a gap hides, not only
how many characters.
Co-authored-by: KoNit-K <konit.block@protonmail.com>
_sample_summary_input accepted either a flat string or a record
sequence and, for strings, rebuilt records with content.split("\n\n").
The PR's own third commit established that "\n\n" is not a record
delimiter (message bodies contain blank lines), so that fallback would
re-introduce split records for any caller that reached it. The only
production caller already passes records to _sample_summary_records;
drop the dual-signature wrapper and move the test callers to records.
The record split added .rstrip("\n") to every serialized record. That
changes the bytes _serialize_for_summary returns for legacy tail_mode
and for agent/micro_compaction.py's _micro_summarize_one, which never
sample and never needed it: a message with trailing newlines was no
longer reproduced verbatim in the summarizer prompt. Sampling receives
records structurally, so an empty pseudo-record can no longer become the
tail anchor and the strip has no remaining purpose.
Preserve record framing structurally so messages containing internal blank lines are not split into pseudo-records.
- agent/context_compressor.py:
* Extract _serialize_records_for_summary() to retain turn boundaries as a sequence of records.
* Add _sample_summary_records() to sample across serialized turn records directly without lossy double-newline splitting.
* Correct separator accounting in omission markers at gap boundaries so separator counts match full serialized text.
* Update _sample_summary_input() to support structural records while preserving flat-string fallback with trailing fragment pruning.
* Wire _generate_summary() to use _serialize_records_for_summary() and _sample_summary_records() in lean mode.
- tests/agent/test_context_compressor.py:
* Test multi-paragraph serialized messages maintain role boundaries and keep newest message paragraphs together.
* Test newest message ending in blank lines does not produce empty tail anchor.
* Test exact omission marker separator accounting.
* Test end-to-end _generate_summary prompt assembly with multi-paragraph messages.
(cherry picked from commit f68f1bfcf037dc5ddecf6cc4032023cd88c496c8)
The compound refusal check `_response_refusal_text(response) or
_is_summary_refusal(content)` was spelled out verbatim at both call sites
(agent/context_compressor.py:3639 in `_call_summary_llm` and
agent/micro_compaction.py:157 in `_micro_summarize_one`, the latter through
two separate `_cc()` facade lookups). `_classify_summary_failure` already
describes it as one concept ("refusal content (prose or provider `refusal`
field)"), and the ordering invariant — an explicit provider `refusal` field
wins over summary-shaped content — lived in two callers instead of one place.
Add `_is_refusal_response(response, content)` next to the detectors and call
it from both sites (micro: one `_cc()` lookup). `_is_summary_refusal` and
`_response_refusal_text` stay as-is for the unit tests. Behaviour unchanged.
Proof: guard + micro test files green (62 passed); mutating the new predicate
to `return False` fails exactly the batch refusal-body test, the provider
refusal-field test and the micro refusal test (3 failed, 59 passed).
`_response_refusal_text` (agent/context_compressor.py:199-201) returned ""
early when `_coerce_llm_message` yielded None or a str. That branch is dead:
`_message_field` (agent/auxiliary_client.py:8014) is
`msg.get(name) if isinstance(msg, dict) else getattr(msg, name, None)`, and
`getattr(None, "refusal", None)` / `getattr("x", "refusal", None)` are both
None, which the `isinstance(refusal, str)` tail already maps to "". So the
guard could never change the result — invariant: a non-dict, non-object
message has no `refusal` attribute, so the field lookup is None.
Probe (None, "x", bare dict, dict-with-choices, choices=None, dict refusal,
str message, refusal=None) returns ['', '', 'no', 'nope', '', 'r', '', '']
both before and after this change; the guard test file stays green.
Repo convention: no defensive guards around code that cannot fail.
agent/context_compressor.py:_classify_summary_failure (L697 marker list)
maps the "refusal content" RuntimeError onto empty_content without saying
so in the docstring, which makes the "LLM returned empty content" fallback
log line look wrong when the real cause was a refusal. Document the
intentional reuse (cooldown + main-model fallback + abort) in one place.
Gate finding: B.2ab (misleading fallback log for refusals).
agent/context_compressor.py:_is_summary_refusal (L177-184) matched on the
opener only, so a genuine templated summary that starts with a hedge such as
"I cannot see the earlier turns, but here is the summary:\n## Goal ..."
was classified as a refusal and compression aborted (gate probe: the
[preamble] compress() run aborted with "refusal content"). A refusal-only body
never carries the summary template's "## " section headings, while a real
summary always does, so exempt any body containing a "## " heading after the
prefix match.
Test: TestSummaryRefusalGuard::test_narrowing_and_heading_exemption. Red when
the heading exemption is removed (L91 assert True is False) and red when the
summary-term narrowing is dropped via an early `return True` (L85 assert True
is False, pins gate mutation M3); green at head (25 passed). Live probe:
[preamble] now compresses 25->7 with prev_summary_set=True.
Gate finding: B.2ab (false positive on prose-preambled summary).
agent/context_compressor.py:_SUMMARY_REFUSAL_PREFIX_RE (L170) only matched
"cannot"/"can't", so the two-word "I can not provide a summary" slipped
through, and _is_summary_refusal (L184) looked for the exact terms
summary/summarize/checkpoint, so British "I can't summarise this" was
accepted as a checkpoint. Allow optional whitespace in can(?:\s*not|['’]t)
and match the stems ("summar", "checkpoint") instead.
Test: two new params on TestSummaryRefusalGuard::test_rejects_refusal_body.
Red with the previous prod file (2 failed: assert False is True), green at
head (24 passed).
Gate finding: B.2c (detector misses "can not" / "summarise").
agent/context_compressor.py:_response_refusal_text (L187-201) re-implemented
the choices[0].message traversal that agent/auxiliary_client.py already
exposes as _coerce_llm_message (L7997) + _message_field (L8014) and that the
sibling extract_content_or_reasoning uses on the very same response. Use the
shared helpers so the two extractors cannot drift on response shape
(dict-shaped proxies vs ChatCompletion objects vs bare messages).
Behaviour kept: None / SDK object / dict-with-choices / missing attr /
refusal=None -> "" ; the dict-unwrap (message/reason/text) + isinstance(str)
tail is unchanged. Gate live probe ($D/gate/B_live_probe.py) output is
byte-identical before and after; the guard test file is green.
Gate finding: B.2c (duplicate response traversal).
OpenAI-style structured-output refusals arrive in
`choices[0].message.refusal` with `content` empty or filler, so the
prose detector never sees the refusal and the filler would be committed
as the compaction checkpoint. Read that field (str, or the dict shapes
some proxies emit) in both the batch (`_call_summary_llm`) and micro
(`_micro_summarize_one`) paths and route it through the same
"refusal content" failure so it can never become `_previous_summary`.
Also widen the prose opener with #118406's extra shapes — `we` as
subject, `refuse to`, `could not` / `couldn't`, `apologise`, and a bare
`I'm unable` / `I am not able` opener. #118406's `could(?:not|['’]t)`
missed "couldn't" (it only matched "couldnot"/"could't"); spelled out
here as `could\s*not|couldn['’]t`.
Co-authored-by: Mohamad Kanso <91088196+MohamadKanso@users.noreply.github.com>
The refusal guard in _micro_summarize_one already returns None, which
routes through _record_micro_failure like every other failed
summarization. Snapshotting and restoring _micro_compact_cursor around
that path changed the behaviour of EVERY summarize_failed outcome, not
just refusals, to satisfy one test's cursor assertion. The cursor write
in _resolve_compact_cursor is an idempotent re-resolution, so there is
nothing to undo.
The test now resolves the cursor via _next_exchange before snapshotting
it, so it still proves the source exchange survives a refusal without
needing production code to rewind state.
Builds on #72026 (@PRATHAMESH75): list content carries the turn's memory-prefetch /
pre_llm_call context as a durable text part appended once in the prologue, in every
api mode (MoA and codex_app_server included), so the request, the persisted row,
compaction and a later resume all see the message the model saw.
Persistence gap from the #72026 review: in-place preflight compaction (and a
close/early flush that races the prologue) writes the current user row BEFORE the
part exists and the crash persist identity-skips that dict, so a resumed session
replayed the turn without the context. The list branch now pushes the appended part
into that row via set_user_message_content under the same _row_id-under-lock
protocol as the string sidecar backfill, keeping the writer's shape (compaction: raw
parts; flush: text projection).
Titling moves before the injection step so a list turn's title is derived from the
user's ask, not the injected tail. Tests trimmed to one invariant per layer: hook
edit reaches the wire on a list turn and replays after reload; in-place compaction
+ reload keeps the part (red without the backfill); memory query flattens parts.
The delivery half of this PR lets a multimodal (list) turn carry the memory /
pre_llm_call context via a durable text part. But the execution half never ran
on those turns: `_memory_turn_start_and_prefetch` keyed the query off
`isinstance(str)`, so a list turn collapsed to `_query = ""` — `on_turn_start`
saw an empty turn and `is_trivial_prompt("")` skipped `prefetch_all` entirely.
So on an image+text turn recall never fired and the sidecar had nothing to
deliver, silently (no `recall` audit rows, no `prefetch failed` warning).
Flatten str/list content to its text via `flatten_message_text` before building
the query. An image-only turn still flattens to "" and is correctly treated as
trivial (no semantic text to query on); a text+image turn now runs prefetch on
its text. Adds unit coverage for the flattening and the trivial-prompt gate.
Reported by @albert748 on #72026.
compose_user_api_content returns None for non-string (multimodal) user
content, so the pre_llm_call plugin context and memory-prefetch block —
which ride the string api_content sidecar — were silently dropped on
image-only/attachment turns. A profile plugin's pre_llm_call {"context":
...} return could route text turns but never image-only turns.
Deliver the composed context as a durable text part on multimodal turns
via the existing append_notes_to_multimodal_content channel (the same one
gateway must-deliver notes already use), so the injected context reaches
the model and the wire stays byte-identical to the persisted/replayed
transcript. String turns keep the api_content sidecar path unchanged; the
MoA/codex_app_server guards are preserved. Extract the shared injection
composition into _context_injection_parts so both paths inject byte-
identical context.
skill_view loads SKILL.md whole and the content then rides in context for
every later call of the session, so body size is paid per turn. The only
size signal was the 100k hard cap in skill_manage, and agent-authored skills
grew by small patches until they sat right under it (31 of 432 local skills
over 40k chars, 7 at 100-115k; skill_view results averaging 32k chars).
- skill_linter: advisory `oversized-body` past _BODY_SOFT_BUDGET_CHARS (24k,
~3x the ~200-line standard; bundled skills average ~20k) naming the size,
the token estimate and the references/ split.
- skill_manage patch: attach lint findings the write INTRODUCED (diff of
rules before/after), so the crossing patch reports it once and a clean
patch on an already-large skill stays quiet. Create keeps reporting all.
- curator prompt: a body over the budget is itself a consolidation target.
- docs: skills.md linter paragraph.
A plugin's transform_llm_output replacement reached final_response only.
agent/turn_finalizer.py::finalize_turn fired the hook after the assistant
row had already been persisted — and the row is first written even earlier,
in agent/turn_final_response.py::finish_text_response's durable flush — so
messages[-1], the SQLite/JSON session, /resume and the next turn's replay all
kept the raw model text while the user had seen the rewritten one.
Writing the transformed text back after that flush cannot work: SQLite treats
a non-blank assistant row as settled (resolve_and_repair_transcript_batch
adopts the stored content instead of overwriting it), so the only correct seam
is BEFORE the row is first persisted. apply_llm_output_transform (new, in
turn_finalizer) fires the hook once per turn_id and records the outcome;
finish_text_response calls it ahead of append+flush and writes the result into
the row (api_content for the promoted-reasoning sidecar), finalize_turn's
_persist_step calls it ahead of the recovery-path tail close, and
_apply_output_hooks reads the recorded outcome (firing only when no earlier
seam saw a response) before post_llm_call. Only the current turn's not-yet-
written text changes — earlier turns and the system prompt are untouched.
post_llm_call is unchanged: it is an observer whose return is ignored by
contract, so there is nothing of it to persist (#14913/#44253's premise).
Fixes#44239
Slim redo of #44244 (AIalliAI, earliest; same sync-then-persist idea, moved to
the pre-flush seam) — also supersedes #65921 (SingleVirgin, sibling fix).
Co-authored-by: AIalliAI <285906080+AIalliAI@users.noreply.github.com>
(cherry picked from commit 5fe02a0aeecef422a4ffb4ff4385015f6a1528a0)
Agent-runtime tools (todo_list, session_search, memory, clarify,
delegate_task, the preview/terminal readers, context-engine and
memory-provider tools) are dispatched inline and never reach
handle_function_call, which is the only place transform_tool_result ran.
A registered transform silently did nothing for them, although the hook
is documented as applying to every tool.
Apply the same helper on both runtime executor paths, after the terminal
post_tool_call so the observer still sees the untransformed result:
the concurrent path in invoke_tool's inline branch, and the sequential
path in _publish_sequential_result. The sequential registry dispatch
marks itself transform_applied so handle_function_call stays the single
invocation for registry tools and nothing double-fires.
The sequential result classification (failure detection and the logged
result length) moves below the transform so it reads the result the
model actually sees, matching the registry and concurrent paths.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit bc31efff0a6e2fd8177edd73cfe1a7112df9f78d)
Fixes#72836
Salvages #115397 (thomasscottbeck-sudo); supersedes #72860 (webtecnica, earliest — sequential path only).
(cherry picked from commit d6b5e969b3e08a557a62ed85c8b828537857b3d8)
`agent/shell_hooks.py::_parse_pre_tool_call` translated only the block and
modify dialects, so a shell hook printing the documented
`{"action": "approve", ...}` parsed to None and the tool ran with no approval
prompt — silently, with exit 0, valid JSON and `hermes hooks doctor` green.
The Python-plugin side already accepts approve and routes it through
`_resolve_block_from_details` → `request_tool_approval`; the shell parser now
yields the same `{"action": "approve", "message"?, "rule_key"?}` shape (optional
fields kept only as non-empty stripped strings), so `hermes hooks test` prints
it under `parsed:` and the dispatcher escalates it. the `decision` dialect's
`{"decision": "approve"}` means auto-ALLOW, not "ask a human", so it is
deliberately not mapped; that dialect has no top-level ask dialect to mirror.
Slim redo with credit: #92562 (earliest) bundled a larger policy-authority
rework; #110325 carried the same parser change plus an unrelated rule_key
default change and 10+ tests.
Fixes#92553
Supersedes #92562
Supersedes #110325
Co-authored-by: fangliquanflq <fangliquan@qq.com>
A general-plugin context engine is one shared instance; agent init copied it per agent
with copy.deepcopy() only. Engines that hold a SQLite connection or lock (hermes-lcm)
already expose clone_for_agent() for exactly this, but it was never called, so every
init logged "could not be safely copied … falling back to built-in compressor" and the
engine was unusable through the plugin system.
ContextEngine grows clone_for_agent() (default: deepcopy, the previous behaviour) and
_select_context_engine calls it; the failure message now names the hook to override.
Docs: context-engine-plugin.md documents the per-agent clone contract.
Test change (existing on main): tests/agent/test_context_engine.py::
test_agent_init_source_deepcopies_singleton_not_aliases was a source-reading pin on the
literal `copy.deepcopy(_candidate)` line, which this fix intentionally replaces. It is
superseded by tests/agent/test_plugin_context_engine_clone.py, which drives the real
_select_context_engine seam and asserts the invariant it guarded (child update_model()
never mutates the shared singleton) plus the new clone_for_agent() path.
Fixes#99640
credit: @stephenschoettler #62374
credit: @686f6c61 #99677
`memory.provider: builtin` (also `default`, `built-in`, `none`) selects the built-in
store, yet doctor short-circuited only on an empty name and printed
"⚠ builtin plugin not found run: hermes memory setup", and the provider migration
(`hermes update`, agent init) printed "⚠ Memory provider 'builtin' is configured but
not installed and not in the plugin catalog". update_cmd_deps and web_server_memory
each carried their own (differing) inline sentinel set.
One predicate, `agent.memory_provider.is_core_memory_provider`, now owns the sentinel
set next to the MemoryProvider ABC; doctor_state, memory_provider_migration,
update_cmd_deps, web_server_memory and agent_init's provider activation all use it.
Fixes#75647Fixes#115113
credit: @Christopher-Schulze #75679
credit: @Luna161 #83317
credit: @kokhlo #115117
credit: @KnowBotDev #115302
Under a multiplexed gateway / serve backend the launch profile's own turns bound
secrets and terminal policy but NO HERMES_HOME override (launch_profile_runtime_scope,
_profile_runtime_scope_tokens(None)). Once multiplexing is active an unset override
is the fail-closed "unbound context" signal (serves_routed_profile, per-home slots,
third-party runtime bindings such as OMH's pre_tool_call gate), so every plugin hook
a launch-profile turn fired looked unscoped and a fail-closed plugin vetoed every
tool call ("OMH runtime binding unavailable", #118538). Routed turns already bind
theirs via gateway/run.py::_profile_runtime_scope; the launch profile is a tenant
like any other, so its scope now binds the launch home too.
Sibling gap in the two hook kinds delivered off-turn by long-lived worker threads:
agent.plugin_stream_hooks (on_stream_*/on_interim_message) and the plugin event bus
(plugins_dispatch._dispatch_event) ran callbacks in the worker's empty context, so
even a routed turn's observers saw no scope. Both now capture the emitter's
contextvars at enqueue and run each delivery under a copy.
Single-profile hosts are unchanged (no override until activation).
Closes#118538
Deleting an idle scratch entry left two things behind. Headless browsers
started by a lane's e2e run kept running for days with a "(deleted)" cwd
(about 20 on one host; the process registry never knew them because they
were grandchildren of a shell that had exited). Repos whose linked
worktree lived in the entry kept a dangling registration until someone
ran `git worktree prune` by hand (10 in one repo).
The prune now lives in hermes_constants_scratch (hermes_constants keeps the
entry point). Before an idle entry is removed, same-user processes whose
cwd is inside it, or inside any scratch path that no longer exists, are
TERMed then KILLed; the deleted-cwd sweep runs on every pass so orphans
from earlier deletions are caught too. `.git` files found in the entry
name their repo, which gets `git worktree prune` after the rmtree.
cache/terminal used its own 72h fixed-age sweep; it now shares the 24h
idle rule and the subtree check. `hermes doctor` warns about cache-root
directories over 1 GiB that neither pruner covers, since finished campaign
trees parked there sat for weeks (95 GB on one host). System-prompt
scratch line updated to match.
cron/scheduler.py and tui_gateway/prompt_turn.py each parsed
turn_exit_reason for max_iterations_reached( inline. Both now read
agent.turn_failure_copy.is_max_iteration_handoff so cron delivery and the
/goal judge gate cannot drift on what counts as a non-failed handoff.
