e0ef0eb9c3
* feat(connections): manage_connections covers local MCP servers; setup_mcp leaves the schema
One model tool now connects the user to apps of both kinds. A target
`{"name": "linear", "mcp": true}` is a locally configured MCP server;
`install` / `enable` / `authorize` are its verbs. Bare strings and
`{"name": ...}` stay managed connectors and that leg is unchanged.
MCP targets run through one backend-owned connection operation
(tools/connections_tool_operation.py): created with a server-side
deadline from the new config key `connections.wait_timeout_seconds`
(default 120, floor 5, no ceiling), per-target state, and exactly-once
settlement (all resolved / Continue / deadline / interrupt). Unresolved
targets freeze as `not_connected` with the settle reason.
Why the fold works now: the approval card is reached through
`agent.connection_callback` via the agent-level inline executor table,
which is the only path that carries a GUI callback. Registry dispatch
(every non-GUI surface) settles MCP targets as `unavailable` with the
`hermes mcp install / login` hint; managed targets in the same call
are unaffected.
`setup_mcp` is removed from every advertised toolset and from the
deferral list; an inline-table shim keeps calls from conversations
opened before this change dispatching (prompt-cache protection).
`_LEGACY_TOOL_ALIASES` is not the mechanism: inline tools bypass it.
Gateway: `mcp.setup.request/respond` are replaced by
`connection.request/respond/expire` (no wire compat; desktop ships
with this). The bridge waits exactly the operation's deadline. The
`session.resume` snapshot gains `pending_connection` so a reopened
window restores the card with the original deadline.
`manage_connections` joins `_SEQUENTIAL_DEADLINE_EXEMPT_TOOLS`: the
operation owns its wait; the 420s guard must not report `tool_timeout`
while the card is live.
The portal `check_fn` on the tool is dropped in favour of a
handler-level gate on the managed leg, so signed-out sessions can still
approve local MCPs.
* wip(desktop): connection.request store, resume restore, card routing for MCP targets
Renderer half of the setup_mcp fold, first slice: connection-request store
(mirrors clarify), connection.request/expire handling, pending_connection
resume restore, mcpTargets() + isCardTool(name, args) so MCP-target
manage_connections calls classify as cards. Not yet: the card component
rewrite (mcp-setup-tool.tsx), mcp-directory.ts removal, vitest, docs.
Does not typecheck until the card rewrite lands.
* fix(config): hermes update turns on the connections toolset for saved toolset lists
`hermes tools` writes an explicit `platform_toolsets.<platform>` list, and the
resolver reads absence from that list as "unchecked". The `connections`
toolset (#106842) shipped after most users last saved, so `manage_connections`
is stripped from the schema on every install that ever opened the picker.
The Nous entitlement gate never runs; the agent reports the tool as missing.
Migration 44 -> 45 (renumbered when folded into #109517; main was already at 44) appends `connections` to each explicit per-platform list
that lacks it and records the offer in `known_builtin_toolsets` where that
record exists, so a later uncheck reads as a decline. It skips: platforms
whose record already holds `connections` (the user saw the checkbox and left
it off), bare composite lists ([hermes-cli]) that already inherit it, platforms
where the toolset is not allowed, and any config whose `agent.disabled_toolsets`
names `connections` (Blank Slate, `hermes tools --disable`), because the
resolver subtracts that list last and the enable would never take effect.
The explicit-list test is the resolver's own: any configurable or plugin key.
`hermes update` runs migrations post-pull for the active profile and every
sibling, so one update is enough. Fresh installs and composite users were
never affected.
* refactor: anti-slop pass on the desktop slice; shorten added comments
Parse connection.request at the boundary with a typed wire interface instead of
unknown + typeof; mcpTargets reuses connectorText; comments cut to one or two
lines. slop-ratchet: no net-new findings in 13 touched files.
* feat(desktop): the MCP approval card answers manage_connections; MCP Directory removed
The existing card (mcp-setup-tool.tsx) now reads the connection-request store,
renders for manage_connections calls with mcp:true targets, answers through
connection.respond with a per-target outcome, and no longer calls reload.mcp
after Install; the new server's tools arrive on the between-turns refresh.
A settled operation renders the first target's frozen state.
session.resume restores a pending card with its original deadline on both the
activate and cold-resume paths.
lib/mcp-directory.ts is deleted along with its two fallback branches
(suggestion provider, card install). The catalog was already primary in both;
a catalog miss now yields no suggestion / a notInCatalog error. The GitHub
never-suggest test is rewritten on catalog-shaped data.
vitest: connection-request store (6), suggestion provider, clarify restore.
slop-ratchet: no net-new findings in 19 touched files.
* chore: drop __pycache__ files swept in by an over-broad git add
* fix(desktop): correlate the connection.request row with the model's tool call by reason
The synthetic row from connection.request and the tool.start row carried
different ids and no shared match value (op_id is not in the model's args),
so the card mounted twice. reason is the arg both sides carry.
* docs: manage_connections covers local MCP servers; connections.wait_timeout_seconds
* fix(connections): settle reason derives from target state, never from the renderer
A card that answers one of two targets and claims all_resolved must settle as
continue with the other target not_connected; found live with a two-target call.
* fix(desktop): a pending connection card re-arms on resume and activate
The store entry was restored but the transcript row was not, so navigating
away and back (or reloading) lost the card while the backend kept waiting.
restorePendingClarifyToolCall's core is generalized to any blocking tool
name and both resume paths project the connection row through it.
Verified live: card restored after navigate-away and after a full renderer
reload, deadline_at unchanged, approve settles connected.
* style: literal wording in added comments, docstrings and docs
* fix: shared gateway-event contract and config-schema category for the connection events
connection.request/expire replace mcp.setup.* in apps/shared gateway-events
(json list, BACKEND_EVENT_NAMES, GatewayEventMap) so the renderer's event
union includes them and the tui_gateway contract test passes. The new
`connections` config section folds into the agent tab like the other
single-field sections.
* style: import order (perfectionist) in the desktop and shared files this PR touches
* chore: retrigger CI (zero-job dispatch failure, auto-heal)
core_tool_deferral — live A/B harness for tool-visibility changes
Built for the PR #97979 maintainer battery (core-tool deferral behind the
tool_search bridge). Runs REAL in-process AIAgents from two pinned
checkouts and grades task outcomes programmatically — accuracy, api turns,
tokens, wall, bridge-call counts — across any set of models.
Original verdict + full numbers: results/SUMMARY.md and the PR #97979 body
(288 runs; gpt-5.6-terra / glm-5.3-flash / qwen3.8-27b).
Layout
tasks.py— 14-task battery: one task per deferred tool, multistep (todo discipline, GUI chains), long-range (session_search → backup → cron → todo), a destructive-ambiguity clarify trap, an eager-only control, and a false-discovery distractor. Each task carries fixtures, a programmatic grader (0–1 partial credit), and scripted user replies.worker.py— one (arm, model, task, rep) cell in an isolated subprocess: temp HERMES_HOME + workspace, hermetic env (only OPENROUTER_API_KEY survives), seeded session DB (targets + decoys), deterministic desktop-surface stubs (desktop_ui emitter + agent callbacks), computer_use/image_generate stubbed at the registry handler. Terminal/files/cron/process/session-DB are REAL. Exit 3 = infra/config error (never scored).orchestrator.py— battery runner: resume-safe, per-task wall timeouts, parallel cells, errored-record retry, 3-infra-abort fuse.report.py— per-task table both arms (score spread, turns, tok, wall, bridge calls), mean-of-task-means, noise/error accounting.
Running
# 1. Two plain checkouts pinned to the SHAs under test (never pip install -e)
git worktree add /tmp/abdefer-base <baseline-sha>
git worktree add /tmp/abdefer-pr <pr-sha>
export ABDEFER_BASE_TREE=/tmp/abdefer-base
export ABDEFER_PR_TREE=/tmp/abdefer-pr
export OPENROUTER_API_KEY=... # the only key the worker keeps
# 2. Smoke one cheap cell first
python3 worker.py base openai/gpt-5.6-terra config_grep_distractor 1 /tmp/smoke.json
# 3. Battery (per model; start with the STRONGEST model to validate variance)
python3 orchestrator.py openai/gpt-5.6-terra 3 --parallel=5
python3 orchestrator.py z-ai/glm-5.3-flash 3 --parallel=5
python3 orchestrator.py qwen/qwen3.8-27b 3 --parallel=5
# 4. Readout
python3 report.py
ABDEFER_PYTHON overrides the worker interpreter (defaults to the
orchestrator's own); ABDEFER_RESULTS overrides the results root.
Discipline (from the readtool/session_search harness lineage)
- Verify model slugs against the live OpenRouter list before launching.
- Interactive fairness: if the agent ends its turn with a plain-text
question, the worker sends the scripted reply (max 2, counted as
user_roundtrips) — without this, every clarify-shaped task scores 0 unfairly and the battery is poisoned (the first terra run was discarded for exactly this). - Same-denominator rule: errored runs score 0 and STAY in the accuracy denominator; they are excluded from efficiency means.
- Extend contested cells (score spread at n=3) to n=6 before concluding.
- For discovery-rate regressions, always check base-arm usage on the same tasks first — a tool models skip even when visible is not a deferral regression.
- Audit anomalous cells from
*.transcript.jsonbefore publishing.
results/ is gitignored except SUMMARY.md — rep JSONs are rebuildable,
verdicts are the artifact.