Files
hermes-agent/website/docs/reference/toolsets-reference.md
Siddharth Balyan ee2f5629b8 Desktop connect runs on the connection operation: one card, no link to the model, no renderer polling (NS-868) (#110574)
* refactor(connectors): cut comments that restate the code

Connector modules (tools/connectors, tui_gateway connector RPCs, desktop
connector card/store) keep only comments that carry a non-derivable why or
a cross-module contract. No behaviour change.

* feat(connectors): managed connect runs on the connection operation

Managed `connect` / `reconnect` mint one ConnectionOperation for every target and, on a
desktop session, block the tool turn until the operation settles; the result is per-target
outcomes and never carries a connect link. Off the desktop the result carries the links and
returns at once (PR3 delivers them as their own message).

Why: the previous leg handed the model a URL and a `wait` verb, and the renderer ran its own
2s poller on top of the backend's 5s one; both walked the whole gateway catalog at two vendor
calls per page to read one row (~3 Composio calls/s per pending target). A hidden composer
message started the model's `wait` on the user's behalf. None of it was observable from the
operation the MCP leg already used.

What the operation looks like now:
- `contract.py`: TargetState / Actor / SettleReason enums and the `(kind, from) -> {to: actor}`
  transition table. `operation.transition()` enforces it; a card cannot claim a managed
  target `connected`, only the backend watcher can.
- `live.py`: one open operation per session, found by `op_id`. `connectors.operation.status`
  reads it, `connection.respond` drives it, `pending_connection` on resume replays it.
- `run.py`: the one lifecycle for both target kinds (prepare -> card -> wake/observe loop ->
  settle -> result). The managed `observe` hook polls the gateway list once per tick for the
  whole operation; the exact-status route replaces that call when the gateway ships it.
- `connection.update` is emitted on every transition and on settlement; registered in the
  shared event contract with the operation vocabulary typed on the TS side.
- `wait`, `_rendered_links`, `_seen_instructions`, the just-minted bounce and `_clamp_timeout`
  are deleted. `force` on `reconnect` always reinitiates; plain `reconnect` repairs only what
  the gateway reports disconnected.
- `connections.wait_timeout_seconds` is removed from config defaults, the example and the
  docs. The deadline is `OPERATION_DEADLINE_SECONDS = 300` in `operation.py`; the key was
  added on this unmerged train so no migration is needed.
- Wire model: `statusReason` parsed on connection results; the seven-state `connectionStatus`
  is typed on list items and an unknown value fails validation; `CONNECTION_REQUIRED` carries
  `connect_card_available` instead of the link when the session platform is `desktop`.

Session platform, not callback presence, decides whether a card exists: the GUI bridge
attaches callbacks to every backend session, terminal TUI included.

* feat(desktop): connector card subscribes to the connection operation

The card renders from the backend's operation instead of driving its own: `connector-flow.ts`
(the renderer's 2s `connectors.list` poller, its 120s client deadline and `keepWaiting`) is
deleted, and both hidden composer submits in `connector-tool.tsx` go with it. The model is
never nudged into a `wait`; the tool call is blocked on the backend until the operation
settles.

- `connection-request.ts` is the operation store: keyed by `op_id`, one entry per session,
  `applyOperationStatus` / `applyConnectionUpdate` as pure reducers, `respond` leaves the
  entry in place (the backend answers with `connection.update`), `ConnectionTargetOutcome`
  is a discriminated union the backend's transition table accepts.
- `input-requests.ts` applies `connection.update`; `connection.expire` and the resume
  snapshot correlate by `op_id` (a snapshot has no `request_id`).
- `ConnectorOffer` renders one `ConnectorCard` per target from a single
  `Record<ConnectionTargetState, phase>` table; Connect opens the stored link, Try again on
  failed / expired reissues through `connectors.connect` on the open operation, Not now is a
  per-target `skipped`, Continue settles. A settled operation renders `ConnectorSummary` rows
  with no live control.
- `tool-render-class.ts`: `manage_connections` renders the card regardless of
  `HERMES_GUEST_ONBOARDING`; the flag still gates the onboarding flow, not the card. The
  backend gate already decided admission; a card only exists because the tool was admitted.
- `mcp-setup-tool.tsx` speaks the same outcome vocabulary (connected / skipped / failed).
- `ConnectorRow.connectionStatus` is the seven-state literal union, not `string | null`.
- The guided-onboarding poller (`first-build-connectors.ts`) keeps its own row/phase types
  and compiles unchanged; PR3 moves it onto the operation.

anti-slop: no net-new findings (17 touched files vs 11d1a12472).

* fix(connectors): the card never parks the tool thread; every update carries the snapshot

Found by the pre-PR adversarial review and a real-path E2E test (both left in the tree).

- The desktop `connection_callback` was still `_block("connection.request", ...)`, which parked
  the tool thread on a private request-id Event until a `_respond` that no longer exists for
  this event. `connection.respond` settled the operation but the tool waited its full deadline
  before the watcher loop even started. The callback now only emits the card; the operation's
  own wake loop is the wait. The MCP leg's blocking bridge goes with it: the card answers
  through `connection.respond` like every other card.
- `connection.request` and every `connection.update` frame carry the full target snapshot
  (state, link, detail). The initial mint happened before the card existed, so the renderer
  never saw the links and Connect stayed disabled; a Continue settlement stamped
  `not_connected` on the backend while the card still showed `initiated`. The store now
  overlays the snapshot; no state is reconstructed from deltas.
- The `connection.update` emitter is a class-level `on_change` slot on the operation, set
  once by `register()` (a second `register()` no longer stacks wrappers); session lookup takes
  `_sessions_lock`; a re-minted link on an `initiated` target goes through `refresh_link()`
  and emits, instead of a bare attribute write.
- `session.interrupt` is checked before the first observe, so an interrupted call settles
  `interrupt`, not `all_resolved`.
- A gateway list reporting `expired` for an initiated target is recorded with actor `clock`
  (the contract's owner of that edge); it raised `IllegalTransition` before.
- Dead `keepWaiting` i18n keys from the deleted renderer poller removed.

tests/tui_gateway/test_connector_operation_e2e.py runs the desktop lifecycle through the real
tool, registry, gateway RPC handlers and callback bridge with only the HTTP client faked.

* docs(connectors): prompts and docs describe the operation, not the deleted wait verb

The onboarding prompts told the model to call action="wait" with timeout_seconds and to
expect a hidden [setup]/[connectors] note; both are gone. tool-search.md and
toolsets-reference.md said the model gets a connect link on the desktop. tui_gateway/AGENTS.md
gains the connection-operation row of the surface table.

* fix(connectors): the panel re-mints only a dead link

Try again on a failed or expired target mints a fresh link on the open operation. A waiting
target keeps the link it was minted with; the card reopens it and connectors.connect refuses
to spend a second mint (LINK_STILL_VALID). The unused refresh_link() goes. The package
docstring names the new siblings; the nine-name public surface is unchanged.

* test(connectors): the local-batch test answers the operation the way the card does

The callback stopped returning an answer in f782b26d98 (the card answers through
connection.respond); this test still returned one and waited out the 300s deadline in CI.

* ci: retrigger

* fix(connectors): the desktop card appears outside guided onboarding

Live on a signed-in macOS desktop, the two-app connect never showed a card. Three
defects, each hidden by a test that bound state the running app never binds.

The backend read the surface from HERMES_SESSION_PLATFORM only. The desktop and TUI
gateway bind it as HERMES_SESSION_SOURCE (_set_session_context), so session_platform()
was "" and managed connects took the off-desktop branch: links in the model's message,
no operation. session_platform() now reads platform, then source. The E2E test binds
through server._set_session_context instead of set_session_vars(platform="desktop").

The renderer routed manage_connections to the card only under isOnboardingEnabled(),
the HERMES_GUEST_ONBOARDING launch flag, in message-parts.tsx and the run splitter in
fallback.tsx. tool-render-class.ts had already dropped that gate in this PR; the two
routers had not. Both now route on the tool name alone.

ConnectorTool resolved the session owner by the runtime id. Owner routes, hints and
session rows are keyed by the stored id, so in registry topology the owner never
resolved and the card rendered null while the tool blocked. It now resolves by the
stored id, matching the PR1.5 card and every other owner lookup.

message-parts-connectors.test.tsx mounts the real Fallback router with the onboarding
flag off and distinct runtime/stored ids; red before each renderer fix, green after.

* style(connectors): shorter comments, no module mock in the card router test

The router test mocked isOnboardingEnabled to false; jsdom has no preload bridge, so the
real function already returns false. Comments that restated the code are cut to one line.

anti-slop: no net-new findings (25 touched files)

* fix(connectors): Connect on a waiting row opens the stored link

ConnectorCard derived the button's loading state from the phase label, so a managed row that
read "Finish connecting in your browser" (every row, since links are minted up front) had a
disabled Connect button. Nothing on the desktop could open the sign-in link; every managed
connect ended skipped, not_connected, or at the deadline.

The card now takes `busy` for "the action itself is running" and keeps `phase` as a label.
The MCP card passes its in-flight flag; the connector card passes the re-mint wait. Red before:
the Connect button on an initiated row rendered disabled and a click opened nothing.

* fix(connectors): a settled card stays dead; the card binds to its tool call only

A second connect for the same apps revived the finished card on the old tool row. The
connection.request payload carried no id, so the renderer fell back to matching rows by
connector names, and any row with those names qualified, settled or not.

The operation now records the model's tool_call_id and sends it in connection.request and in
the resume snapshot. The card binds to the tool row with that id and to nothing else; the
name-match fallback is deleted. A payload without the id is rejected by the store.

`reason` is removed from the tool: it was the only text the card ever showed from the model
and its absence forked a second tool part, since `reason` doubled as the row-correlation key
in tool-parts.ts. The card never needed it.

`connection.expire` is deleted from the contract and from _EXPIRING_REQUESTS: the card is
raised with _emit, not _block, so nothing has emitted it since the operation lifecycle landed.

Sid's rule of record: a resolved card is fully dead; no path brings it back.

* fix(connectors): the watch loop settles once, on time, and never raises into the result

Three findings from the live review, one loop.

Continue racing a finished sign-in: the loop ran the gateway read, then settled. A read that
returned `connected` for an already-settled or failed target raised IllegalTransition out of
the tool and the model got a generic error instead of the per-app outcomes. The read now skips
targets that are not live (pending, initiated) and skips a settled operation; the loop checks
`settled` after every read.

Settle reason as row text: `settle()` wrote `continue`/`deadline` into each unresolved target's
`detail`, and the card printed it in red. The reason stays on the operation only.

Stop and the deadline waited for the next tick: `/stop` sets a per-thread flag with no wake
hook, so the sleep is sliced at 250 ms and the flag and clock are read each slice. The clock is
also checked before each read, not only after.

Tests: a failed mint that later reads connected settles cleanly; Continue during a read keeps
the settled result; no reason in detail; an interrupt settles within the same second.

* fix(connectors): MCP setup off the desktop returns unavailable instead of blocking

run_mcp_operation treated a non-None connection_callback as "a card exists". Every tui_gateway
session has that callback, the Ink TUI included, so an MCP install from the terminal UI blocked
until the 300 s deadline while the docs promised `unavailable` with the terminal commands.

The MCP path now reads the session surface the same way the managed path does; the callback is
never the predicate. Test binds the surface to `tui` with the callback attached.

* fix(connectors): a failed Try again shows the failure, not the old dead link

The panel's re-mint ignored the gateway's per-app status and moved the row to `initiated` with
whatever link came back, `None` included, so a mint that failed again rendered as waiting on the
link that had already died.

One reader of a mint response now serves both the first mint and Try again
(`managed.mint`, with the actor as a parameter). A repeated failure keeps the row `failed`,
drops the link, and carries the vendor's new text through `operation.refresh`, which emits a
frame without a state change so the card redraws.

* fix(connectors): a forced reconnect waits for the new sign-in before it reports connected

`reconnect` with `force: true` is the account switch. The vendor keeps the old account active
while the new link waits, so the first list read after the mint said `connected` and the
operation settled at once: the new link was dropped and the model was told the switch was done.

A forced target is marked awaiting_new_attempt after the mint. The watcher ignores its row until
the list shows the new attempt (`connectionStatus: initiated`) once, then trusts `connected`.

* fix(connectors): the operation registers under the gateway session key

The tool registered the operation under the agent's session_id; every RPC (connection.respond,
connectors.operation.status, the panel's connectors.connect) and the update emitter looked it up
by the gateway's session key. Those agree until compaction rotates the agent id mid-turn; then
the card's clicks find nothing, no update reaches it, and the tool waits out the deadline.

The registration key is now the bound HERMES_SESSION_KEY, with the agent id as the fallback for
callers with no gateway (unit tests, a bare CLI). The E2E passes a rotated agent id and drives
the card by the gateway key.

* fix(connectors): the forced-reconnect gate reads any non-active row; a failed re-mint of an expired row is failed

Three follow-ups from the verification of the fix pass.

The awaiting_new_attempt gate cleared only on the literal `connectionStatus: initiated`. The
field is optional on the wire and `initializing`, `failed`, `expired` are valid values, so a
forced reconnect could wait the full 300 s and swallow a failed new attempt. The gate now holds
only while the row still reads as the old account (`connected` or `active`) and releases on
anything else.

Try again on an `expired` row whose re-mint fails raised IllegalTransition (no expired → failed
edge). The re-mint steps through `initiated` as the user's attempt, then `failed`, then drops the
dead link.

`detail` never carries a state name any more: `failed` as detail rendered as the row label and
made agent/display.py tag the settled result as a tool error. Only vendor text goes there.

`connection.expire` removed from the renderer's unscoped-stream set; nothing emits it.
2026-09-15 00:41:14 +05:30

13 KiB

sidebar_position, title, description
sidebar_position title description
4 Toolsets Reference Reference for Hermes core, composite, platform, and dynamic toolsets

Toolsets Reference

Toolsets are named bundles of tools that control what the agent can do. They're the primary mechanism for configuring tool availability per platform, per session, or per task.

How Toolsets Work

Every tool belongs to exactly one toolset. When you enable a toolset, all tools in that bundle become available to the agent. Toolsets come in three kinds:

  • Core — A single logical group of related tools (e.g., file bundles read_file, write_file, patch, search_files)
  • Composite — Combines multiple core toolsets for a common scenario (e.g., debugging bundles file, terminal, and web tools)
  • Platform — A complete tool configuration for a specific deployment context (e.g., hermes-cli is the default for interactive CLI sessions)

Configuring Toolsets

Per-session (CLI)

hermes chat --toolsets web,file,terminal
hermes chat --toolsets debugging        # composite — expands to file + terminal + web
hermes chat --toolsets all              # everything

Per-platform (config.yaml)

toolsets:
  - hermes-cli          # default for CLI
  # - hermes-telegram   # override for Telegram gateway

Interactive management

hermes tools                            # curses UI to enable/disable per platform

Or in-session:

/tools list
/tools disable browser
/tools enable homeassistant

Core Toolsets

Toolset Tools Purpose
browser browser_back, browser_cdp, browser_click, browser_console, browser_dialog, browser_get_images, browser_navigate, browser_press, browser_scroll, browser_snapshot, browser_type, browser_vision, web_search Core browser automation. Includes web_search as a fallback for quick lookups. browser_cdp and browser_dialog are gated at runtime — registered only when a CDP endpoint is reachable at session start (via /browser connect, browser.cdp_url config, Browserbase, or Camofox). browser_dialog works together with the pending_dialogs and frame_tree fields that browser_snapshot adds when a CDP supervisor is attached.
clarify clarify Ask the user a question when the agent needs clarification.
code_execution execute_code Run Python scripts that call Hermes tools programmatically.
connections manage_connections Connect the user to apps: managed connector accounts through the Nous gateway, and local MCP servers from the catalog. On the desktop every action is a card that blocks until the user settles it; off the desktop managed actions return a connect link per app.
coding composite (file + terminal + search + web + skills + browser + todo + memory + session_search + clarify + code_execution + delegation + vision) Coding-focused bundle for software work: file editing, terminal, search, web docs, skills, browser, delegate, and code execution.
cronjob cronjob Schedule and manage recurring tasks.
debugging composite (file + terminal + web) Debug bundle — file, process/terminal, web extract/search.
delegation delegate_task Spawn isolated subagent instances for parallel work.
discord discord Core Discord text/embed/DM actions (gateway-only). Active on the hermes-discord toolset.
discord_admin discord_admin Discord moderation (bans, role changes, channel management). Active on the hermes-discord toolset; requires the bot to hold the relevant Discord permissions.
feishu_doc feishu_doc_read Read Feishu/Lark document content. Used by the Feishu document-comment intelligent-reply handler.
feishu_drive feishu_drive_add_comment, feishu_drive_list_comments, feishu_drive_list_comment_replies, feishu_drive_reply_comment Feishu/Lark drive comment operations. Scoped to the comment agent; not exposed on hermes-cli or other messaging toolsets.
file patch, read_file, search_files, write_file File reading, writing, searching, and editing.
homeassistant ha_call_service, ha_get_state, ha_list_entities, ha_list_services Smart home control via Home Assistant. Only available when HASS_TOKEN is set.
computer_use computer_use Background desktop control via cua-driver — does not steal cursor/focus. Works with any tool-capable model. macOS, Windows, and Linux; requires cua-driver on $PATH.
context_engine (varies) Runtime tools exposed by the active context-engine plugin (empty until a plugin populates it).
image_gen image_generate Text-to-image generation via FAL.ai (with opt-in OpenAI / xAI backends).
video_gen video_generate, xai_video_edit, xai_video_extend Text-to-video and image-to-video via plugin-registered backends (xAI Grok-Imagine, FAL.ai Veo 3.1 / Pixverse v6 / Kling 3.0 / Kling O3). Pass image_url to animate an image; omit it for text-to-video. xai_video_edit / xai_video_extend are provider-specific edit/extend tools, gated on xAI Imagine credentials.
kanban kanban_attach, kanban_attach_url, kanban_attachments, kanban_block, kanban_comment, kanban_complete, kanban_create, kanban_heartbeat, kanban_link, kanban_list, kanban_request_changes, kanban_request_review, kanban_show, kanban_unblock Multi-agent coordination tools. Registered for dispatcher-spawned task workers (HERMES_KANBAN_TASK) and for platforms whose saved selection lists kanban (hermes tools enable kanban --platform <p>; the all/* wildcard does not enable it). Workers mark tasks done, request first-class review, block, heartbeat, comment, and create/link follow-up tasks; orchestrator profiles additionally get board-routing tools like list/unblock. delegate_task children are not Kanban run owners: their schema strips/disables this toolset and runtime guards reject direct board mutations, even if parent HERMES_KANBAN_* env vars are present.
memory memory Persistent cross-session memory management.
desktop_ui annotate_preview, close_preview, close_terminal, drive_preview, focus_pane, open_preview, react_to_message, read_preview, read_terminal, read_window_below, tour Affordances that act on the Hermes desktop app itself — read/close the embedded terminal pane, open, read, close, interact with, and annotate the in-app browser, identify the OS window behind the app, reveal a pane, react to a message, run a guided tour (highlight + narrate UI elements in the app or the preview pane). Enabled for sessions whose source is the desktop app, whichever backend it's connected to (local, SSH, URL, or Hermes Cloud). Never present on CLI, TUI, messaging, or cron sessions.
project project_create, project_list, project_switch Create and switch desktop Projects (named, multi-folder workspaces). GUI / desktop sessions only.
safe image_generate, vision_analyze, web_extract, web_search (via includes) Read-only research + media generation. No file writes, no terminal, no code execution.
search web_search Web search only (without extract).
session_search session_search Search past conversation sessions.
skills skill_manage, skill_view, skills_list Skill CRUD and browsing.
spotify spotify_albums, spotify_devices, spotify_library, spotify_playback, spotify_playlists, spotify_queue, spotify_search Native Spotify control (playback, queue, search, playlists, albums, library). Registered by the bundled spotify plugin.
terminal process, terminal Shell command execution and background process management.
todo todo Task list management within a session.
tts text_to_speech Text-to-speech audio generation.
vision vision_analyze Image analysis via vision-capable models.
video video_analyze Video analysis and understanding tools (opt-in, not in the default toolset — add explicitly via --toolsets).
web web_extract, web_search Web search and page content extraction.
x_search x_search Read-only public X discovery via xAI's built-in x_search Responses tool. Use the xurl skill for authenticated X API reads and account actions. Off by default; opt in via hermes tools. Schema only registered when xAI credentials (SuperGrok OAuth or XAI_API_KEY) are configured.
yuanbao yb_query_group_info, yb_query_group_members, yb_search_sticker, yb_send_dm, yb_send_sticker Yuanbao DM/group actions and sticker search. Registered only on hermes-yuanbao.

Platform Toolsets

Platform toolsets define the complete tool configuration for a deployment target. Most messaging platforms use the same set as hermes-cli:

Toolset Differences from hermes-cli
hermes-cli Full toolset — the default for interactive CLI sessions. Includes file, terminal, web, browser, memory, skills, vision, image_gen, todo, tts, delegation, code_execution, cronjob, session_search, clarify, computer_use, Home Assistant, and the kanban tools (all check_fn-gated at runtime).
hermes-acp Drops clarify, cronjob, image_generate, text_to_speech, computer_use, all four Home Assistant tools, and the kanban tools. Focused on coding tasks in IDE context.
hermes-api-server Drops clarify, text_to_speech, computer_use, and the kanban tools. Keeps everything else — suitable for programmatic access where user interaction isn't possible.
hermes-cron Same as hermes-cli.
hermes-telegram Same as hermes-cli.
hermes-discord Adds discord and discord_admin on top of hermes-cli.
hermes-slack Same as hermes-cli.
hermes-whatsapp Same as hermes-cli.
hermes-signal Same as hermes-cli.
hermes-matrix Same as hermes-cli.
hermes-mattermost Same as hermes-cli.
hermes-email Same as hermes-cli.
hermes-sms Same as hermes-cli.
hermes-bluebubbles Same as hermes-cli.
hermes-dingtalk Same as hermes-cli.
hermes-feishu Adds the five feishu_doc_* / feishu_drive_* tools (only used by the document-comment handler, not the regular chat adapter).
hermes-qqbot Same as hermes-cli.
hermes-wecom Same as hermes-cli.
hermes-wecom-callback Same as hermes-cli.
hermes-weixin Same as hermes-cli.
hermes-yuanbao Adds the five yb_* tools (DM/group/sticker) on top of hermes-cli.
hermes-homeassistant Same as hermes-cli (the Home Assistant tools are already present by default and activate when HASS_TOKEN is set).
hermes-webhook Restricted safe subset — only web_search, web_extract, vision_analyze, and clarify. Webhook-triggered runs get no terminal, file, or browser access.
hermes-gateway Internal gateway orchestrator toolset — union of every hermes-<platform> toolset; used when the gateway needs to accept any message source.

Dynamic Toolsets

MCP server toolsets

Each configured MCP server generates a mcp-<server> toolset at runtime. For example, if you configure a github MCP server, a mcp-github toolset is created containing all tools that server exposes.

# config.yaml
mcp_servers:
  github:
    command: npx
    args: ["-y", "@modelcontextprotocol/server-github"]

This creates a mcp-github toolset you can reference in --toolsets or platform configs. The bare server name (github) works as an alias. If a server is named like a built-in toolset (homeassistant, browser), that name resolves to the built-in tools plus the server's mcp__<server>__* tools; neither side shadows the other.

Plugin toolsets

Plugins can register their own toolsets via ctx.register_tool() during plugin initialization. These appear alongside built-in toolsets and can be enabled/disabled the same way.

Custom toolsets

Define custom toolsets in config.yaml to create project-specific bundles:

toolsets:
  - hermes-cli
custom_toolsets:
  data-science:
    - file
    - terminal
    - code_execution
    - web
    - vision

Wildcards

  • all or * — expands to every registered toolset (built-in + dynamic + plugin)

A handful of tools have an additional availability check on top of toolset membership and are not turned on by all/* alone:

  • Capability-gated tools (browser, computer_use, code_execution, Feishu, Home Assistant, cronjob) appear only when their backend/credential prerequisite is configured.
  • Workflow-gated tools — the kanban toolset — are deliberately opt-in. all/* does not enable kanban; you must list kanban explicitly (or be a dispatcher-spawned worker with HERMES_KANBAN_TASK set). Kanban tools mutate shared board state, so they stay off by default even under all.

Relationship to hermes tools

The hermes tools command provides a curses-based UI for toggling individual tools on or off per platform. This operates at the tool level (finer than toolsets) and persists to config.yaml. Disabled tools are filtered out even if their toolset is enabled.

See also: Tools Reference for the complete list of individual tools and their parameters.