Commit Graph

99 Commits

Author SHA1 Message Date
Jeff J Hunter
65625dfe87 fix(computer_use): attach element_token when the driver schema accepts it
cua-driver 0.21 refuses a bare element_index:

    click: bare element_index is not accepted; pass element_token,
    or snapshot_id together with element_index

_maybe_attach_element_token gated solely on the trycua/cua#1961 capability
vocabulary. 0.21 stopped publishing per-tool capability sets — every tool
reports an empty set — while still accepting element_token in its input
schema. The gate therefore fails closed on 0.21.x and we send the bare
index, so the driver rejects the call.

The effect is total: every element-targeted click is refused, leaving
agents with only blind pixel coordinates. Observed against cua-driver
0.21.0 on X11, where a capture returned 762 elements with 762 tokens
cached and every subsequent click still failed with snapshot_id_required.

Check the live input schema first — supports_input_property() already
exists for exactly this, and its docstring notes it "deliberately inspects
tools/list rather than ... requiring a capability token the driver never
shipped". The capability check is retained as a fallback so drivers that
did ship the vocabulary are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 17:34:31 -07:00
Teknium
d63e380324 compat(plugins): warn once per name when a plugin resolves an old import path; lint step restored in CI
Every PLUGIN-COMPAT __getattr__ now calls hermes_cli.plugin_compat.warn_once(facade, name, target) before
resolving, emitting a HermesPluginCompatWarning (FutureWarning) once per process per name: old path, new
path, removal target. Importing a facade for its live API stays silent; only resolving a moved name warns.
COMPAT_MANIFEST.md documents the warning and how to silence it during migration.

Verified the runtime never routes through a pointer: every entry point (run_agent, cli, hermes_cli.main,
gateway.run, tui_gateway.server, web_server, model_tools + tool discovery, hermes_state, cron.scheduler,
browser_tool, mcp_tool, kanban, auth) imports clean and `hermes doctor` runs end to end with the warning
promoted to an error.

Also restores the check_compat_pointers CI step to .github/workflows/lint.yml, which a0be177aac dropped
when the compat layer was regenerated (the lint script itself was present; the workflow step was not).

hermes_cli/plugin_compat.py, tests/test_plugin_compat_warning.py and the two-line insert per facade are
part of the compat layer and go away with it.
2026-09-04 00:15:16 -07:00
Teknium
2776813df3 compat(plugins): temporary import-path shims for external plugins — ONE commit, revert on schedule
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:

    git revert <this sha>

removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.

What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
  so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
  tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
  relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)

Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
2026-09-03 17:13:22 -07:00
Teknium
5c3db6750f simplify(compat): tools/computer_use/cua_backend — drop 24 re-exports + shutil re-import, repoint 5 callers + 14 test files 2026-09-03 13:22:51 -07:00
Teknium
e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium
0071ba9965 Merge origin/main (561b053f79) into simp/forwardport: forward-port 220 main commits into the simplified tree 2026-09-03 03:31:03 -07:00
Teknium
23b9ffc4fa fix(integration): restore subprocess stdin=DEVNULL / utf-8 encoding guards and windows-footgun gates dropped by round-3 compaction
Repo scanners (check_subprocess_stdin, check-windows-footguns --all) flagged 21 sites where
the r3 single-line collapses lost stdin=DEVNULL, encoding='utf-8'/errors='replace', the
'# windows-footgun: ok' same-line marker, or the getattr(os, 'geteuid') gate. Each guard is
restored at the call site (real portability/hang fixes, not suppressions).
2026-09-03 02:46:19 -07:00
Zane Chee
b1bf099f14 fix(computer-use): distinguish doctor and gateway environments 2026-09-02 21:49:23 -07:00
Teknium
fa0ff15614 refactor(computer_use): final cua backend/driver/daemon folds — inline discovery reasons, contextlib suppress, one-line probes; strings frozen 2026-09-02 21:47:13 -07:00
Teknium
22def3f4a9 refactor(computer_use): contextlib rollback/suppress in cua backend + daemon; _json_object helper; daemon startup-failure helper; tighten prose 2026-09-02 21:02:05 -07:00
Teknium
95b0ae9121 refactor(computer_use): compact cua_backend/driver/daemon docstrings, banners and imports; fold contract-reason logic 2026-09-02 20:37:04 -07:00
Teknium
ea7e75f65a refactor(computer_use): unify cua quiet-subprocess probes via _run_quiet; table-drive discovery reasons; trim re-imports, contract gate, daemon teardown 2026-09-02 20:25:17 -07:00
Teknium
2a9e16fded refactor(computer_use): compact cua_backend re-export imports (AST-identical) 2026-09-02 16:46:34 -07:00
Teknium
8f877d40f9 refactor(computer_use): single-blank layout between top-level defs (AST-identical) 2026-09-02 16:35:33 -07:00
Teknium
c3d789893f refactor(computer_use): compact cua_backend docstrings/comments (WHYs preserved) 2026-09-02 16:18:47 -07:00
Teknium
337267f00e refactor(computer_use): compact cua_backend origin (best-effort helper, trimmed re-exports) 2026-09-02 15:54:29 -07:00
Teknium
0360b71926 refactor(computer_use): split CuaDriverBackend into capture/input mixins and a driver-resolution module 2026-09-02 15:42:36 -07:00
Teknium
b6a3d3f440 refactor(tools): restore stdin=DEVNULL on subprocess probes dropped during compaction
voice_mode WSL playback probe, subagent_worktree._run_git, cua_backend
loginctl/xprop probes lost their explicit stdin= (and one line-wrapped past
the encoding= same-line rule); lazy_deps._run carries it via _SUBPROCESS_KW
so mark it for the guard. Fixes tests/tools/test_subprocess_stdin_guard.py
and tests/scripts/test_footgun_subprocess_encoding.py (green on base).
2026-09-02 14:45:42 -07:00
Teknium
33ae442e94 refactor(tools/computer_use): split cua_backend into daemon/actions/... siblings; dispatch tables; compact doctor/tool 2026-09-02 14:45:15 -07:00
Zane Chee
b2e24b986f fix(computer-use): stop launching retired browser-grant runtimes 2026-08-29 18:35:17 -07:00
fangliquanflq
6662b3618d fix(computer-use): accept current notarised CUA Driver 2026-08-27 20:21:34 +08:00
Teknium
f780cb36d8 refactor(computer_use): drop the cua_browser_* route — browser work goes through browser_exec
Real-profile browsing routes all in-page browser work through the Browser Use
CLI (browser_exec), which obsoletes the cua-driver typed-browser surface baked
into computer_use. Remove it so computer_use is a pure DESKTOP-control tool
(screenshots / mouse / keyboard / window management) and every call's schema
drops ~24 browser-only params + 9 actions.

- schema.py: 9 cua_browser_* actions and the typed-browser param block removed;
  14 desktop actions + shared params kept; description drops the browser rung.
- tool.py: cua_browser entries out of _SAFE/_DESTRUCTIVE_ACTIONS; the whole
  cua_browser dispatch block deleted; {"type","cua_browser_type"} → "type"
  (desktop typing untouched); _config_preauthorized (browser-prepare-only, a
  no-op for every desktop action) and the browser-page escalation hint removed.
- browser_route.py deleted (no importers outside the package); cua_backend.py
  drops the import + typed_browser_* methods; backend.py drops the non-abstract
  defaults.
- tests: browser-route/contract suites removed; browser assertions trimmed.

Desktop control unchanged. 233 computer_use tests pass; the 1 remaining failure
(test_gateway_session_key_yolo_maps_to_unrestricted_mode) is a pre-existing
cross-test state leak — fails identically on origin/main, passes in isolation.
2026-08-26 19:25:33 -07:00
Teknium
45db70a80a fix(computer-use): fail closed on unverified CuaDriver.app + background launch
Hardening on top of the TCC daemon-identity salvage:
- _validate_cua_driver_app_signature: codesign -dv gate requiring EXACT
  Identifier=com.trycua.driver and the official team (4YEC26S9KF) before
  /usr/bin/open hands the bundle to LaunchServices — the identity fix must
  not double as a launcher for arbitrary/impostor bundles (suffixed
  identifiers and wrong teams rejected; unsigned dev builds only via
  computer_use.allow_unsigned_driver: true in config.yaml).
- _resolve_cua_driver_app_path: derive the bundle ONLY from the resolved
  driver binary — the /Applications fallback could launch a DIFFERENT
  install than the manifest resolved.
- open -n -g: don't activate/steal focus when launching the daemon.
- 7 new tests incl. sabotage-verified exact-match assertions.

Grafted from #76433's review direction (@Chadmc9889's original fail-closed
validation requirement).

Co-authored-by: Chadmc9889 <Chadmc9889@users.noreply.github.com>
2026-08-26 03:21:37 -07:00
projetsjsl
4746f614be fix(computer-use): preserve macOS TCC daemon identity
Launch private computer-use daemons through CuaDriver.app so Screen
Recording authorization remains attached to its stable bundle identity
instead of Hermes' ad-hoc signature.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
2026-08-26 03:21:37 -07:00
cvillarroel2
1a7f83a73b fix(computer_use): disable embedded daemon overlay 2026-08-26 00:54:36 -07:00
YappLeCunt
c6ae9325ec fix(computer-use): default the cursor overlay off on Linux X11
cua-driver maps the agent-cursor overlay as a fullscreen, always-on-top,
all-workspaces X11 window (save-unders composited). When a computer-use
session ends uncleanly — an agent interrupted mid-capture, a stale target
window, a driver error — that window can be left stuck above every app on
every workspace, wedging desktop input until the app is restarted. This
is the same failure class as the HUD's transparent always-on-top window on
Mutter/X11 (#83473), and it bit a real user: an interrupted capture froze
the desktop, the app had to be force-restarted, and the overlay window was
still mapped fullscreen afterwards.

The overlay is cosmetic (a tinted cursor sprite); the driver, captures,
and synthetic input all work without it. `_cua_no_overlay()` already
defaulted it off on macOS (idle CPU redraw loop, #28152/#47032) and
headless/WSL2 Linux; this extends the same auto-detect to X11 desktop
sessions, where raw X11 stacking has no compositor-owned surface to tear
down with the driver's connection. Wayland keeps the overlay: the
compositor owns the layer-surface lifecycle, so a dead driver cannot leave
a stuck top window.

Behavior contract unchanged: an explicit `computer_use.no_overlay: false`
still restores the cursor on any platform, and `true` forces it off.

Tests: X11 (DISPLAY set, no Wayland env) and XDG_SESSION_TYPE=x11
auto-detect off; Wayland keeps the overlay; explicit false overrides
auto-detection on X11.
2026-08-25 23:14:34 -07:00
2ndNatureAI
aeac982223 fix(computer_use): route explicit screen capture to get_desktop_state
'Screenshot my screen' previously resolved the 'screen' sentinel to the OS
shell window (Progman/WorkerW) via list_windows — capturing the wallpaper +
icons layer, never the windows actually displayed. cua-driver's
get_desktop_state does a real composited full-screen grab; the
screen/fullscreen/all sentinels now route there directly, bypassing window
enumeration (which also keeps screenshots working when Windows UIA
enumeration hangs — trycua/cua#2110/#2113).

app='desktop' keeps the shell-window lane so desktop icons/taskbar stay
clickable.

Salvaged from PR #60081 by @2ndNatureAI (surgical reapply; original branch
predates the capture-routing refactor).
2026-08-25 02:31:34 -07:00
BlackishGreen33
c73d721b1d fix(computer-use): recreate CUA session suspect after MCP timeout (#74799)
An MCP call_tool deadline hit left the cua-driver session wedged for
all later computer-use calls. Mark the session suspect on a
concurrent.futures.TimeoutError and tear down + recreate it before the
next non-lifecycle call; healthy sessions are never restarted.

Fail-closed: the timed-out action may still have taken effect on the
remote screen, so it is never silently replayed — the error result
carries structuredContent.code=timeout_outcome_unknown with
next_step=fresh_state.

Informed by #74877 by BlackishGreen33.

Co-authored-by: BlackishGreen33 <s5460703@gmail.com>
2026-08-25 01:47:21 +05:30
elphamale
11a9dcf567 feat(mcp): migrate to the mcp 2.x SDK
mcp 2.0.0 implements MCP revision 2026-07-28 and makes three breaking
changes Hermes sits on top of: `mcp.server.fastmcp` is gone, every model
field is renamed to snake_case (camelCase survives only as a
serialization alias, which pydantic does not expose to attribute
access), and the SDK's own HTTP stack moved from `httpx` to `httpx2`.

Bump the pin across the dev/mcp/computer-use extras and port the tree:

- `mcp_serve.py` and `agent/transports/hermes_tools_mcp_server.py` move
  from `FastMCP` to `mcp.server.MCPServer`, which has the same
  decorator/add_tool surface. The hermes-tools server already
  synthesised `__signature__` from Hermes' JSON Schema, which is exactly
  what 2.0's `add_tool` reads.
- SDK model reads go through `mcp_field(obj, snake, camel)`, which reads
  both spellings. A single-spelling read fails *silently* on the other
  generation — empty tool schemas, dropped structured content, tool
  results vanishing from sampling conversations — and `mcp` is an
  optional extra users install at their own version.
- `sdk_httpx()` resolves the httpx flavour from the SDK's own transport
  module, so objects handed to `streamable_http_client`, the `sse_client`
  factory, and the OAuth metadata helpers come from the module the
  installed SDK actually imports.
- HTTP support is gated on either streamable-HTTP entry point, not just
  the deprecated alias 2.0 removed.
- OAuth: `OAuthClientProvider` lost its `timeout` argument (the
  configured `oauth.timeout` now bounds the callback waiter's own poll
  loop, where the browser round-trip was always awaited), and
  `callback_handler` must return `AuthorizationCodeResult` rather than a
  tuple. 2.0 also validates the RFC 9207 `iss` parameter, so the
  callback handler and paste fallback capture it.

`mcp`/`mcp-types` 2.0.0 are inside the 14-day `exclude-newer` window, so
two narrow `exclude-newer-package` entries unblock `uv lock`, annotated
for removal on or after 2026-08-11. `httpx2` needs no exemption: 2.7.0 is
already outside the window and satisfies mcp's floor.

Refs #69931

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:26:10 -07:00
Teknium
00c12dac61 fix(computer-use): auto-repair an installed driver that fails the runtime contract
A same-day version-floor bump (0.20 runtime contract) left every install
with an older cua-driver hard-failing on all computer_use calls: the
start() gate fails closed, while the `hermes update` refresh defers to the
driver's own check-update verb — whose ~20h cache routinely answers "no
update available" right after we raise the floor. Hermes knew it required
0.20+ but never acted on that knowledge.

Two changes:

- tools_config.install_cua_driver(): a contract-failed installed driver is
  repaired on the upgrade=True path too (previously only upgrade=False).
  The contract failure itself is the confirmation, so the
  require_confirmed_update gate and the check-update short-circuit are
  bypassed for repairs — an indeterminate or stale-cached check can no
  longer pin users on an unusable driver.

- cua_backend.CuaDriverBackend.start(): when the contract gate fails on an
  installed binary, attempt one automatic repair per process via the
  standard install path, then re-probe. HERMES_CUA_DRIVER_CMD overrides
  are never repaired (explicit override is authoritative even when broken)
  and a missing binary still just reports the install hint. A failing
  installer can't loop: the second start() surfaces the original error.

Tests: contract-repair coverage in test_computer_use.py (auto-repair
success, failed repair surfaces the original error, once-per-process
guard, override never repaired, missing binary never repaired) and
test_install_cua_driver.py (incompatible driver repairs despite an
indeterminate check-update, check-update not consulted). All new tests
verified to fail against the unfixed source (sabotage run).
2026-08-16 13:01:20 -07:00
Francesco Bonacci
cdbea83e1c fix(computer-use): keep a v3 capability manifest on approval-bypassed runs
`--yolo` / `-z` route the session onto a private embedded daemon in
`unrestricted` mode. That daemon was constructed without the configured
capability manifest, and the serve command only attached
`--capability-manifest` when the mode was exactly `bounded`. So the moment a
run was bypassed, the user's declared ceiling was dropped:

    without -z:  --permission-mode bounded --capability-manifest ...
    with -z:     --permission-mode unrestricted --dangerously-bypass-approvals

No manifest, no warning. The most carefully configured run - a reviewed
ceiling, written by hand - became the least constrained one, silently, and
it failed open.

That was never a driver limitation. cua-driver documents the manifest as a
ceiling across modes ("A manifest can narrow a profile but never widen it";
its own authorization table calls it `optional_capability_manifest_ceiling`),
and accepts it alongside `--permission-mode unrestricted`.

The forwarding is version-aware, because the two manifest schemas differ
(cua-driver session_manifest.rs):

* v1/v2 are legacy and must declare `mode: bounded`. Handing one to an
  unrestricted runtime aborts startup with "legacy capability manifest mode
  must be bounded", so a naive forward would turn a working session into a
  hard failure. These are forwarded for bounded only, and a warning names
  the migration when one cannot apply.
* v3 must not declare a mode. It is the mode-independent ceiling, and it now
  rides along with unrestricted.

Unreadable or unparseable manifests are not forwarded outside bounded, on
the same fail-safe reasoning; bounded still forwards unconditionally and
lets the driver be the authority there.

Verified against cua-driver 0.20.0 on Windows. Launch args now carry
`--permission-mode unrestricted --dangerously-bypass-approvals
--capability-manifest <v3> --approve-capability-manifest`, and the ceiling
is enforced in the bypassed run - a tool outside the manifest is refused
("outside the capability manifest for this session ... blocked as a
protected resource") where the same config previously ran unbounded. A
legacy manifest was confirmed to abort driver startup when forwarded, which
is what the version gate prevents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 11:34:40 -07:00
Francesco Bonacci
9a96fdc5b8 fix(computer-use): enforce existing-profile grant, unblock the opt-in
Live-testing the Cua Driver 0.20 convergence on Windows 11 (session 2,
cua-driver 0.20.0) surfaced three defects in the existing-profile browser
path and in install status.

1. The config grant was silently nullified by an approval bypass.

`--yolo` / `-z` map onto a private unrestricted daemon, which answers every
browser_prepare. Because the host delegated the entire existing-profile
decision to the driver, that bypass also nullified
`computer_use.grant_existing_profile: false`: a plain `hermes -z` attached
to the user's real Chrome profile and read live page content over CDP, with
the driver reporting it as "the approved existing Chromium profile". It was
never approved.

An approval bypass is consent to skip prompts, not consent to read an
existing profile's pages, cookies, and storage. CuaTypedBrowserRoute.prepare
now enforces the key itself, regardless of permission mode. bounded stays
exempt - its reviewed capability manifest is the authorization boundary.
The authorization inputs are resolved in the backend from config and the
backend's immutable mode, never from model-supplied kwargs.

2. The grant, once set, still could not be used.

With `grant_existing_profile: true` the runtime is launched
`--grant existing-profile` correctly, but cua_browser_prepare then hit a
runtime approval prompt anyway - re-asking the user to authorize what the
config already authorized, and making the documented opt-in unusable on any
non-interactive run, where the prompt has nobody to answer it and the call
dies on approval timeout. The durable, file-backed grant now stands in for
that prompt. Scope is narrow: only the existing-profile prepare, only when
the grant is present; isolated launches still prompt and any resolution
failure falls closed to prompting.

3. `computer-use status` hid a custom override and spliced its output.

With HERMES_CUA_DRIVER_CMD pointed at cmd.exe, status printed the child's
multi-line banner and prompt inside the one-line version field, never
mentioned the override, and advised `hermes computer-use install` - which
install itself (correctly) refuses to run against an overridden path. It now
names the override and mirrors install's update-or-unset guidance, and
version output is reduced to one bounded line.

Verified on the reported host: `-z` existing-profile attach now refuses and
names the key; `grant: true` no longer prompts (33s vs a 300s approval
timeout); status names the override and prints one line. No change to the
reconciliation path - driver SHA256 unchanged end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 11:34:40 -07:00
Francesco Bonacci
81af2ef013 fix(computer-use): reconcile existing cua-driver installs 2026-08-16 11:34:40 -07:00
Francesco Bonacci
a403fe6f92 feat(computer-use): support Cua Driver 0.20 runtime contracts 2026-08-16 11:34:40 -07:00
Teknium
460d345642 fix(computer-use): flag macOS zero-display capture in doctor + discovery reason
A headless Mac or asleep built-in panel leaves ScreenCaptureKit with 0
shareable displays while TCC grants pass — health_report stays ok and
every capture silently returns 0x0 (#67165). Guard at the report seam
(_apply_display_count_guard, both real and fallback paths): flips the
screen_capture_capability check to fail with recovery actions (wake
display / HDMI dummy / virtual display) and downgrades ok -> degraded.
The empty-discovery reason ladder gains the matching darwin rung.

Composed from #52949 (sujeet111) and #67259 (webtecnica); both PRs
predate the doctor rewrite and the envelope normalization on main, so
this reimplements their shared intent at the current seams.

Co-authored-by: Sujeet <64351924+sujeet111@users.noreply.github.com>
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
2026-08-15 18:20:55 -07:00
0xGr1mm
d303e18d09 fix(computer_use): ignore placeholder pid/window_id ids
capture() read any non-None pid/window_id as a request for exact-window
targeting. Several providers emit every declared schema property on every
tool call, zero-filling unused optional integers, so those calls arrive as
pid=0, window_id=0. The exact-target branch was then entered, the caller's
app= was discarded, _positive_int(0) returned None for both ids, and the
capture failed with a message pointing at pid/window_id. For that class of
model capture(app=...) and frontmost capture never worked at all.

Normalize non-positive ids to None before the branch decision so dispatch
falls through to app/frontmost discovery. Malformed non-numeric ids are
deliberately not treated as placeholders: they still reach the existing
validation error instead of being silently ignored.

Fixes #81333
2026-08-15 18:20:55 -07:00
Teknium
95aa709606 fix(computer-use): diagnose empty window discovery; fail fast on dead-daemon CLI fallback
Two live-QA findings from a locked KDE desktop (real cua-driver 0.20.0):

1. capture() with zero discovered windows returned a bare
   'capture mode=ax 0x0' — no hint that the desktop session was LOCKED,
   which freezes renderers and hides windows. New
   _empty_discovery_reason() names the dominant causes in order: locked
   session (loginctl LockedHint probe, fail-safe), missing DISPLAY,
   else a pointer at hermes computer-use doctor. Surfaced through the
   existing window_title -> summary path, so the model and the user see
   it inline.

2. _call_tool_via_cli retried 'daemon is not running' 4x with ~3.5s of
   backoff sleeps — a permanent condition for that invocation (the CLI
   transport needs the machine-wide daemon; Hermes' MCP runtime does
   not). Now fails fast on the first attempt with a message naming the
   split. Transient empty output (EAGAIN congestion) keeps the retry
   loop — pinned by test.

Live-verified on the locked desktop: capture now reports the lock and
the unlock action; CLI fallback errors immediately with the transport
explanation. 7 new tests; 191 sibling tests green.
2026-08-15 17:11:53 -07:00
Teknium
20cf326bd1 fix(computer-use): align browser authorization with live-verified cua-driver 0.19.3 contract
Live-tested against the real cua-driver 0.19.3 binary (Linux x86_64):

- bounded serve flags corrected: the daemon accepts
  --session-policy/--approve-session-policy, not the docs'
  --capability-manifest names (which it rejects). Verified end-to-end:
  a bounded daemon with a real policy file starts and reports running.
- browser-approve verified real but interactive-only (refuses without a
  TTY) and its token is a legacy compatibility path disabled by default
  on current drivers (per the live browser_prepare schema). Kept as a
  passthrough; no longer presented as the primary route.
- NEW primary standard-mode route, verified live: launch the runtime
  with cua-driver's trusted-launcher grant. config opt-in
  computer_use.grant_existing_profile: true appends
  --grant existing-profile to the standard-mode MCP spawn (MCP
  initialize verified accepting the flag). Default false = attachment
  keeps failing closed. Never applied to bounded/unrestricted daemons.
- Skill, system prompt, tool schema, and docs updated to the verified
  ladder: config grant > bounded manifest > YOLO; token = legacy.
2026-08-15 15:04:32 -07:00
Teknium
48dd9c87cf feat(computer-use): user-facing authorization for cua-driver browser attachment
Completes the typed cua_browser_* route (PR #74166 lineage) with the
authorization surface that makes existing-profile attachment and
repeatable bounded automation reachable by real users:

- hermes computer-use browser-approve: CLI passthrough that mints
  cua-driver's five-minute single-use attachment token for one exact
  (pid, window_id). The user, never the model, is the token source.
- approval_token passthrough on cua_browser_prepare (schema + dispatch +
  browser_route), forwarded only for existing_profile and only as a
  non-empty string.
- computer_use.permission_mode: bounded + capability_manifest config:
  private per-session embedded daemon launched with
  --capability-manifest/--approve-capability-manifest; missing manifest
  fails loudly. 'unrestricted' is deliberately NOT a config value —
  it stays bound to the explicit per-session YOLO toggle.
- Skill + system-prompt + docs guidance for the three authorization
  rungs and the isolated-profile-first default.

E2E-verified against a temp HERMES_HOME: real config resolution to
bounded, loud failure without a manifest, real argparse path driving a
fake cua-driver binary, standard default preserved.
2026-08-15 15:04:32 -07:00
Teknium
825a9753c1 fix(computer_use): resolve cua-driver at its official Windows installer path (#85038) 2026-08-12 21:30:42 -07:00
Francesco Bonacci
c268397752 feat(computer_use): align cua-driver 0.10 permission modes 2026-07-29 12:19:37 -07:00
Francesco Bonacci
847e401b74 feat(computer_use): align cua-driver 0.9 contracts
Salvaged from PR #67807 by @f-trycua onto current main.

- Foreground gate: discover delivery_mode support from the live tools/list
  inputSchema.properties (fail closed), not the never-shipped
  input.delivery_mode capability token
- bring_to_front: standalone strict-schema MCP tool (inject_session=False),
  separate approval scope, requires foreground
- Verdict precedence: confirmed > unverifiable (verify before retry) >
  suspected_noop/refusal (escalate); surfaced as explicit verdict field
- Typed cua_browser_* route inside computer_use (browser_route.py) with
  exact-binding, adapter-injected session, snapshot-scoped refs
- Per-Hermes-session backend isolation + release_computer_use_session seam
  wired into AIAgent.close()
- Recorded 0.9 tools/list fixture replaces fabricated capability tokens
2026-07-29 12:19:37 -07:00
teknium1
5b751dc0ad chore: remove unused imports and dead locals (ruff F401/F841 sweep)
Cleans F401 unused imports and F841 dead local assignments across
root *.py, agent/, hermes_cli/, tools/, gateway/, cron/, tui_gateway/
(tests/, plugins/, skills/ excluded).

Intentionally KEPT (false positives / test-patch surfaces):
- agent/transports/__init__.py package re-exports
- cli.py browser_connect re-exports (DEFAULT_BROWSER_CDP_URL area,
  used by tests/cli/test_cli_browser_connect.py)
- hermes_cli/main.py _prompt_auth_credentials_choice /
  _model_flow_bedrock_api_key (accessed via main_mod attr in tests)
- gateway/run.py aliased replay_cleanup + whatsapp_identity re-exports
  and _PORT_BINDING_PLATFORM_VALUES (test-referenced)
- hermes_cli/web_server.py get_running_pid (tests monkeypatch it) and
  _OAUTH_TOKEN_URL availability probe
- hermes_cli/config.py get_process_hermes_home re-export (noqa'd F811
  chain) and yaml availability-probe import
- hermes_cli/nous_subscription.py managed_nous_tools_enabled
  (tests patch hermes_cli.nous_subscription.managed_nous_tools_enabled)
- try/except ImportError availability probes (env_loader, tts_tool,
  mcp_tool, web_server anthropic OAuth block)
- tools/web_tools.py noqa F401 re-exports
- hermes_cli/setup_whatsapp_cloud.py:263 'proceed' skipped: possible
  missing-guard bug, flagged for separate review
- unused function parameters (signature changes out of scope)

Side-effect RHS calls preserved where only the binding was dead
(e.g. web_server proc = _spawn_hermes_action -> bare call).
2026-07-29 11:53:39 -07:00
Teknium
16720dc45b fix(computer-use): hide the --no-overlay help probe console too
Follow-up to the #62821 salvage: the _cua_driver_supports_no_overlay
--help probe is another Windows-reachable spawn; give it the same
windows_hide_flags() treatment. Also adjust the status test to stub
_resolve_driver_cmd (permissions.py resolves via that helper, not
shutil.which).
2026-07-29 11:35:33 -07:00
motoblurr
7bc9956660 fix(computer-use): normalize Windows manifest paths in WSL
A Windows-installed cua-driver can return an absolute
``C:\Users\...\cua-driver.exe`` mcp_invocation.command to a Hermes
process running inside WSL. POSIX spawning can't use the raw Windows
string even though the binary is reachable through DrvFS. Translate
``<drive>:\...`` to ``/mnt/<drive>/...`` in _resolve_mcp_invocation
(before the path-separator check, since backslash is not a separator
on POSIX), only when actually running under WSL.

Salvaged from #63532 by @motoblurr (original commit carried a
placeholder 'Hermes Agent <hermes@local>' identity; re-authored).
Fixes #63938 premise.
2026-07-29 11:35:33 -07:00
ZundamonnoVRChatkaisetu
50e27abdd3 fix(computer-use): hide Windows cua-driver subprocess consoles
Apply windows_hide_flags() (CREATE_NO_WINDOW; 0 on POSIX) at the
Windows-reachable cua-driver subprocess boundaries: manifest probe,
update checker, CLI fallback transport, doctor health-report spawn,
and the permissions/status runner. Prevents OpenConsole/Windows
Terminal windows flashing into the foreground when spawned from
GUI-backed Gateway/Desktop processes.

The env-probe half of the original PR was already implemented on main
and is not re-applied here.

Salvaged from #62821 by @ZundamonnoVRChatkaisetu (original commits
carried a placeholder 'Claude Code Enterprise' identity; re-authored
to the contributor's GitHub identity).
2026-07-29 11:35:33 -07:00
Koho Zheng
f2a4ca9637 fix(computer-use): normalize cua-driver result envelopes
cua-driver 0.7.x can return list_windows/list_apps payloads under
structuredContent.windows, data.windows, data._legacy_windows, or
top-level windows/_legacy_windows (direct CLI responses). The wrapper
only read structuredContent.windows, so discovery came back empty
(capture 0x0, list_apps []) while raw cua-driver calls worked.

- add _windows_from_tool_result(): walks the known envelope shapes in
  priority order, skipping empty higher-priority envelopes
- route _load_windows() (MCP + CLI re-fetch paths) through the helper,
  covering capture() and focus_app()
- harden _ingest_windows(): skip non-dict members, normalize untrusted
  app_name/title/z_index fields
- list_apps(): prefer structuredContent.apps, fall through populated
  data/top-level envelopes, derive unique apps from window-shaped
  payloads via _apps_from_windows(), keep the text-line fallback last
- tests for every envelope shape, precedence, malformed records, and
  app derivation

Salvaged from #63037 (Reaper-Legion), which itself preserved the
original implementation from #57961 (kohoj); #73007 (umi008)
independently proposed the same normalization later.

Fixes #57905

Co-authored-by: Reaper <248977840+Reaper-Forge@users.noreply.github.com>
Co-authored-by: Ulises Millan Guerrero <ulises.millanguerrero@gmail.com>
2026-07-29 11:08:58 -07:00
Atakan
56bda4529b fix(computer_use): revive ended cua-driver sessions once 2026-07-28 14:24:30 -07:00
teknium1
920facdc4c fix(update): never blind-reinstall cua-driver during hermes update
'Refreshing cua-driver (Computer Use)...' could hang for minutes on
Windows: when the driver's native check-update verb returned an
indeterminate result (old driver without the verb, offline, GitHub
rate-limited, or the probe timing out), install_cua_driver(upgrade=True)
fell through to the full upstream installer — a silent, output-captured
run with a 660s ceiling, plus install.ps1's 600s concurrency-lock wait
on Windows on top. Every 'hermes update' paid that cost.

Two changes:

- install_cua_driver() grows require_confirmed_update: with it set, an
  indeterminate check keeps the installed version and returns fast,
  printing the force path (hermes computer-use install --upgrade).
  'hermes update' passes it; the explicit --upgrade CLI keeps the old
  fall-through so a force refresh still works when the check can't
  answer.
- cua_driver_update_check() default timeout is now 25s on Windows
  (8s unchanged on POSIX): first-spawn of the exe under Defender /
  SmartScreen routinely exceeds 8s, and a false timeout is exactly the
  indeterminate result that used to trigger the multi-minute reinstall.
2026-07-26 13:47:35 -07:00
teknium1
d4b867cf9f fix(windows): sweep remaining unguarded text-mode subprocess sites codebase-wide
AST-driven pass over every subprocess.run/Popen/check_output/check_call/call
with text=True (or universal_newlines=True) and no explicit encoding=:
append encoding='utf-8', errors='replace' at the kwarg site. 136 call
sites across 28 files (cli.py, hermes_cli/main.py, tools_config.py,
environments, computer_use, gateway, scripts, skills helpers, agent/*).

Together with the salvaged #55339/#60741 commits this closes out issue
#53428's bug class; the salvaged #60751 linter rule in
check-windows-footguns.py now enforces it repo-wide (verified: 807 files
scanned, zero findings).
2026-07-24 11:45:57 -07:00