Commit Graph

156 Commits

Author SHA1 Message Date
Teknium
06fb390212 fix(integration): restore stdin=DEVNULL on computer_use subprocess calls dropped in simplification 2026-09-02 17:48:26 -07:00
Teknium
1b29d9fe81 Merge branch 'simp/r2-tools-b-cua' into simp/integration2 2026-09-02 17:05:18 -07:00
Teknium
0223eeca66 refactor(computer_use): walrus-fold guard checks, wrap wide lines (931->929 LOC) 2026-09-02 17:03:36 -07:00
Teknium
21372a16b2 refactor(computer_use): compact response helpers, NoopBackend recorder, docstrings/comments (976->931 LOC) 2026-09-02 17:00:40 -07:00
Teknium
015639a6d7 refactor(computer_use): compact capture mixin (name helper, flat signatures/log calls) 2026-09-02 16:53:14 -07:00
Teknium
18fa584366 refactor(computer_use): _CaptureView shares capture-derived facts across response branches; _detach_locked, _bounds_hints, _best_effort_write dedupe (997->976 LOC) 2026-09-02 16:52:47 -07:00
Teknium
2a9e16fded refactor(computer_use): compact cua_backend re-export imports (AST-identical) 2026-09-02 16:46:34 -07:00
Teknium
a61298a21c refactor(computer_use): doctor — shared structured/first-line helpers, unified version message 2026-09-02 16:45:39 -07:00
Teknium
8f877d40f9 refactor(computer_use): single-blank layout between top-level defs (AST-identical) 2026-09-02 16:35:33 -07:00
Teknium
343269e125 Merge branch 'simp/r2-cua-sess' into simp/r2-tools-b-cua 2026-09-02 16:32:08 -07:00
Teknium
00958829bb refactor(computer_use): collapse embedded daemon branch/field boilerplate 2026-09-02 16:28:40 -07:00
Teknium
7c144b8ad8 refactor(computer_use): extract _wait_or_kill and _socket_ready from the embedded daemon; compact prose 2026-09-02 16:26:07 -07:00
Teknium
49cef0bae3 refactor(computer_use): unify cua parse image sniff with backend.image_dimensions_from_bytes; inline _first_nonempty_list; compact prose 2026-09-02 16:23:58 -07:00
Teknium
c3d789893f refactor(computer_use): compact cua_backend docstrings/comments (WHYs preserved) 2026-09-02 16:18:47 -07:00
Teknium
3fefdd879b refactor(computer_use): collapse redundant locals and branches in cua_backend_session 2026-09-02 16:18:32 -07:00
Teknium
d2add66648 refactor(computer_use): tool.py — verdict-field table, payload merge, stub/record cleanups 2026-09-02 16:16:54 -07:00
Teknium
2086987cdc refactor(computer_use): route unknown-outcome results directly through _outcome_unknown 2026-09-02 16:16:30 -07:00
Teknium
9cf2c364ab refactor(computer_use): _recreate_session helper for the two restart+restore paths 2026-09-02 16:14:11 -07:00
Teknium
04403a5632 refactor(computer_use): compact schema.py literal (value byte-identical; verified via tool-schema dump) 2026-09-02 16:12:51 -07:00
Teknium
66ecaa31e7 refactor(computer_use): dedupe capability-state reset, tighten CLI result and error-text helpers 2026-09-02 16:12:02 -07:00
Teknium
67ac6cc619 refactor(computer_use): table-drive unknown-outcome messages; hoist _tool_field helper 2026-09-02 16:09:13 -07:00
Teknium
a42c78bd2e Merge branch 'simp/r2-cua-misc' into simp/r2-tools-b-cua 2026-09-02 16:08:20 -07:00
Teknium
a739007d77 Merge branch 'simp/r2-cua-tool' into simp/r2-tools-b-cua 2026-09-02 16:08:17 -07:00
Teknium
c43eb6a8d5 refactor(computer_use): shared driver-JSON probe + flattened MCP invocation resolution 2026-09-02 16:07:50 -07:00
Teknium
eb837d2e36 refactor(computer_use): unify session redeclare path, inline single-use session helpers 2026-09-02 16:07:05 -07:00
Teknium
5629d0fe3d refactor(computer_use): compact cua_backend_session prose and multi-line expressions 2026-09-02 16:04:11 -07:00
Teknium
38ef892613 refactor(computer_use): split _CuaDriverSession CLI fallback into _cli_command/_cli_run_json/_cli_result 2026-09-02 16:01:25 -07:00
Teknium
7e939af627 refactor(computer_use): compact permissions/vision_routing/backend/__init__ docstrings and spacing 2026-09-02 15:54:54 -07:00
Teknium
337267f00e refactor(computer_use): compact cua_backend origin (best-effort helper, trimmed re-exports) 2026-09-02 15:54:29 -07:00
Teknium
d4b59e044c refactor(computer_use): tool.py — nullcontext in _stop_backend 2026-09-02 15:54:13 -07:00
Teknium
6417d5dfbc refactor(computer_use): doctor — inline fallback seam into run_doctor, shared CLI output helper 2026-09-02 15:51:02 -07:00
Teknium
1d676837e0 refactor(computer_use): tool.py — reflow long literals, tighten signatures 2026-09-02 15:50:49 -07:00
Teknium
b083383770 refactor(computer_use): dedupe capture/input mixins (shared CLI re-fetch, refusal + window helpers) 2026-09-02 15:48:22 -07:00
Teknium
a1314db032 refactor(computer_use): tool.py — single-blank layout, **delivery kwargs for input handlers, staged try/except merge 2026-09-02 15:47:52 -07:00
Teknium
04de344f41 refactor(computer_use): compact backend/permissions/vision_routing/__init__ docs; tighten doctor helpers 2026-09-02 15:45:55 -07:00
Teknium
e3b79addc6 refactor(computer_use): tool.py — compact comments/docstrings, hoist vision prompt, tighten layout 2026-09-02 15:43:11 -07:00
Teknium
0360b71926 refactor(computer_use): split CuaDriverBackend into capture/input mixins and a driver-resolution module 2026-09-02 15:42:36 -07:00
Teknium
80affc7ee3 refactor(computer_use): doctor — reuse permissions._child_env, split fallback/report rendering into helpers 2026-09-02 15:37:58 -07:00
Teknium
cf9f454cdc refactor(computer_use): tool.py — _reject_unsafe, _pop_session_locked, _cache_file, summary/envelope helpers, dispatch table for _summarize_action 2026-09-02 15:37:47 -07:00
Teknium
b6a3d3f440 refactor(tools): restore stdin=DEVNULL on subprocess probes dropped during compaction
voice_mode WSL playback probe, subagent_worktree._run_git, cua_backend
loginctl/xprop probes lost their explicit stdin= (and one line-wrapped past
the encoding= same-line rule); lazy_deps._run carries it via _SUBPROCESS_KW
so mark it for the guard. Fixes tests/tools/test_subprocess_stdin_guard.py
and tests/scripts/test_footgun_subprocess_encoding.py (green on base).
2026-09-02 14:45:42 -07:00
Teknium
33ae442e94 refactor(tools/computer_use): split cua_backend into daemon/actions/... siblings; dispatch tables; compact doctor/tool 2026-09-02 14:45:15 -07:00
Zane Chee
b2e24b986f fix(computer-use): stop launching retired browser-grant runtimes 2026-08-29 18:35:17 -07:00
fangliquanflq
6662b3618d fix(computer-use): accept current notarised CUA Driver 2026-08-27 20:21:34 +08:00
Teknium
f780cb36d8 refactor(computer_use): drop the cua_browser_* route — browser work goes through browser_exec
Real-profile browsing routes all in-page browser work through the Browser Use
CLI (browser_exec), which obsoletes the cua-driver typed-browser surface baked
into computer_use. Remove it so computer_use is a pure DESKTOP-control tool
(screenshots / mouse / keyboard / window management) and every call's schema
drops ~24 browser-only params + 9 actions.

- schema.py: 9 cua_browser_* actions and the typed-browser param block removed;
  14 desktop actions + shared params kept; description drops the browser rung.
- tool.py: cua_browser entries out of _SAFE/_DESTRUCTIVE_ACTIONS; the whole
  cua_browser dispatch block deleted; {"type","cua_browser_type"} → "type"
  (desktop typing untouched); _config_preauthorized (browser-prepare-only, a
  no-op for every desktop action) and the browser-page escalation hint removed.
- browser_route.py deleted (no importers outside the package); cua_backend.py
  drops the import + typed_browser_* methods; backend.py drops the non-abstract
  defaults.
- tests: browser-route/contract suites removed; browser assertions trimmed.

Desktop control unchanged. 233 computer_use tests pass; the 1 remaining failure
(test_gateway_session_key_yolo_maps_to_unrestricted_mode) is a pre-existing
cross-test state leak — fails identically on origin/main, passes in isolation.
2026-08-26 19:25:33 -07:00
Teknium
3da5897c39 refactor(computer_use): diet schema + delete prompt block (~1.4K tok/call); remove max_elements, ladder moves to response verdicts 2026-08-26 04:50:13 -07:00
Teknium
65605d4a7a fix(computer_use): pitch background-FIRST (not background-only) in schema, prompt block, and skill 2026-08-26 04:50:13 -07:00
Teknium
45db70a80a fix(computer-use): fail closed on unverified CuaDriver.app + background launch
Hardening on top of the TCC daemon-identity salvage:
- _validate_cua_driver_app_signature: codesign -dv gate requiring EXACT
  Identifier=com.trycua.driver and the official team (4YEC26S9KF) before
  /usr/bin/open hands the bundle to LaunchServices — the identity fix must
  not double as a launcher for arbitrary/impostor bundles (suffixed
  identifiers and wrong teams rejected; unsigned dev builds only via
  computer_use.allow_unsigned_driver: true in config.yaml).
- _resolve_cua_driver_app_path: derive the bundle ONLY from the resolved
  driver binary — the /Applications fallback could launch a DIFFERENT
  install than the manifest resolved.
- open -n -g: don't activate/steal focus when launching the daemon.
- 7 new tests incl. sabotage-verified exact-match assertions.

Grafted from #76433's review direction (@Chadmc9889's original fail-closed
validation requirement).

Co-authored-by: Chadmc9889 <Chadmc9889@users.noreply.github.com>
2026-08-26 03:21:37 -07:00
projetsjsl
4746f614be fix(computer-use): preserve macOS TCC daemon identity
Launch private computer-use daemons through CuaDriver.app so Screen
Recording authorization remains attached to its stable bundle identity
instead of Hermes' ad-hoc signature.

Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
2026-08-26 03:21:37 -07:00
cvillarroel2
1a7f83a73b fix(computer_use): disable embedded daemon overlay 2026-08-26 00:54:36 -07:00
YappLeCunt
c6ae9325ec fix(computer-use): default the cursor overlay off on Linux X11
cua-driver maps the agent-cursor overlay as a fullscreen, always-on-top,
all-workspaces X11 window (save-unders composited). When a computer-use
session ends uncleanly — an agent interrupted mid-capture, a stale target
window, a driver error — that window can be left stuck above every app on
every workspace, wedging desktop input until the app is restarted. This
is the same failure class as the HUD's transparent always-on-top window on
Mutter/X11 (#83473), and it bit a real user: an interrupted capture froze
the desktop, the app had to be force-restarted, and the overlay window was
still mapped fullscreen afterwards.

The overlay is cosmetic (a tinted cursor sprite); the driver, captures,
and synthetic input all work without it. `_cua_no_overlay()` already
defaulted it off on macOS (idle CPU redraw loop, #28152/#47032) and
headless/WSL2 Linux; this extends the same auto-detect to X11 desktop
sessions, where raw X11 stacking has no compositor-owned surface to tear
down with the driver's connection. Wayland keeps the overlay: the
compositor owns the layer-surface lifecycle, so a dead driver cannot leave
a stuck top window.

Behavior contract unchanged: an explicit `computer_use.no_overlay: false`
still restores the cursor on any platform, and `true` forces it off.

Tests: X11 (DISPLAY set, no Wayland env) and XDG_SESSION_TYPE=x11
auto-detect off; Wayland keeps the overlay; explicit false overrides
auto-detection on X11.
2026-08-25 23:14:34 -07:00