A finished (non-timeout) agent-browser failure at the backend level — the CLI
exiting 101 against a stale session daemon, or empty/non-JSON output from a dead
one — returned normally as {"success": False}, so the cached local session record
was never marked suspect and every later browser command re-ran against the same
dead daemon until the process was recycled by hand (#115184).
- _interpret_browser_command_output carries `returncode` on its three
protocol-level failure dicts (parsed page-level errors never carry one)
- _is_recoverable_local_backend_failure classifies those on plain local Chromium
sessions only (cloud/CDP/real-profile/Lightpanda own their recovery)
- _recycle_local_session is the local half of _handle_browser_command_timeout,
extracted so the timeout path and the new path share the alive→suspect /
dead→tree-kill+evict split
- _run_browser_command retries once on the replacement session; `close` is
exempt (a dead daemon is already closed and cleanup must not spawn a session
just to close it)
- argv construction + spawn moved into _dispatch_browser_command so the retry
loop stays a loop over one call
Based on #115206 by @liuhao1024 (returncode on the failure dict, the
recoverability predicate, the one-retry loop); reshaped so the recycle helper is
shared with the timeout path instead of calling the timeout handler.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
The janitor reaped the bot's headed Chromium after 120 s of AGENT inactivity — which is
exactly the state a human takeover (login, 2FA) puts the agent in — so the browser died
under the human mid-login. The janitor now counts a human-held lease as activity for the
browser the human shares with the bot, and the agent-browser daemon's own idle timer
(which cannot see the lease) steps back on the Bot Desktop so the lease-aware janitor
owns that browser's lifetime; a crashed hermes still leaves it to the orphan reaper.
Fixes#110064
_build_browser_env() is shared by the npx cache warmer (hermes update / doctor
--fix), the lazy Chromium auto-installer, the Lightpanda engine's env and
browser_use_cli. Hooking bot_desktop.auto_start there made every one of those
block up to 15 s spawning Xvnc+Xfce with browser.headed on, against
desktop_env()'s own "never starts anything" contract.
The hook now sits where a headed Chromium is actually spawned for a tool
action, mirroring computer_use dispatch: _spawn_and_collect (the first
agent-browser command forks the daemon; Lightpanda engine excluded), the Chrome
fallback from Lightpanda, and the real-profile Chrome launch. The regression
test asserts both halves: the env builder never starts the screen, the headed
Chromium spawn does, a headless or Lightpanda spawn does not.
The base64 guard covered only `eval` scripts. cmd.exe re-parses every
argument the .cmd shim forwards, so multi-line text sent through `fill`
(browser_type) was truncated at its first line and %VAR% expanded the same
way (#113838). Any non-eval command whose argv carries a newline or % now
runs as `agent-browser batch --json` with the command as a JSON array on
stdin (served from a temp file like stdout/stderr), and the single batch
entry is unwrapped to the usual {success, data, error} shape. The shim test
now drives _run_browser_command with _spawn_and_collect captured, so the
call-site wiring is guarded, not just the helper.
Follow-up to the cherry-picked one-line ``_GET_IMAGES_JS`` (#113844, @KoNit-K),
closing the class the issue asked to audit. Supersedes the earlier #82278 (@Clubheader),
which reached the same shim-truncation diagnosis via ``eval --stdin``; base64 needs no
stdin plumbing and also survives cmd.exe ``%VAR%`` expansion:
- ``browser_tool_session._shim_safe_eval_args``: when the resolved argv[0] is a
``.cmd``/``.bat`` shim (``npx.cmd``, npm's ``agent-browser.cmd`` on Windows)
the ``eval`` script is sent as ``--base64 <b64>`` (``agent-browser eval -b``,
present since the 0.26 floor). cmd.exe re-parses the child command line —
a newline ends the argument and ``%VAR%`` expands even inside quotes — so
this is the only lossless transport for model-authored ``browser_console``
expressions and the vault ``eval`` fallback, not just the bundled constant.
Every other spawn target (native binary, POSIX shim) keeps the raw argv.
- Tests: two invariants in ``tests/tools/test_browser_eval_shim_args.py``
(shim → base64 round trip with native/POSIX/non-eval controls; every
``*_JS`` constant across ``tools/browser_*`` is single-line). The
contributor's get_images regression test is dropped as subsumed by the
module-wide constant scan.
Host-specific (Windows): code-path proof. Live on this host: real
``agent-browser --json eval -b <b64>`` of the collapsed script returns the
image list (data: URIs filtered); ``eval "JSON.stringify("`` — the first line
the shim delivers — reproduces the reporter's exact
``SyntaxError: Unexpected end of input``.
Co-authored-by: Clubheader <Clubheader@users.noreply.github.com>
`computer_use` carried two Bot Screen actions, `request_handoff` and `wait_for_human`, that let
the model ask for the screen and then block a tool call until the human handed it back. Both only
make sense when a person is guaranteed to be watching the Desktop pane; from Telegram, the CLI or
a cron worker the request lands nowhere and the wait burns minutes before returning. The blocking
wait also fought the sequential tool deadline (600 s default vs 420 s), so the model saw a timeout
before the wait returned while the thread stayed parked.
The agent now simply says what it needs in its reply and ends the turn; the user takes over from
the pane, does the step, hands back and tells it to continue. Take over / hand back and every fence
(actions refused with human_has_control, epoch-voided results, suppressed thumbnails) are
unchanged. Removes the handoff module, the schema entries and their host-conditional rewriter,
the lease's pending_handoff field and wait helpers, and the "Bot needs you" badge in the pane.
launcher.sh received the dock browser as one shell line and recovered the
executable with `${3%% *}`: a Chromium under '/opt/Google Chrome/' or a
profile dir under a HERMES_HOME with a space split at the first blank, the
existence check failed or Exec= became garbage, and the dock had no working
Browser icon.
Python now hands the launcher the bare executable (HERMES_BD_BROWSER_EXEC, for
the `command -v` check) and a ready-made Exec= line
(HERMES_BD_BROWSER_EXEC_LINE) built by `browser.dock_exec_line`: each argument
double-quoted, reserved characters backslash-escaped inside the quotes and the
backslashes string-escaped once more, per the Desktop Entry spec. `dock_argv`
is the single source of the dock's arguments (incl. the root sandbox flags).
Test: tests/tools/test_bot_desktop_browser.py — a spaced executable and a
spaced, quote-bearing profile dir produce a correctly quoted Exec= line
(AttributeError on the previous commit); the launcher seed test keeps running
the real script (`bash -n` clean).
`browser.executable()` always preferred Playwright's bundled Chromium. Its
`chrome_sandbox` is not setuid, so for a non-root user on Ubuntu 23.10+
(`kernel.apparmor_restrict_unprivileged_userns=1`) the dock's Browser icon died
with `FATAL: No usable sandbox!` even when a distro chromium with the sandbox
helper was installed. And when the only bundle is chromium_headless_shell (the
official image) `executable()` is None and launcher.sh silently skipped the
dock entry — nothing anywhere said "no headed browser".
Now: non-root under the userns restriction tries a system chrome/chromium first
and falls back to the Playwright build (no --no-sandbox for non-root, by
ruling: a loud failure beats a sandbox-less browser). `DesktopStatus.browser`
carries the resolved executable (None = no headed browser) through `as_dict()`
so the pane and `hermes computer-use screen status --json` can show it.
The root sandbox-bypass flags are ONE list, `browser_tool_session.
CHROMIUM_SANDBOX_BYPASS_ARGS`: agent-browser gets it via AGENT_BROWSER_ARGS and
the dock command appends the same flags as root, so the human's click and the
agent's launch start the same binary the same way. The sysctl reader is shared
as `apparmor_restricts_unprivileged_userns()`.
Tests: tests/tools/test_bot_desktop_browser.py — restricted non-root prefers
the system chromium and keeps Playwright's when alone (AttributeError on
bc36ddb5f9); root dock args are a superset of the agent's (dock lacked
--no-sandbox); status exposes browser / None (field missing).
`browser_console(expression=...)` answers over the CDP supervisor's persistent
WebSocket and returns BEFORE `_run_browser_command`, which is where the Bot
Desktop lease fence lived. With a human holding the lease every other browser
command returned `human_has_control` while the one command that evaluates
arbitrary JS still read the page the human was typing into.
The fence is now ONE helper, `browser_tool_session.run_fenced(session_info, fn)`
(admit -> run -> epoch check), used by both the subprocess path and the eval
fast path, so a future third path cannot fork the policy again.
Test: tests/tools/test_bot_desktop_browser_fence.py — with a human lease and a
fake supervisor returning a value, browser_console must return
human_has_control and never evaluate the expression (red on bc36ddb5f9).
The dock's Browser launched raw Chromium on the shared user-data-dir with no automation
endpoint. When the human opened it first and handed back, agent-browser's own launch was
forwarded into their instance by Chromium's ProcessSingleton and exited 21 without a
DevToolsActivePort, so every browser_navigate failed until the human closed their window.
The reverse order worked, which is why it slipped through.
- The dock command carries --remote-debugging-port=0, so a human-started instance advertises a
port in <user-data-dir>/DevToolsActivePort (browser.dock_command; runtime.start reads it).
- browser.running_instance_cdp_port() trusts that file only when SingletonLock's pid is alive
AND the port accepts a connection (both files outlive a closed Chromium), and never for the
instance the calling agent-browser session launched itself: handing that daemon --cdp makes
it treat the launch as a config change, close its browser and attach to the port that just
died with it (seen live).
- The local argv builder appends --cdp <port> to the --session launch when such an instance
exists, so the same daemon (and its snapshot refs) drives the human's window.
Live, HERMES_HOME=/tmp/bs-f-home on this host: human-first — dock instance up, navigate x2
succeeded, one Chromium main process (same pid) throughout; agent-first — navigate, dock
click, navigate x2 succeeded, one process throughout. Before the fix human-first returned
"Chrome exited early (exit code: 21) ... Failed to create SingletonLock".
(cherry picked from commit d732fad0af005ffd38eca153dd29c0f7e9dd6bc7)
Real-profile local sessions attach over a loopback cdp_url, yet that
Chrome is launched with the Bot Desktop DISPLAY, so it is the very
browser a human who took over is typing into; keying the fence on "no
cdp_url" let every command through. Fence whenever the session carries
the `local` feature and exempt only remote/cloud/user-supplied CDP. Also
fence while a human holds the lease even when the published DISPLAY is
gone (dead Xvnc), matching computer_use instead of silently unfencing.
(cherry picked from commit 7b8825bf32a67229666f1f848d3ac171ff3fbc93)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
