31 Commits

Author SHA1 Message Date
Teknium
399d956903 feat(bot_desktop): Bot Screen, computer_use and the browser run inside the terminal backend (#121169)
* feat(docker): publish nousresearch/hermes-sandbox:desktop for terminal backends

The terminal backends (docker, modal, daytona, singularity) all default to
nikolaik/python-nodejs:python3.11-nodejs20, a bare Python+Node base. For Bot
Screen, computer_use and the browser to run INSIDE that sandbox instead of on
the gateway host, the sandbox image needs the display stack.

docker/sandbox-desktop.Dockerfile is that base plus:
  - the everyday tools it lacked (jq, ripgrep, fd, tmux, less, nano, vim,
    zip, rsync, tree, procps, htop, sudo for the base's uid-1000 `pn`)
  - the exact package set the Hermes -desktop image installs (TigerVNC,
    Xfce components, dbus, xauth, fonts)
  - Playwright's headed Chromium (same build as the -desktop image)
  - cua-driver 0.28.2 from its pinned release tarball

No Hermes inside; the default user stays root like the base so nothing
changes for people who just switch docker_image. Desktop processes run as
`pn`. 4.27 GB on amd64.

docker.yml gains a `sandbox` variant with its own cache scope and repository
(nousresearch/hermes-sandbox:desktop, :main-desktop, :<release>-desktop);
the docker-integration suite is skipped for it (no Hermes to test) and
docker/sandbox-desktop-smoke.sh runs instead: as `pn`, every launcher and
cua-driver binary resolves, the real launcher.sh publishes :20, the RFB
socket completes the 3.8 handshake relayed over `docker exec -i` stdio, and
a headed Chromium maps a window on that display. hadolint lints the new
Dockerfile in docker-lint.yml.

* feat(docker): sandbox desktop base on python3.13-nodejs26

Matches the Hermes image (Python 3.13 / Node 26) and the top of requires-python;
the default docker_image tag it inherited was Python 3.11 / Node 20. Same pn
uid 1000, Debian 13; smoke (launcher, RFB relay, headed Chromium) passes.

* feat(docker): bake agent-browser into hermes-sandbox:desktop

The browser tools drive the agent-browser CLI; when the browser follows the
terminal backend that CLI has to exist inside the sandbox. Pinned to the same
^0.26.0 range the gateway resolves, --ignore-scripts like the gateway's npx path.

* feat(bot_desktop): the screen, computer_use and the browser follow the terminal backend

A user who sandboxes `terminal` (docker/ssh/singularity) had the agent's
screen, cua-driver and Chromium running on the gateway HOST beside that
sandbox: Bot Screen gave a headless host a display, the Xfce panel carries
xfce4-terminal, and `computer_use` could open a shell outside the boundary
the sandbox exists for.

Now the desktop lives where the terminal lives:

- tools/environments/streams.py: one primitive per spawn-per-call backend, the
  local argv prefix that runs its remainder inside the sandbox with stdio open
  (`docker exec -i`, `ssh`, `apptainer exec`). SDK backends (modal, daytona,
  vercel) have none and report so.
- tools/bot_desktop/sandbox_host.py: launcher.sh runs inside the sandbox as
  the image's `pn`; the pane's RFB bytes ride a 12-line python relay over that
  prefix; `cua-driver mcp` is the prefix + the sandbox image's own driver.
- tools/bot_desktop/placement.py + `bot_desktop.placement` (auto|terminal|
  gateway). `auto` follows the backend; a sandbox that cannot host a screen
  REFUSES with the opt-in named instead of silently using the host.
- runtime.start/stop/status/published_env branch on placement; the pane,
  lease, epoch fencing and CLI are unchanged.
- cua_backend: the MCP invocation is the sandbox one when the screen is
  there; the host driver's runtime contract is irrelevant then; check_fn is
  true under a terminal placement without a host binary.
- browser_tool_session: agent-browser invocations are wrapped in the prefix
  with the daemon, socket dir and profile inside the sandbox; screenshots are
  fetched back so MEDIA: paths keep working; recycle closes the sandbox
  daemon.
- web_routers/display.py: the bridge pumps a relay's stdio when the screen is
  in a sandbox, a unix socket otherwise.

Live on docker with nousresearch/hermes-sandbox:desktop: start/observe/RFB
handshake through the dashboard bridge, human takeover fences the agent
(HumanHasControl) and keystrokes reach the sandbox Xvnc, handback restores,
three start/stop rounds leave zero desktop processes; computer_use capture
and list_windows see only the sandbox's Xfce; browser_navigate/snapshot/
vision run with Chromium and agent-browser inside the container and zero
host processes on the bot profile; modal + auto refuses naming the opt-in.

* feat(desktop): Screen pane shows where a sandbox-placed screen runs; Install is host-only

DesktopStatus gains placement ('gateway' | 'terminal:<backend>'). A sandbox
image lacking the stack is a blocker naming hermes-sandbox:desktop, shown in
place of Start; install_command stays None there because the pane's Install
button runs the package manager on the gateway host, the wrong machine, and
display.install refuses for the same reason. The pane header carries
'Screen runs inside the docker sandbox, with the terminal' (4 locales).

* fix(bot_desktop): "is the screen in the sandbox" is a disk check on hot paths, never a config read

Every browser command and CUA spawn asked in_sandbox(), which resolves placement by
loading config, which initializes HERMES_HOME. Under a test's fake home that raised
HomeInitializationError from _run_browser_command; on a real host it read config per
click. Hot paths now ask sandbox_screen_running(): the start marker on disk, written
only by a sandbox start. Policy (in_sandbox) stays for start/install, where config is
the question. display.observe gates on "an RFB endpoint exists" for either placement.

* test(moa): late-accounting sink test asserts the wedged slot's row, not sink order

Under CI load the poll loop can see the interrupt before collecting the fast slot, so the
fast slot also arrives late and first; the test then failed on sink_calls[0]. The
contract is that the wedged slot's real usage reaches the sink.

* chore: retrigger CI (zero-job dispatch failure, auto-heal)

* fix(bot_desktop): a sandbox that died under a live screen fails loudly, never falls to the host

sandbox_screen_running() drops a start marker whose terminal environment is no longer
registered (stale after a process restart). When the environment object outlives its
container, the browser's sandbox wrap now checks the published DISPLAY and raises
"the screen inside the terminal backend's sandbox is gone; start it again" instead of
KeyError('AGENT_BROWSER_PROFILE'). Live: fresh sandbox navigate ok; docker rm -f the
container; next navigate returns that error; zero host Chromium either way.

* feat(terminal): nousresearch/hermes-sandbox:desktop is the default container sandbox

Every container backend (docker, modal, daytona, singularity) now defaults to the
sandbox image with the desktop stack, so Bot Screen, computer_use and the browser
run inside the sandbox for everyone who never chose an image; Python 3.13 / Node 26
match the Hermes image. One constant (DEFAULT_SANDBOX_IMAGE) replaces six copies of
the old literal. Migration 47 moves saved configs still holding the OLD default and
never touches an image the user pinned. Docker reuse recreates a container built
from another image, or the flip would silently never take effect for anyone with a
persisted container (live: old container removed, new one on 3.13 / Node 26 with
Xvnc, cua-driver, agent-browser present).

* chore: retrigger CI (zero-job dispatch failure)

* chore: retrigger CI (zero-job dispatch failure, auto-heal)

* chore: retrigger CI (zero-job dispatch failure, auto-heal)

* chore(config): template stamps v47 and shows the new default sandbox image

The template is what install.sh / docker / doctor --fix seed; a stamp behind
DEFAULT_CONFIG makes every fresh install migrate on first run.

* feat(sandbox): the default image change is a decision, not a surprise

A persisted Docker sandbox on another image is kept when docker_image is unset;
only a written docker_image (an explicit pin) recreates it. The pin verdict
travels as TERMINAL_DOCKER_IMAGE_PINNED through both terminal bridges (process
env and per-profile scope) and the container-config allowlist.

Approval surfaces, all through hermes_cli.sandbox_image_switch: the interactive
CLI asks once at startup (y = pin the new image, n = pin the current one, Enter =
ask later); the Screen pane shows the same choice with Switch / Keep buttons via
display.switchSandboxImage; `hermes config set terminal.docker_image …` is the
same answer from any shell. Gateways and cron never decide: they keep the sandbox
and log the notice.

Migration 47 now unsets a saved image equal to the OLD default instead of
rewriting it to the new one — that value was the template copied, not a pin, and
rewriting it would have made the runtime recreate existing sandboxes unasked.

Modal restores its snapshot and Daytona reuses its labeled sandbox regardless of
the configured image, so existing sandboxes there were already untouched.

* fix(config): both plain defaults that preceded the desktop sandbox image are template copies

main pinned nikolaik/python-nodejs:python3.14-nodejs22 (cd0f97f833) without a migration;
a saved config holding either literal is unset by migration 47, so it follows the default
and existing sandboxes get the keep-or-switch decision instead of a silent recreate.

* ci(docker): build the sandbox image on release/dispatch, not every main push

Leaves docker.yml exactly as on main. The sandbox image carries no Hermes code,
so two 4 GB multi-arch builds per merge bought nothing. sandbox-image.yml builds
and smokes on a PR that edits its own Dockerfile/smoke, and publishes only on a
release or a manual dispatch with publish=true. Stable tag stays :desktop.

* fix(config): keep main's config.py/config_defaults.py edits under the sandbox-image delta

The rebase resolved both files wholesale with the branch side, dropping main's move to
hermes_yaml (the 3.14 runtime venv has no PyYAML) and the 3.14 base pin. This is main's
version plus exactly the branch's own changes: DEFAULT_SANDBOX_IMAGE, the pin verdict in the
env bridge, placement defaults and the v47 stamp.

* test: sandbox-image tests read config.yaml through hermes_yaml (no PyYAML on the 3.14 runtime)

* fix(bot_desktop): read the sandbox marker BOM-tolerantly (windows footgun lint)

* fix(bot_desktop): placement is the authority; sandbox screen survives restarts

Review findings on the sandbox-hosted Bot Screen, each reproduced live first.

Authority. The browser preflight and the CUA invocation keyed off screen
LIVENESS, so `placement: terminal` with the screen not yet up handed the tool an
unchanged host command. `runtime.tool_placement()` is now the one resolver:
terminal placement starts the sandbox screen on demand (no auto_start opt-in
inside the user's own sandbox), refused placement raises its reason, and neither
ever yields the host. placement.resolve() answers a local backend from env alone
so the common case costs no config load on the spawn path.

Restart. sandbox_screen_running() deleted the marker whenever the process-local
terminal registry was empty, i.e. after every gateway restart, while Xvnc kept
running in the container; stop() then returned False and left it. The marker
now records the owning container; liveness comes from `docker inspect` on it,
stop/status re-attach to the recorded owner (even after the placement setting
moved), and only a container that is gone drops the marker.

SSH. remote_argv emitted `bash -c <script>` as three words; OpenSSH joins them
and the remote login shell ran `bash -c export` and the rest itself. The script
travels as one quoted word for ssh (remote_command knows the backend); docker
and apptainer keep argv.

CDP reach. agent-browser inside the sandbox reports the sandbox's loopback;
the Browser Use harness, browser_exec and the vault supervisor connect from the
host and got connection refused. streams.forward_port() proxies a local port
over the exec stream (same relay as the RFB bridge) and the CDP URL is rewritten
to the local end.

pids limit. --pids-limit 256 counts threads; measured on the desktop image the
desktop stack is 44, one Chromium tab 212, the agent's browser with two tabs
488. Past the cap every further docker exec died with "procReady not received".
Default is 2048 with the measurements in the comment.

Replacement. An approved image switch force-removed the old container before
`docker run` tried the new image; a private tag or registry outage left nothing.
The image is inspected/pulled first and the old container kept on failure.

Desktop integration. The sandbox start never passed the dock's browser launcher
(no Browser icon) and the thumbnail needed a host launcher pid + host ImageGrab
(always None). The dock runs the sandbox's Playwright Chromium on the shared
profile; the thumbnail is grabbed inside the sandbox (Pillow baked into the
image). The browser profile moves from /tmp — a 512 MB tmpfs emptied on every
container stop — to the desktop user's home, so logins follow the container.

Pin provenance. A TERMINAL_DOCKER_IMAGE written in a routed profile's .env is a
pin even when it spells the default; the scope compared values before.

* docs(bot-screen): no literal tmp path in the profile-location note

* fix(bot_desktop): docker inspect liveness probe closes stdin (TUI subprocess guard)

* fix(bot_desktop): adopting a screen the sandbox kept records the marker

Live ssh probe: after the host's state was lost while the sandbox kept its
Xvnc, start() took the idempotent early return (display already published)
and never wrote the host marker, so status/thumbnail/stop lost the screen.
Record the adopted display like a fresh launch.

Docs: what an ssh host of your own must carry, and why a Dockerfile ENV is
not enough for a login session (PLAYWRIGHT_BROWSERS_PATH via /etc/environment).

* docker(sandbox-desktop): login sessions find the browser (PLAYWRIGHT_BROWSERS_PATH via /etc/environment)

* docs(bot-screen): what the Apptainer path inherits from the image and what it does not

* chore(config): sandbox-image migration is 47→48 (main took 47 for compression.threshold_tokens)

* chore: retrigger CI (zero-job dispatch failure, auto-heal)
2026-09-28 03:34:07 -07:00
ethernet
2be53ecd7b fix(encoding): read kernel pseudo-files as plain utf-8
utf-8-sig exists to tolerate BOMs that Windows tooling adds to files
users edit. /proc and /sys files are generated by the Linux kernel, never
BOM'd and absent on Windows, so -sig there only muddies the read/write
policy. Switch every literal /proc/ and /sys/ read to utf-8 and teach
the footgun read rule that string literals starting with /proc/ or
/sys/ are exempt (user-edited files keep utf-8-sig).
2026-09-24 11:50:30 -04:00
ethernet
3f4b8840f1 Merge remote-tracking branch 'origin/main' into ethie/pm-clean
# Conflicts:
#	pyproject.toml
#	tests/fixtures/resolution_allowlist.json
2026-09-23 07:36:31 -04:00
teknium1
7ed6534c7e Merge origin/main: browser fence composed with the dispatch/retry split (#115184); server registration, i18n, contracts 2026-09-23 03:25:06 -07:00
ethernet
9f2ba1b74d merge origin/main (779 commits) into ethie/pm-clean
Branch semantics kept where main and PM disagree: update_cmd_deps.py,
constraints-termux.txt, the Electron update-api-check module and the
post-swap hand-off test stay deleted; the pending-fleet-restart catch-up
and the local_runtime tag/download ladder stay retired (PM owns engines).

Ported from main onto the branch's shape: profile_scoped_chore for the
auto-archive and plugin-update housekeeping chores, the local-runtime
cross-process boot lock and residency cap, the checkpoint tmp_pack sweep,
the cua daemon-liveness status probe, the remote-served Desktop update
flag (posix.sh / windows.ps1), sign-in for env-pinned remote gateways
(urlDisabled on RemoteSetupFields), the uvloop extra split (uvicorn
without [standard]), and the umask-scoping spawn test.

uv.lock regenerated with pm.build_env --lock-only; new utf-8 reads from
main switched to utf-8-sig (check-windows-footguns).
2026-09-21 00:58:39 -04:00
teknium1
73c4dc151d fix(browser): recycle a poisoned local session after a protocol-level agent-browser failure
A finished (non-timeout) agent-browser failure at the backend level — the CLI
exiting 101 against a stale session daemon, or empty/non-JSON output from a dead
one — returned normally as {"success": False}, so the cached local session record
was never marked suspect and every later browser command re-ran against the same
dead daemon until the process was recycled by hand (#115184).

- _interpret_browser_command_output carries `returncode` on its three
  protocol-level failure dicts (parsed page-level errors never carry one)
- _is_recoverable_local_backend_failure classifies those on plain local Chromium
  sessions only (cloud/CDP/real-profile/Lightpanda own their recovery)
- _recycle_local_session is the local half of _handle_browser_command_timeout,
  extracted so the timeout path and the new path share the alive→suspect /
  dead→tree-kill+evict split
- _run_browser_command retries once on the replacement session; `close` is
  exempt (a dead daemon is already closed and cleanup must not spawn a session
  just to close it)
- argv construction + spawn moved into _dispatch_browser_command so the retry
  loop stays a loop over one call

Based on #115206 by @liuhao1024 (returncode on the failure dict, the
recoverability predicate, the one-retry loop); reshaped so the recycle helper is
shared with the timeout path instead of calling the timeout handler.

Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
2026-09-20 12:47:50 -07:00
teknium1
b2629519e7 fix(bot-screen): a human holding the lease keeps the shared browser alive
The janitor reaped the bot's headed Chromium after 120 s of AGENT inactivity — which is
exactly the state a human takeover (login, 2FA) puts the agent in — so the browser died
under the human mid-login. The janitor now counts a human-held lease as activity for the
browser the human shares with the bot, and the agent-browser daemon's own idle timer
(which cannot see the lease) steps back on the Bot Desktop so the lease-aware janitor
owns that browser's lifetime; a crashed hermes still leaves it to the orphan reaper.

Fixes #110064
2026-09-18 21:07:18 -07:00
teknium1
ada0538199 fix(bot-screen): only a real headed browser command auto-starts the screen; the env builder stays pure
_build_browser_env() is shared by the npx cache warmer (hermes update / doctor
--fix), the lazy Chromium auto-installer, the Lightpanda engine's env and
browser_use_cli. Hooking bot_desktop.auto_start there made every one of those
block up to 15 s spawning Xvnc+Xfce with browser.headed on, against
desktop_env()'s own "never starts anything" contract.

The hook now sits where a headed Chromium is actually spawned for a tool
action, mirroring computer_use dispatch: _spawn_and_collect (the first
agent-browser command forks the daemon; Lightpanda engine excluded), the Chrome
fallback from Lightpanda, and the real-profile Chrome launch. The regression
test asserts both halves: the env builder never starts the screen, the headed
Chromium spawn does, a headless or Lightpanda spawn does not.
2026-09-18 19:20:01 -07:00
ethernet
a6ae6ace51 Merge remote-tracking branch 'origin/main' into ethie/pm-clean
# Conflicts:
#	.github/workflows/js-tests.yml
#	agent/model_metadata.py
#	apps/desktop/electron/main.ts
#	apps/desktop/scripts/bundle-electron-main.mjs
#	apps/desktop/src/app/settings/about-settings.tsx
#	apps/desktop/src/app/settings/gateway-settings.test.tsx
#	apps/desktop/src/app/settings/gateway-settings.tsx
#	apps/desktop/src/app/updates-overlay.tsx
#	gateway/shutdown_flush.py
#	hermes_bootstrap.py
#	hermes_cli/local_runtime/binaries.py
#	hermes_cli/main.py
#	hermes_cli/managed_uv.py
#	hermes_cli/update_cmd.py
#	hermes_cli/update_cmd_deps.py
#	hermes_cli/update_cmd_fleet.py
#	hermes_cli/update_cmd_maint.py
#	hermes_cli/update_receipt.py
#	hermes_cli/update_serve_obligations.py
#	hermes_constants.py
#	tests/hermes_cli/test_doctor.py
#	tests/hermes_cli/test_managed_uv.py
#	tests/hermes_cli/test_pending_supervisor_recovery.py
#	tests/hermes_cli/test_startup_fast_guards.py
#	tests/hermes_cli/test_update_desktop_stale_warning.py
#	tests/hermes_cli/test_update_fleet_restart_pending.py
#	tests/hermes_state/test_hermes_state.py
#	tests/tools/test_tirith_security.py
#	tools/bot_relay.py
#	tools/checkpoint_manager.py
#	tools/write_approval.py
#	website/docs/getting-started/updating.md
#	website/docs/reference/environment-variables.md
2026-09-18 17:26:10 -04:00
teknium1
b1be493b41 Merge origin/main: SDK export list (WORKSPACE_PAGE_HEADER_AREA beside the profile-group exports) 2026-09-18 14:09:20 -07:00
teknium1
6c9e583860 fix(browser): route any newline/% argument past a Windows .cmd shim losslessly
The base64 guard covered only `eval` scripts. cmd.exe re-parses every
argument the .cmd shim forwards, so multi-line text sent through `fill`
(browser_type) was truncated at its first line and %VAR% expanded the same
way (#113838). Any non-eval command whose argv carries a newline or % now
runs as `agent-browser batch --json` with the command as a JSON array on
stdin (served from a temp file like stdout/stderr), and the single batch
entry is unwrapped to the usual {success, data, error} shape. The shim test
now drives _run_browser_command with _spawn_and_collect captured, so the
call-site wiring is guarded, not just the helper.
2026-09-18 10:29:47 -07:00
teknium1
35a03bce14 fix(browser): eval scripts reach a Windows .cmd shim base64-encoded; tests trimmed
Follow-up to the cherry-picked one-line ``_GET_IMAGES_JS`` (#113844, @KoNit-K),
closing the class the issue asked to audit. Supersedes the earlier #82278 (@Clubheader),
which reached the same shim-truncation diagnosis via ``eval --stdin``; base64 needs no
stdin plumbing and also survives cmd.exe ``%VAR%`` expansion:

- ``browser_tool_session._shim_safe_eval_args``: when the resolved argv[0] is a
  ``.cmd``/``.bat`` shim (``npx.cmd``, npm's ``agent-browser.cmd`` on Windows)
  the ``eval`` script is sent as ``--base64 <b64>`` (``agent-browser eval -b``,
  present since the 0.26 floor). cmd.exe re-parses the child command line —
  a newline ends the argument and ``%VAR%`` expands even inside quotes — so
  this is the only lossless transport for model-authored ``browser_console``
  expressions and the vault ``eval`` fallback, not just the bundled constant.
  Every other spawn target (native binary, POSIX shim) keeps the raw argv.
- Tests: two invariants in ``tests/tools/test_browser_eval_shim_args.py``
  (shim → base64 round trip with native/POSIX/non-eval controls; every
  ``*_JS`` constant across ``tools/browser_*`` is single-line). The
  contributor's get_images regression test is dropped as subsumed by the
  module-wide constant scan.

Host-specific (Windows): code-path proof. Live on this host: real
``agent-browser --json eval -b <b64>`` of the collapsed script returns the
image list (data: URIs filtered); ``eval "JSON.stringify("`` — the first line
the shim delivers — reproduces the reporter's exact
``SyntaxError: Unexpected end of input``.

Co-authored-by: Clubheader <Clubheader@users.noreply.github.com>
2026-09-18 10:29:47 -07:00
teknium1
88bd29213b refactor(bot-screen): drop the agent-side handoff actions; takeover is always human-initiated
`computer_use` carried two Bot Screen actions, `request_handoff` and `wait_for_human`, that let
the model ask for the screen and then block a tool call until the human handed it back. Both only
make sense when a person is guaranteed to be watching the Desktop pane; from Telegram, the CLI or
a cron worker the request lands nowhere and the wait burns minutes before returning. The blocking
wait also fought the sequential tool deadline (600 s default vs 420 s), so the model saw a timeout
before the wait returned while the thread stayed parked.

The agent now simply says what it needs in its reply and ends the turn; the user takes over from
the pane, does the step, hands back and tells it to continue. Take over / hand back and every fence
(actions refused with human_has_control, epoch-voided results, suppressed thumbnails) are
unchanged. Removes the handoff module, the schema entries and their host-conditional rewriter,
the lease's pending_handoff field and wait helpers, and the "Bot needs you" badge in the pane.
2026-09-13 09:30:43 -07:00
teknium1
7fd5f59ba2 fix(bot-screen): dock Browser entry quotes its Exec= line, so spaced paths work
launcher.sh received the dock browser as one shell line and recovered the
executable with `${3%% *}`: a Chromium under '/opt/Google Chrome/' or a
profile dir under a HERMES_HOME with a space split at the first blank, the
existence check failed or Exec= became garbage, and the dock had no working
Browser icon.

Python now hands the launcher the bare executable (HERMES_BD_BROWSER_EXEC, for
the `command -v` check) and a ready-made Exec= line
(HERMES_BD_BROWSER_EXEC_LINE) built by `browser.dock_exec_line`: each argument
double-quoted, reserved characters backslash-escaped inside the quotes and the
backslashes string-escaped once more, per the Desktop Entry spec. `dock_argv`
is the single source of the dock's arguments (incl. the root sandbox flags).

Test: tests/tools/test_bot_desktop_browser.py — a spaced executable and a
spaced, quote-bearing profile dir produce a correctly quoted Exec= line
(AttributeError on the previous commit); the launcher seed test keeps running
the real script (`bash -n` clean).
2026-09-13 06:04:43 -07:00
teknium1
dff8929194 fix(bot-screen): dock Browser picks a browser that can start, and status says when there is none
`browser.executable()` always preferred Playwright's bundled Chromium. Its
`chrome_sandbox` is not setuid, so for a non-root user on Ubuntu 23.10+
(`kernel.apparmor_restrict_unprivileged_userns=1`) the dock's Browser icon died
with `FATAL: No usable sandbox!` even when a distro chromium with the sandbox
helper was installed. And when the only bundle is chromium_headless_shell (the
official image) `executable()` is None and launcher.sh silently skipped the
dock entry — nothing anywhere said "no headed browser".

Now: non-root under the userns restriction tries a system chrome/chromium first
and falls back to the Playwright build (no --no-sandbox for non-root, by
ruling: a loud failure beats a sandbox-less browser). `DesktopStatus.browser`
carries the resolved executable (None = no headed browser) through `as_dict()`
so the pane and `hermes computer-use screen status --json` can show it.

The root sandbox-bypass flags are ONE list, `browser_tool_session.
CHROMIUM_SANDBOX_BYPASS_ARGS`: agent-browser gets it via AGENT_BROWSER_ARGS and
the dock command appends the same flags as root, so the human's click and the
agent's launch start the same binary the same way. The sysctl reader is shared
as `apparmor_restricts_unprivileged_userns()`.

Tests: tests/tools/test_bot_desktop_browser.py — restricted non-root prefers
the system chromium and keeps Playwright's when alone (AttributeError on
bc36ddb5f9); root dock args are a superset of the agent's (dock lacked
--no-sandbox); status exposes browser / None (field missing).
2026-09-13 06:02:35 -07:00
teknium1
74fca28fe6 fix(bot-screen): browser_console obeys the screen lease on the supervisor fast path
`browser_console(expression=...)` answers over the CDP supervisor's persistent
WebSocket and returns BEFORE `_run_browser_command`, which is where the Bot
Desktop lease fence lived. With a human holding the lease every other browser
command returned `human_has_control` while the one command that evaluates
arbitrary JS still read the page the human was typing into.

The fence is now ONE helper, `browser_tool_session.run_fenced(session_info, fn)`
(admit -> run -> epoch check), used by both the subprocess path and the eval
fast path, so a future third path cannot fork the policy again.

Test: tests/tools/test_bot_desktop_browser_fence.py — with a human lease and a
fake supervisor returning a value, browser_console must return
human_has_control and never evaluate the expression (red on bc36ddb5f9).
2026-09-13 05:51:46 -07:00
teknium1
7a55ef4b6b fix(bot-screen): agent attaches to a human-started dock Browser instead of dying on the profile singleton
The dock's Browser launched raw Chromium on the shared user-data-dir with no automation
endpoint. When the human opened it first and handed back, agent-browser's own launch was
forwarded into their instance by Chromium's ProcessSingleton and exited 21 without a
DevToolsActivePort, so every browser_navigate failed until the human closed their window.
The reverse order worked, which is why it slipped through.

- The dock command carries --remote-debugging-port=0, so a human-started instance advertises a
  port in <user-data-dir>/DevToolsActivePort (browser.dock_command; runtime.start reads it).
- browser.running_instance_cdp_port() trusts that file only when SingletonLock's pid is alive
  AND the port accepts a connection (both files outlive a closed Chromium), and never for the
  instance the calling agent-browser session launched itself: handing that daemon --cdp makes
  it treat the launch as a config change, close its browser and attach to the port that just
  died with it (seen live).
- The local argv builder appends --cdp <port> to the --session launch when such an instance
  exists, so the same daemon (and its snapshot refs) drives the human's window.

Live, HERMES_HOME=/tmp/bs-f-home on this host: human-first — dock instance up, navigate x2
succeeded, one Chromium main process (same pid) throughout; agent-first — navigate, dock
click, navigate x2 succeeded, one process throughout. Before the fix human-first returned
"Chrome exited early (exit code: 21) ... Failed to create SingletonLock".

(cherry picked from commit d732fad0af005ffd38eca153dd29c0f7e9dd6bc7)
2026-09-12 18:59:14 -07:00
teknium1
d1a1f9952d fix(browser): fence the bot's browser by provenance, not by cdp_url
Real-profile local sessions attach over a loopback cdp_url, yet that
Chrome is launched with the Bot Desktop DISPLAY, so it is the very
browser a human who took over is typing into; keying the fence on "no
cdp_url" let every command through. Fence whenever the session carries
the `local` feature and exempt only remote/cloud/user-supplied CDP. Also
fence while a human holds the lease even when the published DISPLAY is
gone (dead Xvnc), matching computer_use instead of silently unfencing.

(cherry picked from commit 7b8825bf32a67229666f1f848d3ac171ff3fbc93)
2026-09-12 18:58:05 -07:00
teknium1
15c51a0307 fix(bot-screen): browser tools obey the lease, epoch-only fence, dropped viewer link keeps exclusion 2026-09-12 18:57:50 -07:00
ethernet
5e4a2a3d24 refactor(pm): remove legacy dependency and launch managers
Competing installers and checkout-local venv assumptions bypassed PM
selection, install consent, and generation lifetimes. Route consumers
through PM and installation-bound launchers. Refresh source launchers
before obsolete Python entries can be collected.

Remove Node, browser, and CUA acquisition engines, obsolete venv-holder
handling, detached sync, and unused PM APIs. Keep historical updater
exports inert and preserve external tool ownership and native integration.

Share product freshness and prepared inputs across builders. Align plugin
admission, Docker provisioning, setup instructions, and behavioral tests.

Verified targeted Python and JavaScript tests, desktop and web typechecks,
scoped lint, real product builds, and the Docker frontend smoke test.
The missed post-setup test cleanup is included and verified.

Native Windows/macOS execution, full Rust compilation, and the complete
repository suite remain unverified. Historical compatibility requirements
were preserved and extended, not fully rescanned.
2026-09-12 14:57:38 -04:00
ethernet
6756d11b5f fix(pm): ship full chromium without headless shell
Full Chromium serves both headed and headless sessions. The separate
shell duplicates the browser payload and is not needed for either mode.

Remove the shell from PM and Docker. Select the managed Chromium
executable for agent-browser and the full Chromium channel for direct
Playwright callers. Route setup through PM and remove retired packages
from cached bundle stores without changing the user's tool store.

Update signing, architecture checks, launch probes and install guidance.
Leave llama packages and Docker archive cleanup unchanged.

Verification:
- Real agent-browser navigation, clicks, DOM reads and screenshots pass
  in headed and headless modes with the same Chromium executable.
- The direct Playwright doctor probe passes.
- Focused Python and desktop packaging tests pass, as do six Docker
  checks and both real-browser task-scroll tests.
- The built linux/amd64 image is 1.393 GB compressed, 223.6 MB smaller.
- The broader PM suite and two unrelated setup tests still fail.
  Those failures reproduce on unchanged HEAD.
- Five updated eval scripts parse; their full scenarios were not run.
2026-09-10 16:08:03 -04:00
ethernet
25ce047f98 fix(footguns): utf-8-sig on all reads flagged by check-windows-footguns (164 sites)
The 4,272-commit upstream merge brought 164 new utf-8 read violations
across 96 files. Class-fixed: reads utf-8-sig, writes unchanged.
check-windows-footguns.py --all: 164 -> 0.
2026-09-04 14:46:24 -04:00
Teknium
de60f789a7 simplify(compat): tools/browser_tool + browser_supervisor — drop 114 re-exports + 6 legacy aliases + PEP 562 requests/call_llm hook, repoint 21 non-test callers; siblings read sibling names directly 2026-09-03 14:16:54 -07:00
Teknium
e83816a4d1 review-fix(comments): restore lost #NNNN rationale comments across non-test source (mechanical sweep, condensed, code unchanged)
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
2026-09-03 09:44:26 -07:00
Teknium
35f8514afc refactor(tools): browser_tool — compact docstrings/comments across browser_tool_* modules (keep every WHY) 2026-09-02 23:53:53 -07:00
Teknium
2b41af97a1 refactor(tools): browser_tool — vision capture phase helper, session/lifecycle body compaction, tighter constant tables 2026-09-02 23:37:37 -07:00
Teknium
ed9476cf40 refactor(tools): browser_tool — consolidate config caches, navigate/eval/console helpers, lifecycle best-effort wrapper, compact session/vision bodies 2026-09-02 23:27:55 -07:00
Teknium
408cafe06f refactor(tools): browser_tool — data-driven tool table, guarded-action helper, compact re-export blocks, _pid_exists delegates to gateway.status 2026-09-02 23:01:32 -07:00
Teknium
d3523096fa refactor(tools): browser_tool — origin proxy replaces per-call _bt lookups, unified JSON/error builders, cached-config helper, dead shim removal 2026-09-02 22:39:28 -07:00
Teknium
6a9387f6f4 refactor(browser): table-driven registry.register loop for the 10 browser tools; bracket-hug compaction across browser_* modules (AST-identical) 2026-09-02 16:26:33 -07:00
Teknium
fdaaa87ea4 refactor(browser): move session/daemon command execution to tools/browser_tool_session.py, CDP override + supervisor lifecycle to tools/browser_tool_cdp.py, vision helpers to tools/browser_tool_vision.py 2026-09-02 16:21:07 -07:00