b41c8ddfc4
A bot running on a headless Linux gateway now gets its own desktop (TigerVNC Xvnc + a minimal Xfce session, one per profile) that Hermes Desktop streams live. The user can watch the bot work, take over to type a login / 2FA code / CAPTCHA, and hand control back; the bot refuses every computer_use action (screenshots included) while a human holds the screen, then resumes with the session cookies the human just created. Why this shape: - The screen lives on the machine Hermes runs on, not in a vendor cloud browser, so it works for any app the bot drives and keeps the session on the user's host. - Xfce components are launched individually (xfwm4, xfce4-panel, xfdesktop, xfsettingsd) under a private dbus session rather than xfce4-session/the metapackage: no screensaver, power manager or polkit agent to lock or prompt a headless desktop. - Transport is raw RFB over a WebSocket beside /api/ws, authenticated with a one-shot ticket minted through the already-authenticated RPC channel; noVNC runs in the Electron renderer. The bridge parses the RFB client stream and drops keyboard/pointer/clipboard (incl. QEMU Extended KeyEvent, which noVNC switches to once Xvnc advertises it) from any viewer that does not hold the lease, so viewOnly is enforced server-side, not by the client. - One lease per profile (agent | human viewer) is the single truth for the RFB bridge, the computer_use tool and the Desktop UI; taking control evicts other viewers' input with close code 4000 control-taken. - Auto-start happens only at the computer_use tool boundary (headless host, packages present, bot_desktop.auto_start=true); env builders stay pure so status probes and tests never spawn X servers. tests/tools/conftest.py pins the binaries to "missing" for the same reason the browser-use fixture does. Surfaces: Desktop (Bots → right-click → Open Screen; Take over / Hand back), CLI (`hermes computer-use screen status|start|stop|install-deps`), tool (`computer_use` actions request_handoff / wait_for_human), gateway RPCs (display.status/start/stop/observe/lease.acquire/lease.release + display.lease event), docs page user-guide/features/bot-screen.
6 lines
288 B
Python
6 lines
288 B
Python
"""Bot Desktop: per-profile headless Xfce desktop over RFB, viewed from Hermes Desktop.
|
|
|
|
``runtime`` owns the Xvnc/Xfce process and the published env; ``lease`` owns who may drive the
|
|
screen (agent vs. human); ``rfb_filter`` is the byte-level input gate the WebSocket bridge applies.
|
|
"""
|