Files
hermes-agent/agent
Brooklyn Nicholson 3ead0f8dc1 feat(desktop): widget clicks reach the agent as hidden user turns — the widget updating IS the response
An inline ::preview widget could render and be clicked, but the click went
nowhere: the sandbox has no channel to the agent, so an interactive chart
was a dead end. Now the frame injects a second script beside the measurer
that gives the page one voice:

  window.hermes.send('get-price eth')
  <button data-hermes-send="get-price eth">ETH</button>  (zero-script form)

The prompt rides postMessage up tagged with the mount token, then goes
through the composer's own send path (requestComposerSubmit -> prompt.submit)
flagged display_kind=hidden — the same row-typing auto-continue and internal
notifications already use. The agent wakes and takes a real turn; the
durable row persists (context, resume, DB audit); but NO bubble renders,
live or on reload. The user clicks ETH and the chart just changes — the
off-screen loop is click -> hidden turn -> agent rewrites the widget file ->
frame hot-swaps.

Trust boundary matches size reports and is tighter where it matters: mount
token required (frames can't forge each other's intents), string-only,
trimmed, capped at 500 chars, throttled to one intent per second per frame.
The gateway whitelists display_kind to "hidden" — the RPC can't mint
arbitrary row types — and the flag threads through both turn paths (inline
and compute-host isolation) so isolated sessions don't resurrect bubbles on
resume.

The desktop platform hint teaches the model to wire interactive widgets
with data-hermes-send and to answer clicks by updating the widget's file
rather than with prose; the SDK doc documents the contract.
2026-08-17 16:08:18 -05:00
..
2026-08-12 13:37:04 -04:00