Files
hermes-agent/tools/computer_use
Adrian Lastra 519ec7b3b3 fix(computer_use): parse (label) and = "value" AX element label forms
The SOM/AX element list dropped labels for two extremely common cua-driver
render forms, leaving the model unable to target elements by name:
  - [79] AXButton (Dark)              -> parenthesised label
  - [4]  AXStaticText = "Wi-Fi"       -> = "value" form
  - [92] AXPopUpButton = "Automatic"  -> = "value" form
The old regex only matched quoted "label" and id=Label, so System Settings
buttons/text/popups all surfaced with empty labels. That's why selecting the
macOS Appearance 'Dark' button by element index required guessing — the
labels weren't available to aim with.

Fix: extend _ELEMENT_LINE_RE to capture all four label forms (= "value",
"quoted", (parenthesised), id=Label), skipping a pure-digit (N) order number
in favour of the id= label. Verified live against System Settings: the
Appearance buttons now surface as Auto/Light/Dark.

Adds a regression test covering all label forms. Full suite: 84 passed.
2026-07-05 05:36:31 -07:00
..