5eea87882a
* refactor(desktop): split clarify-tool.tsx into a clarify/ folder
Pure moves, no behaviour change. Parsing, the question-card core
(shell, choice rows, question block), the delivery watchdog, and each
pending/settled card get their own file so the core can be reused.
* feat(clarify): one questions[] shape with per-question status and one outcome
The clarify tool now takes only questions=[{question, choices?, multi_select?}].
A wrong shape is a tool error that names the right one, so a model that
sends the old top-level question/choices corrects itself on the next call.
Every result has the same shape on every surface: each response carries
status (answered, skipped, unanswered) with user_response null unless
answered, and the result carries one outcome (submitted, cancelled,
timed_out, undelivered) plus an optional surface-written notice. The
timeout and cancel sentences that used to pose as the user's answer are
gone, and the compressor reads status instead of matching their prefixes.
The callback contract is callback(questions) -> {answers, outcome, notice?}
(None = skipped, missing qid = unanswered). tui_gateway settles a batch the
same way for the last lock, a cancel, an interrupt and the deadline, and
clarify.lock keeps a null answer as a skip. A multi-select answer that is
not a JSON array counts as one typed ("Other") answer. The tool-row preview
reads the first question.
* feat(clarify): every surface asks through the one question card
CLI: the batch panel is the only panel; Enter on an empty field skips a
question, Ctrl+C cancels and keeps the locked answers, the deadline returns
timed_out. -q and -z return undelivered with a no-user notice.
Ink TUI: the single-question prompt is gone; the card supports multi-select
(Space or a digit toggles, Enter locks, typed Other text joins the picks),
an empty submit skips, Esc cancels, and "Batch" names are dropped.
Desktop: the single-question card is deleted and its keyboard handling moved
into the one card (arrows, letters and digits, auto-advance, Enter picks then
confirms). Confirm enables at one answer; blank questions lock null; Skip and
a composer message cancel; the settled card shows No answer for unanswered
questions. The bots room card follows the same contract.
Messaging: one card per question with a "Reply skip to skip" line in every
locale; timeouts and delivery failures map to timed_out and undelivered.
* fix(clarify): cancelled for a released messaging card, labels win over the skip word
A prose reply to a messaging choice card, /new and session end released
the wait with "", which read as timed_out. They now resolve with a
CANCELLED marker and the tool gets outcome cancelled.
A typed reply that matches a choice label ("Skip") now resolves to that
choice; the skip word applies only when no choice matches.
The bots room card sends the picked labels as they are; the tool already
strips the recommendation label. Unused OUTCOMES and a new docstring and
comment are removed.
* fix(tui): re-editing a multi-select answer keeps its picks
The Ink card stored a multi-select answer as display text ("A, B"), so
revisiting the question put the whole string into Other and re-locked it
as one item. The answers map now keeps the raw JSON array, the card
restores the picks on revisit, and only the display lines format it.
Comments that earlier edits reworded are restored to their original
words, minus the phrases the change made wrong. The compute-host clarify
lock accepts None for a skipped question.
* test(clarify): align three checks with the one-answer confirm and the regenerated keys
The desktop Cmd+Enter test now expects a send with one answer (the blank
question locks null), the compressor test expects the single-answer summary
the code gives, and locales/_keys.desktop.json is regenerated for the
removed and added clarify strings.
* test(tui): the one-question card header is singular
413 lines
15 KiB
Python
413 lines
15 KiB
Python
"""Tests for the gateway-side clarify primitive (tools/clarify_gateway.py).
|
|
|
|
The clarify tool needs to ask the user a question and block the agent
|
|
thread until they respond. These tests cover the module-level state
|
|
machine: register, wait, resolve via button, resolve via text-fallback,
|
|
"Other"-button text-capture flip, timeout, session boundary cleanup.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import threading
|
|
import time
|
|
from concurrent.futures import ThreadPoolExecutor
|
|
|
|
|
|
def _clear_clarify_state():
|
|
"""Reset module-level state between tests."""
|
|
from tools import clarify_gateway as cm
|
|
with cm._lock:
|
|
cm._entries.clear()
|
|
cm._session_index.clear()
|
|
cm._notify_cbs.clear()
|
|
|
|
|
|
class TestClarifyPrimitive:
|
|
"""Core register/wait/resolve mechanics."""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def test_button_choice_resolves_wait(self):
|
|
"""resolve_gateway_clarify unblocks wait_for_response with the chosen string."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("id1", "sk1", "Pick one", ["A", "B", "C"])
|
|
|
|
def resolver():
|
|
time.sleep(0.05)
|
|
cm.resolve_gateway_clarify("id1", "B")
|
|
|
|
threading.Thread(target=resolver).start()
|
|
result = cm.wait_for_response("id1", timeout=10.0)
|
|
assert result == "B"
|
|
|
|
def test_first_resolution_wins(self):
|
|
"""A late cancellation must not overwrite an already-selected choice."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register("id-race", "sk-race", "Pick one", ["A", "B"])
|
|
|
|
assert cm.resolve_gateway_clarify("id-race", "A") is True
|
|
assert cm.resolve_gateway_clarify("id-race", "") is False
|
|
assert entry.response == "A"
|
|
|
|
def test_open_ended_auto_awaits_text(self):
|
|
"""Clarify with no choices is in text-capture mode immediately."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register("id2", "sk2", "Free form?", None)
|
|
assert entry.awaiting_text is True
|
|
|
|
# get_pending_for_session returns the entry so the gateway
|
|
# text-intercept can find it.
|
|
pending = cm.get_pending_for_session("sk2")
|
|
assert pending is not None
|
|
assert pending.clarify_id == "id2"
|
|
|
|
|
|
def test_include_choice_prompts_returns_multi_choice_entry(self):
|
|
"""Gateway typed replies must see active choice prompts too."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("id3b", "sk3b", "Pick", ["X", "Y"])
|
|
pending = cm.get_pending_for_session("sk3b", include_choice_prompts=True)
|
|
assert pending is not None
|
|
assert pending.clarify_id == "id3b"
|
|
|
|
|
|
def test_clear_session_cancels_pending_entries(self):
|
|
"""clear_session unblocks blocked threads."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("id7", "sk7", "Q?", ["A"])
|
|
|
|
def waiter():
|
|
return cm.wait_for_response("id7", timeout=10.0)
|
|
|
|
with ThreadPoolExecutor(1) as pool:
|
|
fut = pool.submit(waiter)
|
|
time.sleep(0.05)
|
|
cancelled = cm.clear_session("sk7")
|
|
assert cancelled == 1
|
|
result = fut.result(timeout=10.0)
|
|
assert result == cm.CANCELLED
|
|
|
|
|
|
def test_clear_session_preserves_resolved_response(self):
|
|
"""clear_session must not clobber an answer that already won.
|
|
|
|
First-writer-wins (doryani-ai on PR #75732): a button callback that
|
|
resolved the entry before session cleanup must keep its response.
|
|
clear_session only cancels entries whose event is not yet set, so
|
|
the racing waiter observes the real answer, not the empty sentinel.
|
|
"""
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("id-race", "sk-race", "Pick one", ["A", "B"])
|
|
|
|
def waiter():
|
|
return cm.wait_for_response("id-race", timeout=10.0)
|
|
|
|
with ThreadPoolExecutor(1) as pool:
|
|
fut = pool.submit(waiter)
|
|
time.sleep(0.05)
|
|
# Button wins the race first...
|
|
assert cm.resolve_gateway_clarify("id-race", "B") is True
|
|
# ...then session cleanup runs before the waiter wakes.
|
|
cancelled = cm.clear_session("sk-race")
|
|
assert cancelled == 0
|
|
result = fut.result(timeout=10.0)
|
|
# The real answer must survive cleanup, not the "" cancellation.
|
|
assert result == "B"
|
|
|
|
|
|
def test_session_index_isolation(self):
|
|
"""Entries from different sessions don't leak across get_pending lookups."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("idA", "alpha", "Q?", None) # auto-await text
|
|
cm.register("idB", "beta", "Q?", None) # auto-await text
|
|
|
|
a = cm.get_pending_for_session("alpha")
|
|
b = cm.get_pending_for_session("beta")
|
|
assert a is not None and a.clarify_id == "idA"
|
|
assert b is not None and b.clarify_id == "idB"
|
|
|
|
|
|
|
|
class TestGatewayTextIntercept:
|
|
"""The gateway's _handle_message intercepts text replies to pending clarifies."""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def test_get_pending_for_session_returns_oldest_text_awaiting(self):
|
|
"""When two clarifies are pending, get_pending_for_session returns the
|
|
first that is awaiting_text (the older one if both)."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
# Older multi-choice (not awaiting text)
|
|
cm.register("first", "sk", "Q1?", ["A"])
|
|
# Newer open-ended (awaiting text)
|
|
cm.register("second", "sk", "Q2?", None)
|
|
|
|
pending = cm.get_pending_for_session("sk")
|
|
# The newer one is awaiting text; the older isn't.
|
|
assert pending is not None
|
|
assert pending.clarify_id == "second"
|
|
|
|
# Now flip the first to text mode too. Both are awaiting text,
|
|
# FIFO returns the older one.
|
|
cm.mark_awaiting_text("first")
|
|
pending2 = cm.get_pending_for_session("sk")
|
|
assert pending2 is not None
|
|
assert pending2.clarify_id == "first"
|
|
def test_text_fallback_enables_awaiting_text_for_multi_choice(self):
|
|
"""When base send_clarify renders choices as text, mark_awaiting_text
|
|
is called so the gateway text-intercept can capture the reply."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register("id-tf", "sk-tf", "Pick one", ["A", "B", "C"])
|
|
# Initially, multi-choice does NOT await text (button path)
|
|
assert entry.awaiting_text is False
|
|
|
|
# After the base send_clarify text fallback calls mark_awaiting_text:
|
|
flipped = cm.mark_awaiting_text("id-tf")
|
|
assert flipped is True
|
|
|
|
# Now get_pending_for_session should find it
|
|
pending = cm.get_pending_for_session("sk-tf")
|
|
assert pending is not None
|
|
assert pending.clarify_id == "id-tf"
|
|
|
|
# Clean up
|
|
cm.clear_session("sk-tf")
|
|
|
|
|
|
class TestCoverageGaps:
|
|
"""Cover remaining branches: unknown-id wait, timeout config exception."""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def test_wait_for_response_unknown_id_returns_none(self):
|
|
"""wait_for_response on a non-existent id returns None immediately."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
assert cm.wait_for_response("nonexistent-id", timeout=0.1) is None
|
|
|
|
|
|
def test_get_clarify_timeout_exception_returns_default(self, monkeypatch):
|
|
"""get_clarify_timeout returns 3600 when load_config raises."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
monkeypatch.setattr("hermes_cli.config.load_config",
|
|
lambda: (_ for _ in ()).throw(RuntimeError("boom")))
|
|
assert cm.get_clarify_timeout() == 3600
|
|
|
|
|
|
class TestClarifyTimeoutResolution:
|
|
"""resolve_clarify_timeout is the single source of truth for the clarify
|
|
timeout, shared by the CLI, TUI/desktop, and messaging-gateway paths."""
|
|
|
|
|
|
|
|
def test_cli_defaults_do_not_shadow_agent_clarify_timeout(self):
|
|
"""The classic CLI builds its config from ``cli._cli_config_defaults()``; a seeded
|
|
legacy ``clarify.timeout`` there wins over the user's ``agent.clarify_timeout`` in
|
|
the resolver, capping every CLI modal at that value (120 s) while the docs promise
|
|
3600 s / unlimited. The CLI defaults must leave the legacy key unset."""
|
|
import cli
|
|
from tools import clarify_gateway as cm
|
|
|
|
defaults = cli._cli_config_defaults()
|
|
assert (defaults.get("clarify") or {}).get("timeout") is None
|
|
assert cm.resolve_clarify_timeout(defaults) == 3600
|
|
cli._merge_file_config(defaults, {"agent": {"clarify_timeout": 900}})
|
|
assert cm.resolve_clarify_timeout(defaults) == 900
|
|
|
|
|
|
def test_non_positive_preserved_as_unlimited_sentinel(self):
|
|
"""<= 0 is passed through verbatim — the waiting loops read it as
|
|
'unlimited', so the resolver must not clamp it to a positive default."""
|
|
from tools import clarify_gateway as cm
|
|
|
|
assert cm.resolve_clarify_timeout({"agent": {"clarify_timeout": 0}}) == 0
|
|
assert cm.resolve_clarify_timeout({"clarify": {"timeout": -1}}) == -1
|
|
|
|
|
|
class TestUnlimitedWait:
|
|
"""timeout <= 0 makes wait_for_response block until the answer arrives
|
|
instead of auto-skipping."""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def test_zero_timeout_waits_until_resolved(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
cm.register("u1", "sk", "Q?", ["A", "B"])
|
|
result_box = {}
|
|
|
|
def waiter():
|
|
result_box["r"] = cm.wait_for_response("u1", timeout=0)
|
|
|
|
t = threading.Thread(target=waiter)
|
|
t.start()
|
|
# An unlimited wait cannot finish while nothing resolves it: still
|
|
# running after a comfortable margin (old code auto-skipped at once).
|
|
t.join(timeout=0.3)
|
|
assert t.is_alive()
|
|
|
|
# Once resolved, the unlimited wait returns the real answer.
|
|
cm.resolve_gateway_clarify("u1", "B")
|
|
t.join(timeout=5.0)
|
|
assert not t.is_alive()
|
|
assert result_box["r"] == "B"
|
|
|
|
|
|
class TestMultiSelectTextFallback:
|
|
"""Multi-select clarifies via the gateway text fallback.
|
|
|
|
The adapter's numbered-list fallback asks the user to reply with
|
|
comma/space-separated numbers; _coerce_text_response must map those to a
|
|
JSON array of choice labels (which _parse_multi_select_response on the
|
|
tool side decodes into a list).
|
|
"""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def _register_multi(self, cid="m1", choices=("A", "B", "C")):
|
|
from tools import clarify_gateway as cm
|
|
entry = cm.register(cid, "sk", "Pick some", list(choices), multi_select=True)
|
|
# Text fallback path always flips awaiting_text on.
|
|
cm.mark_awaiting_text(cid)
|
|
return entry
|
|
|
|
|
|
|
|
def test_multi_select_without_choices_is_ignored(self):
|
|
"""multi_select on an open-ended clarify is meaningless — dropped."""
|
|
from tools import clarify_gateway as cm
|
|
entry = cm.register("s2", "sk", "Q?", None, multi_select=True)
|
|
assert entry.multi_select is False
|
|
|
|
|
|
def test_duplicate_selections_deduped(self):
|
|
import json
|
|
from tools import clarify_gateway as cm
|
|
entry = self._register_multi()
|
|
coerced = cm._coerce_text_response(entry, "1, 1, 2")
|
|
assert json.loads(coerced) == ["A", "B"]
|
|
|
|
def test_resolve_text_response_end_to_end(self):
|
|
"""resolve_text_response_for_session delivers the JSON array to the waiter."""
|
|
import json
|
|
from tools import clarify_gateway as cm
|
|
self._register_multi(cid="m3")
|
|
result_box = {}
|
|
|
|
def waiter():
|
|
result_box["r"] = cm.wait_for_response("m3", timeout=5)
|
|
|
|
t = threading.Thread(target=waiter)
|
|
t.start()
|
|
time.sleep(0.05)
|
|
assert cm.resolve_text_response_for_session("sk", "1,2") is True
|
|
t.join(timeout=5)
|
|
assert json.loads(result_box["r"]) == ["A", "B"]
|
|
|
|
def test_single_select_regression_numeric(self):
|
|
"""Single-select coercion unchanged: '2' maps to the choice label string."""
|
|
from tools import clarify_gateway as cm
|
|
entry = cm.register("s3", "sk", "Q?", ["A", "B", "C"])
|
|
assert cm._coerce_text_response(entry, "2") == "B"
|
|
|
|
def test_single_select_regression_label(self):
|
|
from tools import clarify_gateway as cm
|
|
entry = cm.register("s4", "sk", "Q?", ["A", "B"])
|
|
assert cm._coerce_text_response(entry, "b") == "B"
|
|
|
|
|
|
class TestNativeRejectClassification:
|
|
"""Rejected typed replies must distinguish free prose from bad selections.
|
|
|
|
Free prose cancels/falls through (deadlock break). Selection-shaped but
|
|
invalid replies (out-of-range number, unrecognised comma-list) keep the
|
|
pending clarify armed so the user can retry.
|
|
"""
|
|
|
|
def setup_method(self):
|
|
_clear_clarify_state()
|
|
|
|
def test_multi_select_out_of_range_is_invalid_selection(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register(
|
|
"ms-oor", "sk-ms", "Pick some", ["A", "B", "C"], multi_select=True,
|
|
)
|
|
assert entry.awaiting_text is False
|
|
value, reason = cm._coerce_text_response_detailed(entry, "99")
|
|
assert value is None
|
|
assert reason == "invalid_selection"
|
|
assert cm.attempt_text_response_for_session("sk-ms", "99") == (
|
|
cm.TEXT_REJECTED_SELECTION
|
|
)
|
|
pending = cm.get_pending_for_session("sk-ms", include_choice_prompts=True)
|
|
assert pending is not None
|
|
assert not pending.event.is_set()
|
|
|
|
def test_multi_select_bad_comma_list_is_invalid_selection(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register(
|
|
"ms-bad", "sk-ms2", "Pick some", ["A", "B", "C"], multi_select=True,
|
|
)
|
|
value, reason = cm._coerce_text_response_detailed(entry, "1,99")
|
|
assert value is None
|
|
assert reason == "invalid_selection"
|
|
assert cm.attempt_text_response_for_session("sk-ms2", "nope,nope") == (
|
|
cm.TEXT_REJECTED_SELECTION
|
|
)
|
|
pending = cm.get_pending_for_session("sk-ms2", include_choice_prompts=True)
|
|
assert pending is not None
|
|
assert not pending.event.is_set()
|
|
|
|
def test_multi_select_free_prose_is_rejected_prose(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register(
|
|
"ms-prose", "sk-ms3", "Pick some", ["A", "B"], multi_select=True,
|
|
)
|
|
value, reason = cm._coerce_text_response_detailed(
|
|
entry, "just checking the visual UI, no need to pass any data",
|
|
)
|
|
assert value is None
|
|
assert reason == "prose"
|
|
assert cm.attempt_text_response_for_session(
|
|
"sk-ms3", "just checking the visual UI, no need to pass any data",
|
|
) == cm.TEXT_REJECTED_PROSE
|
|
|
|
def test_single_select_out_of_range_is_invalid_selection(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register("ss-oor", "sk-ss", "Pick one", ["A", "B"])
|
|
value, reason = cm._coerce_text_response_detailed(entry, "9")
|
|
assert value is None
|
|
assert reason == "invalid_selection"
|
|
assert cm.attempt_text_response_for_session("sk-ss", "9") == (
|
|
cm.TEXT_REJECTED_SELECTION
|
|
)
|
|
|
|
def test_single_select_prose_is_rejected_prose(self):
|
|
from tools import clarify_gateway as cm
|
|
|
|
entry = cm.register("ss-prose", "sk-ss2", "Pick one", ["A", "B"])
|
|
value, reason = cm._coerce_text_response_detailed(
|
|
entry, "one more unrelated thought",
|
|
)
|
|
assert value is None
|
|
assert reason == "prose"
|