Files
hermes-agent/tests/tui_gateway/test_warning_callback_conservation.py
Siddharth Balyan 5eea87882a Clarify: one question shape and one result shape on every surface (skip, cancel, timeout and undelivered are told apart) (#127760)
* refactor(desktop): split clarify-tool.tsx into a clarify/ folder

Pure moves, no behaviour change. Parsing, the question-card core
(shell, choice rows, question block), the delivery watchdog, and each
pending/settled card get their own file so the core can be reused.

* feat(clarify): one questions[] shape with per-question status and one outcome

The clarify tool now takes only questions=[{question, choices?, multi_select?}].
A wrong shape is a tool error that names the right one, so a model that
sends the old top-level question/choices corrects itself on the next call.

Every result has the same shape on every surface: each response carries
status (answered, skipped, unanswered) with user_response null unless
answered, and the result carries one outcome (submitted, cancelled,
timed_out, undelivered) plus an optional surface-written notice. The
timeout and cancel sentences that used to pose as the user's answer are
gone, and the compressor reads status instead of matching their prefixes.

The callback contract is callback(questions) -> {answers, outcome, notice?}
(None = skipped, missing qid = unanswered). tui_gateway settles a batch the
same way for the last lock, a cancel, an interrupt and the deadline, and
clarify.lock keeps a null answer as a skip. A multi-select answer that is
not a JSON array counts as one typed ("Other") answer. The tool-row preview
reads the first question.

* feat(clarify): every surface asks through the one question card

CLI: the batch panel is the only panel; Enter on an empty field skips a
question, Ctrl+C cancels and keeps the locked answers, the deadline returns
timed_out. -q and -z return undelivered with a no-user notice.

Ink TUI: the single-question prompt is gone; the card supports multi-select
(Space or a digit toggles, Enter locks, typed Other text joins the picks),
an empty submit skips, Esc cancels, and "Batch" names are dropped.

Desktop: the single-question card is deleted and its keyboard handling moved
into the one card (arrows, letters and digits, auto-advance, Enter picks then
confirms). Confirm enables at one answer; blank questions lock null; Skip and
a composer message cancel; the settled card shows No answer for unanswered
questions. The bots room card follows the same contract.

Messaging: one card per question with a "Reply skip to skip" line in every
locale; timeouts and delivery failures map to timed_out and undelivered.

* fix(clarify): cancelled for a released messaging card, labels win over the skip word

A prose reply to a messaging choice card, /new and session end released
the wait with "", which read as timed_out. They now resolve with a
CANCELLED marker and the tool gets outcome cancelled.

A typed reply that matches a choice label ("Skip") now resolves to that
choice; the skip word applies only when no choice matches.

The bots room card sends the picked labels as they are; the tool already
strips the recommendation label. Unused OUTCOMES and a new docstring and
comment are removed.

* fix(tui): re-editing a multi-select answer keeps its picks

The Ink card stored a multi-select answer as display text ("A, B"), so
revisiting the question put the whole string into Other and re-locked it
as one item. The answers map now keeps the raw JSON array, the card
restores the picks on revisit, and only the display lines format it.

Comments that earlier edits reworded are restored to their original
words, minus the phrases the change made wrong. The compute-host clarify
lock accepts None for a skipped question.

* test(clarify): align three checks with the one-answer confirm and the regenerated keys

The desktop Cmd+Enter test now expects a send with one answer (the blank
question locks null), the compressor test expects the single-answer summary
the code gives, and locales/_keys.desktop.json is regenerated for the
removed and added clarify strings.

* test(tui): the one-question card header is singular
2026-09-29 18:33:44 +05:30

155 lines
8.4 KiB
Python

"""Real callback and queue boundaries; renderer preference never hides operator logs."""
import logging
import queue
from types import SimpleNamespace
import pytest
import hermes_yaml as yaml
from agent.status_output import StatusOutputMixin
from tui_gateway import server
from tests.tui_gateway.test_auto_continue import turn_env, marker_home, _session
CLARIFY_QUESTIONS = [{"qid": "q0", "question": "Continue recovery?", "choices": ["yes", "no"], "multi_select": False}]
class Agent(StatusOutputMixin):
log_prefix = ""
suppress_status_output = True
platform = "tui"
_print_fn = None
def _touch_activity(self, *args):
pass
@pytest.mark.parametrize("setting", [None, False, True])
def test_real_tui_callbacks_filter_only_diagnostics_at_owner_sink(tmp_path, monkeypatch, setting):
launch, owner = tmp_path / "launch", tmp_path / "owner"
launch.mkdir()
owner.mkdir()
monkeypatch.setenv("HERMES_HOME", str(launch))
(launch / "config.yaml").write_text(yaml.safe_dump({"display": {"suppress_warning_notifications": setting is not True}}))
(owner / "config.yaml").write_text(yaml.safe_dump({} if setting is None else {"display": {"suppress_warning_notifications": setting}}))
frames = []
monkeypatch.setattr(server, "write_json", lambda frame: frames.append(frame) or True)
for home, muted in [(owner, setting is True), (launch, setting is not True), (owner, setting is True)]:
session = _session(profile_home=str(home))
monkeypatch.setitem(server._sessions, "warning-callback", session)
agent = Agent()
for key, callback in server._agent_cbs("warning-callback").items():
setattr(agent, key, callback)
# Intentionally invoke outside the owning context, like a callback worker.
frames.clear()
agent._emit_warning("unfamiliar warning wording")
agent._emit_diagnostic_status("unfamiliar fallback wording")
agent._emit_diagnostic_wait("unfamiliar retry wording")
agent._emit_notice(SimpleNamespace(level="error", text="notice detail", kind="custom", ttl_ms=0, key="custom", id="n"))
diagnostic_frames = list(frames)
agent._emit_status("⚠ user-requested progress with warning punctuation")
assert bool(diagnostic_frames) is not muted
assert len(diagnostic_frames) == (0 if muted else 4)
assert frames[-1]["params"]["payload"]["text"] == "⚠ user-requested progress with warning punctuation"
@pytest.mark.parametrize("setting", [None, False, True])
def test_post_turn_drain_uses_owner_and_preserves_real_clarify(turn_env, marker_home, monkeypatch, setting, caplog):
from tools.process_registry import process_registry
from tui_gateway import server_requests
owner = marker_home / "owner"
owner.mkdir()
monkeypatch.setenv("HERMES_HOME", str(marker_home))
(marker_home / "config.yaml").write_text(yaml.safe_dump({"display": {"suppress_warning_notifications": setting is not True}}))
(owner / "config.yaml").write_text(yaml.safe_dump({} if setting is None else {"display": {"suppress_warning_notifications": setting}}))
frames, work, controls = [], [], []
def wire(frame):
frames.append(frame)
if frame.get("method") == "clarify":
assert server_requests.resolve_response({"id": frame["id"], "result": {"answers": {"q0": "yes"}}})
return True
monkeypatch.setattr(server, "write_json", wire)
monkeypatch.setattr(server_requests, "_write", wire)
monkeypatch.setattr(server, "_start_usage_ticker", lambda *a: (server.threading.Event(), SimpleNamespace(join=lambda: None)))
monkeypatch.setattr(process_registry, "completion_queue", queue.Queue())
agent = SimpleNamespace(session_id="session-key", clear_interrupt=lambda: None)
agent.clarify_callback = server._agent_cbs("drain-owner")["clarify_callback"]
def run(message, **kwargs):
work.append(message)
controls.append(agent.clarify_callback(CLARIFY_QUESTIONS))
logging.getLogger("operator").warning("watch diagnostic retained")
return {"final_response": "diagnostic final", "messages": []}
agent.run_conversation = run
session = _session(agent=agent, profile_home=str(owner))
monkeypatch.setitem(server._sessions, "drain-owner", session)
process_registry.completion_queue.put({"type": "watch_disabled", "session_id": "proc", "session_key": "session-key",
"origin_ui_session_id": "drain-owner", "message": "watch diagnostic"})
with caplog.at_level(logging.WARNING, logger="operator"):
server._run_post_turn_followups("request", "drain-owner", session, {}, None)
assert len(work) == 1
assert [c["answers"] for c in controls] == [{"q0": "yes"}]
assert process_registry.completion_queue.empty()
assert "watch diagnostic retained" in caplog.text
presentation = [f for f in frames if f.get("params", {}).get("type") in {"status.update", "message.complete", "message.start", "error"}]
assert bool(presentation) is not (setting is True)
assert len([f for f in frames if f.get("method") == "clarify"]) == 1
@pytest.mark.parametrize("setting", [None, False, True])
def test_diagnostic_turn_callbacks_from_real_worker_keep_controls_and_logs(turn_env, marker_home, monkeypatch, setting, caplog):
from tui_gateway import server_requests
monkeypatch.setenv("HERMES_HOME", str(marker_home))
(marker_home / "config.yaml").write_text(yaml.safe_dump(
{} if setting is None else {"display": {"suppress_warning_notifications": setting}}))
frames, controls, failures = [], [], []
def wire(frame):
frames.append(frame)
if frame.get("method") == "clarify":
assert server_requests.resolve_response({"id": frame["id"], "result": {"answers": {"q0": "yes"}}})
return True
monkeypatch.setattr(server, "write_json", wire)
monkeypatch.setattr(server_requests, "_write", wire)
monkeypatch.setattr(server, "_start_usage_ticker", lambda *a: (server.threading.Event(), SimpleNamespace(join=lambda: None)))
# The normal worker starts after the submitter releases _sessions_lock.
# The shared inline-thread fixture instead holds that lock while joining
# this real callback thread, creating a fixture-only deadlock.
monkeypatch.setattr(server.threading, "Thread", server._RealThread)
agent = Agent()
agent.session_id = "session-key"
agent.clear_interrupt = lambda: None
session = _session(agent=agent, profile_home=str(marker_home), running=True)
monkeypatch.setitem(server._sessions, "thread-owner", session)
for key, callback in server._agent_cbs("thread-owner").items():
setattr(agent, key, callback)
def run(message, **kwargs):
def worker():
try:
# Concrete callbacks can be retained by provider/helper workers.
agent._emit_status("plain wake progress")
agent._emit_wait_notice("plain wake thinking")
agent._emit_notice(SimpleNamespace(level="info", text="wake notice", kind="custom", ttl_ms=0, key="custom", id="n"))
kwargs["stream_callback"]("wake stream")
controls.append(agent.clarify_callback(CLARIFY_QUESTIONS))
logging.getLogger("operator").warning("worker diagnostic retained")
except BaseException as exc:
failures.append(exc)
thread = server._RealThread(target=worker)
thread.start()
thread.join(timeout=5)
assert not thread.is_alive()
return {"final_response": "wake result", "messages": []}
agent.run_conversation = run
with caplog.at_level(logging.WARNING, logger="operator"):
server._run_prompt_submit("request", "thread-owner", session, "engine failure",
display_metadata={"notification_category": "diagnostic"})
session["_run_thread"].join(timeout=10)
assert not session["_run_thread"].is_alive()
assert not failures
assert [c["answers"] for c in controls] == [{"q0": "yes"}]
assert "worker diagnostic retained" in caplog.text
presentation = [f for f in frames if f.get("params", {}).get("type") in
{"status.update", "thinking.delta", "notification.show", "message.delta", "message.complete"}]
assert len(presentation) == (0 if setting is True else 5)
# Once the turn completes, the same sink must present the next requested progress.
agent._emit_status("next requested progress")
assert frames[-1]["params"]["payload"]["text"] == "next requested progress"