feat: the setup handoff carries what setup learned, and the task chat talks first (#129419)
start_chat from the setup profile now appends a short "What setup learned about me" block under the model's handoff message: the name, app, plugin and layout picks (recorded by setup_choose into the conversation's card facts) and the interpreted scan lines (machine, apps in use, runs agents, level) that the /initiate-setup turn records beside the fork rows. The task chat gets them even when the model drops a part, and never sees raw scan values. The handoff plan now asks the task chat to talk first: acknowledge what setup learned, ask one plain-text question with two to four concrete directions, offer to connect the apps they need, and build only after a pick or "go". The forced "one HTML page" line is gone; the no-mock, no-routing-around-connectors and ask-before-acting rules stay.
This commit is contained in:
Siddharth Balyan
GitHub
@@ -97,7 +97,7 @@ def build_initiate_setup_prompt(surface: str, tools, primary_profile: str, sessi
|
||||
return f"{HEADER}\n\n{skill}\n\n```json\n{facts}\n```"
|
||||
|
||||
|
||||
_NAME_QUESTION = "What should I call you?"
|
||||
NAME_QUESTION = "What should I call you?"
|
||||
# A closed card counts as answered: the skill takes the default and never re-asks it.
|
||||
_ANSWERED = ("submitted", "cancelled")
|
||||
|
||||
@@ -118,7 +118,7 @@ def _opening_step(call) -> str | None:
|
||||
args = _json_object(function.get("arguments"))
|
||||
if args.get("kind") == "accent":
|
||||
return "accent"
|
||||
return "name" if args.get("kind") == "question" and args.get("question") == _NAME_QUESTION else ""
|
||||
return "name" if args.get("kind") == "question" and args.get("question") == NAME_QUESTION else ""
|
||||
|
||||
|
||||
def _opening_so_far(history):
|
||||
@@ -172,7 +172,7 @@ def _opening(suggested: str | None, said: set, replies: dict):
|
||||
reply = replies.get("name")
|
||||
if reply is None:
|
||||
# ``options`` is required by the tool schema; the history must carry it even when empty.
|
||||
card = {"kind": "question", "question": _NAME_QUESTION,
|
||||
card = {"kind": "question", "question": NAME_QUESTION,
|
||||
"options": [{"id": "suggested", "label": suggested}] if suggested else [], "multi_select": False}
|
||||
reply = _json_object((yield "" if INTRO in said else INTRO, "setup_choose", card))
|
||||
if "accent" in replies:
|
||||
|
||||
@@ -192,7 +192,7 @@ One short sentence: the work gets its own chat so it has room, and this one stay
|
||||
|
||||
- `profile`: `primary_profile`. Without it the task opens here, with no terminal or file tools.
|
||||
- `title`: the task's name.
|
||||
- `message`: the handoff message, written exactly as the fork result's `handoff.message` says, with `handoff.plan` as its plan.
|
||||
- `message`: the handoff message, written exactly as the fork result's `handoff.message` says, with `handoff.plan` as its plan. The app appends a "What setup learned" block (their picks and the scan) under it; do not write that part. The task chat talks first: it offers a few directions in plain text and builds once they pick.
|
||||
|
||||
### Beat 8: after the handoff
|
||||
|
||||
@@ -220,7 +220,7 @@ When `surface` is not `desktop` or a tool is not in `tools_present`:
|
||||
## Pitfalls
|
||||
|
||||
- Repeating the card's question in text.
|
||||
- A handoff message missing a part: the task chat sees nothing else.
|
||||
- A handoff message missing a part: the task chat sees only it and the app's "What setup learned" block.
|
||||
|
||||
## Verification
|
||||
|
||||
|
||||
@@ -798,11 +798,34 @@ def _handoff(facts: dict) -> dict:
|
||||
"machine": '"' + " ".join(part for part in parts if part) + '"'}
|
||||
|
||||
|
||||
_LEVELS = {"beginner": "new to AI agent apps", "power-user": "a power user", "expert": "an expert"}
|
||||
|
||||
|
||||
def _learned(facts: dict) -> list[str]:
|
||||
"""The scan, in plain first-person lines, for the block ``start_chat`` appends to the task chat's first
|
||||
message. Only interpreted fields: no names, paths or raw values from the scan."""
|
||||
scan, machine = facts.get("scan") or {}, facts["machine"]
|
||||
ram = f", {machine['ram_gb']} GB RAM" if machine.get("ram_gb") else ""
|
||||
lines = [f"This {facts['signals']['machine_kind']}: {facts['signals']['description']}{ram}."]
|
||||
if scan.get("apps_used"):
|
||||
lines.append(f"Apps I use on it: {', '.join(scan['apps_used'])}.")
|
||||
if scan.get("runs_agents") is True:
|
||||
lines.append("I already run AI agents here.")
|
||||
level = _LEVELS.get(scan.get("user_level"), "")
|
||||
if scan.get("developer") is True:
|
||||
level = f"{level}, a developer" if level else "a developer"
|
||||
if level:
|
||||
lines.append(f"I am {level}.")
|
||||
return lines
|
||||
|
||||
|
||||
def setup_cards(facts: dict) -> dict:
|
||||
"""What the setup cards take from these facts: the fork rows, the rows the apps and plugins cards start
|
||||
with picked (apps seen in use; Blender when it is here), and the handoff text the fork result carries."""
|
||||
with picked (apps seen in use; Blender when it is here), the handoff text the fork result carries, and
|
||||
the scan lines ``start_chat`` appends to the task chat's first message."""
|
||||
used = (facts.get("scan") or {}).get("apps_used") or []
|
||||
return {
|
||||
"learned": _learned(facts),
|
||||
"fork": facts["fork"],
|
||||
"preselected": {
|
||||
"connectors": [_SCAN_CONNECTORS[app] for app in used if app in _SCAN_CONNECTORS],
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## message
|
||||
|
||||
The handoff message is the new chat's first user message. It is visible, so write it as their own ask, in their language, in the first person. Nothing else reaches the task chat: no memory, no hidden note.
|
||||
The handoff message is the new chat's first user message. It is visible, so write it as their own ask, in their language, in the first person. The app adds one block under it, "What setup learned about me": their name, app, plugin and layout picks, and what the scan saw. Do not list those picks or scan facts again. Nothing else reaches the task chat: no memory, no hidden note.
|
||||
|
||||
The install list: the plugins they picked that the task needs (all of them for machine setup or a task naming the app), plus a picked first task's own `plugins`. The connect list: apps the task needs that setup did not connect; none for machine setup.
|
||||
|
||||
@@ -12,14 +12,14 @@ Write these parts in order, one paragraph each (part names are not text). Copy t
|
||||
|
||||
1. The ask, in one or two sentences, in their words where they gave them, keeping the named app and the outcome. No machine specs.
|
||||
2. "Call me <name>." Then what they are working on, if they said it.
|
||||
3. "I use: <all app picks, exact ids>. Connected during setup: <apps setup connected>."
|
||||
4. Only with an install or connect list: "In your first turn, before anything else: install <install list> with one `manage_catalog` install call carrying all of them, then connect <connect list> with one `manage_connections` connect call carrying all of them. The ids are exact; skip search and status checks, and start the work when they return. If I skip some, go on without them and tell me in one line what each would have added; do not offer them again." Drop the install or the connect clause when its list is empty. With an install list, add: "Find plugin tools with `tool_search` and read a plugin's skill with `skill_view` by its exact name; if its app is not running, tell me plainly." With other plugin picks, add: "Picked during setup, not needed yet: <other plugin picks>."
|
||||
3. "Connected during setup: <apps setup connected>."
|
||||
4. Only with an install or connect list: "Once I pick a direction or say go, before building: install <install list> with one `manage_catalog` install call carrying all of them, then connect <connect list> with one `manage_connections` connect call carrying all of them. The ids are exact; skip search and status checks, and start the work when they return. If I skip some, go on without them and tell me in one line what each would have added; do not offer them again." Drop the install or the connect clause when its list is empty. With an install list, add: "Find plugin tools with `tool_search` and read a plugin's skill with `skill_view` by its exact name; if its app is not running, tell me plainly." With other plugin picks, add: "Picked during setup, not needed yet: <other plugin picks>."
|
||||
5. The plan below, word for word.
|
||||
6. Last (drop the first sentence when `scan.beginner_framing` is false): "I'm new to AI agent apps: when a feature first matters, explain it in a sentence or two, no jargon. As you start, tell me in one short sentence that you'll ask for permissions as you go and I can say no or redirect you. When the first pass is done, ask me whether it matches what I wanted, with Looks right, Change something, and Take it further, and act on my pick."
|
||||
6. Last (drop the first sentence when `scan.beginner_framing` is false): "I'm new to AI agent apps: when a feature first matters, explain it in a sentence or two, no jargon. When you start building, tell me in one short sentence that you'll ask for permissions as you go and I can say no or redirect you. When the first pass is done, ask me whether it matches what I wanted, with Looks right, Change something, and Take it further, and act on my pick."
|
||||
|
||||
## build
|
||||
|
||||
"Plan briefly, then build: scaffold, research, first artifact. Use real data only from connected apps and find their tools with `tool_search`; tools already signed in on this computer, like a logged-in gh, are fair to use, and say so in one line. Never route around a connector: no IMAP client, app password or other way into the same account; if I decline one, that app is out of this build. Ask before sending, deleting or scheduling anything, and set up no recurring job unless I asked. Make the result something I can open: one HTML page if the idea allows it." With a connect list, add: "Include at least one real reading or action through a connected app." Without one, add: "Make this first version finishable with no account I have not connected: web research with the browser visible to me, scripts, a small app, a file-based tracker, a generated page. If the idea needs an account, build the no-account core first and offer the connection next."
|
||||
"Talk with me before you build. In your first reply, say in two or three plain lines what you take from what setup learned, then ask me in plain text, not a card, one short question with two to four concrete directions drawn from my apps and this computer, like: Should I start with A, B or C? Or tell me more about what you want. Name the apps each direction needs and offer to connect them. Build only after I pick one or say go; then plan briefly and build. Use real data only from connected apps and find their tools with `tool_search`; tools already signed in on this computer, like a logged-in gh, are fair to use, and say so in one line. Never route around a connector: no IMAP client, app password or other way into the same account; if I decline one, that app is out of this build. Ask before sending, deleting or scheduling anything, and set up no recurring job unless I asked. Never use mock or sample data." With a connect list, add: "Include at least one real reading or action through a connected app." Without one, add: "Make this first version finishable with no account I have not connected: web research with the browser visible to me, scripts, a small app, a file-based tracker, a generated page. If the idea needs an account, build the no-account core first and offer the connection next."
|
||||
|
||||
## machine
|
||||
|
||||
|
||||
@@ -147,6 +147,19 @@ def _follow_up(kind: str, card: str, result: dict, rows: Optional[list], cards:
|
||||
return extra, state
|
||||
|
||||
|
||||
# The picks start_chat hands the task chat from the setup profile, keyed by card kind.
|
||||
_REMEMBERED = frozenset({"connectors", "plugins", "layout"})
|
||||
|
||||
|
||||
def _remember(kind: str, question: str, result: dict, state: dict) -> dict:
|
||||
from agent.initiate_setup_prompt import NAME_QUESTION
|
||||
|
||||
key = "name" if kind == "question" and question == NAME_QUESTION else kind if kind in _REMEMBERED else None
|
||||
if key is None or result["outcome"] != "submitted":
|
||||
return state
|
||||
return {**state, "picks": {**state.get("picks", {}), key: result.get("label") or result["picked"]}}
|
||||
|
||||
|
||||
# App-owned parts of a card, filled here from the recorded facts so the model can neither drop nor edit them.
|
||||
_APP_FILLED: dict[str, Callable[[dict], dict]] = {
|
||||
"tour": lambda cards: {"options": _TOUR_ROWS, "multi_select": False},
|
||||
@@ -223,6 +236,7 @@ def setup_choose_tool(kind: str = "", question: str = "", options=None, multi_se
|
||||
reply = callback(payload)
|
||||
result = _result(reply, payload["options"])
|
||||
extra, state = _follow_up(kind, card, result, payload["options"], cards)
|
||||
state = _remember(kind, text, result, state)
|
||||
if session_id and state != cards:
|
||||
record_cards(session_id, state)
|
||||
return json.dumps({**result, **extra}, ensure_ascii=False)
|
||||
|
||||
@@ -9,11 +9,29 @@ def _rejected(reason: str, retryable: bool = False) -> str:
|
||||
return json.dumps({"status": "rejected", "reason": reason, "retryable": retryable})
|
||||
|
||||
|
||||
# The setup picks the handoff block names, in its order.
|
||||
_PICK_LINES = (("name", "Name"), ("connectors", "Apps I picked"), ("plugins", "Plugins I picked"), ("layout", "Layout"))
|
||||
|
||||
|
||||
def _setup_learned(cards: dict) -> str:
|
||||
"""The "What setup learned" block for a handoff from the setup profile: the user's card picks and the
|
||||
interpreted scan lines its ``/initiate-setup`` turn recorded, so the task chat has them whatever the model
|
||||
wrote. Empty when nothing was recorded."""
|
||||
picks = cards.get("picks") or {}
|
||||
lines = [f"- {label}: {', '.join(map(str, value)) if isinstance(value, list) else value}"
|
||||
for key, label in _PICK_LINES if (value := picks.get(key))]
|
||||
lines += [f"- {line}" for line in cards.get("learned") or ()]
|
||||
if not lines:
|
||||
return ""
|
||||
return "\n".join(["What setup learned about me:", *lines,
|
||||
"Start from this; look at my computer again only when the task needs more."])
|
||||
|
||||
|
||||
def start_chat(args: dict, caller_id: str | None = None) -> str:
|
||||
from agent.onboarding import PROFILE_BUILD_FLAG, mark_seen
|
||||
from gateway.session_context import get_session_env
|
||||
from hermes_cli.profiles import SETUP_PROFILE_MARKER
|
||||
from hermes_cli.setup_profile import mark_completed
|
||||
from hermes_cli.setup_profile import mark_completed, read_cards
|
||||
from hermes_constants import profile_name_for_home
|
||||
from tui_gateway import server
|
||||
from tui_gateway.transport import bind_transport, reset_transport
|
||||
@@ -34,6 +52,11 @@ def start_chat(args: dict, caller_id: str | None = None) -> str:
|
||||
except server.ProfileUnavailableError as exc:
|
||||
return _rejected(str(exc))
|
||||
target_home = Path(home or server._hermes_home)
|
||||
from_setup = (caller_home / SETUP_PROFILE_MARKER).is_file()
|
||||
if from_setup:
|
||||
with server._session_profile_runtime_scope({"profile_home": str(caller_home)}, hydrate_secrets=False):
|
||||
learned = _setup_learned(read_cards(caller.get("session_key") or ""))
|
||||
message = f"{message}\n\n{learned}" if learned else message
|
||||
token = bind_transport(caller.get("transport"))
|
||||
try:
|
||||
with server._session_profile_runtime_scope({"profile_home": str(home) if home else None}):
|
||||
@@ -49,7 +72,7 @@ def start_chat(args: dict, caller_id: str | None = None) -> str:
|
||||
return _rejected(submitted["error"]["message"], retryable=True)
|
||||
finally:
|
||||
reset_transport(token)
|
||||
if (caller_home / SETUP_PROFILE_MARKER).is_file() and target_home.resolve() != caller_home.resolve():
|
||||
if from_setup and target_home.resolve() != caller_home.resolve():
|
||||
mark_completed()
|
||||
name = result["info"]["profile_name"]
|
||||
return json.dumps({
|
||||
|
||||
@@ -209,7 +209,7 @@ One short sentence: the work gets its own chat so it has room, and this one stay
|
||||
|
||||
- `profile`: `primary_profile`. Without it the task opens here, with no terminal or file tools.
|
||||
- `title`: the task's name.
|
||||
- `message`: the handoff message, written exactly as the fork result's `handoff.message` says, with `handoff.plan` as its plan.
|
||||
- `message`: the handoff message, written exactly as the fork result's `handoff.message` says, with `handoff.plan` as its plan. The app appends a "What setup learned" block (their picks and the scan) under it; do not write that part. The task chat talks first: it offers a few directions in plain text and builds once they pick.
|
||||
|
||||
### Beat 8: after the handoff
|
||||
|
||||
@@ -237,7 +237,7 @@ When `surface` is not `desktop` or a tool is not in `tools_present`:
|
||||
## Pitfalls
|
||||
|
||||
- Repeating the card's question in text.
|
||||
- A handoff message missing a part: the task chat sees nothing else.
|
||||
- A handoff message missing a part: the task chat sees only it and the app's "What setup learned" block.
|
||||
|
||||
## Verification
|
||||
|
||||
|
||||
Reference in New Issue
Block a user