hermes-admin/hermes-agent
1
0
Fork 0
Code Issues Pull Requests Actions 29 Packages Projects Releases Wiki Activity

chore: remove old plan files

Browse Source
This commit is contained in:
ethernet
2026-08-09 13:22:25 -04:00
parent 244d296646
commit 0e2b5f3835
15 changed files with 0 additions and 2092 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
.github/pr-screenshots/39327/providers-collapsed.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 36 KiB

BIN
.github/pr-screenshots/39327/providers-expanded.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 40 KiB

BIN
.github/pr-screenshots/39327/tools-collapsed.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
.github/pr-screenshots/39327/tools-expanded.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 36 KiB

BIN
.github/pr-screenshots/45449/billing-confirm.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 138 KiB

BIN
.github/pr-screenshots/45449/billing-overview.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 148 KiB

BIN
.github/pr-screenshots/telegram-overflow/topic-final-response-clipped.jpg vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 428 KiB

146
.plans/message-reactions.md
View File

@@ -1,146 +0,0 @@
# Message reactions (desktop tapbacks)
Two-way emoji reactions on individual messages in the desktop transcript: the
user reacts to any message, the agent reacts to a user message, and both sides
read the other's reactions as conversational signal.
## What already exists
Hermes already models reactions on the **platform** side — the desktop is the
only surface without them.
| Surface | Reaction support | Where |
|---|---|---|
| Agent → platform message | `send_message(action="react"/"unreact")` | `tools/send_message_tool.py:266` `_handle_react()` |
| Photon / iMessage | tapbacks in + out, routed only for messages we sent | `plugins/platforms/photon/adapter.py:1240-1283` |
| Telegram | `setMessageReaction`, config-gated | `plugins/platforms/telegram/adapter.py:9669+` |
| Slack / Matrix / Feishu / Discord | inbound reaction events → hooks | `gateway/run.py:4688` `_handle_reaction_event()` → `HookRegistry.emit("reaction:added")` |
| Adapter contract | `add_reaction()` / `remove_reaction()` coroutines, `set_reaction_handler()` | `gateway/platforms/base.py:3330` |
| Core "affection" detector | regex on user text → `vibe`, drives CLI pet / TUI heart / desktop hearts | `agent/reactions.py`, `agent/turn_context.py:592-604` |
Two things follow from that table:
1. **The agent-facing verb already exists.** `send_message(action="react")` is
the established shape. A desktop reaction should extend that tool, not add a
new core tool — every new tool ships on every API call (AGENTS.md footprint
ladder).
2. **The inbound convention already exists.** Photon turns a tapback into a
normal message event with `reply_to_message_id` + `reply_to_is_own_message`,
and the gateway prefixes `[Replying to your previous message: "…"]`
(`gateway/run.py:13125-13132`). Desktop reactions should read the same way to
the model.
Nothing exists on the desktop side: `grep -ri reaction` across `apps/desktop`
finds only the pet-overlay hearts.
## Prior art
**iOS Tapback** ([Apple](https://support.apple.com/guide/iphone/react-with-tapbacks-iph018d3c336/ios)):
double-tap or touch-and-hold a message → floating pill above the bubble with
heart / thumbs-up / thumbs-down / haha / ‼️ / ❓, swipe left for suggested emoji
and stickers, or tap the emoji button for the full keyboard. **One tapback per
message per person** — tapping the same one again removes it, tapping a
different one replaces it. Multiple people's tapbacks stack on the badge.
**Platform data models** converge on the same shape:
| Platform | Model | Add / remove |
|---|---|---|
| Slack | `{name, count, users[]}` | [`reactions.add`](https://docs.slack.dev/reference/methods/reactions.add) / `reactions.remove`, emits `reaction_added` |
| Discord | `{emoji, count, me}` on the message object | `PUT`/`DELETE .../reactions/{emoji}/@me` |
| Telegram | `reaction: [{type:"emoji", emoji:"👍"}]` — replaces the whole set | `setMessageReaction`, `is_big` for the big animation |
Telegram's "set the whole array" is the closest match to iOS semantics and the
simplest thing to persist.
**assistant-ui has no reaction primitive.** `@assistant-ui/react` 0.14.24 (MIT,
vendored at `apps/desktop/node_modules`): zero hits for "reaction" in `core/src`,
`react/src`, `dist/`, or the 2.2 MB `llms-full.txt` docs dump. What exists is a
hard-coded binary `FeedbackAdapter` (`"positive" | "negative"`,
`core/src/adapters/feedback.ts`) that throws when unconfigured and only writes
back onto assistant messages. Not usable for emoji, not usable on user messages.
**But `metadata.custom` is the supported extension channel** and this repo
already uses it: `ThreadUserMessage`/`ThreadAssistantMessage`/`ThreadSystemMessage`
all carry `metadata.custom: Record<string, unknown>` (`core/src/types/message.ts:319-366`),
and `chat-runtime.ts:397` already ships `custom: { attachmentRefs }` through it.
**Emoji picker survey** (npm week of 2026-07-22, sizes measured from the
published ESM entry):
| Library | License | Weekly DL | gzip | Headless | Latest |
|---|---|---|---|---|---|
| **frimousse** | MIT | 573k | **8.5 kB** | ✅ fully unstyled, composable parts | 0.3.0 · 2025-07-15 |
| emoji-picker-react | MIT | 1.31M | 87 kB | ❌ own CSS-in-JS (flairup) | 4.19.1 · 2026-04-27 |
| emoji-mart | MIT | 2.22M | ~120 kB w/ data | ❌ Preact + shadow styling | 5.6.0 · **2024-04-25**, 217 open issues |
| emoji-picker-element | Apache-2.0 | 183k | — | ❌ Web Component / Shadow DOM | 1.29.1 · 2026-03-01 |
No picker is currently a dependency (only `emoji-regex`, transitive). Already
paid for and reusable: `radix-ui` (Popover), `motion`, `@tanstack/react-virtual`,
Tailwind v4.
## Recommendation
**Hand-roll the tapback pill; add frimousse only behind the "+".** Six fixed
emoji in a pill is ~40 lines of JSX against existing tokens — pulling 87 kB of
`emoji-picker-react` to render six buttons, plus a CSS engine that fights
`DESIGN.md`, is backwards. frimousse is headless, dependency-free, 10× smaller,
and exposes `emojibaseUrl` so the data can be bundled as a Vite asset instead of
hitting jsDelivr (Electron must work offline).
### Data model
One reaction per author per message, Telegram-style whole-set replacement:
```ts
type MessageReaction = { emoji: string; author: 'user' | 'agent'; at: number }
```
Persisted in the existing `messages.display_metadata` JSON column
(`hermes_state_common.py:215`) — no new table. It already survives insert,
compaction, and every read projection, and
`set_latest_matching_message_display_kind()` (`hermes_state.py:5292`) is the
precedent for stamping metadata onto an already-persisted row.
### Model context
Reactions must reach the model **without breaking prompt caching**. The
`api_messages` build loop strips `display_metadata` from every outgoing copy
(`agent/conversation_loop.py:1443-1446`) precisely so display state never
becomes a provider field. Two candidate paths:
| Path | Cache impact | Notes |
|---|---|---|
| Rewrite the reacted-to message's content to carry the annotation | **Breaks the cached prefix** — mutates past context | Rejected. AGENTS.md: prompt caching is sacred. |
| Deliver the reaction as the *next* turn's leading annotation, mirroring photon | Prefix untouched; only the new turn carries it | Matches `[Replying to your previous message: "…"]` (`gateway/run.py:13125`), which the agent already understands |
The second is the same trick the platform adapters already use, so the model
sees a familiar shape and no existing conversation is rewritten.
### Attach points
| Concern | File | Lines |
|---|---|---|
| Assistant hover bar | `apps/desktop/src/components/assistant-ui/thread/assistant-message.tsx` | 134–175 |
| User hover cluster | `apps/desktop/src/components/assistant-ui/thread/user-message.tsx` | 296–336 |
| Callback threading (ref caveat 79–99) | `apps/desktop/src/components/assistant-ui/thread/index.tsx` | 109–133 |
| `metadata.custom` → runtime | `apps/desktop/src/lib/chat-runtime.ts` | 384–432 |
| RPC client ↔ server pattern | `sidebar/session-actions-menu.tsx:62-89` ↔ `tui_gateway/server.py:8322` | — |
| Persistence | `hermes_state_common.py:192-216`, `hermes_state.py:5292-5324` | — |
| Prompt injection / strip | `agent/conversation_loop.py` | 1430–1529 |
### Known gaps to solve first
- **No durable message id crosses the gateway RPC path.** `_history_to_messages()`
(`tui_gateway/server.py:6545`) builds `{"role", "text"}` and drops the id. The
REST path carries `messages.id` incidentally via `SELECT *` but TS
`SessionMessage` (`types/hermes.ts:513-533`) doesn't declare it. Renderer ids
are ephemeral and change shape between rehydrated (`<ts>-<i>-<role>`), live
(`assistant-<ms>`), and optimistic (`user-<ms>-<rand>`) messages. A reaction
needs a stable key — this is the first thing to fix.
- **WeakMap identity cache** in `apps/desktop/src/app/chat/runtime-repository.ts:26-66`
keys normalized `ThreadMessage` by `ChatMessage` identity. A reaction change
must produce a **new** `ChatMessage` object or the UI renders stale.
- **Rewind rewrites rows** (`replace_messages`), so anything keyed by row id
needs cascade handling — an argument for keeping reactions in
`display_metadata` on the row itself rather than a side table.

291
.plans/openai-api-server.md
View File

@@ -1,291 +0,0 @@
# OpenAI-Compatible API Server for Hermes Agent
## Motivation
Every major chat frontend (Open WebUI 126k★, LobeChat 73k★, LibreChat 34k★,
AnythingLLM 56k★, NextChat 87k★, ChatBox 39k★, Jan 26k★, HF Chat-UI 8k★,
big-AGI 7k★) connects to backends via the OpenAI-compatible REST API with
SSE streaming. By exposing this endpoint, hermes-agent becomes instantly
usable as a backend for all of them — no custom adapters needed.
## What It Enables
```
┌──────────────────┐
│ Open WebUI │──┐
│ LobeChat │ │ POST /v1/chat/completions
│ LibreChat │ ├──► Authorization: Bearer <key> ┌─────────────────┐
│ AnythingLLM │ │ {"messages": [...]} │ hermes-agent │
│ NextChat │ │ │ gateway │
│ Any OAI client │──┘ ◄── SSE streaming response │ (API server) │
└──────────────────┘ └─────────────────┘
```
A user would:
1. Set `API_SERVER_ENABLED=true` in `~/.hermes/.env`
2. Run `hermes gateway` (API server starts alongside Telegram/Discord/etc.)
3. Point Open WebUI (or any frontend) at `http://localhost:8642/v1`
4. Chat with hermes-agent through any OpenAI-compatible UI
## Endpoints
| Method | Path | Purpose |
|--------|------|---------|
| POST | `/v1/chat/completions` | Chat with the agent (streaming + non-streaming) |
| GET | `/v1/models` | List available "models" (returns hermes-agent as a model) |
| GET | `/health` | Health check |
## Architecture
### Option A: Gateway Platform Adapter (recommended)
Create `gateway/platforms/api_server.py` as a new platform adapter that
extends `BasePlatformAdapter`. This is the cleanest approach because:
- Reuses all gateway infrastructure (session management, auth, context building)
- Runs in the same async loop as other adapters
- Gets message handling, interrupt support, and session persistence for free
- Follows the established pattern (like Telegram, Discord, etc.)
- Uses `aiohttp.web` (already a dependency) for the HTTP server
The adapter would start an `aiohttp.web.Application` server in `connect()`
and route incoming HTTP requests through the standard `handle_message()` pipeline.
### Option B: Standalone Component
A separate HTTP server class in `gateway/api_server.py` that creates its own
AIAgent instances directly. Simpler but duplicates session/auth logic.
**Recommendation: Option A** — fits the existing architecture, less code to
maintain, gets all gateway features for free.
## Request/Response Format
### Chat Completions (non-streaming)
```
POST /v1/chat/completions
Authorization: Bearer hermes-api-key-here
Content-Type: application/json
{
"model": "hermes-agent",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What files are in the current directory?"}
],
"stream": false,
"temperature": 0.7
}
```
Response:
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1710000000,
"model": "hermes-agent",
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "Here are the files in the current directory:\n..."
},
"finish_reason": "stop"
}],
"usage": {
"prompt_tokens": 50,
"completion_tokens": 200,
"total_tokens": 250
}
}
```
### Chat Completions (streaming)
Same request with `"stream": true`. Response is SSE:
```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Here "},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"are "},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
### Models List
```
GET /v1/models
Authorization: Bearer hermes-api-key-here
```
Response:
```json
{
"object": "list",
"data": [{
"id": "hermes-agent",
"object": "model",
"created": 1710000000,
"owned_by": "hermes-agent"
}]
}
```
## Key Design Decisions
### 1. Session Management
The OpenAI API is stateless — each request includes the full conversation.
But hermes-agent sessions have persistent state (memory, skills, tool context).
**Approach: Hybrid**
- Default: Stateless. Each request is independent. The `messages` array IS
the conversation. No session persistence between requests.
- Opt-in persistent sessions via `X-Session-ID` header. When provided, the
server maintains session state across requests (conversation history,
memory context, tool state). This enables richer agent behavior.
- The session ID also enables interrupt support — a subsequent request with
the same session ID while one is running triggers an interrupt.
### 2. Streaming
The agent's `run_conversation()` is synchronous and returns the full response.
For real SSE streaming, we need to emit chunks as they're generated.
**Phase 1 (MVP):** Run agent in a thread, return the complete response as
a single SSE chunk + `[DONE]`. This works with all frontends — they just see
a fast single-chunk response. Not true streaming but functional.
**Phase 2:** Add a response callback to AIAgent that emits text chunks as the
LLM generates them. The API server captures these via a queue and streams them
as SSE events. This gives real token-by-token streaming.
**Phase 3:** Stream tool execution progress too — emit tool call/result events
as the agent works, giving frontends visibility into what the agent is doing.
### 3. Tool Transparency
Two modes:
- **Opaque (default):** Frontends see only the final response. Tool calls
happen server-side and are invisible. Best for general-purpose UIs.
- **Transparent (opt-in via header):** Tool calls are emitted as OpenAI-format
tool_call/tool_result messages in the stream. Useful for agent-aware frontends.
### 4. Authentication
- Bearer token via `Authorization: Bearer <key>` header
- Token configured via `API_SERVER_KEY` env var
- Optional: allow unauthenticated local-only access (127.0.0.1 bind)
- Follows the same pattern as other platform adapters
### 5. Model Mapping
Frontends send `"model": "hermes-agent"` (or whatever). The actual LLM model
used is configured server-side in config.yaml. The API server maps any
requested model name to the configured hermes-agent model.
Optionally, allow model passthrough: if the frontend sends
`"model": "anthropic/claude-sonnet-4"`, the agent uses that model. Controlled
by a config flag.
## Configuration
```yaml
# In config.yaml
api_server:
enabled: true
port: 8642
host: "127.0.0.1" # localhost only by default
key: "your-secret-key" # or via API_SERVER_KEY env var
allow_model_override: false # let clients choose the model
max_concurrent: 5 # max simultaneous requests
```
Environment variables:
```bash
API_SERVER_ENABLED=true
API_SERVER_PORT=8642
API_SERVER_HOST=127.0.0.1
API_SERVER_KEY=your-secret-key
```
## Implementation Plan
### Phase 1: MVP (non-streaming) — PR
1. `gateway/platforms/api_server.py` — new adapter
- aiohttp.web server with endpoints:
- `POST /v1/chat/completions` — Chat Completions API (universal compat)
- `POST /v1/responses` — Responses API (server-side state, tool preservation)
- `GET /v1/models` — list available models
- `GET /health` — health check
- Bearer token auth middleware
- Non-streaming responses (run agent, return full result)
- Chat Completions: stateless, messages array is the conversation
- Responses API: server-side conversation storage via previous_response_id
- Store full internal conversation (including tool calls) keyed by response ID
- On subsequent requests, reconstruct full context from stored chain
- Frontend system prompt layered on top of hermes-agent's core prompt
2. `gateway/config.py` — add `Platform.API_SERVER` enum + config
3. `gateway/run.py` — register adapter in `_create_adapter()`
4. Tests in `tests/gateway/test_api_server.py`
### Phase 2: SSE Streaming
1. Add response streaming to both endpoints
- Chat Completions: `choices[0].delta.content` SSE format
- Responses API: semantic events (response.output_text.delta, etc.)
- Run agent in thread, collect output via callback queue
- Handle client disconnect (cancel agent)
2. Add `stream_callback` parameter to `AIAgent.run_conversation()`
### Phase 3: Enhanced Features
1. Tool call transparency mode (opt-in)
2. Model passthrough/override
3. Concurrent request limiting
4. Usage tracking / rate limiting
5. CORS headers for browser-based frontends
6. GET /v1/responses/{id} — retrieve stored response
7. DELETE /v1/responses/{id} — delete stored response
## Files Changed
| File | Change |
|------|--------|
| `gateway/platforms/api_server.py` | NEW — main adapter (~300 lines) |
| `gateway/config.py` | Add Platform.API_SERVER + config (~20 lines) |
| `gateway/run.py` | Register adapter in _create_adapter() (~10 lines) |
| `tests/gateway/test_api_server.py` | NEW — tests (~200 lines) |
| `cli-config.yaml.example` | Add api_server section |
| `README.md` | Mention API server in platform list |
## Compatibility Matrix
Once implemented, hermes-agent works as a drop-in backend for:
| Frontend | Stars | How to Connect |
|----------|-------|---------------|
| Open WebUI | 126k | Settings → Connections → Add OpenAI API, URL: `http://localhost:8642/v1` |
| NextChat | 87k | BASE_URL env var |
| LobeChat | 73k | Custom provider endpoint |
| AnythingLLM | 56k | LLM Provider → Generic OpenAI |
| Oobabooga | 42k | Already a backend, not a frontend |
| ChatBox | 39k | API Host setting |
| LibreChat | 34k | librechat.yaml custom endpoint |
| Chatbot UI | 29k | Custom API endpoint |
| Jan | 26k | Remote model config |
| AionUI | 18k | Custom API endpoint |
| HF Chat-UI | 8k | OPENAI_BASE_URL env var |
| big-AGI | 7k | Custom endpoint |

705
.plans/streaming-support.md
View File

@@ -1,705 +0,0 @@
# Streaming LLM Response Support for Hermes Agent
## Overview
Add token-by-token streaming of LLM responses across all platforms. When enabled,
users see the response typing out live instead of waiting for the full generation.
Streaming is opt-in via config, defaults to off, and all existing non-streaming
code paths remain intact as the default.
## Design Principles
1. **Feature-flagged**: `streaming.enabled: true` in config.yaml. Off by default.
When off, all existing code paths are unchanged — zero risk to current behavior.
2. **Callback-based**: A simple `stream_callback(text_delta: str)` function injected
into AIAgent. The agent doesn't know or care what the consumer does with tokens.
3. **Graceful degradation**: If the provider doesn't support streaming, or streaming
fails for any reason, silently fall back to the non-streaming path.
4. **Platform-agnostic core**: The streaming mechanism in AIAgent works the same
regardless of whether the consumer is CLI, Telegram, Discord, or the API server.
---
## Architecture
```
stream_callback(delta)
│
┌─────────────┐ ┌─────────────▼──────────────┐
│ LLM API │ │ queue.Queue() │
│ (stream) │───►│ thread-safe bridge between │
│ │ │ agent thread & consumer │
└─────────────┘ └─────────────┬──────────────┘
│
┌──────────────┼──────────────┐
│ │ │
┌─────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ CLI │ │ Gateway │ │ API Server│
│ print to │ │ edit msg │ │ SSE event │
│ terminal │ │ on Tg/Dc │ │ to client │
└───────────┘ └───────────┘ └───────────┘
```
The agent runs in a thread. The callback puts tokens into a thread-safe queue.
Each consumer reads the queue in its own context (async task, main thread, etc.).
---
## Configuration
### config.yaml
```yaml
streaming:
enabled: false # Master switch. Default off.
# Per-platform overrides (optional):
# cli: true # Override for CLI only
# telegram: true # Override for Telegram only
# discord: false # Keep Discord non-streaming
# api_server: true # Override for API server
```
### Environment variables
```
HERMES_STREAMING_ENABLED=true # Master switch via env
```
### How the flag is read
- **CLI**: `load_cli_config()` reads `streaming.enabled`, sets env var. AIAgent
checks at init time.
- **Gateway**: `_run_agent()` reads config, decides whether to pass
`stream_callback` to the AIAgent constructor.
- **API server**: For Chat Completions `stream=true` requests, always uses streaming
regardless of config (the client is explicitly requesting it). For non-stream
requests, uses config.
### Precedence
1. API server: client's `stream` field overrides everything
2. Per-platform config override (e.g., `streaming.telegram: true`)
3. Master `streaming.enabled` flag
4. Default: off
---
## Implementation Plan
### Phase 1: Core streaming infrastructure in AIAgent
**File: run_agent.py**
#### 1a. Add stream_callback parameter to __init__ (~5 lines)
```python
def __init__(self, ..., stream_callback: callable = None, ...):
self.stream_callback = stream_callback
```
No other init changes. The callback is optional — when None, everything
works exactly as before.
#### 1b. Add _run_streaming_chat_completion() method (~65 lines)
New method for Chat Completions API streaming:
```python
def _run_streaming_chat_completion(self, api_kwargs: dict):
"""Stream a chat completion, emitting text tokens via stream_callback.
Returns a fake response object compatible with the non-streaming code path.
Falls back to non-streaming on any error.
"""
stream_kwargs = dict(api_kwargs)
stream_kwargs["stream"] = True
stream_kwargs["stream_options"] = {"include_usage": True}
accumulated_content = []
accumulated_tool_calls = {} # index -> {id, name, arguments}
final_usage = None
try:
stream = self.client.chat.completions.create(**stream_kwargs)
for chunk in stream:
if not chunk.choices:
# Usage-only chunk (final)
if chunk.usage:
final_usage = chunk.usage
continue
delta = chunk.choices[0].delta
# Text content — emit via callback
if delta.content:
accumulated_content.append(delta.content)
if self.stream_callback:
try:
self.stream_callback(delta.content)
except Exception:
pass
# Tool call deltas — accumulate silently
if delta.tool_calls:
for tc_delta in delta.tool_calls:
idx = tc_delta.index
if idx not in accumulated_tool_calls:
accumulated_tool_calls[idx] = {
"id": tc_delta.id or "",
"name": "", "arguments": ""
}
if tc_delta.function:
if tc_delta.function.name:
accumulated_tool_calls[idx]["name"] = tc_delta.function.name
if tc_delta.function.arguments:
accumulated_tool_calls[idx]["arguments"] += tc_delta.function.arguments
# Build fake response compatible with existing code
tool_calls = []
for idx in sorted(accumulated_tool_calls):
tc = accumulated_tool_calls[idx]
if tc["name"]:
tool_calls.append(SimpleNamespace(
id=tc["id"], type="function",
function=SimpleNamespace(name=tc["name"], arguments=tc["arguments"]),
))
return SimpleNamespace(
choices=[SimpleNamespace(
message=SimpleNamespace(
content="".join(accumulated_content) or "",
tool_calls=tool_calls or None,
role="assistant",
),
finish_reason="tool_calls" if tool_calls else "stop",
)],
usage=final_usage,
model=self.model,
)
except Exception as e:
logger.debug("Streaming failed, falling back to non-streaming: %s", e)
return self.client.chat.completions.create(**api_kwargs)
```
#### 1c. Modify _run_codex_stream() for Responses API (~10 lines)
The method already iterates the stream. Add callback emission:
```python
def _run_codex_stream(self, api_kwargs: dict):
with self.client.responses.stream(**api_kwargs) as stream:
for event in stream:
# Emit text deltas if streaming callback is set
if self.stream_callback and hasattr(event, 'type'):
if event.type == 'response.output_text.delta':
try:
self.stream_callback(event.delta)
except Exception:
pass
return stream.get_final_response()
```
#### 1d. Modify _interruptible_api_call() (~5 lines)
Add the streaming branch:
```python
def _call():
try:
if self.api_mode == "codex_responses":
result["response"] = self._run_codex_stream(api_kwargs)
elif self.stream_callback is not None:
result["response"] = self._run_streaming_chat_completion(api_kwargs)
else:
result["response"] = self.client.chat.completions.create(**api_kwargs)
except Exception as e:
result["error"] = e
```
#### 1e. Signal end-of-stream to consumers (~5 lines)
After the API call returns, signal the callback that streaming is done
so consumers can finalize (remove cursor, close SSE, etc.):
```python
# In run_conversation(), after _interruptible_api_call returns:
if self.stream_callback:
try:
self.stream_callback(None) # None = end of stream signal
except Exception:
pass
```
Consumers check: `if delta is None: finalize()`
**Tests for Phase 1:** (~150 lines)
- Test _run_streaming_chat_completion with mocked stream
- Test fallback to non-streaming on error
- Test tool_call accumulation during streaming
- Test stream_callback receives correct deltas
- Test None signal at end of stream
- Test streaming disabled when callback is None
---
### Phase 2: Gateway consumers (Telegram, Discord, etc.)
**File: gateway/run.py**
#### 2a. Read streaming config (~15 lines)
In `_run_agent()`, before creating the AIAgent:
```python
# Read streaming config
_streaming_enabled = False
try:
# Check per-platform override first
platform_key = source.platform.value if source.platform else ""
_stream_cfg = {} # loaded from config.yaml streaming section
if _stream_cfg.get(platform_key) is not None:
_streaming_enabled = bool(_stream_cfg[platform_key])
else:
_streaming_enabled = bool(_stream_cfg.get("enabled", False))
except Exception:
pass
# Env var override
if os.getenv("HERMES_STREAMING_ENABLED", "").lower() in ("true", "1", "yes"):
_streaming_enabled = True
```
#### 2b. Set up queue + callback (~15 lines)
```python
_stream_q = None
_stream_done = None
_stream_msg_id = [None] # mutable ref for the async task
if _streaming_enabled:
import queue as _q
_stream_q = _q.Queue()
_stream_done = threading.Event()
def _on_token(delta):
if delta is None:
_stream_done.set()
else:
_stream_q.put(delta)
```
Pass `stream_callback=_on_token` to the AIAgent constructor.
#### 2c. Telegram/Discord stream preview task (~50 lines)
```python
async def stream_preview():
"""Progressively edit a message with streaming tokens."""
if not _stream_q:
return
adapter = self.adapters.get(source.platform)
if not adapter:
return
accumulated = []
token_count = 0
last_edit = 0.0
MIN_TOKENS = 20 # Don't show until enough context
EDIT_INTERVAL = 1.5 # Respect Telegram rate limits
try:
while not _stream_done.is_set():
try:
chunk = _stream_q.get(timeout=0.1)
accumulated.append(chunk)
token_count += 1
except queue.Empty:
continue
now = time.monotonic()
if token_count >= MIN_TOKENS and (now - last_edit) >= EDIT_INTERVAL:
preview = "".join(accumulated) + " ▌"
if _stream_msg_id[0] is None:
r = await adapter.send(
chat_id=source.chat_id,
content=preview,
metadata=_thread_metadata,
)
if r.success and r.message_id:
_stream_msg_id[0] = r.message_id
else:
await adapter.edit_message(
chat_id=source.chat_id,
message_id=_stream_msg_id[0],
content=preview,
)
last_edit = now
# Drain remaining tokens
while not _stream_q.empty():
accumulated.append(_stream_q.get_nowait())
# Final edit — remove cursor, show complete text
if _stream_msg_id[0] and accumulated:
await adapter.edit_message(
chat_id=source.chat_id,
message_id=_stream_msg_id[0],
content="".join(accumulated),
)
except asyncio.CancelledError:
# Clean up on cancel
if _stream_msg_id[0] and accumulated:
try:
await adapter.edit_message(
chat_id=source.chat_id,
message_id=_stream_msg_id[0],
content="".join(accumulated),
)
except Exception:
pass
except Exception as e:
logger.debug("stream_preview error: %s", e)
```
#### 2d. Skip final send if already streamed (~10 lines)
In `_process_message_background()` (base.py), after getting the response,
if streaming was active and `_stream_msg_id[0]` is set, the final response
was already delivered via progressive edits. Skip the normal `self.send()`
call to avoid duplicating the message.
This is the most delicate integration point — we need to communicate from
the gateway's `_run_agent` back to the base adapter's response sender that
the response was already delivered. Options:
- **Option A**: Return a special marker in the result dict:
`result["_streamed_msg_id"] = _stream_msg_id[0]`
The base adapter checks this and skips `send()`.
- **Option B**: Edit the already-sent message with the final response
(which may differ slightly from accumulated tokens due to think-block
stripping, etc.) and don't send a new one.
- **Option C**: The stream preview task handles the FULL final response
(including any post-processing), and the handler returns None to skip
the normal send path.
Recommended: **Option A** — cleanest separation. The result dict already
carries metadata; adding one more field is low-risk.
**Platform-specific considerations:**
| Platform | Edit support | Rate limits | Streaming approach |
|----------|-------------|-------------|-------------------|
| Telegram | ✅ edit_message_text | ~20 edits/min | Edit every 1.5s |
| Discord | ✅ message.edit | 5 edits/5s per message | Edit every 1.2s |
| Slack | ✅ chat.update | Tier 3 (~50/min) | Edit every 1.5s |
| WhatsApp | ❌ no edit support | N/A | Skip streaming, use normal path |
| HomeAssistant | ❌ no edit | N/A | Skip streaming |
| API Server | ✅ SSE native | No limit | Real SSE events |
WhatsApp and HomeAssistant fall back to non-streaming automatically because
they don't support message editing.
**Tests for Phase 2:** (~100 lines)
- Test stream_preview sends/edits correctly
- Test skip-final-send when streaming delivered
- Test WhatsApp/HA graceful fallback
- Test streaming disabled per-platform config
- Test thread_id metadata forwarded in stream messages
---
### Phase 3: CLI streaming
**File: cli.py**
#### 3a. Set up callback in the CLI chat loop (~20 lines)
In `_chat_once()` or wherever the agent is invoked:
```python
if streaming_enabled:
_stream_q = queue.Queue()
_stream_done = threading.Event()
def _cli_stream_callback(delta):
if delta is None:
_stream_done.set()
else:
_stream_q.put(delta)
agent.stream_callback = _cli_stream_callback
```
#### 3b. Token display thread/task (~30 lines)
Start a thread that reads the queue and prints tokens:
```python
def _stream_display():
"""Print tokens to terminal as they arrive."""
first_token = True
while not _stream_done.is_set():
try:
delta = _stream_q.get(timeout=0.1)
except queue.Empty:
continue
if first_token:
# Print response box top border
_cprint(f"\n{top}")
first_token = False
sys.stdout.write(delta)
sys.stdout.flush()
# Drain remaining
while not _stream_q.empty():
sys.stdout.write(_stream_q.get_nowait())
sys.stdout.flush()
# Print bottom border
_cprint(f"\n\n{bot}")
```
**Integration challenge: prompt_toolkit**
The CLI uses prompt_toolkit which controls the terminal. Writing directly
to stdout while prompt_toolkit is active can cause display corruption.
The existing KawaiiSpinner already solves this by using prompt_toolkit's
`patch_stdout` context. The streaming display would need to do the same.
Alternative: use `_cprint()` for each token chunk (routes through
prompt_toolkit's renderer). But this might be slow for individual tokens.
Recommended approach: accumulate tokens in small batches (e.g., every 50ms)
and `_cprint()` the batch. This balances display responsiveness with
prompt_toolkit compatibility.
**Tests for Phase 3:** (~50 lines)
- Test CLI streaming callback setup
- Test response box borders with streaming
- Test fallback when streaming disabled
---
### Phase 4: API Server real streaming
**File: gateway/platforms/api_server.py**
Replace the pseudo-streaming `_write_sse_chat_completion()` with real
token-by-token SSE when the agent supports it.
#### 4a. Wire streaming callback for stream=true requests (~20 lines)
```python
if stream:
_stream_q = queue.Queue()
def _api_stream_callback(delta):
_stream_q.put(delta) # None = done
# Pass callback to _run_agent
result, usage = await self._run_agent(
..., stream_callback=_api_stream_callback,
)
```
#### 4b. Real SSE writer (~40 lines)
```python
async def _write_real_sse(self, request, completion_id, model, stream_q):
response = web.StreamResponse(
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await response.prepare(request)
# Role chunk
await response.write(...)
# Stream content chunks as they arrive
while True:
try:
delta = await asyncio.get_event_loop().run_in_executor(
None, lambda: stream_q.get(timeout=0.1)
)
except queue.Empty:
continue
if delta is None: # End of stream
break
chunk = {"id": completion_id, "object": "chat.completion.chunk", ...
"choices": [{"delta": {"content": delta}, ...}]}
await response.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Finish + [DONE]
await response.write(...)
await response.write(b"data: [DONE]\n\n")
return response
```
**Challenge: concurrent execution**
The agent runs in a thread executor. SSE writing happens in the async event
loop. The queue bridges them. But `_run_agent()` currently awaits the full
result before returning. For real streaming, we need to start the agent in
the background and stream tokens while it runs:
```python
# Start agent in background
agent_task = asyncio.create_task(self._run_agent_async(...))
# Stream tokens while agent runs
await self._write_real_sse(request, ..., stream_q)
# Agent is done by now (stream_q received None)
result, usage = await agent_task
```
This requires splitting `_run_agent` into an async version that doesn't
block waiting for the result, or running it in a separate task.
**Responses API SSE format:**
For `/v1/responses` with `stream=true`, the SSE events are different:
```
event: response.output_text.delta
data: {"type":"response.output_text.delta","delta":"Hello"}
event: response.completed
data: {"type":"response.completed","response":{...}}
```
This needs a separate SSE writer that emits Responses API format events.
**Tests for Phase 4:** (~80 lines)
- Test real SSE streaming with mocked agent
- Test SSE event format (Chat Completions vs Responses)
- Test client disconnect during streaming
- Test fallback to pseudo-streaming when callback not available
---
## Integration Issues & Edge Cases
### 1. Tool calls during streaming
When the model returns tool calls instead of text, no text tokens are emitted.
The stream_callback is simply never called with text. After tools execute, the
next API call may produce the final text response — streaming picks up again.
The stream preview task needs to handle this: if no tokens arrive during a
tool-call round, don't send/edit any message. The tool progress messages
continue working as before.
### 2. Duplicate messages
The biggest risk: the agent sends the final response normally (via the
existing send path) AND the stream preview already showed it. The user
sees the response twice.
Prevention: when streaming is active and tokens were delivered, the final
response send must be suppressed. The `result["_streamed_msg_id"]` marker
tells the base adapter to skip its normal send.
### 3. Response post-processing
The final response may differ from the accumulated streamed tokens:
- Think block stripping (`<think>...</think>` removed)
- Trailing whitespace cleanup
- Tool result media tag appending
The stream preview shows raw tokens. The final edit should use the
post-processed version. This means the final edit (removing the cursor)
should use the post-processed `final_response`, not just the accumulated
stream text.
### 4. Context compression during streaming
If the agent triggers context compression mid-conversation, the streaming
tokens from BEFORE compression are from a different context than those
after. This isn't a problem in practice — compression happens between
API calls, not during streaming.
### 5. Interrupt during streaming
User sends a new message while streaming → interrupt. The stream is killed
(HTTP connection closed), accumulated tokens are shown as-is (no cursor),
and the interrupt message is processed normally. This is already handled by
`_interruptible_api_call` closing the client.
### 6. Multi-model / fallback
If the primary model fails and the agent falls back to a different model,
streaming state resets. The fallback call may or may not support streaming.
The graceful fallback in `_run_streaming_chat_completion` handles this.
### 7. Rate limiting on edits
Telegram: ~20 edits/minute (~1 every 3 seconds to be safe)
Discord: 5 edits per 5 seconds per message
Slack: ~50 API calls/minute
The 1.5s edit interval is conservative enough for all platforms. If we get
429 rate limit errors on edits, just skip that edit cycle and try next time.
---
## Files Changed Summary
| File | Phase | Changes |
|------|-------|---------|
| `run_agent.py` | 1 | +stream_callback param, +_run_streaming_chat_completion(), modify _run_codex_stream(), modify _interruptible_api_call() |
| `gateway/run.py` | 2 | +streaming config reader, +queue/callback setup, +stream_preview task, +skip-final-send logic |
| `gateway/platforms/base.py` | 2 | +check for _streamed_msg_id in response handler |
| `cli.py` | 3 | +streaming setup, +token display, +response box integration |
| `gateway/platforms/api_server.py` | 4 | +real SSE writer, +streaming callback wiring |
| `hermes_cli/config.py` | 1 | +streaming config defaults |
| `cli-config.yaml.example` | 1 | +streaming section |
| `tests/test_streaming.py` | 1-4 | NEW — ~380 lines of tests |
**Total new code**: ~500 lines across all phases
**Total test code**: ~380 lines
---
## Rollout Plan
1. **Phase 1** (core): Merge to main. Streaming disabled by default.
Zero impact on existing behavior. Can be tested with env var.
2. **Phase 2** (gateway): Merge to main. Test on Telegram manually.
Enable per-platform: `streaming.telegram: true` in config.
3. **Phase 3** (CLI): Merge to main. Test in terminal.
Enable: `streaming.cli: true` or `streaming.enabled: true`.
4. **Phase 4** (API server): Merge to main. Test with Open WebUI.
Auto-enabled when client sends `stream: true`.
Each phase is independently mergeable and testable. Streaming stays
off by default throughout. Once all phases are stable, consider
changing the default to enabled.
---
## Config Reference (final state)
```yaml
# config.yaml
streaming:
enabled: false # Master switch (default: off)
cli: true # Per-platform override
telegram: true
discord: true
slack: true
api_server: true # API server always streams when client requests it
edit_interval: 1.5 # Seconds between message edits (default: 1.5)
min_tokens: 20 # Tokens before first display (default: 20)
```
```bash
# Environment variable override
HERMES_STREAMING_ENABLED=true
```

240
docs/plans/2026-06-09-003-fix-telegram-stream-overflow-continuations-plan.md
View File

@@ -1,240 +0,0 @@
---
title: "fix: Prevent Telegram streamed replies from ending after first overflow chunk"
status: active
date: 2026-06-09
type: fix
target_repo: hermes-agent
origin: user-reported Telegram topic screenshot
---
# fix: Prevent Telegram streamed replies from ending after first overflow chunk
## Summary
Fix a Telegram gateway bug where a long streamed assistant reply can appear to stop mid-answer in a topic after the first overflow chunk. The reported screenshot shows a long Hermes response in the `Nehemiah - Coding` Telegram topic ending at `- The visible tool-call summary`, followed by the user noting that the previous message did not finish streaming to that Telegram topic.
The plan targets the streamed edit overflow path, not general model generation. A completed assistant response must either reach Telegram in full across all continuation messages or leave enough state for the gateway fallback path to deliver the remaining content instead of marking the turn complete after a partial delivery.
---
## Problem Frame
Telegram limits message text to 4096 UTF-16 code units. Hermes streams gateway responses by editing a message and, when a streamed message grows past the limit, splitting the overflow into additional Telegram messages. The adapter already has a split-and-deliver path for oversized edits, but the partial-continuation failure contract is weak: if chunk 1 is edited successfully and a later continuation fails, the adapter can still report success for the operation. The stream consumer may then mark the final response delivered even though the visible topic only contains the first part.
This is especially visible in Telegram forum topics because a long final response can be split below tool-progress bubbles, and a missing continuation looks exactly like the stream stopped mid-answer.
---
## Requirements
- R1. Long streamed Telegram replies must preserve all final content across overflow chunks.
- R2. If any continuation chunk fails after the first overflow edit lands, the gateway must not mark the final response as fully delivered.
- R3. Continuation chunks must remain routed to the same Telegram topic/thread as the original response.
- R4. The fix must avoid duplicate full-answer sends when all overflow chunks were delivered successfully.
- R5. Tests must cover the reported failure shape: a final streamed reply that exceeds Telegram's limit, succeeds on the first edit, fails on a continuation, and must not be treated as complete.
---
## Key Technical Decisions
- Treat overflow delivery as all-or-not-complete. `_edit_overflow_split` should only return a successful final-delivery result when every planned chunk reaches Telegram. Partial delivery is a distinct outcome that downstream code can recover from.
- Carry partial-overflow metadata through `SendResult.raw_response` rather than adding a new public dataclass field unless implementation proves the existing result shape is insufficient. The stream consumer already inspects `SendResult` after adapter edits, so a small raw response contract can keep the change contained.
- Make the stream consumer responsible for final-delivery truth. The adapter knows which chunks landed, but the consumer owns `_final_response_sent`, `_final_content_delivered`, `_fallback_prefix`, and fallback final-send behaviour.
- Keep routing inside Telegram adapter helpers. Continuation sends should continue to use `_thread_kwargs_for_send(...)` with metadata-derived `message_thread_id` and reply anchors so forum topic behaviour stays consistent.
---
## High-Level Technical Design
```mermaid
sequenceDiagram
participant C as GatewayStreamConsumer
participant T as TelegramAdapter.edit_message
participant B as Telegram Bot API
C->>T: finalize/edit long accumulated response
T->>B: edit original message with chunk 1
loop remaining chunks
T->>B: send continuation in same topic/thread
end
alt all chunks delivered
T-->>C: success, last message id, continuation ids
C->>C: mark final response delivered
else any continuation failed
T-->>C: partial overflow failure with delivered prefix metadata
C->>C: do not mark final delivered
C->>B: fallback sends missing tail or full final response safely
end
```
---
## Implementation Units
### U1. Add a partial-overflow contract for Telegram edit splits
**Goal:** Make `TelegramAdapter._edit_overflow_split` distinguish complete overflow delivery from partial delivery.
**Requirements:** R1, R2, R4
**Dependencies:** None
**Files:**
- `gateway/platforms/telegram.py`
- `tests/gateway/test_telegram_send.py` or the existing Telegram adapter test module that already covers `edit_message` overflow behaviour
**Approach:**
- Keep the successful path unchanged when every chunk is delivered: return `SendResult(success=True, message_id=<last chunk>, continuation_message_ids=(...))`.
- When a continuation fails after the first edit, return a result that clearly indicates partial delivery instead of plain success. Prefer `success=False`, `retryable=True`, and `raw_response` metadata such as delivered chunk count, total chunk count, last delivered message id, and the visible delivered prefix.
- Preserve logging, but do not rely on logs as the only signal. The caller must be able to tell partial delivery happened.
- Ensure the first edited chunk and all successful continuation chunks still include the existing Markdown/plain-text fallback behaviour.
**Patterns to follow:**
- Existing overflow handling in `TelegramAdapter.edit_message` and `_edit_overflow_split`.
- Existing `SendResult` semantics in `gateway/platforms/base.py`, especially `retryable`, `raw_response`, and `continuation_message_ids`.
**Test scenarios:**
- Oversized finalized edit where all continuations succeed returns success, the last continuation id, and all continuation ids.
- Oversized finalized edit where the first continuation send fails returns a partial-overflow failure and does not report success.
- Oversized finalized edit where one continuation succeeds and a later continuation fails reports the last delivered continuation id and delivered count in raw metadata.
- A continuation MarkdownV2 formatting failure still retries plain text before being treated as a delivery failure.
**Verification:** Adapter tests prove complete overflow remains successful and partial overflow is observable by the caller.
### U2. Teach the stream consumer to recover from partial overflow
**Goal:** Ensure a partial Telegram overflow does not set `_final_response_sent` or `_final_content_delivered` unless the full response reached the user.
**Requirements:** R1, R2, R4, R5
**Dependencies:** U1
**Files:**
- `gateway/stream_consumer.py`
- `tests/gateway/test_stream_consumer.py` or a focused new `tests/gateway/test_stream_consumer_telegram_overflow.py`
**Approach:**
- In `_send_or_edit`, when `adapter.edit_message(...)` returns a partial-overflow failure, update consumer state to reflect the last visible prefix/message and enter fallback delivery for the missing content.
- Avoid treating `_already_sent` as final delivery. A partial visible message can be true while final delivery is false.
- Use the delivered-prefix metadata if available so `_send_fallback_final(...)` sends only the missing tail. If implementation finds the prefix is unreliable after Markdown formatting, prefer sending the complete final response as a fresh fallback message rather than silently dropping the tail.
- Keep the existing success handling for `continuation_message_ids` when the adapter delivered all chunks.
**Patterns to follow:**
- Existing fallback mode in `GatewayStreamConsumer._send_or_edit` and `_send_fallback_final`.
- Existing comments around `_final_response_sent`, `_final_content_delivered`, and `_fallback_prefix` for prior partial-delivery regressions.
**Test scenarios:**
- A final streamed response that overflows and receives a complete-success edit split sets final-delivery flags and does not invoke fallback.
- A final streamed response whose adapter reports partial overflow does not set final-delivery flags immediately.
- After partial overflow, fallback delivery sends the remaining tail and then marks final content delivered only if the fallback send succeeds.
- If fallback delivery also fails, the consumer leaves final-delivery false so the gateway's non-streaming final-send safety path can still run.
**Verification:** Stream consumer tests reproduce the screenshot shape by simulating first chunk visible and continuation failure, then assert the final answer is not suppressed.
### U3. Preserve Telegram topic/thread routing for overflow and fallback continuations
**Goal:** Ensure overflow recovery messages land in the same Telegram forum topic or DM topic fallback context.
**Requirements:** R3
**Dependencies:** U1, U2
**Files:**
- `gateway/platforms/telegram.py`
- `gateway/stream_consumer.py`
- `tests/gateway/test_stream_consumer_thread_routing.py`
- Relevant Telegram adapter routing tests, if existing coverage is closer there
**Approach:**
- Keep passing `metadata` through every overflow continuation and fallback send.
- Keep reply anchors where valid, but do not let a missing reply anchor drop the `message_thread_id` for normal forum topics.
- For private DM topic fallback metadata, preserve the existing stricter anchor behaviour documented in the adapter comments.
**Patterns to follow:**
- `TelegramAdapter._thread_kwargs_for_send(...)`.
- Existing tests around Telegram topic recovery and stream consumer thread routing.
**Test scenarios:**
- Overflow continuations include `message_thread_id` for a forum topic.
- A continuation retry after `reply message not found` keeps forum topic routing when allowed.
- Partial-overflow fallback sends receive the same metadata passed to the original stream consumer.
**Verification:** Thread-routing assertions inspect fake bot calls and confirm all continuation/fallback messages carry the expected topic metadata.
### U4. Add issue evidence and PR body traceability
**Goal:** Make the upstream issue and PR clearly trace the user-visible bug and verification evidence.
**Requirements:** R5
**Dependencies:** U1, U2, U3
**Files:**
- GitHub issue body created via `gh issue create`
- PR body using `.github/PULL_REQUEST_TEMPLATE.md`
**Approach:**
- Create a GitHub issue with the screenshot evidence: the long message in the `Nehemiah - Coding` Telegram topic stops at `- The visible tool-call summary`, and the user's reply says the previous message did not finish streaming to that Telegram topic.
- Reference affected component as Gateway and platform as Telegram.
- In the PR body, link the issue with `Fixes #...`, describe the split-delivery contract change, and include the screenshot or attach it if GitHub upload is available.
- Follow `CONTRIBUTING.md` and the repository PR template exactly.
**Patterns to follow:**
- `.github/ISSUE_TEMPLATE/bug_report.yml`
- `.github/PULL_REQUEST_TEMPLATE.md`
**Test scenarios:**
- Test expectation: none, this is tracker and PR documentation work.
**Verification:** The GitHub issue exists with screenshot evidence or an explicit screenshot reference, and the PR body links the issue and lists the tests run.
---
## Scope Boundaries
### In Scope
- Telegram streamed response overflow splitting and recovery.
- Stream consumer final-delivery truth for partial overflow delivery.
- Topic/thread metadata preservation for overflow and fallback continuation sends.
- Focused unit tests around adapter and stream consumer behaviour.
### Out of Scope
- Changing model streaming semantics in `run_agent.py`.
- Reworking Telegram draft streaming, which is DM-only and not the forum-topic path in the screenshot.
- Changing general platform message splitting for Discord, Slack, WhatsApp, or Matrix unless a shared helper must be corrected for the Telegram fix.
- Altering tool-progress display settings or terminal progress rendering.
### Deferred to Follow-Up Work
- Broader observability for gateway delivery completeness across all messaging platforms.
- A user-facing resend/recover command for a previous truncated response.
---
## Risks & Mitigations
- Risk: fallback recovery duplicates already-visible first chunks. Mitigation: use delivered-prefix metadata where reliable and add tests for no-duplicate complete-success behaviour.
- Risk: preserving forum topic routing while dropping invalid reply anchors is easy to regress. Mitigation: include fake bot call assertions for `message_thread_id` and reply behaviour.
- Risk: MarkdownV2 formatting can alter visible/raw prefix comparisons. Mitigation: keep fallback conservative; duplicate content is preferable to silently missing content, but tests should keep the common path tail-only.
---
## Sources & Research
- User-provided screenshot at `/root/.hermes/image_cache/img_f664e68f6ddf.jpg`.
- `gateway/stream_consumer.py` streamed edit, overflow, fallback, and final-delivery state handling.
- `gateway/platforms/telegram.py` Telegram send/edit overflow splitting and topic routing helpers.
- `gateway/platforms/base.py` `SendResult` contract and shared message chunking helper.
- `tests/gateway/test_stream_consumer.py`, `tests/gateway/test_stream_consumer_thread_routing.py`, and Telegram adapter tests for focused regression coverage.
---
## Verification Strategy
- Run focused Telegram adapter overflow tests.
- Run focused stream consumer overflow/fallback tests.
- Run topic-routing tests affected by metadata changes.
- Run the gateway test subset around Telegram send/edit, stream consumer, and run progress if touched.
- Before PR creation, ensure `git diff` contains only the plan, implementation, tests, and PR/issue-relevant documentation for this bug.

160
hermes-already-has-routines.md
View File

@@ -1,160 +0,0 @@
# Hermes Agent Has Had "Routines" Since March
Anthropic just announced [Claude Code Routines](https://claude.com/blog/introducing-routines-in-claude-code) — scheduled tasks, GitHub event triggers, and API-triggered agent runs. Bundled prompt + repo + connectors, running on their infrastructure.
It's a good feature. We shipped it two months ago.
---
## The Three Trigger Types — Side by Side
Claude Code Routines offers three ways to trigger an automation:
**1. Scheduled (cron)**
> "Every night at 2am: pull the top bug from Linear, attempt a fix, and open a draft PR."
Hermes equivalent — works today:
```bash
hermes cron create "0 2 * * *" \
"Pull the top bug from the issue tracker, attempt a fix, and open a draft PR." \
--name "Nightly bug fix" \
--deliver telegram
```
**2. GitHub Events (webhook)**
> "Flag PRs that touch the /auth-provider module and post to #auth-changes."
Hermes equivalent — works today:
```bash
hermes webhook subscribe auth-watch \
--events "pull_request" \
--prompt "PR #{pull_request.number}: {pull_request.title} by {pull_request.user.login}. Check if it touches the auth-provider module. If yes, summarize the changes." \
--deliver slack
```
**3. API Triggers**
> "Read the alert payload, find the owning service, post a triage summary to #oncall."
Hermes equivalent — works today:
```bash
hermes webhook subscribe alert-triage \
--prompt "Alert: {alert.name} — Severity: {alert.severity}. Find the owning service, investigate, and post a triage summary with proposed first steps." \
--deliver slack
```
Every use case in their blog post — backlog triage, docs drift, deploy verification, alert correlation, library porting, bespoke PR review — has a working Hermes implementation. No new features needed. It's been shipping since March 2026.
---
## What's Different
| | Claude Code Routines | Hermes Agent |
|---|---|---|
| **Scheduled tasks** | ✅ Schedule-based | ✅ Any cron expression + human-readable intervals |
| **GitHub triggers** | ✅ PR, issue, push events | ✅ Any GitHub event via webhook subscriptions |
| **API triggers** | ✅ POST to unique endpoint | ✅ POST to webhook routes with HMAC auth |
| **MCP connectors** | ✅ Native connectors | ✅ Full MCP client support |
| **Script pre-processing** | ❌ | ✅ Python scripts run before agent, inject context |
| **Skill chaining** | ❌ | ✅ Load multiple skills per automation |
| **Daily limit** | 5-25 runs/day | **Unlimited** |
| **Model choice** | Claude only | **Any model** — Claude, GPT, Gemini, DeepSeek, Qwen, local |
| **Delivery targets** | GitHub comments | Telegram, Discord, Slack, SMS, email, GitHub comments, webhooks, local files |
| **Infrastructure** | Anthropic's servers | **Your infrastructure** — VPS, home server, laptop |
| **Data residency** | Anthropic's cloud | **Your machines** |
| **Cost** | Pro/Max/Team/Enterprise subscription | Your API key, your rates |
| **Open source** | No | **Yes** — MIT license |
---
## Things Hermes Does That Routines Can't
### Script Injection
Run a Python script *before* the agent. The script's stdout becomes context. The script handles mechanical work (fetching, diffing, computing); the agent handles reasoning.
```bash
hermes cron create "every 1h" \
"If CHANGE DETECTED, summarize what changed. If NO_CHANGE, respond with [SILENT]." \
--script ~/.hermes/scripts/watch-site.py \
--name "Pricing monitor" \
--deliver telegram
```
The `[SILENT]` pattern means you only get notified when something actually happens. No spam.
### Multi-Skill Workflows
Chain specialized skills together. Each skill teaches the agent a specific capability, and the prompt ties them together.
```bash
hermes cron create "0 8 * * *" \
"Search arXiv for papers on language model reasoning. Save the top 3 as Obsidian notes." \
--skills "arxiv,obsidian" \
--name "Paper digest"
```
### Deliver Anywhere
One automation, any destination:
```bash
--deliver telegram # Telegram home channel
--deliver discord # Discord home channel
--deliver slack # Slack channel
--deliver sms:+15551234567 # Text message
--deliver telegram:-1001234567890:42 # Specific Telegram forum topic
--deliver local # Save to file, no notification
```
### Model-Agnostic
Your nightly triage can run on Claude. Your deploy verification can run on GPT. Your cost-sensitive monitors can run on DeepSeek or a local model. Same automation system, any backend.
---
## The Limits Tell the Story
Claude Code Routines: **5 routines per day** on Pro. **25 on Enterprise.** That's their ceiling.
Hermes has no daily limit. Run 500 automations a day if you want. The only constraint is your API budget, and you choose which models to use for which tasks.
A nightly backlog triage on Sonnet costs roughly $0.02-0.05. A monitoring check on DeepSeek costs fractions of a cent. You control the economics.
---
## Get Started
Hermes Agent is open source and free. The automation infrastructure — cron scheduler, webhook platform, skill system, multi-platform delivery — is built in.
```bash
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
```
Set up a scheduled task in 30 seconds:
```bash
hermes cron create "0 9 * * 1" \
"Generate a weekly AI news digest. Search the web for major announcements, trending repos, and notable papers. Keep it under 500 words with links." \
--name "Weekly digest" \
--deliver telegram
```
Set up a GitHub webhook in 60 seconds:
```bash
hermes gateway setup # enable webhooks
hermes webhook subscribe pr-review \
--events "pull_request" \
--prompt "Review PR #{pull_request.number}: {pull_request.title}" \
--skills "github-code-review" \
--deliver github_comment
```
Full automation blueprints gallery: [hermes-agent.nousresearch.com/docs/reference/automation-blueprints-catalog](https://hermes-agent.nousresearch.com/docs/reference/automation-blueprints-catalog)
Documentation: [hermes-agent.nousresearch.com](https://hermes-agent.nousresearch.com)
GitHub: [github.com/NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent)
---
*Hermes Agent is built by [Nous Research](https://nousresearch.com). Open source, model-agnostic, runs on your infrastructure.*

157
plugins/hermes-achievements/docs/achievements-performance-implementation-plan.md
View File

@@ -1,157 +0,0 @@
# Hermes Achievements Performance Implementation Plan
Status: Ready for execution after hackathon review window
Constraint: Plugin remains frozen until judging is complete
Decision: `/overview` and top-banner slots are out of scope and will be removed.
---
## Phase 0 — Baseline & Safety (no behavior change)
### Task 0.1: Add perf benchmark script (local)
Objective: Repro baseline before/after.
Acceptance:
- Can print endpoint timings for `/achievements` (3 runs each, cold + warm).
### Task 0.2: Define acceptance thresholds
Objective: Lock success criteria now.
Acceptance:
- Documented SLOs:
- `/achievements` p95 < 1s (cached)
- max active scan jobs = 1
---
## Phase 1 — Remove unused overview/slot surface (highest certainty)
### Task 1.1: Remove `/overview` backend route
Objective: Eliminate duplicate heavy endpoint path.
Acceptance:
- `plugin_api.py` no longer exposes `/overview`.
### Task 1.2: Remove slot registration and SummarySlot frontend code
Objective: Remove cross-tab banner fetch behavior.
Acceptance:
- No `registerSlot(..."sessions:top"...)` or `registerSlot(..."analytics:top"...)`.
- No frontend call to `api("/overview")`.
### Task 1.3: Update plugin manifest
Objective: Reflect final UI scope.
Acceptance:
- `manifest.json` removes `slots` declarations.
- Tab registration remains intact.
---
## Phase 2 — Shared snapshot persistence + single-flight for `/achievements`
### Task 2.1: Introduce snapshot store abstraction + on-disk persistence
Objective: Single source of truth for Achievements data that survives process restarts.
Acceptance:
- One structure contains dataset consumed by `/achievements`.
- Repeated requests do not recompute when cache is fresh.
- Snapshot persisted at `~/.hermes/plugins/hermes-achievements/scan_snapshot.json`.
### Task 2.2: Single-flight scan coordinator
Objective: Prevent concurrent recomputes.
Acceptance:
- Simultaneous requests result in one compute run.
### Task 2.3: Refactor `/achievements` to read snapshot
Objective: Remove direct repeated compute from request path.
Acceptance:
- `/achievements` does not run independent full recompute per request when cache is valid.
---
## Phase 3 — Stale-While-Revalidate
### Task 3.1: TTL state (`FRESH`/`STALE`)
Objective: Serve immediately when stale, refresh in background.
Acceptance:
- Cached response returned quickly even when expired.
- Refresh is asynchronous.
### Task 3.2: Add `scan-status` endpoint (optional)
Objective: Let UI/ops inspect scan state.
Acceptance:
- Returns state, last success time, last duration, last error.
### Task 3.3: Add metadata fields to `/achievements`
Objective: Improve transparency.
Acceptance:
- Response includes `generated_at`, `is_stale`, maybe `scan_id`.
---
## Phase 4 — Incremental Scanning (optional but recommended)
### Task 4.1: Add per-session checkpoint file
Objective: Track session-level changes, not just global scan time.
Acceptance:
- Checkpoint persisted at `~/.hermes/plugins/hermes-achievements/scan_checkpoint.json`.
- For each session: `session_id`, fingerprint (`updated_at`/message_count/hash), and cached contribution.
### Task 4.2: Incremental aggregation
Objective: Recompute only changed/new sessions and reuse unchanged contributions.
Acceptance:
- Typical refresh time drops materially below full scan.
- Aggregate rebuild uses: subtract old contribution + add new contribution for changed sessions.
### Task 4.3: Full rebuild fallback
Objective: Preserve correctness.
Acceptance:
- Manual full rescan always possible.
- Schema/version changes invalidate checkpoint and force full rebuild.
---
## Test Plan
1. Unit tests
- Snapshot lifecycle transitions
- Dedupe logic under parallel requests
- `/achievements` response compatibility
2. Integration tests
- Opening Achievements repeatedly causes <=1 heavy scan while in-flight
- `/achievements` warm-cache load is fast
- manual rescan updates snapshot and timestamps
3. Manual benchmarks
- Compare pre/post `/achievements` timings with same history dataset
---
## Rollout Plan
1. Release internal branch with Phase 1 (remove overview/slots).
2. Validate no UI regression in Achievements tab.
3. Add Phase 2 snapshot/dedupe.
4. Add Phase 3 stale-while-revalidate + status metadata.
5. Optional: incremental scanner.
Rollback: keep old compute path behind temporary feature flag for one release window.
---
## Definition of Done
- Achievements tab remains fully functional (counts, latest, tiers, cards, filters).
- No `/overview` endpoint or slot calls remain.
- Repeated Achievements loads feel immediate after warm cache.
- Metrics/unlocks remain unchanged versus baseline.

219
plugins/hermes-achievements/docs/achievements-performance-implementation-spec.md
View File

@@ -1,219 +0,0 @@
# Hermes Achievements Implementation Spec (Detailed)
This document is implementation-facing detail to execute the performance refactor later.
Decision scope: keep only Achievements tab flow; remove `/overview` + top-banner slot integration.
---
## A) Current Behavior Summary
- `evaluate_all()` performs:
- full `scan_sessions()`
- `SessionDB.list_sessions_rich(...)`
- `db.get_messages(session_id)` for each session
- text/tool regex analysis + aggregation + evaluation
- `/overview` and `/achievements` both currently call `evaluate_all()` directly.
- slot calls (`sessions:top`, `analytics:top`) currently invoke `/overview`.
Consequence: repeated full recomputes and contention.
---
## B) De-scope/Removal Changes
1. Remove backend route:
- `GET /overview`
2. Remove frontend slot usage:
- `SummarySlot` component
- `registerSlot("sessions:top")`
- `registerSlot("analytics:top")`
3. Remove manifest slot declarations:
- `"slots": ["sessions:top", "analytics:top"]`
4. Keep:
- tab route/page for Achievements
- `/achievements` endpoint and full tab rendering
---
## C) Target Internal Interfaces
### 1) `SnapshotStore`
Responsibilities:
- hold latest computed snapshot in memory
- persist/load snapshot from disk
- expose age and staleness checks
Storage path:
- `~/.hermes/plugins/hermes-achievements/scan_snapshot.json`
Methods (conceptual):
- `get()` -> snapshot | null
- `set(snapshot)`
- `is_stale(ttl_seconds)`
### 2) `ScanCoordinator`
Responsibilities:
- single-flight guard for compute jobs
- track scan status
Methods:
- `run_if_needed(force: bool = false)`
- `get_status()`
State fields:
- `state`: `idle|running|failed`
- `started_at`, `finished_at`
- `last_error`
- `run_count`
### 3) `build_snapshot()`
Responsibilities:
- execute current compute logic once
- on first run, perform full scan and materialize per-session contributions
- on subsequent runs, process only changed/new sessions via checkpoint fingerprints
- produce shape consumed by `/achievements`
Output:
- `achievements`
- count fields
- optional `scan_meta`
---
## D) Endpoint Behavior Matrix (No `/overview`)
| Endpoint | Cache fresh | Cache stale | No cache | Force rescan |
|---|---|---|---|---|
| `/achievements` | return cached | return stale + trigger bg refresh | blocking bootstrap scan | n/a |
| `/rescan` | trigger refresh | trigger refresh | trigger refresh | yes |
| `/scan-status` | status only | status only | status only | status only |
Notes:
- At most one scan run active.
- Other callers either await same run or receive stale snapshot according to policy.
---
## E) Data Shape (Proposed)
```json
{
"generated_at": 0,
"is_stale": false,
"scan_meta": {
"duration_ms": 0,
"sessions_scanned": 0,
"messages_scanned": 0,
"mode": "full",
"error": null
},
"achievements": [],
"unlocked_count": 0,
"discovered_count": 0,
"secret_count": 0,
"total_count": 0,
"error": null
}
```
Compatibility guidance:
- Keep existing `/achievements` keys.
- Add metadata keys without breaking old callers.
Checkpoint file (new):
- `~/.hermes/plugins/hermes-achievements/scan_checkpoint.json`
Suggested checkpoint shape:
```json
{
"schema_version": 1,
"generated_at": 0,
"sessions": {
"<session_id>": {
"fingerprint": {
"updated_at": 0,
"message_count": 0,
"hash": "optional"
},
"contribution": {
"metrics": {}
}
}
}
}
```
Notes:
- fingerprint mismatch => recompute that session contribution only.
- unchanged fingerprint => reuse stored contribution.
---
## F) Concurrency Contract
- Any request path that needs fresh data must pass through single-flight coordinator.
- If a scan is running:
- do not start second scan
- either await in-flight run (bounded) or serve stale snapshot immediately
- lock scope must include scan start/finish state transitions.
---
## G) Error Handling Contract
- If refresh fails and prior snapshot exists:
- return prior snapshot with `is_stale=true` and error metadata
- If refresh fails and no prior snapshot:
- return explicit error response (current behavior equivalent)
- `scan-status` should always return last known state/error.
---
## H) Frontend Integration Contract
- Achievements page:
- one fetch on mount to `/achievements`
- optional background refresh indicator if stale
- no top-banner slot integration
- avoid duplicate in-flight calls during fast navigation by cancellation/debounce.
---
## I) Validation Checklist
- [ ] `/overview` route removed
- [ ] manifest has no `sessions:top`/`analytics:top` slots
- [ ] frontend has no `api("/overview")` calls
- [ ] repeated Achievements navigation does not create multiple heavy scans
- [ ] average warm load times meet SLOs
- [ ] unlock totals match pre-refactor baseline for same history
- [ ] no schema regression in `/achievements` response
---
## J) Suggested File Placement for Future Work
- backend changes: `dashboard/plugin_api.py`
- optional extraction:
- `dashboard/perf_snapshot.py`
- `dashboard/perf_scan_coordinator.py`
- frontend request hygiene: `dashboard/dist/index.js` (or source if available)
- plugin metadata: `dashboard/manifest.json`
- persisted runtime files:
- `~/.hermes/plugins/hermes-achievements/state.json` (existing unlock state)
- `~/.hermes/plugins/hermes-achievements/scan_snapshot.json` (new)
- `~/.hermes/plugins/hermes-achievements/scan_checkpoint.json` (new)
---
## K) Post-Implementation Reporting Template
Record:
- dataset size (sessions/messages/tool calls)
- pre/post `/achievements` timings (cold/warm)
- whether single-flight dedupe triggered under repeated tab open
- any behavioral diffs in unlock counts

174
plugins/hermes-achievements/docs/achievements-performance-spec.md
View File

@@ -1,174 +0,0 @@
# Hermes Achievements Performance Spec (Post-Hackathon)
Status: Draft (no code changes yet)
Owner: hermes-achievements plugin
Scope: `dashboard/plugin_api.py` + `dashboard/dist/index.js` request behavior
Decision: **Drop `/overview` and top-banner slots**; keep only Achievements tab data path.
---
## 1) Problem Statement
Current plugin endpoints `/achievements` and `/overview` both execute a full history recomputation (`evaluate_all()`), which performs a full SessionDB scan each request.
Observed on this machine/repo:
- ~83 sessions
- ~7,125 messages
- ~3,623 tool calls
- `evaluate_all()` ~13–16s per call
- `/achievements` ~13–15s per call
- `/overview` ~12–15s per call
- Overlap between endpoints increases perceived wait.
Given current product direction, `/overview` and cross-tab top-banner slots are not needed.
---
## 2) Goals
- Keep achievement correctness unchanged.
- Keep all Achievements-tab UX/data (unlocked/discovered/secrets/highest/latest/cards).
- Remove unused summary path (`/overview`) and slot wiring.
- Make Achievements tab faster by avoiding duplicate endpoint pathways.
- Ensure at most one heavy scan can run at a time.
Non-goals (phase 1):
- Rewriting achievement rules.
- Changing badge semantics/states.
---
## 3) Endpoint Semantics (Target)
### `GET /api/plugins/hermes-achievements/achievements`
Single source endpoint for Achievements UI.
Returns full payload used by the tab:
- `achievements`
- `unlocked_count`
- `discovered_count`
- `secret_count`
- `total_count`
- `error`
### `POST /api/plugins/hermes-achievements/rescan` (optional)
Manual refresh trigger.
Prefer async trigger + immediate status response.
### `GET /api/plugins/hermes-achievements/scan-status` (optional new)
Reports scan state for UX/ops.
### Removed
- `GET /api/plugins/hermes-achievements/overview`
---
## 4) UI Scope (Target)
Keep:
- Achievements page/tab (`/achievements` in plugin tab manifest)
- All existing Achievements tab stats/cards/filters
Remove:
- Top-banner summary slot components using `sessions:top` and `analytics:top`
- Any frontend call path to `/overview`
---
## 5) Runtime State Machine (for `/achievements`)
- `FRESH`: cached snapshot age <= TTL
- `STALE`: snapshot exists but expired
- `SCANNING`: background recompute running
- `FAILED`: last recompute failed, last good snapshot still served
Rules:
1. FRESH -> serve immediately.
2. STALE + not scanning -> serve stale snapshot immediately and launch background refresh.
3. SCANNING -> do not start another scan; join single-flight in-flight job.
4. No snapshot yet -> allow one blocking bootstrap scan.
---
## 6) Caching & Invalidation
### Phase 1
- In-memory cache + persisted snapshot file.
- TTL: 60–180 seconds (configurable).
- Single-flight dedupe for scan requests.
- Persist plugin data under:
- `~/.hermes/plugins/hermes-achievements/scan_snapshot.json`
### Phase 2
- Incremental scan checkpoints with per-session fingerprints.
- Persist checkpoint data under:
- `~/.hermes/plugins/hermes-achievements/scan_checkpoint.json`
- Checkpoint stores, per session:
- `session_id`
- fingerprint (`updated_at`, message_count, or hash)
- cached per-session contribution used for aggregate recomposition
- Scan policy:
- First run: full scan and materialize snapshot + checkpoint.
- Next runs: process only new/changed sessions, reuse unchanged contributions.
- Full rebuild only on:
- schema/version change
- checkpoint corruption
- explicit full rescan
---
## 7) Frontend Contract
- Achievements tab requests `/achievements` once on mount.
- No slot-based summary fetches.
- If response says `is_stale=true`, UI may display “Updating in background”.
- Avoid duplicate mount-triggered calls and cancel stale requests on navigation.
---
## 8) SLO Targets
- `/achievements` p95 < 1s (cached)
- Max concurrent heavy scans: 1
- Background refresh should not block UI
---
## 9) Observability Requirements
Track:
- scan count
- scan duration avg/p95
- dedupe hit count (joined in-flight scans)
- stale-served count
- failures + last error
Expose minimal diagnostics in `/scan-status`.
---
## 10) Backward Compatibility
- Keep `/achievements` response shape backward-compatible.
- Removing `/overview` is acceptable because slot UI is intentionally removed.
- If temporary compatibility is needed, `/overview` can return static deprecation response for one release.
---
## 11) Risks
- Stale data confusion -> mitigate with `generated_at` and explicit refresh status.
- Cache invalidation bugs -> start with conservative TTL + manual rescan.
- Concurrency bugs -> protect scan section with lock/single-flight guard.
- Session mutation edge cases -> use per-session fingerprint invalidation (not global timestamp only).
---
## 12) Persistence Files (Explicit)
Plugin state directory:
- `~/.hermes/plugins/hermes-achievements/`
Files:
- `state.json` (existing): unlock tracking
- `scan_snapshot.json` (new): latest materialized achievements payload
- `scan_checkpoint.json` (new): per-session fingerprints + contributions for incremental refresh