The agent flush row (_db_flush_row) always sets token_count, None for a replayed
message, so keying the keep on the key's absence missed the main production writer
and a digest-matched rewrite still nulled the stored count. Key it on the value.
Replay dicts from _rows_to_conversation never decode token_count, so the
serialized row always carried NULL. On the unaddressed (message_uid + CAS
snapshot) path that meant every legitimate rewrite nulled the original row's
token_count, and a dict whose content was unchanged still differed in
token_count and triggered a needless _rewrite_row. A missing key means
"unknown", so keep the stored value before comparing owned columns.
Also reuse message_metadata.message_uid_or_none at the lookup call site instead
of a second hand-rolled uid guard, and reflow an over-wide comment.
Stamping DB_ROW_SNAPSHOT on every repair_alternation load also moved the
include_row_ids=True resume loaders (resume conversations, rewind, TUI
gateway, turn lease) onto the CAS-match rewrite path. That path writes
every owned column from the live dict, and replay never decodes
token_count, so a rewrite of a resumed row nulled the stored
token_count. Only the unaddressed live replay needs the digest (it is
the proof that recovers a lost _row_id), so stamp it only when
repair_alternation and not include_row_ids.
The token_count column stays in _CONVERSATION_ROW_COLUMNS: the digest
hashes every owned column of these rows, so the projection must stay a
superset of _OWNED_COLUMNS; the comment now states that contract.
Also index (session_id, message_uid) over active rows so the logical
identity lookup, which runs inside the write lock, is a SEARCH instead
of a full-session scan; tighten the twin test so a copied uid without a
snapshot is still not adopted.
Found by the pre-merge correctness and hermes-specific review of this stack:
- A fold after a row whose one response repeated a provider id (one shared uid) appended the
absorbed turn's uid in the wrong slot. The shared uid now fills each of the row's own
occurrences before the fold appends (per_occurrence_tool_call_uids). A single response keeps one
shared uid for a repeated id: every result carries it, so no call looks unanswered.
- A digest-less re-flush of a restored fold (replay heal, adopt after a digest mismatch) let the
stored pre-fold map overwrite the longer live list, so the later occurrence's results paired
with nothing. A live list that starts with every stored occurrence is kept (stored uid or list).
- A dict that lost its uid map (a clone) rewrote tool_call_uids to NULL. The row-addressed
rewrite now fills stored uids for the calls the dict still names; a call it dropped keeps none.
- append_messages_batch(chunk_rows=...) reset the pairing index per chunk, so a result in the chunk
after its call was stored without tool_call_uid (branch/seed copies). The whole batch is paired
before it is chunked.
- _merge_assistant_into keeps its dict guard on the absorbed turn's map (a plugin-built dict with a
non-dict map no longer raises).
- without_persistence_fields() replaces the two hand-written strips (context-engine selection,
token-estimate fingerprint).
- mint_uid() is the one uid minter; uid_list() the one uid-list normalizer (message_metadata's
_absorbed_uids and the codec's _uid_list share it).
- _live_or_column() replaces four copies of 'live key first, column name second'; the absorbed
list writer takes the dict, not a (live_key, column) pair its one caller always passed the same.
- _restore_row_identity() is the one stored-identity-onto-live-dict step, shared by restore and
the row-addressed rewrite (transcript_repair had a hand copy). Restore skips the decoders for
NULL columns (the common row).
- index_tool_call_uids() returns the variants it named, so the flush walk stops computing them
twice; tool_call_uid_from_history drops guards its one caller can't trip.
- rewind returns the replacement dict (the insert already stamps _row_id and message_uid)
instead of re-querying last_insert_rowid.
- _remember_absorbed_row's folded is required, so every call site states the witness rule.
- The backfill reads/deletes state_meta through _meta_row/_delete_meta; chat_completions strips
MESSAGE_UID by constant.
No behaviour change.
hermes_state_messages.py crossed 2000 lines with the v31 identity columns. The codec between
the live identity keys and the four columns (_uid_list/_uid_map and their JSON writers, the
restore decoder) moves to its own sibling; transcript repair imports _uid_map from there.
_json_or moves to hermes_state_common so both modules share it (same 'hermes_state' logger).
No behaviour change.
The salvage carried two copies of the uid reader (message_metadata._uid_or_none and
hermes_state_messages._message_uid_or_none), two tolerant uid-map parsers
(transcript_repair._json_map and hermes_state_messages._uid_map), and the same four
identity-field copy blocks in _db_flush_row and sync_flushed_message_markers.
message_metadata now owns message_uid_or_none, stamp_message_uid (moved off SessionDB so
non-store callers can mint before a write) and copy_identity_fields; transcript_repair
reuses _uid_map.
- Strip persistence-only fields from a select_context() selection: the request copy was stripped before
the hook, but an engine handing back the conversation_messages clones put timestamp, _row_id and the
new ids on the wire (pre-existing for the old fields).
- Every row carries a message_uid whoever wrote it: an AFTER INSERT trigger mints one for a row inserted
without it (an older build writing into a v31 store); the one-time backfill is gated by its own
state_meta marker so a store whose schema version cannot advance (no FTS5) never rescans.
- One merge-witness helper (record_absorbed_message) for every fold of a durable message: alternation
repair's user and assistant merges, the compressor's in-flight restatement onto the carrier, the real
user anchor folded into a scaffolding turn (the anchor's uid leads), micro-compaction's adjacent-user
merge.
- tool_call_uids follows tool_calls: it is a payload column, so an assistant merge's unioned map is
rewritten with the row instead of being overwritten by the stale stored map on adoption.
- A result pairs with the nearest call naming its provider id: each assistant row's ids shadow earlier
occurrences in the batch, restore and live-list indexes, a call row without a map leaves its result
unpaired, and nothing pairs across a user turn.
- A composite rewind returns and installs the replacement row's uid on the live scaffold.
- Tests: import_foreign_history, hermes sessions recover and backup import carry the ids; the ACP
restore test accounts for the uid on restored rows.
(cherry picked from commit b30636a49cb29fdd21ba4100eef9d810897338be)
Provider tool-call ids repeat: Hermes mints deterministic call_<12hex> ids for identical calls and models
reuse ids, so tool_call_id cannot identify an occurrence. An assistant row now carries
messages.tool_call_uids, a {tool_call_id: uuid4} map for its tool_calls, and each tool-result row carries
the matching messages.tool_call_uid. The provider-facing tool_calls JSON is untouched, so row identity,
display identity and the CAS digest are unchanged and nothing nested reaches the wire.
Minted at the assistant row's first insert; paired onto the result in the same batch, from the live list
when the result is flushed later, or on restore from the preceding assistant row; copied by every clone
and re-insert; restored as the live _tool_call_uids / _tool_call_uid; stripped from provider requests.
(cherry picked from commit aa49c0f59788676d4beca288181711c558fca165)
A merge survivor's _absorbed_message_uids lived only in memory, so an engine that restarted after a
merge was back to parsing "\n\n" to learn which stored messages a composite user row contains. Store
the list in messages.absorbed_message_uids (an owned column: the survivor's row-addressed rewrite and
every re-insert carry it), restore it as the live key, and round-trip it through export/import.
(cherry picked from commit 0bc05f2656229b1010f330e1998684b04d5f963f)
The physical messages.id is re-issued by every copy (in-place compaction generations and their
concurrent-tail clones, rotation children, replace_messages), and _row_id is opt-in on restore, so a
context engine that keeps its own per-message state had nothing stable to key on across a restart or a
compaction boundary and re-identified rows by content and timestamp.
Add messages.message_uid (uuid4 hex), minted once at a row's first insert and stamped on the caller's
dict, copied by every SQL clone, kept by every re-insert of the same dict, left alone by row-addressed
rewrites, restored on every projection, and stripped from provider requests. A consecutive-user merge
survivor keeps its own uid and records the absorbed uids in _absorbed_message_uids. Schema v31 backfills
a uid onto rows that predate the column.
(cherry picked from commit 20d67ffb5e50e0ed956e56ddc7f8e28d36059e2e)
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule
The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:
- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
names, restored dead definitions) and the three re-export stub modules
(gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
`plugins.allow_deprecated_imports` escape hatch
An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.
hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).
In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.
* chore: retrigger CI (zero-job startup_failure phantom)
* test: drop resolution allowlist rows for the two deleted which() sites
hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
_insert_message_rows stamps _row_id and the stored-row digest onto the
caller's dicts inside the write transaction. Only append_messages_batch
restored that state on rollback. archive_and_compact, replace_messages
and the rotation handoff left the rolled-back id + digest on the dicts;
SQLite reuses the id, so a later flush found a digest mismatch on the
foreign row, adopted it and silently dropped the user's message.
Move the capture/restore into _execute_transcript_write, used by every
caller that inserts caller-owned dicts: each attempt starts from the
caller's state and a final failure restores it before re-raising.
(Rewind replacement and import insert dicts built inside the txn.)
Also: bind _message_row_params directly on insert instead of the
serialized-dict round-trip, import the public DB_ROW_SNAPSHOT /
CANONICAL_ROW names, set adopt=False once, and reuse target_row
instead of re-SELECTing when nothing was written.
A legacy (no-digest) dict over a non-blank assistant row adopted the whole
decoded DB row: tool_calls / reasoning* / codex_* were overwritten with the
stored JSON (which still holds the escaped lone surrogate the sanitizer just
fixed, re-injecting it into the provider payload) and live-only fields were
popped. Resumed dicts (_rows_to_conversation stamps _row_id without a
digest) and compaction clones hit this path. Adopt content only, as before
this stack, via a content-only canonical handled like the metadata-only one.
_insert_message_rows dropped a clone's parent digest but only the flush
path restamped it, so clones made by archive_and_compact / replace /
rotation handoff / import reached the legacy path and the first live edit
after a clone was not persisted. Stamp the stored-row digest inside
_insert_message_rows (one batched SELECT, cold paths only; the flush path
statement count is unchanged) and drop the duplicate call in
append_messages_batch.
Define the _db_row_snapshot / _canonical_row keys once in
agent/message_metadata.py and import them everywhere instead of repeating
the literals.
The row digest hashed every repair column, so a same-process metadata write
(reaction, display-kind stamp, api_content / codex reasoning backfill,
platform message id) made our own row look like a foreign winner. The
re-flush then adopted the stale DB row: a later live edit (the non-ASCII
strip recovery) was reverted, and unsanitized tool_calls/reasoning were
copied back onto the live dict.
The digest now covers only the owned (non-metadata) columns: it means "the
row is still what we last committed". Match -> write the live owned values
and hand over only presentation metadata the live dict lacks; mismatch ->
genuine other writer, adopt as before. The r3 "stored content equals the
durable form of live" special case is subsumed and removed.
Also:
- _insert_message_rows drops a carried digest when it assigns a new row id
(compaction/replace/import clones carried the parent's version).
- append_messages_batch restores each message's _row_id / digest /
timestamp and pops the adopted row at the top of every _execute_write
attempt, so a rolled-back attempt cannot resolve to a foreign row.
- message_id is no longer synced onto live (int -> str flip, spurious
platform_message_id).
- The JSONL divert strips both bookkeeping keys via one frozenset.
Same-process writers (set_message_reaction, display-kind stamping, api_content
backfills, codex reasoning update) change stored columns after a flush without
refreshing the live row digest. The next sanitize + re-flush treated that as a
concurrent winner and copied the lossy durable projection over live multimodal
content, dropping image parts and shifting the prompt-cache prefix. On adoption
we now keep live content when the stored content is just the durable form of
it and sync metadata only; a real concurrent content winner is still adopted.
The legacy blank-assistant path (dict with _row_id but no digest, blank DB row)
again only fills the row from live content, as on main, instead of running the
full canonical sync that wiped live reasoning_content/finish_reason/tool_calls.
Adoption on the legacy path is limited to a non-blank row.
Also: transcript_row_snapshot returns str (the partial-row branch had no
caller), serialization only runs on the digest-match branch that reads it,
stamping reuses hermes_state_common._id_chunks, and _MESSAGE_WRITE_COLUMNS is a
plain top-level import (hermes_state_messages imports this module lazily, so
there is no cycle).
The row-addressed repair stamped the decoded durable row on every
resolved message, so after our own rewrite the sync copied the lossy
durable projection (image parts -> "text\n[screenshot]") back onto the
live dict: multimodal user/tool messages lost their images and the
prompt-cache prefix changed. Adopt the DB row only when another writer
won (digest mismatch) or on the legacy assistant path, as BASE did.
The insert-time digest hashed Python bind values, but SQLite affinity
rewrites them on storage (int message_id -> TEXT, float token_count ->
INTEGER), so live and DB digests never matched and in-place edits were
silently dropped. Hash the stored rows instead, only on the
append_messages_batch flush path that reads the digest (one SELECT per
batch), incrementally (type tag + length prefix) instead of via JSON.
Also skip the no-op UPDATE, fix the _write_columns comment/spacing and
drop the duplicate top-level Optional import (F811).
The CAS row snapshot was a full copy of each message's durable payload
riding on the live dict. The rough token estimator priced it (about 2x
estimates -> premature compaction) and it doubled transcript memory.
Replace it with a 16-byte blake2b digest of the repair columns. The
compare now runs in Python against the target row already read inside
the BEGIN IMMEDIATE transaction, followed by a plain UPDATE. Also:
- add _db_row_snapshot to PERSISTENCE_ONLY_MESSAGE_FIELDS so the
estimator and the outbound request builder both drop it
- derive _REPAIR_COLUMNS/_SYNC_FIELDS from _MESSAGE_WRITE_COLUMNS
- use hermes_state_common._placeholders
- drop the dead resume-path stamp (the SELECT has no token_count, so it
was always None) and the dead tool name assignment in
_decoded_repair_row
- keep the digest out of divert JSONL
The kept active-row test now pins estimate stability across a flush and
the survival of a concurrent writer's row. It goes red on the old
prod files and red when the digest compare is removed.
transcript_row_snapshot annotates Optional, which was only reachable via the
PLUGIN-COMPAT re-export block at the bottom of the module. Internal code must
not depend on that revert-scheduled block, so import it with the other typing names.
The Sep 2026 decomposition (PR #102117) makes internal import paths a non-API: names now live in
the focused modules that define them. This commit is the ONLY thing keeping the old paths alive,
so external plugins have time to update. It is deliberately a single, unsquashed commit:
git revert <this sha>
removes every shim, stub and manifest at once on the announced date. Nothing in-tree may depend on
these pointers: scripts/check_compat_pointers.py (wired into lint.yml) fails CI if it does.
What it adds (see COMPAT_MANIFEST.md, compat_manifest.json):
- 332 facade modules get one delimited `PLUGIN-COMPAT` block appended at the end of the file
- 1,172 moved names resolved lazily via a module `__getattr__` (PEP 562) — never a top-level import,
so no import cycles; facades that already had `__getattr__` get a chained one
- 592 third-party/stdlib names the old modules used to expose, with their original import statements
- 266 public definitions that had been deleted as unused, restored byte-for-byte from the pre-decomposition
tree (+40 private helpers and 16 imports pulled in only because a restored definition needs them)
- 3 deleted modules recreated as re-export stubs (gateway/startup_watchdog, hermes_cli/observability/
relay_runtime, tools/environments/modal_utils)
- private names (`_x`) get no pointer: they were never API (3,792 skipped)
Verified: all 335 touched modules import under a fresh HERMES_HOME and every manifest name resolves;
the lint reports zero in-tree uses; ruff clean; targeted suites unchanged.
- Preserve streamed assistant text in Desktop UI when message.complete delivers empty text.
- Prevent destructive hydration in Desktop useMessageStream over rendered text on empty completion.
- Recover stream buffer in finalize_turn when final_response is empty on healthy turns.
- Unify in-place blank assistant repair, watermark clone resolution, non-blank concurrent winner adoption, and batch row appends into a single atomic guarded SessionDB transaction.
- Synchronize canonical committed content to live in-memory messages dicts and preserve all-or-nothing rollback semantics on persistence failure.
