4854225903
The CAS row snapshot was a full copy of each message's durable payload riding on the live dict. The rough token estimator priced it (about 2x estimates -> premature compaction) and it doubled transcript memory. Replace it with a 16-byte blake2b digest of the repair columns. The compare now runs in Python against the target row already read inside the BEGIN IMMEDIATE transaction, followed by a plain UPDATE. Also: - add _db_row_snapshot to PERSISTENCE_ONLY_MESSAGE_FIELDS so the estimator and the outbound request builder both drop it - derive _REPAIR_COLUMNS/_SYNC_FIELDS from _MESSAGE_WRITE_COLUMNS - use hermes_state_common._placeholders - drop the dead resume-path stamp (the SELECT has no token_count, so it was always None) and the dead tool name assignment in _decoded_repair_row - keep the digest out of divert JSONL The kept active-row test now pins estimate stability across a flush and the survival of a concurrent writer's row. It goes red on the old prod files and red when the digest compare is removed.
45 lines
1.5 KiB
Python
45 lines
1.5 KiB
Python
"""Internal metadata attached to durable conversation messages."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from time import time as wall_time
|
|
from typing import Any, MutableMapping, Optional, TypeVar
|
|
|
|
|
|
# These fields describe Hermes' durable record and timeline display, not
|
|
# provider-visible message content. The request builder strips them from every
|
|
# outgoing copy and the token estimator ignores them: one set, so an estimate
|
|
# never prices bytes the provider never receives (an edit's inline_diff in
|
|
# display_metadata is ~9KB and would trigger premature compaction).
|
|
PERSISTENCE_ONLY_MESSAGE_FIELDS = frozenset(
|
|
{"timestamp", "display_kind", "display_metadata", "_row_id", "_db_row_snapshot"}
|
|
)
|
|
|
|
_Message = TypeVar("_Message", bound=MutableMapping[str, Any])
|
|
|
|
|
|
def stamp_message_timestamp(
|
|
message: _Message,
|
|
*,
|
|
timestamp: Optional[float] = None,
|
|
) -> _Message:
|
|
"""Attach a creation timestamp without replacing source-provided time.
|
|
|
|
Gateway adapters can supply the platform event time; all other callers use
|
|
the local wall clock. Returns the same mapping for use at append sites.
|
|
"""
|
|
if message.get("timestamp") is None:
|
|
message["timestamp"] = wall_time() if timestamp is None else timestamp
|
|
return message
|
|
|
|
|
|
def append_message(
|
|
messages: list[Any],
|
|
message: _Message,
|
|
*,
|
|
timestamp: Optional[float] = None,
|
|
) -> _Message:
|
|
"""Stamp and append one live transcript message."""
|
|
messages.append(stamp_message_timestamp(message, timestamp=timestamp))
|
|
return message
|