fix(relay): native Anthropic stream spans keep their input and cache token counts

On the native Anthropic stream, every Relay LLM span exported prompt, cache-read and
cache-write token counts as missing; only output tokens survived. The SDK's
MessageDeltaUsage serializes its unset fields (input_tokens, cache_read_input_tokens,
cache_creation_input_tokens) as null, and AnthropicStreamAccumulator's message_delta merge
let those nulls overwrite the real values captured from message_start. The Anthropic SDK's
own accumulator merges only non-null delta fields; ours now does the same. Hermes' own
session accounting was unaffected (it reads the SDK snapshot); the loss was the Relay
trace, which is what cache-reuse analysis of exported traces runs on.
This commit is contained in:
kshitijk4poor
2026-10-07 00:06:07 +05:30
committed by kshitij
parent 9ab4a4fdbb
commit 3fc3caeb98

View File

@@ -648,7 +648,9 @@ class AnthropicStreamAccumulator:
if "usage" in payload:
usage, current_usage = payload["usage"], self._message.get("usage")
if isinstance(current_usage, dict) and isinstance(usage, dict):
usage = {**current_usage, **usage}
# The SDK's MessageDeltaUsage serializes unset fields as null; they must not
# erase message_start's input/cache counts.
usage = {**current_usage, **{k: v for k, v in usage.items() if v is not None}}
self._message["usage"] = usage
_EVENT_HANDLERS = {