fix(relay): native Anthropic stream spans keep their input and cache token counts
On the native Anthropic stream, every Relay LLM span exported prompt, cache-read and cache-write token counts as missing; only output tokens survived. The SDK's MessageDeltaUsage serializes its unset fields (input_tokens, cache_read_input_tokens, cache_creation_input_tokens) as null, and AnthropicStreamAccumulator's message_delta merge let those nulls overwrite the real values captured from message_start. The Anthropic SDK's own accumulator merges only non-null delta fields; ours now does the same. Hermes' own session accounting was unaffected (it reads the SDK snapshot); the loss was the Relay trace, which is what cache-reuse analysis of exported traces runs on.
This commit is contained in:
@@ -648,7 +648,9 @@ class AnthropicStreamAccumulator:
|
||||
if "usage" in payload:
|
||||
usage, current_usage = payload["usage"], self._message.get("usage")
|
||||
if isinstance(current_usage, dict) and isinstance(usage, dict):
|
||||
usage = {**current_usage, **usage}
|
||||
# The SDK's MessageDeltaUsage serializes unset fields as null; they must not
|
||||
# erase message_start's input/cache counts.
|
||||
usage = {**current_usage, **{k: v for k, v in usage.items() if v is not None}}
|
||||
self._message["usage"] = usage
|
||||
|
||||
_EVENT_HANDLERS = {
|
||||
|
||||
Reference in New Issue
Block a user