The micro-compaction defrag pass (_defrag_rolling_summary) rewrites the
newest MICRO marker's content and pops _DB_PERSISTED_MARKER from the
LIVE dict in place — the same in-place pop class finalize_turn's fill
site was fixed for in #75170. Without invalidation the bounded
flush-scan cursor identity-skips the rewritten marker row and the
defragged rolling summary never reaches state.db (resume rehydrates a
stale summary).
The compressor holds no agent reference, so the pop site raises
_flush_scan_cursor_invalidated and the finalize_turn micro-compaction
block consumes it, setting agent._db_flush_scan_prefix = None.
The module-scope pop sites (context_compressor.py:175/224) operate on
fresh copies — identity-breaking by construction — and need no flag.
Follow-up to #75170 (fix-the-class sweep of _DB_PERSISTED_MARKER
in-place pops).
The bounded flush-scan in _flush_messages_to_session_db_unlocked skips
the identity-matched prefix of its previous snapshot, on the documented
assumption that no code path pops _DB_PERSISTED_MARKER from a live dict
in place. finalize_turn's pure-tool-call-tail fill is exactly that path:
it pops the marker so the filled content gets re-persisted — but the
cursor then skips the row anyway, so the delivered final response never
reaches state.db and /resume replays content="" (the #43849/#44100
class resurfacing via the perf cursor). Invalidate the cursor at the
pop site so the filled row is re-examined.
Companion to the preflight fix: _estimate_msg_budget_tokens charged
reasoning_details at chars/4 via _REPLAY_BUDGET_KEYS, so the signed/base64
envelope (measured 72% of the reasoning mass on Anthropic-wire sessions)
consumed the tail budget and _find_tail_cut_by_tokens summarized away real
transcript to make room for tokens that are never sent (69 messages on the
measured session; up to ~4.8x budget inflation on thinking-heavy histories).
Per the #51800 counter-argument, actual thinking TEXT stays visible to the
budget: _reasoning_details_text_chars counts thinking/text/summary fields
and skips signature/data/encrypted blobs, and the text is skipped entirely
when reasoning/reasoning_content already carries the identical prose (so it
is charged once, not twice). codex_reasoning_items remains fully charged —
Codex Responses genuinely replays it every request (#55572).
Sabotage-verified: restoring reasoning_details to _REPLAY_BUDGET_KEYS fails
the new envelope and double-charge tests.
The reasoning_details field (OpenRouter/Anthropic thinking blocks +
opaque cryptographic signature blobs) inflates the rough token estimate
by ~4x. Providers do not bill these envelope bytes as prompt tokens.
In a measured Kimi K3 session, reasoning_details held 2,124K chars
vs 281K chars of actual thinking text. The estimator reported ~533K
tokens when real prompt_tokens was ~140K — triggering compression at
~27% of the configured threshold.
Fix: skip reasoning_details in both _estimate_message_chars and
_estimate_message_tokens_without_images, alongside the existing
_anthropic_content_blocks exclusion.
Fixes#73298
Fix#75588
## Root cause
When a short conversation ends in a tool-call/result group and the
protected head alignment reaches the end of the message list,
_find_tail_cut_by_tokens() could return len(messages) + 1. This
happened because the final return used max(cut_idx, head_end + 1)
which could push past the array length when head_end >= len(messages).
The out-of-range value then propagated into _find_context_summaries()
which iterated range(start, end) and indexed messages[idx] without
clamping, raising IndexError and failing the active gateway turn.
## Fix
Two-layer defense:
1. Source fix: _find_tail_cut_by_tokens() now clamps its return to
min(n, ...) so it never exceeds len(messages).
2. Defensive clamp: _find_context_summaries() now bounds start/end
to [0, len(messages)] so even if a future caller passes bad values,
it cannot crash.
## Verification
- 7 new regression tests for the exact boundary conditions
- All 214 existing test_context_compressor.py tests pass
Review follow-up on #75102. The shadow substituted the sidecar whenever
the ``api_content`` key was merely PRESENT, but the wire only substitutes
a non-empty string sidecar on a user/assistant row (see
``turn_context.substitute_api_content``). For any other shape the sidecar
is popped and discarded while the clean ``content`` is sent -- so the
shadow dropped real content from the estimate and UNDERcounted, the
dangerous direction: compaction fires too late and the turn dies on a
hard context-length error instead of merely compressing early.
Gate the substitution on the same predicate, and cover the divergent
shapes (None, empty string, int, list, non-user/assistant role) with a
test that fails against the unconditional version.
Also rename the image test: it never carried a sidecar, so it was not
testing what its name claimed. It is a non-regression pin on the flat
per-image accounting that moved into ``_wire_message_shadow()``, and is
now named for that.
`api_content` is a SUBSTITUTE for `content`, not an addition to it.
`turn_context.substitute_api_content()` pops the sidecar and overwrites
`content` at every API-bound message-build site (the `api_messages` build
in `conversation_loop`, the max-iterations summary in
`chat_completion_helpers`, the chat-completions transport), so exactly one
of the two is ever sent to the provider.
The preflight estimator counted both, because both `_estimate_message_chars`
and `_estimate_message_tokens_without_images` walked every key of the
persisted dict with a single-entry denylist (`_anthropic_content_blocks`).
Any message whose sidecar differs from its clean stored content was counted
twice — exactly 2.00x on a 40KB sidecar.
The sidecar exists to keep the provider prompt-cache prefix byte-stable, so
it is written on precisely the long, cache-pinned messages where the
doubling hurts most. Because `estimate_messages_tokens_rough()` also feeds
the compaction threshold via `context_compressor` and `conversation_loop`,
the inflated estimate makes compression fire on phantom bytes.
Fix: substitute rather than sum, mirroring the wire. The two estimator
helpers had drifted into near-identical copies of the same shadow-building
loop, so this factors the shared logic into `_wire_message_shadow()` and
fixes the class once instead of patching one site and leaving the other.
Image accounting is unchanged: base64 payloads are still replaced with a
placeholder and charged at the flat `_count_image_tokens` rate, and the
`_multimodal` text_summary path is preserved.
Tests: three cases in `TestEstimateMessagesTokensRough` — sidecar equal to
content is counted once, a sidecar that DIFFERS is still counted (a lower
bound, so it fails if the field were dropped rather than substituted, which
would undercount the real request), and a sidecar cannot smuggle raw base64
past the flat image rate.
Verified on Linux (Python 3.11): 53 passed in
tests/agent/test_model_metadata.py, 57 passed with
tests/agent/test_context_breakdown.py, 656 passed / 3 skipped across the
compression/context/token/estimate/prune surface of tests/agent.
Mutation-tested: reverting the substitution fails the new equality test.
`scripts/check-windows-footguns.py` is not applicable — no file I/O,
process management, terminal handling, subprocesses, or signals.
agent_init.py's init-time fallback and chat_completion_helpers.py's
try_activate_fallback() still read key_env via raw os.getenv(), missing the
per-profile secret scope installed by the multiplexed gateway (same bug
fixed for fallback_config.py/auxiliary_client.py in this PR). Both now
delegate to hermes_cli.fallback_config.resolve_entry_api_key(), and the
Ollama Cloud OLLAMA_API_KEY read now goes through
agent.secret_scope.get_secret() too.
agent_init.py's fallback loop had no try/except around key resolution
(unlike the other three call sites), so a fail-closed UnscopedSecretError
under multiplexing would have crashed init instead of skipping to the next
fallback entry — added the same skip-and-continue handling.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
resolve_entry_api_key() and the duplicated _fallback_entry_api_key()
read key_env via a raw os.getenv(), bypassing per-profile secret
scoping in the multiplexed gateway. Under multiplexing this can hand
a fallback request another profile's credential. Both now resolve
through agent.secret_scope.get_secret(), which reads the active
profile scope when multiplexing is on and falls back to os.environ
unchanged when it's off, so single-profile behavior is preserved.
Closes#74311
shutil.move() moves into an existing destination directory instead of
replacing it. When the snapshot extract failed after creating some of its
output, the recovery path moved each staged entry onto a path the extract
had already created, burying the user's own skill one level deeper
(skills/alpha/alpha/) and leaving the snapshot's partial content in its
place. rollback() then returned "snapshot extract failed (state restored)".
Clear the failed extract's output before moving the staged copies back:
entries the original tree never had are dropped, and each staged entry's
destination is removed before the move. When an entry still cannot be
restored, keep the staging dir and name the entries in the message rather
than reporting a restore that did not happen.
AIAgent.__init__ calls set_current_session_id(self.session_id), which
mutated both the task-local ContextVar and the process-global os.environ.
Because _build_child_agent wraps construction in delegated_child_context(),
the ContextVar write is harmless (task-local), but the os.environ write
clobbered the parent's HERMES_SESSION_ID for the rest of the process —
leaking the child id into parent tools and subprocesses spawned after
the child was built.
Root cause of HermesPRDelegationSessionContext: parent
20260729_212118_5d797e dispatched child 20260730_160515_736ea1; later
parent terminal inherited HERMES_SESSION_ID=the child.
Fix: set_current_session_id() skips the process-global os.environ write
when called from within a delegated_child_context(). The child's own
tools and subprocesses still resolve their id through the ContextVar
(task-local), while the parent's process-wide env keeps the parent's
session identity. Root agents (CLI, gateway, cron) retain both paths.
Adds 7 regression tests covering single child, concurrent children (8
parallel), parent-tool observation after construction, and root-agent
session rotation backward compatibility. All pass; ruff clean.
_sync_device_code_entry_to_auth_store used key-presence on the profile
store to decide whether to write-through rotated tokens to the global
root. _store_provider_state unconditionally creates that key, so every
refresh after the first self-disabled the write-through — root kept a
revoked refresh token and every other profile died with
refresh_token_reused / invalid_grant.
Fix: use _load_provider_state_with_source to learn where the grant was
resolved from. When the source is the global root, write back only to
root and skip _store_provider_state so the profile never accrues a
shadowing providers.<id> key that blocks future root fallback.
Add regression test verifying write-through fires on refresh 2+, not
just the first call.
- Bound ALL reads of the on-disk JWT store through one _read_jwt_store()
helper (load, eviction, save-merge) — the 1 MiB cap previously only
covered the load path; eviction and save could still parse an
oversized/corrupt store and rewrite it back out (sweeper finding).
- Fix the class, not the site: the recovery gates checked the literal
provider == "copilot" while /model and profile configs can leave the
alias spelling in place (the reporter's own log shows provider=copilot
AND provider=github-copilot in one session — the aliased turns would
have silently skipped recovery). Single owner:
AIAgent._is_copilot_provider() (slug aliases + Copilot base-URL
fallback), used by both run_agent recovery methods and both
conversation_loop gates.
- Update the salvaged 401 test to current main's client-retirement
contract (release deferred to GC — no synchronous .close()).
- Add copilot_stale_cred_retry_attempted to the TurnRetryState field
contract test; add bounded-store and alias-gate regression tests.
Copilot degrades in two related ways that both abort a turn as non-retryable
and only clear on a gateway restart (a cold process re-runs the token exchange):
1. HTTP 400 model_not_available_for_integrator / model_not_supported — a
raw/degraded token routes to the restricted copilot-language-server
integrator whose allowlist omits enterprise-only models (e.g.
claude-opus-4.8). Because it is a 400 (not 401), the existing 401 refresh
path never fired. Prevented (retry-with-backoff exchange + on-disk JWT
persistence + header guard at the client chokepoint) and self-healed at
runtime (single-shot forced re-exchange + client rebuild + retry before
fallback).
2. HTTP 401 'IDE token expired: unauthorized: token expired' — the short-TTL
*exchanged* IDE token expires mid-turn. The clean-401 path DID fire and call
_try_refresh_copilot_client_credentials(), but that method only re-resolved
the stable raw ghu_ token and rebuilt the client — it never evicted the
cached exchanged JWT or forced a fresh exchange, so the retry put the SAME
expired token back on the wire, 401'd again, and the single-shot guard
aborted the turn. Fix: force a fresh IDE-token exchange (evict cached JWT via
evict_cached_exchanged_token + re-mint via get_copilot_api_token) before the
client rebuild, mirroring the merged auxiliary-path recovery (#59837) and the
400 recovery in this same PR. Graceful fallback to the resolved token if the
exchange endpoint is unreachable; picks up the enterprise base_url on
re-exchange.
Brings main-loop clean-401 recovery to parity with the merged auxiliary path
(#59837), using the newer on-disk-aware evict helper. Companion context: #58743
(this PR, expanded), #51313, #63204 (which assumed the 401 path already
recovered — it reached the method but the method was too weak).
Tests: exchange retry/persist round-trip, restart-blip disk reuse, stale-cred
400 classifier, 400 recovery, and 3 new 401 cases (fresh exchanged token on the
wire; network-blip fallback to resolved token). 58 copilot tests green on
current main.
Disk-full / ENOSPC / SQLITE_FULL on first-message session persist used to be
swallowed as a debug log while prompt.submit still returned streaming, so the
send vanished with no error. Re-raise those failures, return a real RPC error,
and stamp session_persistence_failed turns with error so clients get a terminal
error frame.
build_moa_facade() reused agent.model as the preset name; a session
that had drifted to a fallback model crashed on restore with
MoAPresetNotFoundError. Validate the resolved preset against the
configured presets and fall back to the default preset.
Salvaged from PR #74903 by @liusencomic-cyber.
Narrow the moa_aggregator Relay bypass to CodexAuxiliaryClient and add
coverage that call_llm(stream=True) returns the provider's direct
create() result for Responses-shim clients.
Salvaged from PR #74903 by @liusencomic-cyber.
Convert a completed MoA aggregator response into one valid Chat
Completions delta chunk at the MoA facade boundary, normalize completed
message.tool_calls into indexed stream deltas, and classify these local
MoA adapter-shape errors as non-fallback format errors so a local
compatibility bug cannot silently drift the user's MoA route to a
single model (#55933 follow-up).
Salvaged from PR #74903 by @liusencomic-cyber.
Under managed Relay execution the provider factory runs lazily inside
provider_stream() on the Relay session's event loop. The MoA facade's
auxiliary call_llm(stream=True) is invoked from that callback, so the
eager final_response check added for the non-managed path never fires:
the inner ManagedLlmStream is returned to the outer stream, which then
synchronously iterates it on the same loop thread and dies with
RuntimeError: Cannot run the event loop while another loop is running
(the completed response effectively trapped one level deeper).
stream_current() now detects a running event loop and returns the raw
factory result instead of nesting a ManagedLlmStream: the outer managed
stream already provides Relay tracking for the enclosing attempt, and
its own completed_response_predicate traps the completed response as
final_response — the same contract the main streaming worker consumes
(chat_completion_helpers reads stream.final_response after the chunk
loop). Nested managed streams remain supported for genuinely streaming
providers via the outer stream's own iteration.
Adds managed-execution regressions using the retained relay_turn
fixture: direct completed-response trapping, and the nested
facade-shaped stream_current call.
Port from nanocoai/nanoclaw#2748: Docker's built-in 64 MB /dev/shm silently
breaks shared-memory-hungry workloads inside the sandbox — Chromium/Playwright
renderers crash tabs, and PyTorch DataLoader workers die with 'bus error' /
'insufficient shared memory'. tmpfs is lazily allocated, so the higher ceiling
costs nothing until actually used, and usage still counts against the
container's --memory cgroup limit.
- tools/environments/docker.py: --shm-size 1g in resource args (not
cgroup-gated; tmpfs mount option). Skipped when docker_extra_args already
sets --shm-size, or when configured empty/'0' (Docker default).
- terminal.docker_shm_size config key (DEFAULT_CONFIG + all three
config->TERMINAL_DOCKER_SHM_SIZE env bridges: CLI, gateway, config.py map)
- tests: default emit, custom value, opt-out, extra_args precedence,
helper edge cases (sabotage-verified: default/custom tests fail without
the emit)
Port from openclaw/openclaw#112954. The redactor knew GitHub, Slack,
Google, Stripe, AWS access-key-ID and ~25 other vendor prefixes but had
zero GitLab coverage — glpat-/gloas-/gldt-/glrt-/glrtr-/glcbt-/glptt-/
glft-/glimt-/glagent-/glsoat-/glffct-/glwt- tokens and legacy GR1348941
runner registration tokens passed through display and log surfaces
verbatim. Follow-up explicitly invited when #4541 was closed.
Each pattern keeps a full literal prefix so the _PREFIX_SUBSTRINGS
pre-screen (derived at module load) stays false-negative-free; routable
runner tokens allow dotted segments. Sibling site: skills_guard's
credential-exposure scan gains a gitlab_token_leaked pattern.
Phase 2 review findings on the salvage branch:
C1 (critical): batch and micro summary markers share
COMPRESSED_SUMMARY_METADATA_KEY, and compress() never reset micro state.
After micro absorbed exchanges 1..k, a batch compaction summarizing
1..m (m>k) could fire; the next micro pass's supersede then dropped the
batch marker (whose content the stale rolling summary does NOT contain)
and archive_and_compact immediately made the loss durable. Defrag had
the same hazard: it rewrote "the newest marker" even if that was a
batch marker. Empirically confirmed with a probe (batch marker content
destroyed in one pass).
Fix, three parts:
- Micro-created markers now carry MICRO_COMPACT_MARKER_KEY; supersede
and defrag only ever touch micro-tagged markers. Rehydration in
_resolve_compact_cursor tags the marker it absorbs (containment
proof), which safely covers adopting a batch marker as the new
rolling base after a reset.
- compress() success path resets micro rolling summary/cursor state so
a stale summary can never claim cumulativeness over a batch marker.
- Regression tests for both directions plus the reset.
W4: _splice_micro_compact_result no longer strips _db_persisted stamps
from surviving messages. Micro archives in place under the SAME session
id (unlike batch's child-session rotation, #57491), so surviving stamps
are accurate; stripping them meant an archive_and_compact failure left
every previously-persisted message unstamped and the next append-only
flush re-inserted them all as duplicate active rows.
W5: finalize_turn micro gate now checks agent._persist_disabled —
persistence-isolated fork agents (background review) must not burn an
aux call per review turn, and must never archive_and_compact the
canonical session rows if their compressor ever gains a DB binding.
W1: _serialize_one_exchange now delegates to _serialize_for_summary
(was a ~70-line near-verbatim copy; one serializer, one place to fix).
S4: _find_one_exchange boundary guard rejects only assistant/tool
boundaries (the actual alternation hazard) instead of requiring user —
a stray mid-list system/injected message can no longer wedge the
cursor forever.
5 new regression tests; 38 micro/prune tests, 400 compression-suite
tests, 61 finalize/persist tests pass; ruff clean.
tests/run_agent/test_proactive_prune_loop_wiring.py builds agents with a
MagicMock compressor; getattr(mock, '_micro_compact_enabled', False)
returns a truthy auto-attribute, so the hook called _micro_compact on the
mock and spliced its (empty-iterating) return over the transcript —
wiping all messages before persist (CI slice 7/8 failure).
Gate now requires _micro_compact_enabled is True, a callable
_micro_compact, and a non-empty list result before touching messages.
Same hardening protects production plugin context engines that don't
subclass ContextCompressor.
Two integration bugs found during review of #74522, both confirmed with
empirical probes against the production message-repair path:
1. Alternation: the summary marker was role="user" and an exchange was a
single assistant+tools group, so splicing between two user turns produced
user -> marker(user) -> user. The pre-request repair_message_sequence pass
(conversation_loop.py, runs before EVERY API call) then merged the marker
into the neighbouring real user message: metadata gone, cursor
unrecoverable on resume, and the summary text duplicated into the
transcript on every later pass (the transcript GREW every turn).
Fix: an exchange is now a full agent turn (assistant + tools + follow-up
assistant iterations, bounded by user messages), the marker is
assistant-role, and superseding an old marker deliberately merges the two
adjacent real user turns (plain-text \n\n-join, identical to repair
pass 2) so the returned transcript is alternation-valid by construction.
Probe result: repairs 0 (was 2), marker survives, no summary leakage.
2. Defrag destroyed user messages: _defrag_rolling_summary serialized the
whole remaining middle (user turns included) and spliced it away —
8 of 10 user prompts destroyed in one pass, contradicting the feature's
"your messages are never compacted" invariant. Fix: defrag now
re-summarizes only the rolling summary TEXT and rewrites the marker
content in place; transcript shape, cursor, and user turns untouched.
Probe result: 10 of 10 user prompts survive.
Also: marker provenance is now COMPRESSED_SUMMARY_HAS_USER_TURN_KEY=False —
micro markers absorb only assistant/tool content (#64650 invariant), and
real user turns remain in the transcript for provenance detection.
Adds 5 regression tests (repair-pass integration, alternation on
multi-iteration tool turns, defrag user survival, defrag input scope,
marker provenance); updates the two existing tests and the design doc to
the corrected semantics. 28 tests pass.
The on/off switch was the only knob. A pass fired after every completed turn,
absorbed exactly one exchange, and there was no way to ask for less. Since a
pass is also what breaks the prompt-cache prefix, "how often does it run" and
"how often do I pay a cache break" are the same question, and it had no answer.
Add `compression.micro_compact_every_n_turns` (default 1, clamped to >= 1). At 1
the behaviour is what it was; at 5 you get a fifth of the breaks and a fifth of
the reclaim rate. The counter advances per invocation rather than per committed
pass, so a turn that finds nothing to absorb still moves the cadence along and
cannot wedge it, and a bogus 0 or negative degrades to "every turn" instead of
silently disabling compaction.
Also expose `micro_compact_defrag_threshold_tokens`, which has been a hardcoded
attribute on the compressor with no path from config since it was added.
This does not give micro-compaction the prune's reclaim-size gate -- a pass
still commits whatever the single absorbed exchange saved. It makes the break
frequency tunable, which reaches the same end by absorbing less rather than by
waiting for a bigger win. The docs now say that plainly, including that a
reclaim threshold is the obvious follow-up and does not exist yet.
Tests cover the skip-until-due window, the cursor and prefix staying untouched
on skipped turns, the clamp, and that the feature is off unless enabled.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review raised whether default-on can be reconciled with the prompt-cache
contract in AGENTS.md, which permits mutating past context only for context
compression and treats per-conversation caching as sacred. It cannot, and the
codebase already says so in its own words.
A micro-compaction pass rewrites already-sent history, so it invalidates the
cached prefix every turn rather than at an episodic boundary. That is the exact
cost the proactive prune gates against: `proactive_prune_min_reclaim_tokens`
exists, per its own config comment, to keep rewrites to "one big episodic break
instead of a tiny break every tool iteration." Micro-compaction has no
equivalent gate -- one exchange per turn means one break per turn, by design.
Default to off. An operator who wants the amortized stall can opt in with
`compression.micro_compact: true` and accept the tradeoff knowingly; nobody
inherits a per-turn cache break from installing an update.
Also register the key in config_defaults so it is discoverable and picked up by
the update path's new-options check -- it was previously read by agent_init but
declared nowhere -- and document the cache cost in docs/micro-compaction.md
instead of only the benefit. The measurements behind the feature (occupancy
plateau, zero batch compactions) never priced cache invalidation, and the doc
now says which numbers a reader would need to measure to justify enabling it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rolling summary lives only in memory. A resumed session starts with an
empty one while the marker carrying every previously absorbed exchange is
still in the transcript. The first pass after a resume therefore built a
marker from a single exchange and superseded the marker holding the entire
history -- silently discarding everything micro-compaction had accumulated.
This was introduced by the supersede fix. Before it, markers piled up
wastefully, but nothing was ever lost.
Two changes, so a single failure cannot lose data:
Rehydrate. When the cursor is recovered by scanning the transcript -- the
resume path -- also recover the rolling summary from that marker, so the
next pass merges into the existing history instead of replacing it.
Extraction uses rfind for the heading because SUMMARY_PREFIX references the
heading text itself, so the first occurrence is inside the preamble.
Gate superseding. Earlier markers are dropped only when this pass's summary
is demonstrably cumulative, i.e. the rolling summary was non-empty going in.
If rehydration ever fails, the pass keeps both markers: wasteful, but the
history survives.
Tests cover the resume path, the failed-rehydration fallback, and the
round trip of a summary through a marker.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cursor was set to the pre-splice `exchange_end`. A splice collapses the
absorbed span -- an assistant plus its tool results, often four or more
messages -- into a single marker, and may also drop a superseded marker
further back, so every index after it shifts.
The stale cursor therefore overshot, landing inside a *later* exchange's
tool group. The next pass's `_find_one_exchange` walked forward from there
to the following assistant, so the exchange it had landed inside was never
absorbed at all. On tool-bearing conversations micro-compaction was
silently doing roughly half the work it should.
Traced on a 3-tool-per-exchange transcript: the cursor sat at index 6 when
the marker was at 2, and the message count stalled at 32 instead of
continuing to 28.
Derive the cursor from the marker's actual position in the spliced result
instead, which is self-correcting regardless of how much the splice moved.
Apply it on the defrag path too, which had the same staleness.
Found by a randomized long-horizon harness (480 conversation shapes x 25
passes, varying tool-group sizes and summarizer failure modes) asserting
structural and progress invariants after every pass. Existing tests missed
it because their fixtures have no tool results, so the absorbed span is one
message and nothing shifts. The regression test uses tool groups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tokens saved is the wrong headline for this feature. Micro-compaction is
not an efficiency optimisation — the same summarization work happens either
way. What it buys is (a) that work amortized across turns instead of one
stall, and (b) a window kept low enough that a session runs much further
before needing a hard compaction at all.
Neither shows up in "net tokens saved". A session can save nothing on paper
and still be a clear win on both counts.
So the telemetry now carries occupancy: tokens_after as a share of the
compaction threshold, plus the threshold and resolved window it was
computed from. That is the number that says whether a session has headroom
left. The report leads with it, and cross-references the batch
`compression_attempt` lines already in the log so it can show how often the
long pause actually fired — ideally never.
Occupancy is read from the cached threshold only. The public
`threshold_tokens` property resolves lazily and can issue a synchronous
/models probe (#32221); telemetry must never be the thing that blocks a
turn, so an unresolved window reports null. In practice a pass has already
resolved it via the tail calculation, so the field is populated. A test
pins the no-forcing behaviour directly against the emitter.
The report is pure ASCII: `scripts/check_subprocess_stdin.py` currently
dies on a cp1252 console before printing its results, and a diagnostic tool
that crashes on the platform it is diagnosing is worse than no tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The existing log line reports message counts, which is the least
informative number available here: absorbing one tool-heavy exchange can
drop hundreds of tokens while moving the count by one. There was no way to
answer "is this actually helping?" from a real session.
Emit one content-free JSON line per pass, in the same shape as the batch
compaction telemetry: before/after tokens, the delta, the size of the
absorbed exchange, the rolling summary size, duration, and running
per-session totals so a whole run can be read off the last line. No
transcript content rides along.
Add scripts/micro_compaction_report.py to aggregate those lines into
passes, outcome mix, net tokens saved, mean exchange size and durations,
with an optional per-session breakdown.
Measuring it immediately surfaced something worth documenting: the first
pass in a session normally *costs* tokens. The summary marker carries a
fixed ~400 tokens of scaffolding, paid on pass one against a single
absorbed exchange. From pass two the marker is replaced rather than added,
so the overhead is already paid and each exchange is close to pure saving.
Break-even is typically the second or third pass. Tests cover the
telemetry contract, the cumulative totals, and that first-pass/later-pass
shape so nobody reads a single turn and concludes it made things worse.
The estimator costs ~5 ms at 600 messages and ~20 ms at 1200, taken twice
per pass, post-turn — and only once an exchange is actually in hand, so
turns that no-op early pay nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The `_micro_compact` docstring cited "#82483" for the resume double-load
problem. No such issue exists — the repository's highest number is 74323,
so the reference was invented rather than looked up.
The reasoning it was attached to is correct and stays: the session flush is
append-only, so an in-memory splice alone leaves the original rows active
and a resume loads both the summary and the messages it replaced. Only the
citation was wrong.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`_find_one_exchange`'s docstring described an exchange as "(optional) user
message + assistant message + its tool results", but the walk skips past
user messages and starts at the assistant, so user turns are never absorbed
into the rolling summary.
The code is right and the docstring was wrong. Assistant output is largely
an account of what was done and survives summarising with little loss. The
user's messages are the intent everything else is derived from and cannot be
reconstructed from the work that followed — paraphrasing "use the existing
helper, don't add a new one" into a summary is how an agent ends up doing
the opposite six turns later. They are also cheap: a prompt is normally a
tiny fraction of what one tool result costs.
Correct the docstring, document the property (and its cost — a floor on how
small the middle can get, since user turns accumulate), and add a test so it
stays deliberate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Batch compaction pauses a session for one large summarization once the
window fills. Micro-compaction spreads that cost out: after each completed
turn, `finalize_turn` folds the single oldest un-absorbed exchange
(assistant message plus its tool results) into a rolling summary, so the
work happens in small increments during post-turn idle time instead of one
long stall.
Mechanics:
- a cursor tracks the first message not yet absorbed, recovered from the
transcript's last summary marker when in-memory state is unavailable;
- protected head and tail windows are never touched, so the system prompt
and recent turns stay verbatim;
- the absorbed span is replaced by a marker carrying the usual
`_compressed_summary` metadata, so resume, handoff and `/compress`
treat it exactly like a batch summary;
- `archive_and_compact` keeps the session DB in step, otherwise the
append-only flush would leave the original rows active and a resume
would double-load both summary and originals;
- when the rolling summary itself passes a token threshold it is
defragged: re-summarized in one shot and the cursor jumps to the tail;
- an exchange the summarizer can't handle is retried a bounded number of
times, then skipped, so one poison exchange can't stall every turn.
Keep only the newest summary marker. The rolling summary is cumulative, so
each marker already contains everything the previous ones held; leaving them
stacked near-duplicate copies of the same text, each with its own heading and
end-marker scaffolding, and the transcript grew on every turn instead of
shrinking. Measured over six turns on a 12-exchange conversation with tool
output: 4104 -> 4797 tokens before, 4104 -> 2572 after.
Off switch: `compression.micro_compact: false` (default on).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compaction summary's role was selected against the LITERAL
neighbouring messages (compressed[-1] / tail_messages[0]). Mistral-family
chat templates (Devstral, Mistral Small 3.x, Magistral) enforce
user/assistant alternation but exempt the tool flow (tool results and
assistant messages carrying tool_calls) from the check, so a protected
head ending [user, assistant(tool_calls), tool] pinned the summary to
role="user" while the last role the template counts is "user": the
backend rejects the whole request with a Jinja alternation error
(HTTP 500). The summary persists in the stored conversation, every
retry replays the identical poisoned history, and the session is
permanently unrecoverable. Fires on EVERY compaction against a
Mistral-strict backend, captured byte-exact via a tee-proxy in front of
a llama.cpp/llama-swap Devstral deployment.
Fix: compute both neighbour roles through _template_visible_role(),
which skips template-exempt messages. The #52160 (Anthropic user-first)
and #58753 (zero-user-turn) forced-user guards are preserved; their
forced shapes (summary-user followed only by exempt messages) are
alternation-safe. When the visible head ends "assistant" and the
visible tail opens "user", no standalone role can alternate and the
existing merge-into-tail fallback now correctly fires (the literal
logic emitted a standalone user summary there: a second poisoning
shape).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LGN45sMMbwM8cW9T9ga4ou
Reactions live in the existing messages.display_metadata JSON column (no new
table), with iOS Tapback semantics enforced DB-side: one reaction per author
per message, re-tap retracts, different emoji replaces. The desktop catches up
to the reaction contract five platform adapters already ship.
- SessionDB: set/get_message_reaction, latest_message_row_id (role + offset +
require_text so invisible tool-call-only rows are never targeted),
take_unseen_reactions (announce-exactly-once), get_message_role
- message.react RPC: accepts row_id or newest_role for live messages that
haven't learned their durable id yet
- react_to_message tool: desktop-gated (check_fn), defaults to the user's
latest visible message, messages_back for retroactive reactions
- Model context rides run_message only (beside the speech-interrupted note):
the persisted prompt stays clean, so no [The user reacted …] scaffolding in
transcripts, and no cached prefix ever changes
- Resume projection forwards row_id + reactions; _row_id is stripped from
outgoing API copies next to display_metadata
