30ca0e107c
The floor cut-back (`step > total - floor` -> `step = total - floor`) fires whenever an eight-carrier batch is wider than the fit window: tool results carrying several images, or user uploads filling most of the ceiling. `total - floor` tracks the total, so the retire count stopped being a step function and the frontier moved on every turn again — the per-image prefix rewrite this policy exists to prevent. Replay over 24 turns (prefix collapses; issue #113517 script): carrier shape before after 1 image / tool_result 2 2 2 images / tool_result 3 3 3 images / tool_result 13 7 4 images / tool_result 16 11 15 uploads + 1 / tool_result 15 10 The quantum is now `min(batch, window - floor)` where `window` is how many newest carriers fit. Each advance keeps at least `floor + 1` frames, so the cut-back clause is gone; holding for `window - floor` turns per advance is the most the floor allows. The one-image case (#113517's report) is unchanged: 21/29/37 -> 8/16/24 retired. Regression test: test_frontier_holds_when_tool_results_carry_several_images — red on the previous loop shape, green here.
97 lines
4.6 KiB
Python
97 lines
4.6 KiB
Python
"""Send-path image eviction policy shared by both stateless outbound passes.
|
|
|
|
Two passes retire old tool-result images from the per-request copy of the conversation:
|
|
``agent.context_compressor.evict_stale_outbound_tool_images`` on the OpenAI-shaped list and
|
|
``agent.anthropic_message_convert._evict_old_screenshots`` on the Anthropic wire list. Both run
|
|
from scratch on a fresh clone every request, so they must agree on one policy or the second
|
|
pass re-evicts on a different frontier than the first (#113517). This module is stdlib-only so
|
|
the wire converter, a leaf, can import it without dragging in the compaction stack.
|
|
|
|
Why the trigger is a provider limit, not a keep-newest count: retiring an image edits a message
|
|
the provider has already cached, and Anthropic matches its prompt cache on an exact byte prefix.
|
|
A keep-newest-N window retires one more message on every new image, so every turn is a
|
|
full-prefix miss. Holding images until the request would cross a real API limit and then
|
|
retiring a batch costs one slower turn per batch and nothing below the limit.
|
|
|
|
20 is the documented threshold at which Anthropic applies a stricter per-image dimension cap
|
|
(2000 px) to EVERY image in the request, counting images nested in tool_result content. The hard
|
|
ceilings are higher (100 images per request on 200K-context models, 600 otherwise) but the 32 MB
|
|
request-size limit usually binds first, which the byte budget guards with headroom for text.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Optional, Sequence
|
|
|
|
OUTBOUND_IMAGE_LIMIT = 20
|
|
OUTBOUND_IMAGE_BUDGET_BYTES = 24_000_000
|
|
IMAGE_EVICTION_BATCH = 8
|
|
# Satisfiability floor — see outbound_image_retire_count.
|
|
OUTBOUND_IMAGE_FLOOR = 3
|
|
|
|
|
|
def outbound_image_retire_count(
|
|
carrier_blocks_newest_first: Sequence[int],
|
|
reserved_blocks: int,
|
|
*,
|
|
carrier_bytes_newest_first: Optional[Sequence[int]] = None,
|
|
reserved_bytes: int = 0,
|
|
limit: int = OUTBOUND_IMAGE_LIMIT,
|
|
budget: int = OUTBOUND_IMAGE_BUDGET_BYTES,
|
|
batch: int = IMAGE_EVICTION_BATCH,
|
|
floor: int = OUTBOUND_IMAGE_FLOOR,
|
|
) -> int:
|
|
"""How many of the OLDEST image-bearing tool results to retire.
|
|
|
|
``carrier_blocks_newest_first`` is the image-block count per image-bearing tool result,
|
|
newest first; ``reserved_blocks`` counts images the pass must never rewrite (user
|
|
uploads). The byte dimension is active only when ``carrier_bytes_newest_first`` is
|
|
given (the wire pass has no sizes).
|
|
|
|
The retire count must be a STEP FUNCTION of the overshoot, because the pass is recomputed
|
|
on every request: an exact ``count - limit`` target moves the frontier on every new image,
|
|
and a fixed one-batch retire stops enforcing the limit after the first batch. So the count
|
|
advances in quanta until the request fits.
|
|
|
|
The quantum is ``batch`` capped at ``window - floor``, where ``window`` is how many newest
|
|
carriers fit under the ceiling. A quantum wider than that would step past the newest frames
|
|
on every advance; cutting each step back to exactly ``total - floor`` instead makes the
|
|
retire count track ``total`` again — the per-image frontier this policy exists to avoid,
|
|
visible whenever a tool result carries several images or uploads fill most of the ceiling.
|
|
Holding for ``window - floor`` turns per advance is the most the floor allows.
|
|
|
|
The floor is a SATISFIABILITY floor: it shelters the newest frames only when reserved
|
|
uploads alone breach the block ceiling (no retirement can fix that), and never under byte
|
|
pressure — the request-size limit is hard and the provider answers 413.
|
|
"""
|
|
total = len(carrier_blocks_newest_first)
|
|
sizes = carrier_bytes_newest_first
|
|
blocks_kept = [0] * (total + 1)
|
|
bytes_kept = [0] * (total + 1)
|
|
for i in range(total):
|
|
blocks_kept[i + 1] = blocks_kept[i] + carrier_blocks_newest_first[i]
|
|
bytes_kept[i + 1] = bytes_kept[i] + (sizes[i] if sizes is not None else 0)
|
|
|
|
def _bytes_fit(kept: int) -> bool:
|
|
return sizes is None or reserved_bytes + bytes_kept[kept] <= budget
|
|
|
|
def _fits(kept: int) -> bool:
|
|
return reserved_blocks + blocks_kept[kept] <= limit and _bytes_fit(kept)
|
|
|
|
if _fits(total):
|
|
return 0
|
|
|
|
floor = min(max(floor, 0), total)
|
|
max_retire = total - floor if not _fits(0) and _bytes_fit(floor) else total
|
|
if max_retire <= 0:
|
|
return 0
|
|
window = max(k for k in range(total + 1) if _fits(k)) if _fits(0) else 0
|
|
quantum = max(1, min(batch, window - floor))
|
|
|
|
retire = 0
|
|
while retire < max_retire:
|
|
retire = min(retire + quantum, max_retire)
|
|
if _fits(total - retire):
|
|
break
|
|
return retire
|