fix(agent): quantize eviction by the fit window so heavy tool results hold the frontier

The floor cut-back (`step > total - floor` -> `step = total - floor`) fires whenever an
eight-carrier batch is wider than the fit window: tool results carrying several images,
or user uploads filling most of the ceiling. `total - floor` tracks the total, so the
retire count stopped being a step function and the frontier moved on every turn again —
the per-image prefix rewrite this policy exists to prevent.

Replay over 24 turns (prefix collapses; issue #113517 script):

  carrier shape              before  after
  1 image / tool_result          2      2
  2 images / tool_result         3      3
  3 images / tool_result        13      7
  4 images / tool_result        16     11
  15 uploads + 1 / tool_result  15     10

The quantum is now `min(batch, window - floor)` where `window` is how many newest
carriers fit. Each advance keeps at least `floor + 1` frames, so the cut-back clause is
gone; holding for `window - floor` turns per advance is the most the floor allows. The
one-image case (#113517's report) is unchanged: 21/29/37 -> 8/16/24 retired.

Regression test: test_frontier_holds_when_tool_results_carry_several_images — red on
the previous loop shape, green here.
This commit is contained in:
kshitijk4poor
2026-09-17 13:12:16 +05:30
committed by kshitij
parent 5dcfae44b4
commit 30ca0e107c
2 changed files with 65 additions and 6 deletions

View File

@@ -53,6 +53,13 @@ def outbound_image_retire_count(
and a fixed one-batch retire stops enforcing the limit after the first batch. So the count
advances in quanta until the request fits.
The quantum is ``batch`` capped at ``window - floor``, where ``window`` is how many newest
carriers fit under the ceiling. A quantum wider than that would step past the newest frames
on every advance; cutting each step back to exactly ``total - floor`` instead makes the
retire count track ``total`` again — the per-image frontier this policy exists to avoid,
visible whenever a tool result carries several images or uploads fill most of the ceiling.
Holding for ``window - floor`` turns per advance is the most the floor allows.
The floor is a SATISFIABILITY floor: it shelters the newest frames only when reserved
uploads alone breach the block ceiling (no retirement can fix that), and never under byte
pressure — the request-size limit is hard and the provider answers 413.
@@ -78,14 +85,12 @@ def outbound_image_retire_count(
max_retire = total - floor if not _fits(0) and _bytes_fit(floor) else total
if max_retire <= 0:
return 0
window = max(k for k in range(total + 1) if _fits(k)) if _fits(0) else 0
quantum = max(1, min(batch, window - floor))
retire = 0
while retire < max_retire:
step = min(retire + batch, max_retire)
if step > total - floor and _fits(floor):
# A whole batch would retire the frames the model was just asked about while
# keeping the floor already clears the ceiling; take the smaller edit instead.
step = total - floor
retire = step
retire = min(retire + quantum, max_retire)
if _fits(total - retire):
break
return retire

View File

@@ -150,6 +150,60 @@ class TestOutboundStaleVisionEviction:
"each move rewrites a cached row and restarts the prefix"
)
def test_frontier_holds_when_tool_results_carry_several_images(self):
"""Heavy carriers must still advance the frontier in steps, never per image.
With three images per tool result, an eight-carrier batch is wider than the
fit window, so a step cut back to exactly ``total - floor`` tracks the total
and rewrites a cached row on every turn. The quantum must shrink to the window
instead: the frontier may move at most once per (window - floor) turns.
"""
from agent.conversation_loop import _clone_message_for_send
def surviving(n: int) -> list:
history: list[dict] = [{"role": "user", "content": "start"}]
for i in range(n):
history.append(
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": f"call_{i}",
"type": "function",
"function": {"name": "vision_analyze", "arguments": "{}"},
}
],
}
)
history.append(
{
"role": "tool",
"tool_call_id": f"call_{i}",
"content": [
{"type": "text", "text": f"shot {i}"},
*[
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,A{i}{k}"}}
for k in range(3)
],
],
}
)
msgs = [_clone_message_for_send(m) for m in history]
evict_stale_outbound_tool_images(msgs)
assert _outbound_image_blocks(msgs) <= OUTBOUND_IMAGE_LIMIT
return _image_bearing_tool_ids(msgs)
window = OUTBOUND_IMAGE_LIMIT // 3
span = range(window + 1, window + 1 + 4 * (window - OUTBOUND_IMAGE_FLOOR))
frontier = [surviving(n)[0] for n in span]
moves = sum(a != b for a, b in zip(frontier, frontier[1:]))
assert moves <= len(span) // (window - OUTBOUND_IMAGE_FLOOR), (
f"frontier moved {moves} times over {len(span)} turns (frontier={frontier}); "
"a per-image frontier rewrites the cached prefix every turn"
)
assert all(len(k) >= OUTBOUND_IMAGE_FLOOR for k in (surviving(n) for n in span))
def test_multi_image_tool_results_count_as_blocks(self):
"""The provider limit counts image BLOCKS, not tool messages.