fix(compression): keep hygiene turn-hold worker's commit admission so thinking-model summaries are adopted, not burned

The 10s hygiene_max_turn_hold_seconds budget (#92318) releases the arriving
user turn while the summary model is still streaming. For thinking summary
models (DeepSeek-V4-Flash etc.) whose reasoning prefix alone exceeds 10s,
the abandonment path ALWAYS cancelled the commit fence — 100% of the summary
attempt (including the full thinking prefix) was discarded on every turn,
permanently disabling auto-compression while paying the summary model 10s
of thinking per turn, and the flat 60s retry-after then blocked the
agent-side preflight from a fresh chance.

Structural fix (maintainer-chosen direction in #97963): decouple the turn
from the compression instead of holding the turn longer or making the hold
progress-aware (which would reintroduce the #90845 frozen-turn bug):

- CompressionCommitFence gains mark_commit_watermark_fenced() /
  commit_watermark_fenced; compress_context marks the fence right after
  capturing get_active_message_watermark() under the durable compression
  lock (#75316/#87484) — the property that makes a LATE commit safe: rows
  appended after compression start survive both commit paths verbatim as
  cloned concurrent tail (archive_and_compact watermark= and
  publish_compression_child watermark/watermark_ceiling).
- gateway hygiene turn-hold handler: when the fence is watermark-fenced,
  the detached worker (already kept alive via
  _defer_agent_cleanup_until_future_done) KEEPS its commit admission; the
  user's turn proceeds on the uncompressed transcript at the same 10s
  budget, and the summary is adopted at the worker's own watermark-fenced
  commit boundary. Unfenced workers are cancelled exactly as before —
  never worse than the status quo.
- No retry-after is armed while the kept-admission attempt runs (it would
  block preflight adoption via the same-session cooldown); re-attempt
  spacing is covered by the durable compression lock
  (_session_has_compression_in_flight). If the worker ends WITHOUT
  committing, a done-callback restores the flat non-escalating 60s
  retry-after; a successful adoption resets the hygiene failure streak.
  The streak never advances for a deferral either way.
- Docs: configuration.md hygiene_max_turn_hold_seconds one-liner updated
  to describe deferred adoption and the thinking-model case;
  config_defaults.py comment updated. Knob stays config.yaml-only.

Invariants preserved:
- 10s user-latency cap stays hard (#90845/#92318):
  test_session_hygiene_turn_hold_budget_abandons_streaming_wait passes
  UNMODIFIED (its worker is not watermark-fenced, so it pins the cancel
  path through the public surface).
- Stale-clobber impossible: adoption only rides commits bounded by the
  start watermark; the fence still gates admission and unfenced/late
  results are discarded.

New regression tests (tests/gateway/test_session_hygiene_turnhold_adoption.py):
- watermark-fenced worker keeps admission, late summary is committed,
  turn still released at the budget, no cooldown while running,
  streak reset on adoption;
- kept-admission worker that ends without committing restores the flat
  turn-hold retry-after (<=120s, names turn-hold, streak untouched);
- unfenced worker still cancelled and discarded (status quo).
Sabotage-verified: disabling the keep-admission branch fails the two new
adoption tests and leaves the unfenced-cancel test green.

Fixes #97963
This commit is contained in:
Teknium
2026-08-31 12:38:15 -07:00
parent 92fa0845ee
commit 9de9d7613c
6 changed files with 667 additions and 1 deletions

View File

@@ -699,6 +699,19 @@ compression:
# summarization on a short idle thread. Example: 1800 = compact after 30 min idle.
idle_compact_after_seconds: 0
# Gateway session-hygiene turn-hold budget (default: 10). Max seconds an
# arriving user turn is held while a still-streaming hygiene summary
# finishes. Distinct from hygiene_timeout_seconds (compressor inactivity
# budget): this bounds user-visible latency so chat transports (Telegram
# ~30s) do not drop a silent connection. On expiry the turn proceeds
# uncompressed; the detached worker keeps its commit admission (when the
# commit is watermark-fenced) and the summary is adopted at the next safe
# boundary. Thinking-model summarizers often need longer than 10s to emit
# the first content token — raise to 300 (or >= your summarizer's real
# time-to-first-content) only if you want THIS turn to wait for the
# compression instead of adopting it one turn late.
hygiene_max_turn_hold_seconds: 10
# Proactive tool-result prune (default: 0 = disabled). Opt-in token trigger
# for a deterministic, no-LLM prune of OLD tool-result payloads, run
# independently of `threshold` above. On large-window models (512K/1M) the