fix(compression): keep hygiene turn-hold worker's commit admission so thinking-model summaries are adopted, not burned
The 10s hygiene_max_turn_hold_seconds budget (#92318) releases the arriving user turn while the summary model is still streaming. For thinking summary models (DeepSeek-V4-Flash etc.) whose reasoning prefix alone exceeds 10s, the abandonment path ALWAYS cancelled the commit fence — 100% of the summary attempt (including the full thinking prefix) was discarded on every turn, permanently disabling auto-compression while paying the summary model 10s of thinking per turn, and the flat 60s retry-after then blocked the agent-side preflight from a fresh chance. Structural fix (maintainer-chosen direction in #97963): decouple the turn from the compression instead of holding the turn longer or making the hold progress-aware (which would reintroduce the #90845 frozen-turn bug): - CompressionCommitFence gains mark_commit_watermark_fenced() / commit_watermark_fenced; compress_context marks the fence right after capturing get_active_message_watermark() under the durable compression lock (#75316/#87484) — the property that makes a LATE commit safe: rows appended after compression start survive both commit paths verbatim as cloned concurrent tail (archive_and_compact watermark= and publish_compression_child watermark/watermark_ceiling). - gateway hygiene turn-hold handler: when the fence is watermark-fenced, the detached worker (already kept alive via _defer_agent_cleanup_until_future_done) KEEPS its commit admission; the user's turn proceeds on the uncompressed transcript at the same 10s budget, and the summary is adopted at the worker's own watermark-fenced commit boundary. Unfenced workers are cancelled exactly as before — never worse than the status quo. - No retry-after is armed while the kept-admission attempt runs (it would block preflight adoption via the same-session cooldown); re-attempt spacing is covered by the durable compression lock (_session_has_compression_in_flight). If the worker ends WITHOUT committing, a done-callback restores the flat non-escalating 60s retry-after; a successful adoption resets the hygiene failure streak. The streak never advances for a deferral either way. - Docs: configuration.md hygiene_max_turn_hold_seconds one-liner updated to describe deferred adoption and the thinking-model case; config_defaults.py comment updated. Knob stays config.yaml-only. Invariants preserved: - 10s user-latency cap stays hard (#90845/#92318): test_session_hygiene_turn_hold_budget_abandons_streaming_wait passes UNMODIFIED (its worker is not watermark-fenced, so it pins the cancel path through the public surface). - Stale-clobber impossible: adoption only rides commits bounded by the start watermark; the fence still gates admission and unfenced/late results are discarded. New regression tests (tests/gateway/test_session_hygiene_turnhold_adoption.py): - watermark-fenced worker keeps admission, late summary is committed, turn still released at the budget, no cooldown while running, streak reset on adoption; - kept-admission worker that ends without committing restores the flat turn-hold retry-after (<=120s, names turn-hold, streak untouched); - unfenced worker still cancelled and discarded (status quo). Sabotage-verified: disabling the keep-admission branch fails the two new adoption tests and leaves the unfenced-cancel test green. Fixes #97963
This commit is contained in:
@@ -699,6 +699,19 @@ compression:
|
||||
# summarization on a short idle thread. Example: 1800 = compact after 30 min idle.
|
||||
idle_compact_after_seconds: 0
|
||||
|
||||
# Gateway session-hygiene turn-hold budget (default: 10). Max seconds an
|
||||
# arriving user turn is held while a still-streaming hygiene summary
|
||||
# finishes. Distinct from hygiene_timeout_seconds (compressor inactivity
|
||||
# budget): this bounds user-visible latency so chat transports (Telegram
|
||||
# ~30s) do not drop a silent connection. On expiry the turn proceeds
|
||||
# uncompressed; the detached worker keeps its commit admission (when the
|
||||
# commit is watermark-fenced) and the summary is adopted at the next safe
|
||||
# boundary. Thinking-model summarizers often need longer than 10s to emit
|
||||
# the first content token — raise to 300 (or >= your summarizer's real
|
||||
# time-to-first-content) only if you want THIS turn to wait for the
|
||||
# compression instead of adopting it one turn late.
|
||||
hygiene_max_turn_hold_seconds: 10
|
||||
|
||||
# Proactive tool-result prune (default: 0 = disabled). Opt-in token trigger
|
||||
# for a deterministic, no-LLM prune of OLD tool-result payloads, run
|
||||
# independently of `threshold` above. On large-window models (512K/1M) the
|
||||
|
||||
Reference in New Issue
Block a user