88c6085846
* fix(process): a kill's result stands on every backend, and a failed scope stop never counts as stopped Follow-up to #133434, which fixed these races for local processes only. Non-local backends (spawn_via_env) settle through _env_poller_loop, which still wrote the session's final state without the session lock: - It set exit_code before _finish_exited, so a kill that landed first ended as killed with the sandbox's exit code. The exit now goes through _finish_exited alone, and a committed kill keeps its code. - Its backend-lost path set exited/-1/lost directly, replacing a recorded kill. It now goes through _finish_exited(reason="lost"), so the kill stands and the completion is still published by its owner. - A log delta (or a log-rotation reset) fetched before a kill was applied after it, so the session's output no longer matched the kill's snapshot. Both are now dropped once the session has exited, decided under the lock. A failed systemd scope stop could still count as a successful stop: - A recovered PID that died between get() and _signal_kill ignored the stop result and returned a plain already_exited. It now reports scope_stop_failed. - kill_all counted already_exited as stopped even when the scope stop failed, and only looked at _running, so a session that exited after being picked kept a live scope that no later sweep retried. Every kill path now stops the scope through _stop_scope, which marks a failed stop pending; kill_all does not count a result with scope_stop_failed and also retries finished sessions whose stop is pending. _log_delta_command moves unchanged into tools/process_registry_env_log.py (ProcessRegistry._log_delta_command stays as an alias) to keep process_registry.py under its size cap. Refs #132358. Both gaps were found by @andrexibiza in the review of #133434. * fix(process): pruning keeps a session whose scope stop is pending kill_all retries a failed scope stop through the finished session, but _prune_if_needed still evicted that session by TTL (measured from process start, so a long-running process was prunable the moment it finished) or by capacity, leaving a possibly live scope with no retry target. Sessions with _scope_stop_pending are now exempt from both until the stop succeeds; capacity pruning drops the oldest prunable session instead. Found by the independent review of #133648.
35 lines
2.0 KiB
Python
35 lines
2.0 KiB
Python
"""Incremental sandbox log reads for ProcessRegistry's non-local poller."""
|
|
|
|
|
|
def log_delta_command(quoted_log_path: str, offset: int) -> str:
|
|
"""Shell command that reads only the log bytes written since ``offset``
|
|
(``cat``-ing the whole file every poll re-sends all output over docker/SSH).
|
|
|
|
Prints one header line ``"<size> <offset>"`` then the bytes in [offset, size).
|
|
The size is read first and the tail cut at that same size, so a growing file
|
|
never sends a byte twice; a file that shrank was rotated/truncated, so the
|
|
offset drops to 0 and the reader starts over. The window end is pulled back
|
|
to a UTF-8 character boundary (the backend decodes each ``execute()`` result
|
|
on its own, so a straddling multibyte char would become U+FFFD and break watch
|
|
patterns at the seam): up to 3 trailing continuation bytes are held for the
|
|
next poll and the header reports the trimmed size."""
|
|
return (
|
|
f"O={offset}; "
|
|
f"S=$({{ wc -c < {quoted_log_path}; }} 2>/dev/null | tr -dc '0-9'); "
|
|
f"S=${{S:-0}}; "
|
|
f'if [ "$S" -lt "$O" ]; then O=0; fi; '
|
|
# Scan back up to 3 continuation bytes (octal 200-277) to the lead byte; if
|
|
# the lead's declared length (3xx=2, 34x-35x=3, 36x-37x=4) exceeds the bytes
|
|
# present, trim to before it. Complete sequences and ASCII tails untouched.
|
|
f'N=0; P=$S; while [ "$P" -gt "$O" ] && [ "$N" -lt 3 ]; do '
|
|
f"B=$(tail -c +$P {quoted_log_path} 2>/dev/null | head -c 1 | od -An -to1 | tr -dc '0-9'); "
|
|
f'case "$B" in 2[0-7][0-7]) P=$((P-1)); N=$((N+1));; *) break;; esac; done; '
|
|
f'if [ "$N" -gt 0 ] || [ "$P" -eq "$S" ]; then '
|
|
f"B=$(tail -c +$P {quoted_log_path} 2>/dev/null | head -c 1 | od -An -to1 | tr -dc '0-9'); "
|
|
f'case "$B" in 3[0-3][0-7]) L=2;; 3[4-5][0-7]) L=3;; 3[6-7][0-7]) L=4;; *) L=1;; esac; '
|
|
f'if [ "$L" -gt $((N+1)) ]; then S=$((P-1)); fi; fi; '
|
|
f'echo "$S $O"; '
|
|
f'if [ "$S" -gt "$O" ]; then '
|
|
f"tail -c +$((O+1)) {quoted_log_path} 2>/dev/null | head -c $((S-O)); fi"
|
|
)
|