eafed27cf0
Long-running cron jobs (>60 s, i.e. past one heartbeat interval) intermittently ended with last_status=error / "Interrupted by shutdown before terminal completion." while their output was complete. The fire-claim heartbeat thread took ONE sample that read the claim as not ours, latched lost_ownership, and _FireOwnership.lost() then trusted that latch without asking the store again — so a run whose claim still validated (the very condition under which _record_fire_ownership_lost writes that message) was recorded as interrupted, its output never saved. The reporter's own error string proves the miss was transient: it is only ever written when the owner re-validates True. Why this shape: - _FireOwnership.lost(): an explicit transport cancel stays terminal; a latched heartbeat miss is re-checked against the store, and a claim that still validates keeps the run's real outcome. The owner-fenced mark_job_run and fire_claim_fence remain the authority, so a genuinely re-owned claim still yields (no delivery, no terminal write over the new owner, ledger discard). An unreachable store after a latched miss stays fail-closed. - _heartbeat_loop: a miss is re-sampled once after _FIRE_CLAIM_MISS_CONFIRM_SECONDS before it latches. Latching cancels the live agent run and ends the lease refresh, so a single sample must not do that; a genuinely re-owned claim misses twice and latches ~1 s later. The issue's other hypothesis (worker process identity under a systemd-run --scope worker) is falsified by the code: the owner token is stored in fire_claim.by at claim time and compared to the stored value, never recomputed from _machine_id(), and the heartbeat thread runs in a copied context so it reads the same store. Slimmer redo of #113364 (same direction: revalidate before trusting the latch) without its production-dead isinstance(_CombinedCancelEvent) branch. Co-authored-by: KoNit-K <konit.block@protonmail.com>
Website
This website is built using Docusaurus, a modern static website generator.
Installation
yarn
Local Development
yarn start
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
Build
yarn build
This command generates static content into the build directory and can be served using any static contents hosting service.
Deployment
Using SSH:
USE_SSH=true yarn deploy
Not using SSH:
GIT_USER=<Your GitHub username> yarn deploy
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.
Diagram Linting
CI runs ascii-guard to lint docs for ASCII box diagrams. Use Mermaid (````mermaid`) or plain lists/tables instead of ASCII boxes to avoid CI failures.