A recurring job with a finite repeat count (for example repeat=3, every
24h) lost one repeat slot for each automatic re-run after a fire that
could not reach the model. The failed occurrence counted, and then each
5/15/30-minute re-run of that occurrence counted again, so a half-hour
outage finished a repeat=3 job with zero model calls. With
cron.retry_unreachable off, the same outage costs only the one failed
occurrence.
Cause: cron/jobs.py:_advance_after_run bumps repeat.completed on every
mark_job_run, and the re-runs parked by
cron/unreachable_retry.py:plan_retry go through it like scheduled
occurrences. cron/unreachable_retry.py:will_retry also treated a re-run
on the last slot as the job's final run.
Fix: cron/scheduler.py:_finish_completed_run passes ladder_rung=True
when the dispatched instant is the parked re-run instant (new
is_retry_run, which compares the dispatch snapshot's _scheduled_instant
with the ladder state). mark_job_run then calls _advance_after_run with
count_run=False, and will_retry leaves that run out of its final-run
check the same way. The first failed occurrence still counts, and manual
runs still count.
(cherry picked from commit 27038f7064bb611811efee8c527dddecd4fcb57b)
c84ef16384