salch-cred e70801e736 fix(cron): keep a sticky last_failure stamp so a healed run stays visible ...

A failed cron run records last_status=error and bumps failure_streak
correctly, but the next successful run resets both in the same pass and
nothing durable records that a failure happened — a silent watchdog-style
script that fails every Nth tick collapses to a single tick of visible
failure, and monitors sampling jobs.json report green forever (#118354).

Stamp job[last_failure] = {at, detail} on every failed run, latest
failure wins, and do NOT clear it on success: last_status keeps
describing current health while last_failure preserves the recency
evidence for dashboards and cron list. cron list now prints a warning
line for the stamp (marked recovered when last_status is healthy).

Fixes #118354

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

2026-10-02 20:29:53 -05:00

..

__init__.py

…

conftest.py

test(cron): keep script-run tests off the host's PM store

2026-09-27 15:04:50 +05:30

test_agent_scheduling_gate.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_atomic_paused_creation.py

…

test_blueprint_catalog.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_bot_chat_cli_home.py

…

test_bot_chat_delivery_child_env.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_bot_chat_own_profile_multiplex_home.py

test(cron): own-profile bot-chat turn from a multiplexed tick runs in the ticking home

2026-09-23 08:09:47 -07:00

test_bot_chat_pending_identity.py

…

test_bot_chat_pending.py

fix(cron): degraded-delivery marker is recognised by its record, not its text

2026-09-20 17:55:54 +05:30

test_bot_chat_timeout_marker.py

refactor(cron): degraded marker is a plain body wrapped by the standard cronjob header

2026-09-20 17:55:54 +05:30

test_bot_chat_warning_replay.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_catchup_policy.py

…

test_claim_job_for_fire.py

…

test_cleanup_timeout.py

fix(cron): one host ticker owns every profile's cron, per profile

2026-09-21 02:52:15 -07:00

test_codex_execution_paths.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_compute_next_run_dst.py

…

test_compute_next_run_last_run_at.py

…

test_context_from_response_extraction.py

refactor(cron): build run documents from the shared frame constants

2026-10-02 01:00:00 +05:30

test_continuity_silent_audits.py

…

test_cron_bot_chat_delivery.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 15:26:34 -04:00

test_cron_context_from.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_created_delivery.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_delivery_redaction.py

…

test_cron_direct_api_call_watchdog.py

fix(agent): keep openai-codex stale floor/hard cap for inline cron Codex (#69734)

2026-09-25 21:28:32 +05:30

test_cron_direct_api_call.py

test: drop tests that cannot fail or duplicate a same-file keeper

2026-09-29 15:58:55 +05:30

test_cron_emfile_stall.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_empty_payload.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_empty_turn_stays_silent.py

…

test_cron_failure_deliver.py

…

test_cron_failure_notice_copy.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_failure_summarizer_inactivity.py

…

test_cron_home_target_profile_scope.py

…

test_cron_import_error_skew_hint.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_inactivity_timeout.py

fix(cron): establish suspend baseline before worker startup

2026-10-02 13:50:23 -04:00

test_cron_incidents_resolve.py

…

test_cron_incidents.py

fix(cron): ignore measured durations in the incident error signature

2026-10-01 20:14:31 -04:00

test_cron_kanban_env_isolation.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_live_bot_delivery.py

…

test_cron_live_delivery_confirmation.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_manage_profile_scope.py

…

test_cron_mcp_toolset_empty_block.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_memory_contract.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_cron_multiplex_desktop_ticker_scope.py

fix(cron): detect a routed fire from its home, not a ticker-only marker

2026-09-29 22:57:48 -04:00

test_cron_multiplex_inflight_release.py

fix(cron): release a profile's in-flight claim under the key it registered with

2026-09-21 02:52:15 -07:00

test_cron_multiplex_liveness_and_pools.py

fix(cron): release a profile's in-flight claim under the key it registered with

2026-09-21 02:52:15 -07:00

test_cron_multiplex_profile_route_preflight.py

…

test_cron_multiplex_served_profile_parity.py

…

test_cron_multiplex_shared_route_delivery.py

…

test_cron_multiplex_tick_ownership.py

fix(cron): release a profile's in-flight claim under the key it registered with

2026-09-21 02:52:15 -07:00

test_cron_next_run_profile_timezone.py

…

test_cron_no_agent.py

test(cron): use env bash shebang in the restored routed no_agent tests

2026-09-29 22:57:48 -04:00

test_cron_non_push_origin.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_origin_synthetic_thread.py

test(cron): pin the near one-shot thread origin through the cronjob_manage create handler

2026-09-20 15:54:45 -07:00

test_cron_pinned_job_fallback.py

fix(cron): a pinned job never falls back to the global fallback chain (#120312)

2026-09-23 10:42:18 -04:00

test_cron_pre_agent_fallback_notice.py

…

test_cron_profile_isolation.py

…

test_cron_prompt_injection_skill.py

…

test_cron_provider_pin.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_reasoning_effort.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_reconnect_only_rejection.py

fix(cron): trim the reconnect-only retention salvage to two router-driven invariants (#125363, salvage #125455)

2026-09-28 05:28:05 -07:00

test_cron_relay_delivery_guards.py

…

test_cron_relay_run_forward.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_run_metrics.py

feat(telemetry): gateway platform health, delivery, first-reply latency and cron run metrics

2026-09-28 12:43:03 -07:00

test_cron_run_stale_claim_reap.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_script.py

refactor(cron): simplify the interpreter salvage after review

2026-09-28 02:32:05 +05:30

test_cron_thread_seed_dm_keying.py

…

test_cron_tick_stale_yield.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_cron_ticker_drift.py

…

test_cron_timezone_migration_catchup.py

…

test_cron_windows_venv_abi.py

test(cron,gateway): drop dead Windows venv setup; say why selected_venv is patched (#122183)

2026-09-26 19:57:55 +05:30

test_cron_workdir.py

feat(cron): allow Python scripts to use an external interpreter

2026-09-28 02:32:05 +05:30

test_cronjob_gateway_liveness.py

test: drop tests that cannot fail or duplicate a same-file keeper

2026-09-29 15:58:55 +05:30

test_dead_owner_claim_reclaim.py

refactor(cron): reap throttle keyed by hermes_home_key

2026-09-20 17:59:57 +05:30

test_delivery_queue_retention_race.py

…

test_delivery_queue.py

fix(cron): order execution, delivery and incident history by instant, not by text

2026-09-28 19:07:43 +05:30

test_dispatch_lateness_stamp.py

…

test_due_scan_dst_fold.py

test(cron): keep two DST fall-back invariants (due gate, real elapsed timers)

2026-09-23 10:37:15 -07:00

test_due_stale_cron_edit.py

…

test_duration_dst.py

…

test_estop.py

test: estop gateway stub answers get_command like a real MessageEvent

2026-09-28 12:43:03 -07:00

test_execution_ledger.py

fix(cron): order execution, delivery and incident history by instant, not by text

2026-09-28 19:07:43 +05:30

test_external_worker_secret_sources.py

fix(cron): trim external-worker plugin discovery salvage to the bar (#121929, salvage #121950)

2026-09-28 05:28:05 -07:00

test_file_permissions.py

…

test_fire_claim_lost_after_delivery.py

test: synchronize observable state and reuse prepared PM fixtures

2026-09-21 14:49:05 -04:00

test_fire_forward_failure_stamp.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_idle_tick_config_skip.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_inflight_stale_guard.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_jobs_changed_notify.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_jobs_crossprocess_lock.py

…

test_jobs_file_ownership.py

refactor(cron): resolve the jobs.json symlink inside _stage_jobs_payload

2026-10-02 01:02:00 +05:30

test_jobs_shrink_merge.py

test(utils): pin the mkstemp_beside contract for both symlink-into-other-FS outcomes

2026-10-02 01:02:00 +05:30

test_jobs.py

fix(cron): keep caching bad-expr cron cadences; skip only import-failure None

2026-10-02 01:33:04 +05:30

test_last_failure_stamp.py

fix(cron): keep a sticky last_failure stamp so a healed run stays visible

2026-10-02 20:29:53 -05:00

test_lifecycle_guard_budget.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_lifecycle_guard_heredoc_walk.py

…

test_lifecycle_guard_live_db.py

…

test_lifecycle_guard_profile_scan.py

fix(cron): remove profile scan ambiguity without dropping long commands

2026-09-30 21:33:08 -04:00

test_media_delivery_parity.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_media_send_timeout.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_mirror_origin_fallback.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_misfire_backstop_oneshot_grace.py

…

test_misfire_catchup.py

test(cron): keep two DST fall-back invariants (due gate, real elapsed timers)

2026-09-23 10:37:15 -07:00

test_missed_window_catchup.py

…

test_monitor_kind.py

fix(cron): only accept a Python executable as a job interpreter

2026-09-28 02:32:05 +05:30

test_notepad.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_oneshot_dispatch_failure_run_claim.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_oneshot_grace_due_scan.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_oneshot_guard_warning.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 15:26:34 -04:00

test_owner_start_time_drift.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_parallel_pool.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_persisted_error_rearm_legality.py

…

test_preflight_config.py

fix(cron): a pinned job never falls back to the global fallback chain (#120312)

2026-09-23 10:42:18 -04:00

test_preflight_credential_verdict_names_home.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_preflight_managed_routes.py

fix(cron): satellite route lookup reads the primary's layered config (#121212, salvage #121595)

2026-09-28 05:28:05 -07:00

test_quota_hold.py

refactor(cron): simplify quota-hold recovery plumbing

2026-09-24 16:59:23 +05:30

test_recurring_persisted_error_recovery.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_recurring_wedge_selfheal.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_relay_fronted_delivery.py

…

test_restart_safe_worker_death_evidence.py

fix(cron): retain bounded history after targeted worker recovery

2026-09-30 20:57:21 -04:00

test_restart_safe_worker.py

fix(gateway): count restart-wait cron exclusions per profile-scoped run

2026-10-01 22:25:21 -04:00

test_retirement_dispatch.py

…

test_rewrite_skill_refs.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_run_error_diagnostics.py

…

test_run_one_job.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_scheduled_occurrence.py

…

test_scheduler_completion_verification.py

…

test_scheduler_cron_session_isolation.py

…

test_scheduler_provider.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_scheduler_shutdown_guard.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_scheduler.py

fix(cron): a partly delivered split send is not re-sent by the standalone fallback

2026-10-01 17:18:56 -07:00

test_script_claim_heartbeat.py

test(cron): publish the descendant pid atomically

2026-09-24 12:47:41 -04:00

test_sessiondb_init_hang.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_shutdown_interrupt.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_stale_module_leaf_imports.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_stale_running_recovery.py

test(cron): two invariant tests for the live-owner stale-claim bound

2026-09-20 17:59:57 +05:30

test_suggestions.py

Merge remote-tracking branch 'origin/main' into ethie/pm-clean

2026-09-23 07:02:44 -04:00

test_terminal_job_rearm.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_ticker_stall.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_ticker_startup_survival.py

…

test_timezone.py

Merge origin/main into ethie/pm-clean

2026-09-23 19:51:34 -04:00

test_unreachable_retry.py

test(cron): say which failure's notice the yield case releases

2026-09-26 20:52:42 +05:30

test_upgrade_module_skew.py

…

test_usage_audit_logger.py

test: purge low-value tests, lane py05 (339 removed)

2026-09-23 03:15:26 -07:00

test_warning_delivery_policy.py

…

test_warning_execution_outcome.py

…

test_warning_transport_contract.py

…

test_worktree_maintenance_isolation.py

…