fix(state): second-process maintenance on state.db refuses ANY foreign holder
`hermes doctor --fix`'s WAL checkpoint and `repair_state_db_schema`'s preflight documented themselves as fail-OPEN: `live_writer_holds_db` only refused on unknown/deleted/uninspectable holders and then trusted a `BEGIN IMMEDIATE` probe, which is blind to a `journal_mode=DELETE` reader (SHARED only) and cannot run on a malformed file — exactly the states repair and checkpoint get invoked in. A repair in a second process then REINDEXed / VACUUMed a file the gateway still held (#103339 item 2). - `hermes_state_holders.live_writer_holds_db`: any foreign holder of the DB or a sidecar is a live holder; the probe is only an additional positive signal. - doctor `--fix`: the checkpoint runs on `_exclusive_repair_db_guard`'s connection instead of a bare writable `sqlite3.connect`, so an opener arriving after the scan is refused, not joined; `_session_count` is a `mode=ro` reader. - Normal SessionDB writers are untouched: gateway + dashboard in two processes both keep writing (a process-wide flock on the write path — PR #109270's shape — would break that). Tests: the two-process repair race test releases the test process's own header-probe fd (it is a genuine holder now); the mid-repair writer fixture opens its connection after staging starts (a pre-existing holder is refused up front, which is the point). Refs #103339 #100896
This commit is contained in:
@@ -320,15 +320,14 @@ def live_writer_holds_db(
|
||||
*,
|
||||
connect_repair_durable: Callable[..., sqlite3.Connection],
|
||||
) -> bool:
|
||||
"""Return whether repair lacks proven exclusive ownership of ``db_path``."""
|
||||
foreign_holders = foreign_state_db_holders(db_path)
|
||||
if any(
|
||||
pid < 0
|
||||
or path.startswith("uninspectable holder:")
|
||||
or path.startswith("uninspectable descriptor:")
|
||||
or path.endswith(" (deleted)")
|
||||
for pid, path in foreign_holders
|
||||
):
|
||||
"""Return whether repair lacks proven exclusive ownership of ``db_path``.
|
||||
|
||||
ANY foreign process holding the DB or a sidecar is a live holder (#103339): the lock probe below
|
||||
cannot see a DELETE-mode reader (SHARED only) and cannot run at all on a malformed file, and those
|
||||
are exactly the states repair/VACUUM/checkpoint get invoked in. The holder scan is the authority and
|
||||
fails closed on its own failures (unknown/uninspectable sentinels); the probe only adds a positive
|
||||
lock signal on top."""
|
||||
if foreign_state_db_holders(db_path):
|
||||
return True
|
||||
|
||||
probe = None
|
||||
@@ -342,8 +341,7 @@ def live_writer_holds_db(
|
||||
lowered = str(exc).lower()
|
||||
return "locked" in lowered or "busy" in lowered
|
||||
except sqlite3.DatabaseError:
|
||||
return False
|
||||
except Exception:
|
||||
# Malformed/unreadable with no holder on the scan: nobody else has it open, so repair may run.
|
||||
return False
|
||||
finally:
|
||||
if probe is not None:
|
||||
|
||||
Reference in New Issue
Block a user