fix(doctor): name structural state.db corruption honestly and route it to sessions recover
`hermes doctor` reported every write-health-probe failure as "state.db FTS write corruption" and `--fix` ran the FTS repair ladder — rebuild, REINDEX, sqlite_master surgery + VACUUM — on the damaged file. When the damage is structural (canonical tables/indexes), none of those rungs can fix it, each one writes to the torn file in place, and the operator is then told to "restore from the backup copy beside state.db": a `.malformed-backup` that is a snapshot of the same corrupt image. `hermes sessions recover`, the tool that actually rebuilds canonical rows into a fresh file, was never mentioned (#88587; the 1.7 GB field incident lost days to it). Discriminate before mutating. hermes_state_repair.integrity_damage_is_structural maps `PRAGMA integrity_check` output onto the file: a `Tree N` id resolved through sqlite_master.rootpage, an index named in `row N missing from index X`, or a `Freelist:` line is structural unless the object is a Hermes-owned messages_fts* table/shadow (full-matched, so a user lookalike such as archive_fts_data is never swept into the rebuildable set). state_db_has_structural_damage runs it read-only on a fresh connection; an integrity_check that RAISES under the walk (torn root page) is structural too — no FTS-only fixture does that while sessions/messages read cleanly. doctor's state check consults it first: structural damage becomes a manual issue naming `hermes [-p <profile>] sessions recover --source <this db> --inspect-only` (profile pinned, #105887) and explicitly warning off the .malformed-backup; nothing is mutated and no backup is written. FTS-only damage keeps the existing in-place repair path. Verified against real fixtures: a torn `sessions` root page (before: "FTS write corruption", --fix wrote a 1:1 malformed-backup and failed; after: structural, recover guidance, no writes) and the 16-byte DEADBEEF messages_fts_data stomp (still repaired in place via rebuild_fts). Salvaged from PR #88604 (liuhao1024) onto the split doctor_state.py; the classifier lives beside the repair ladder in hermes_state_repair so the ladder itself can consult it next. Fixes #88587
This commit is contained in:
@@ -12,6 +12,7 @@ import itertools
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import stat
|
||||
@@ -684,6 +685,62 @@ def _schema_not_built(exc: BaseException) -> bool:
|
||||
return any(m in str(exc).lower() for m in ("no such table", "no such column"))
|
||||
|
||||
|
||||
# Hermes-owned FTS5 objects: the virtual tables and their shadow b-trees. Full-matched, so a
|
||||
# user-created lookalike (``archive_fts_data``) is not swept into the rebuildable set.
|
||||
_FTS_OBJECT_RE = re.compile(
|
||||
r"messages_fts(_trigram|_cjk)?(_data|_idx|_content|_docsize|_config|_segdir|_segments)?"
|
||||
)
|
||||
_INTEGRITY_TREE_RE = re.compile(r"\bTree (\d+)\b")
|
||||
_INTEGRITY_MISSING_INDEX_RE = re.compile(r"missing from index (\S+)")
|
||||
|
||||
|
||||
def integrity_damage_is_structural(integrity_lines, master_rows) -> bool:
|
||||
"""True when any damaged object named by ``PRAGMA integrity_check`` output lies outside
|
||||
the FTS shadow set: a ``Tree N`` id mapped through ``sqlite_master.rootpage``, an index in
|
||||
``row N missing from index X``, or the file's own freelist. Unparseable lines are not
|
||||
counted (the FTS wording stays, which is incomplete rather than wrong). #88587: an FTS
|
||||
rebuild cannot repair a canonical b-tree, and the ``.malformed-backup`` it leaves behind
|
||||
is a snapshot of the same damage."""
|
||||
name_by_rootpage = {int(rp): name for rp, _type, name in master_rows if rp}
|
||||
for line in integrity_lines:
|
||||
text = str(line)
|
||||
if text.startswith("Freelist"):
|
||||
return True
|
||||
tree = _INTEGRITY_TREE_RE.search(text)
|
||||
if tree:
|
||||
name = name_by_rootpage.get(int(tree.group(1)), "")
|
||||
if name and not _FTS_OBJECT_RE.fullmatch(name):
|
||||
return True
|
||||
missing = _INTEGRITY_MISSING_INDEX_RE.search(text)
|
||||
if missing and not _FTS_OBJECT_RE.fullmatch(missing.group(1)):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def state_db_has_structural_damage(db_path: Path) -> bool:
|
||||
"""Read-only ``integrity_check`` + ``sqlite_master`` rootpage map on a fresh connection;
|
||||
``integrity_damage_is_structural`` over the result. A check that RAISES instead of
|
||||
reporting (a torn page under the walk) is structural too: no FTS-only fixture does that
|
||||
while ``messages``/``sessions`` read cleanly, and the FTS rebuild ladder cannot help.
|
||||
Cannot-open / locked stays False so the caller keeps the FTS path."""
|
||||
try:
|
||||
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=1.0)
|
||||
except sqlite3.Error:
|
||||
return False
|
||||
try:
|
||||
master_rows = [tuple(r) for r in conn.execute(
|
||||
"SELECT rootpage, type, name FROM sqlite_master WHERE rootpage > 0").fetchall()]
|
||||
lines = [str(r[0]) for r in conn.execute("PRAGMA integrity_check").fetchall()]
|
||||
except sqlite3.OperationalError:
|
||||
return False
|
||||
except sqlite3.DatabaseError:
|
||||
return True
|
||||
finally:
|
||||
conn.close()
|
||||
return integrity_damage_is_structural(
|
||||
itertools.chain.from_iterable(line.splitlines() for line in lines), master_rows)
|
||||
|
||||
|
||||
def _db_opens_cleanly(db_path: Path) -> Optional[str]:
|
||||
"""Probe a DB on a fresh connection. Returns None if healthy, else a reason.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user