Review findings on #110914 (@ehz0ah): the psutil leg compared watched
abspath against the kernel-resolved path psutil reports, so a symlinked
HERMES_HOME on macOS returned no holders and let the fallback probe mint
replacement sidecars under a live writer. Both sides now realpath.
doctor's `file:{path}?mode=ro` truncated at '?'/'#' in a home name;
build the URI with as_uri().
`hermes doctor --fix`'s WAL checkpoint and `repair_state_db_schema`'s
preflight documented themselves as fail-OPEN: `live_writer_holds_db` only
refused on unknown/deleted/uninspectable holders and then trusted a
`BEGIN IMMEDIATE` probe, which is blind to a `journal_mode=DELETE` reader
(SHARED only) and cannot run on a malformed file — exactly the states repair
and checkpoint get invoked in. A repair in a second process then REINDEXed /
VACUUMed a file the gateway still held (#103339 item 2).
- `hermes_state_holders.live_writer_holds_db`: any foreign holder of the DB or
a sidecar is a live holder; the probe is only an additional positive signal.
- doctor `--fix`: the checkpoint runs on `_exclusive_repair_db_guard`'s
connection instead of a bare writable `sqlite3.connect`, so an opener
arriving after the scan is refused, not joined; `_session_count` is a
`mode=ro` reader.
- Normal SessionDB writers are untouched: gateway + dashboard in two processes
both keep writing (a process-wide flock on the write path — PR #109270's
shape — would break that).
Tests: the two-process repair race test releases the test process's own
header-probe fd (it is a genuine holder now); the mid-repair writer fixture
opens its connection after staging starts (a pre-existing holder is refused up
front, which is the point).
Refs #103339#100896
Field evidence (2026-09-07, production host): the host runs two independent
Hermes instances - a main gateway (user ubuntu, HERMES_HOME=/home/ubuntu/.hermes)
and a demo gateway (user demo, HERMES_HOME=/home/demo/.hermes). Because the
demo process is owned by another user, its /proc/<pid>/fd table is unreadable
and foreign_state_db_holders() falls back to cmdline + _looks_like_hermes().
The demo argv matches Hermes patterns exactly, so it was flagged as an
uninspectable holder of the MAIN instance's state.db even though lsof proved
0 open handles on it. Result: _recover_stale_fts was deferred 42 times across
6 gateway restarts, the fts_stale breadcrumb never cleared, and FTS
self-repair stayed permanently disabled.
#92419 removed substring false positives (journalctl/grep mentioning hermes);
a genuine second instance with a DIFFERENT HERMES_HOME was still misjudged.
Add _argv_scoped_to_other_home(): when the argv of an uninspectable Hermes
process proves it lives under a different /.hermes home (or a state.db
sidecar under a different parent) AND no token references our state.db,
sidecars, or home, do not count it as our holder. Applied to all three
uninspectable branches (holder + two descriptor paths). Ambiguous argv
without absolute-path tokens remains fail-closed, preserving the
conservative intent.
References #92401
Follow-up on the #97330 salvage (#97329): main grew `_canonical_sqlite_path`
after the PR's base; the holder module re-implemented it as a closure. Export
`canonical_sqlite_path` from hermes_state_holders and alias the legacy name in
hermes_state. Drop the unused `_read_proc_argv` / `_looks_like_hermes`
re-exports (no in-tree callers).
Move live-holder inspection into a bounded helper and make repair fail closed when a process still owns the state database, including deleted WAL/SHM descriptors and ambiguous procfs reads.
Preserve the contributor lineage from the original four-commit review train while presenting one coherent release object on current main.
Co-authored-by: fangliquanflq <fangliquan@qq.com>
Co-authored-by: ciabata <296402666+ciabata-git@users.noreply.github.com>
