prune_sessions hand-rolled the same "guarded by a live lease/lock"
comprehension as the new _guarded_ids helper, so the two could drift on
the next guard change. Move _guarded_ids next to _write_guards_reject in
the maintenance mixin and have prune call it.
delete_session walked the delegate tree up to three times in one write
transaction; compute the target ids once for both the guard check and the
expected-ids fence. delete_sessions ran a per-root guard walk for every
selected id; do one batched check over all roots and their children first
and only attribute per root when something is actually guarded.
The CLI export --delete message repeated "session 'X'" because the
exception text already names the session.
whole_lineages only adds "AND s.id NOT IN (<continued ancestors>)", and
every id in that set is a parent with end_reason='compression'. Every
writer sets ended_at and end_reason together (and every reopen clears
both), so an open row is never in it: the flag could not change
count_open_prune_matches, it only added a recursive CTE and doubled the
bound params on each prune preview.
Remove the parameter and both call sites (CLI prune, dashboard prune),
and move the dashboard "preview lists what it deletes" comment to the
list_prune_candidates call it describes. Prune behaviour is unchanged.
_note_pinned_skipped diffed count_prune_matches(include_pinned=True/False)
with whole_lineages on for prune. Excluding a pinned tip turns it into an
unmatched continuation, so its unpinned compression ancestors drop out of
the "without" count too and the note reported them as pinned (chain
a -> a-2 -> a-3 with only a-3 pinned said "3 pinned sessions").
Count plain filter matches for the note; the delete itself is unchanged.
count_prune_matches has no other caller that needs whole_lineages, so the
keyword goes back to base's signature.
_prune_where popped whole_lineages out of **filters and prune_sessions
injected it by mutating that dict, so a filter-shaped kwarg silently
changed the SQL shape and a typo'd flag would be ignored rather than
rejected. It is now a keyword-only argument of _prune_where and the
list/count prune helpers; prune_sessions passes whole_lineages=True and
the CLI and dashboard previews pass it explicitly. Behaviour unchanged.
Prune's _CONTINUATION_EDGE_SQL re-implemented SessionDB's
_NON_CONTINUATION_CHILD_FILTER_SQL clause by clause. If a new fork marker
lands in one copy only, prune and compression disagree about which
children continue a lineage and prune can again delete the start of a
live chat. Both constants are now built from
hermes_state_common._non_continuation_child_sql (same pattern as
_legacy_reset_child_sql); the class constant is byte-identical and the
prune edge differs only in whitespace.
Retention prune (the default-on startup auto-prune, `hermes sessions prune`
and the dashboard prune) aged every session row on its own. A conversation
that rotating compression split into segments has an ended, old root by
construction, so once that root passed retention_days it was deleted while
the conversation's live tip was still being written: the pre-compression
turns vanished from the resume/Desktop history and from session_search, and
the tip was orphaned.
Prune now deletes a compression ancestor only together with every
continuation after it (`whole_lineages`), so a lineage ages through its
newest segment and goes as a unit once the whole conversation qualifies.
Branch, delegate, reset and tool children do not count as continuations.
The CLI and dashboard prune previews pass the same flag, so they list what
prune deletes; bulk export keeps its current selection.
(cherry picked from commit b082fc6ffa02607cba219a5dfa361d4f3bf4c266)
Three silent data-correctness bugs on a host where ONE process serves every
profile:
- The gateway constructor ran auto-archive and auto-prune/VACUUM once, on a
handle pinned to the construction-time launch home, so a served secondary
profile's state.db was never pruned or vacuumed by anybody. Both now run per
SERVED profile, inside that profile's runtime scope, against its own
SessionDB, its own `sessions:` config and its own transcript dir.
- The dashboard/`hermes serve` auto-archive sweep archived an arbitrary
profile's store but read the config through the PROCESS HERMES_HOME, so one
profile's sessions.auto_archive_days governed every other profile's
retention. It now loads the config of the home whose store it sweeps.
- Auto-VACUUM admission read "no FOREIGN holder" as "the store is quiet", but
the /proc scan skips our own pid by design. VACUUM plus its TRUNCATE
checkpoint therefore retired a WAL generation another live SessionDB in THIS
process still held. The path-keyed registry now answers the in-process half.
Also: argv naming only the install root no longer dismisses a multiplexer as
"another instance" when the store is a profile store beneath that root — under
one-process-per-host that argv is exactly what the holder looks like.
maybe_auto_prune_and_vacuum() runs the same store rewrite as `hermes sessions optimize`
(VACUUM + TRUNCATE checkpoint) from CLI startup and the gateway constructor, with no holder
scan — so the manual command was gated while the automatic producer of the same #110054
failure was not. The VACUUM branch now runs the same foreign_state_db_holders admission and
SKIPS (debug log + a vacuum_skipped_holders count in the result) when a sibling writer holds
the store or a WAL sidecar. Housekeeping never refuses a turn; it only defers the rewrite to
the next run.
`hermes sessions archive --older-than N` (SessionDB.archive_sessions) selected
every ENDED row matching the filters, which includes the compression ancestors
of a long conversation: they are ended (`end_reason='compression'`) and old by
construction. `set_session_archived` then flipped `archived` across the whole
lineage — including the OPEN, actively written, lease-holding live tip — and
the default `archived=exclude` listing (Desktop sidebar, `hermes sessions
list`, /resume) lost the chat while messages kept flowing.
A lineage is now matched through its tip only: `_prune_filter_where` gains
`lineage_tips_only`, which `archive_sessions` always sets and the CLI sets in
archive mode so the dry-run preview and the confirmation count show exactly
the rows the archive will touch. An idle, ended tip still archives its whole
chain (the lineage stays one unit — the listed row is the root, projected to
the tip, so sparing only the tip would leave the chat hidden anyway).
Prune is unchanged. Docs: the bulk-archive section says how compacted
conversations are matched.
Direction from #115500 by @whyyagswhy (automatic archives must not hide the
open live tip); the mechanism differs because the listing keys on the root.
Co-authored-by: whyyagswhy <166958865+whyyagswhy@users.noreply.github.com>
closes#116244
serialize _raise_if_db_replaced inside self._lock and handle reopen after close across optimize_fts, rebuild_fts, vacuum, _enter_fts_fail_open, and _execute_write error branch so a clean close never falsely poisons handles with deletedwalgenerationerror.
Trim the salvaged tests to two invariants (one through the real cli.py
startup path, one at the SessionDB API boundary covering every prune
entry point), make the skip warning fail-closed by naming the allowed
range and the real disable switch, and document the bound in the
sessions user guide. Follow-up to the cherry-pick from #116362.
A negative older_than_days built last_active_before in the future, so
prune_sessions matched every ended session. sessions.retention_days fed
it from config at startup via maybe_auto_prune_and_vacuum on both the
CLI and gateway paths with auto_prune on by default, and
list_never_active_keyed_sessions had the same unguarded cutoff.
_prune_where and list_never_active_keyed_sessions now raise ValueError
on a negative bound, and maybe_auto_prune_and_vacuum warns and skips on
retention_days < 0 so a bad config never reaches a cutoff.
Prune recency is now the freshest of last_activity_at / latest message /
started_at (via _sql_session_last_active), but four doc sites still said
"latest message, else started_at": the list_prune_candidates docstring,
the --older-than/--newer-than help in session_filters, and two comments
in config_defaults. Reword them to the phrasing archive_stale_sessions
already uses so operators are not told live activity is ignored.
Also make archive_stale_sessions use the module constant _LAST_ACTIVE_SQL
instead of an inline _sql_session_last_active("s") call — same alias,
identical SQL, one fewer place to drift.
The archived-holder carve-out in _set_session_title lets the next Bot open
claim the "Bot Chat" title from an archived canonical row. That is correct
for a deliberate sidebar archive, but archive_stale_sessions (the opt-in
sessions.auto_archive sweep) could also archive an idle hidden Bot Chat with
end_reason NULL, which unarchive_recoverable_session refuses; the next Bot
click then stripped the old row's title, retiring the bot's whole history
on an idle timer with no way back.
Skip the hidden canonical Bot Chat in the sweep SELECT using the same
hidden + exact-title predicate set_session_pinned already uses to protect
it, so only an explicit archive retires a Bot Chat. Reword the bot-mode doc
so it no longer promises the retired (still hidden) chat is reachable from
the archive view.
Review finding: auto_archive sweep could irreversibly retire the canonical Bot Chat; docs over-promised archive-view reachability.
Construction-time maybe_auto_archive / maybe_auto_prune_and_vacuum ran
synchronously with no report_startup_progress lease. A multi-minute
VACUUM of a large state.db accrues near-zero CPU, so the startup
watchdog misread it as a parked deadlock and killed the attempt with
exit 75, live-locking gateway restarts. Renew the lease per long step
(prune, orphan sweep, VACUUM, archive) since leases clamp at 900 s,
plus one lease around the gateway maintenance block.
Fixes#111092
prune_sessions(), delete_empty_sessions() and prune_empty_ghost_sessions()
built one IN (?, ..., ?) clause containing every selected session id (the
parent-orphaning UPDATE), so cleaning more than SQLITE_MAX_VARIABLE_NUMBER
sessions failed atomically with "too many SQL variables". The per-row
DELETE loops that followed are folded into the same 900-id batches.
Same single _execute_write() transaction; only the binding is split.
Hand-ported from PR #100658 (targeted the pre-decomposition hermes_state.py
god file; the methods now live in hermes_state_maintenance.py /
hermes_state_sessions.py). The one-pass transcript-directory sweep from
that PR is not ported (out of scope for the variable-limit bug). Authored
by @Mi55ed; ported under --author.
A quarantined/replaced/split-generation handle must never run a full-file rewrite or an FTS5
'optimize': both read damaged or foreign pages and commit the result back, turning contained,
diagnosable corruption into an amplified one. Same guard _execute_write applies to every write.
Salvaged from #102092 onto current main: the _try_wal_checkpoint half landed via #106315's
_quarantine_reason(), so only the two rewrite sites remain.
Follow-up to #106315. vacuum() ran PRAGMA wal_checkpoint + VACUUM + wal_checkpoint(TRUNCATE) on
self._conn with no quarantine check; the only guard it inherited (optimize_fts raising
DeletedWalGenerationError) was swallowed by its own try/except and the rewrite proceeded on the
split-brain handle. Mutation on main: vacuum() returned 2 and rewrote pages after the write stop.
hermes_state.py: delete every '# noqa: F401 (re-exported...)' import block (hermes_state_common/errors/guard/
readpool/sessions/fts/dbfile/wal/repair/registry + agent.context_compressor _DB_PERSISTED_MARKER_KEY); keep
only the names hermes_state.py itself uses, without noqa.
hermes_state_registry.py: drop get_shared_session_db/release_shared_session_db/close_shared_session_dbs
aliases; every caller (gateway/, tools/, tui_gateway/, cron/, mcp_serve, run_agent, tests) now imports
acquire/release/close_all/release_or_close from hermes_state_registry.
hermes_state_titles.py: drop set_auto_title_if_empty shim (title_generator keeps its getattr fallback).
Re-remove shim-only names restored by 34abf954bd: latest_user_message_row_id (tests call
latest_message_row_id(key, role='user'); role-targeting assertions kept) and get_session_activity (tests
build the snapshot via agent.session_activity.build_activity_snapshot over db.get_session(sid)).
hermes_state_wal._log_once resolves its dedupe sets as module globals instead of via hermes_state;
hermes_state_repair helpers call module globals directly (tests patch hermes_state_repair.<name>).
Frozen updater surface untouched (update_cmd_maint imports only SessionDB from hermes_state).
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
hermes_state_maintenance.py 557->418, hermes_state_dbfile.py 545->404.
- _placeholders() replaces 4 inline ','.join('?'...) builders (identical output)
- _write_guards_reject() unifies the lease/lock probe in sweep_orphaned_sessions
and prune_sessions (same kwargs, same exception set; prune keeps set -= order)
- _page_pragmas() absorbs the try/except-debug shape of logical_size_bytes and
_freelist_ratio (log texts unchanged); _try_checkpoint() for the two WAL
checkpoints in vacuum(); _seconds_since() for the two state_meta float parses
- archived tri-state -> f'string' clause (byte-identical SQL)
- dbfile: contextlib.suppress for pass-only excepts, lock closures collapsed,
unreachable size<0 branch dropped, is_zeroed tail folded to one predicate
- docstrings/comments compacted by hand; every WHY/lock-safety invariant kept
SQL PARITY OK (1120 stmts), MSG PARITY OK, import smoke OK.
