576accd92b
Twelve modules each carried their own sqlite3.connect + PRAGMA + `with conn:` stack. The #69567 fd-leak fix (a `with conn:` commits but never closes, so each call leaked a connection and its WAL/SHM fds until GC) was pasted as code plus docstring into six of them and hosted_room_policy_checkpoint never received it; plugins/plugin_storage.plugin_db was the only production caller issuing a raw `PRAGMA journal_mode=WAL`, bypassing the network-FS fallback, the WAL-reset-bug gate and the never-live-downgrade invariant that hermes_state_wal.apply_wal_with_fallback carries. hermes_cli/sqlite_util.py (already home to add_column_if_missing/write_txn, imported by cron, gateway and hermes_cli alike) gains `open_db(path, *, db_label, busy_timeout_ms, wal, foreign_keys, synchronous_full, row_factory, check_same_thread, wal_lock_retries, initialize)` and `transaction(conn, immediate=)`; cron/ledger.py is deleted and hosted_rooms_common's open_sqlite/connect/transaction become 1-3 line forwarders. Migrated: agent/verification_evidence, cron/{executions,incidents,notepad, delivery_queue}, gateway/{delivery_ledger,hosted_room_policy_checkpoint, hosted_rooms_common (-> hosted_rooms, hosted_room_driver)}, hermes_cli/ projects_db, tools/async_delegation, plugins/plugin_storage. Behavior changes (each module keeps its effective PRAGMA set otherwise): - hosted_room_policy_checkpoint: connection now closed after every use and on init failure (was leaked per call), busy_timeout PRAGMA set explicitly. - projects_db: gains busy_timeout=5000 (was the sqlite3 default 5 s connect timeout with no PRAGMA); explicit and observable. - delivery_ledger / async_delegation: busy_timeout PRAGMA now mirrors the 10 s connect timeout they already had. - plugin_storage.plugin_db: WAL through apply_wal_with_fallback (DELETE on network filesystems / WAL-reset-vulnerable builds instead of raw WAL); busy_timeout=5000. - cron/incidents._redact_error: redact_sensitive_text(force=True) — the error text is persisted to disk. - delivery_ledger's private duplicate-column guard and the unguarded `ALTER TABLE ADD COLUMN` sites (shared_metrics, api_server_run_idempotency, holographic store, kanban model_override) go through add_column_if_missing. - hermes_state.py::_scrub_surrogates: dead byte-copy of hermes_state_messages._scrub_surrogates (0 callers) deleted.
47 lines
2.1 KiB
Python
47 lines
2.1 KiB
Python
"""Per-plugin persistent storage: ``<hermes home>/plugin-data/<name>/``.
|
|
|
|
Plugins must NOT park state in ``<hermes home>/plugins/<name>/`` (the install dir, deleted by
|
|
``remove`` and git-pulled by ``update``). Secrets are deliberately NOT part of this convention —
|
|
credential reads go through ``agent.secret_scope`` / ``.env``.
|
|
Usage: ``plugin_data_dir("my-plugin") / "state.json"``; ``plugin_db("my-plugin")`` → ``data.db``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
import sqlite3
|
|
from pathlib import Path
|
|
|
|
__all__ = ["plugin_data_dir", "plugin_db"]
|
|
|
|
# Mirrors the plugin-name shape `hermes plugins install` accepts (no separators/traversal).
|
|
_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9._-]{0,63}$")
|
|
|
|
|
|
def _validate_name(name: str) -> str:
|
|
if not _NAME_RE.fullmatch(name) or ".." in name:
|
|
raise ValueError(f"invalid plugin name for storage: {name!r}")
|
|
return name
|
|
|
|
|
|
def plugin_data_dir(name: str) -> Path:
|
|
"""Return (and create) ``<hermes home>/plugin-data/<name>/``; resolves ``get_hermes_home()`` on
|
|
every call so it follows the active profile — don't cache across profile switches."""
|
|
from hermes_constants import get_hermes_home
|
|
root = get_hermes_home() / "plugin-data" / _validate_name(name)
|
|
root.mkdir(parents=True, exist_ok=True)
|
|
return root
|
|
|
|
|
|
def plugin_db(name: str, filename: str = "data.db") -> sqlite3.Connection:
|
|
"""Open ``<data dir>/<filename>``. WAL so a dashboard reader and a tool writer coexist;
|
|
``check_same_thread=False`` for the threaded FastAPI/tool env — caller owns transactions."""
|
|
if Path(filename).name != filename or not filename:
|
|
raise ValueError(f"invalid plugin db filename: {filename!r}")
|
|
from hermes_cli.sqlite_util import open_db
|
|
|
|
# WAL via the shared fallback helper: network filesystems degrade to DELETE and WAL-reset-bug
|
|
# builds never enable it, instead of every plugin DB bypassing those rules with a raw PRAGMA.
|
|
return open_db(plugin_data_dir(name) / filename, db_label=f"plugin-data/{name}/{filename}",
|
|
foreign_keys=True, row_factory=None, check_same_thread=False)
|