5e380d95ba
Structural fix after five review rounds put four blockers in the same subsystem. The root cause was representational: consent history is a sequence of on/off intervals, but it was stored as ONE moving day-stamp plus a revoked flag. Every fix had to mutate that scalar at exactly the right moment from exactly the right place, and each round the mutation was missing from some reachable path (write-once stamp in R3; recorded inside a loop that never runs when sending is off in R4; dead code whenever collection was off in R5). Consent is now recorded as explicit intervals (send_consent_windows) and eligibility is a pure derivation: a package is sent only when its whole period falls inside a recorded window. One writer - reconcile_send_consent - derives window state from an observation of (config, now). It is idempotent and order-independent, so the wizard, the relay, and the mid-pass check all call the same function and cannot disagree; there are no edges to detect and no ordering between writers to get wrong. The relay reconciles once per process BEFORE the collection gate, which fixes round-5 D1 (enabled:false made the only idle-path observer unreachable). The claim reads the table and never writes it, removing the read-path mutation (D2's rewrite vector). Timestamp discipline, each rule load-bearing and mutation-tested: - 'obs' high-water mark: monotonic, advanced only by observations; confirms an open window forward (last_confirmed_at). - 'data' high-water mark: advanced only by stored package period_end; clamps window OPENS so a rolled-back clock cannot slide a window under refused packages already on disk (round-5 D2). - A close stamps last_confirmed_at, never "now": consent is asserted only for observed time, so a hand-edited config with no process running for 90 days fails closed (round-5 D1 strongest form). - The gate requires period containment, not period_start >=, so an intra-day revoke/re-enable holds back the day package (round-5 D3). - Unlike the day-stamp, a revoke/re-enable cycle no longer destroys the undelivered backlog from the earlier consented window (round-5 D4). The redesign was validated BEFORE implementation against all 13 reproduced defect scenarios on a real store; the first two drafts each failed scenarios in that harness (v1 leaked the unobserved-gap case by closing at "now"; v2 leaked refused windows by letting data stamps confirm consent). The harness ships as tests/hermes_cli/test_shared_metrics_consent_windows.py. Deleted: OPT_IN_PERIOD_KEY, SEND_REVOKED_KEY, LAST_SEEN_SEND_KEY, opt_in_period(), record_revoked(), the relay edge detector body, and the setup wizard's key bookkeeping (~170 lines of transition machinery). Schema: two additive tables, version deliberately unchanged; verified against a copy of the real production DB (13 rows intact, reopen no-op). Also kills round-5's M8 survivor: the seen-exclusion mutation now fails the suite. New mutation sweep: 8/8 killed, including one vacuous test of my own this round (obs-mark monotonicity was covered only by coincidence of the data mark; now pinned directly). Documented cost: a fresh package waits at most one process start after its period completes before release (fail-closed direction). 270 tests pass; ruff and windows-footguns clean. Staging E2E re-run through the interval gate: both packages 202.
83 lines
2.8 KiB
Python
83 lines
2.8 KiB
Python
"""Tests for shared-metrics configuration discovery and setup."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
|
|
from hermes_cli.config import DEFAULT_CONFIG
|
|
from hermes_cli.setup import setup_telemetry
|
|
from hermes_cli.subcommands.setup import build_setup_parser
|
|
|
|
|
|
def test_shared_metrics_are_registered_disabled_by_default():
|
|
assert DEFAULT_CONFIG["telemetry"]["shared_metrics"]["enabled"] is False
|
|
|
|
|
|
def test_setup_telemetry_enables_shared_metrics(monkeypatch):
|
|
config = {}
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup.prompt_yes_no",
|
|
lambda _question, default: not default,
|
|
)
|
|
|
|
setup_telemetry(config)
|
|
|
|
assert config["telemetry"]["shared_metrics"]["enabled"] is True
|
|
|
|
|
|
def test_disabling_collection_closes_the_send_consent_window(monkeypatch, tmp_path):
|
|
"""`hermes tools` -> disable shared metrics must withdraw send consent.
|
|
|
|
The not-enabled branch returned early without recording anything, so the
|
|
consent window stayed open and re-enabling later would release every
|
|
package collected in between.
|
|
"""
|
|
from hermes_cli.observability.shared_metrics import SharedMetricsStore
|
|
from hermes_cli.observability.shared_metrics_sender import (
|
|
reconcile_send_consent,
|
|
)
|
|
from hermes_cli.sqlite_util import write_txn
|
|
|
|
store = SharedMetricsStore(
|
|
database_path=tmp_path / "m.db", outbox_directory=tmp_path / "o"
|
|
)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.observability.shared_metrics.SharedMetricsStore",
|
|
lambda *a, **k: store,
|
|
)
|
|
|
|
# The user had consented; now they turn collection off entirely.
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup.prompt_yes_no", lambda _question, default: False
|
|
)
|
|
config = {"telemetry": {"shared_metrics": {"enabled": True, "send": True}}}
|
|
# Consent was granted earlier, so a window is open — that is precisely
|
|
# the state whose closure must be recorded.
|
|
with store._connection() as connection:
|
|
with write_txn(connection):
|
|
reconcile_send_consent(connection, True)
|
|
|
|
setup_telemetry(config)
|
|
|
|
assert config["telemetry"]["shared_metrics"]["enabled"] is False
|
|
assert config["telemetry"]["shared_metrics"]["send"] is False
|
|
with store._connection() as connection:
|
|
open_windows = connection.execute(
|
|
"SELECT COUNT(*) FROM send_consent_windows WHERE closed_at IS NULL"
|
|
).fetchone()[0]
|
|
assert open_windows == 0, (
|
|
"disabling collection left the send consent window open"
|
|
)
|
|
|
|
|
|
def test_setup_parser_accepts_telemetry_section():
|
|
parser = argparse.ArgumentParser()
|
|
subparsers = parser.add_subparsers(dest="command")
|
|
handler = object()
|
|
build_setup_parser(subparsers, cmd_setup=handler)
|
|
|
|
args = parser.parse_args(["setup", "telemetry"])
|
|
|
|
assert args.section == "telemetry"
|
|
assert args.func is handler
|