8ddff33e4c
Third independent review. Both blockers reproduced against a real store before and after the fix. BLOCKER 1 — head-of-line starvation. The claim query is LIMIT 1, and a package already handled this pass was rejected AFTER the fetch, so _claim_next returned None and send_pending read that as 'queue empty'. Any row that sorts first and becomes eligible again mid-pass therefore terminated the pass. This is reachable normally: a 429 with a short Retry-After, or a pass outliving the 15-minute failure backoff (a legal pass runs ~1900s). Measured: 10 of 19 healthy packages silently dropped. The seen-set is now excluded IN SQL, so None genuinely means no eligible work. Same scenario now delivers 19 of 19. BLOCKER 2 — revoking consent leaked once it was re-granted. opt_in_period was write-once, so packages collected while the user had send: false still had period_start >= the ORIGINAL opt-in day; re-enabling released the whole refused window. Reproduced: 5 packages from a 5-day opted-out window transmitted on re-enable. Turning sending off now closes the consent window, and the next enabled pass opens a new one from that day. Recorded both in the setup wizard and in the sender itself, because config.yaml can be hand-edited where the wizard never sees it. Also: a send_attempts ceiling (a poisoned head row burned ~160 requests over 30 days, unbounded), _defer clamps to >= 1s so it cannot write a past deadline, and the dead skipped_not_due field is removed. Test-quality fixes, since vacuous tests have been the recurring problem: - the lease test asserted only 'in the future', passing for a 1s lease; it now requires the lease to outlast one package's worst legal case - test_shutdown_joins_the_send_thread grepped getsource for a method name — a change-detector AGENTS.md rejects — and is now behavioural - gzip determinism was unguarded: both retries in one pass compress in the same second, so removing mtime=0 was caught by nothing. Now compares output across a real second boundary. All five new regressions are mutation-verified: reintroducing each bug fails its test. The first attempt-ceiling test SURVIVED its mutation (the seeded row was excluded by another predicate) and was rewritten to drive the real loop. 251 tests pass. Staging E2E re-run: both packages 202.
90 lines
3.0 KiB
Python
90 lines
3.0 KiB
Python
"""Tests for the `hermes tools` shared-metrics consent toggle.
|
|
|
|
AGENTS.md requires outbound telemetry to be reachable from a config gate, the
|
|
setup prompt, AND `hermes tools`. These cover the third surface.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from hermes_cli.tools_config import (
|
|
_configure_shared_metrics_interactive,
|
|
_shared_metrics_menu_label,
|
|
_shared_metrics_state,
|
|
)
|
|
|
|
|
|
def _config(**shared):
|
|
return {"telemetry": {"shared_metrics": shared}}
|
|
|
|
|
|
class TestState:
|
|
def test_missing_telemetry_section_is_off(self):
|
|
assert _shared_metrics_state({}) == (False, False)
|
|
|
|
def test_malformed_section_does_not_raise(self):
|
|
assert _shared_metrics_state({"telemetry": "nonsense"}) == (False, False)
|
|
|
|
def test_reads_both_flags(self):
|
|
assert _shared_metrics_state(_config(enabled=True, send=True)) == (True, True)
|
|
|
|
|
|
class TestMenuLabel:
|
|
def test_off_state(self):
|
|
assert "off" in _shared_metrics_menu_label({})
|
|
|
|
def test_local_only_state(self):
|
|
label = _shared_metrics_menu_label(_config(enabled=True))
|
|
assert "collecting locally" in label
|
|
assert "Nous" not in label
|
|
|
|
def test_sending_state_names_the_destination(self):
|
|
label = _shared_metrics_menu_label(_config(enabled=True, send=True))
|
|
assert "sending to Nous" in label
|
|
|
|
|
|
class TestToggle:
|
|
def test_enabling_send_persists(self, monkeypatch):
|
|
config = _config(enabled=True)
|
|
saved = {}
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup.prompt_yes_no", lambda *_a, **_k: True
|
|
)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup._record_send_consent_change", lambda **_k: None
|
|
)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.tools_config.save_config",
|
|
lambda cfg: saved.update({"cfg": cfg}),
|
|
)
|
|
_configure_shared_metrics_interactive(config)
|
|
assert config["telemetry"]["shared_metrics"]["send"] is True
|
|
assert saved, "a consent change must be written to disk"
|
|
|
|
def test_no_write_when_nothing_changed(self, monkeypatch):
|
|
config = _config(enabled=False, send=False)
|
|
saved = []
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup.prompt_yes_no", lambda *_a, **_k: False
|
|
)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.tools_config.save_config", lambda cfg: saved.append(cfg)
|
|
)
|
|
_configure_shared_metrics_interactive(config)
|
|
assert saved == []
|
|
|
|
def test_disabling_collection_also_disables_sending(self, monkeypatch):
|
|
"""The toggle must not leave send=true with nothing to send."""
|
|
config = _config(enabled=True, send=True)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.setup.prompt_yes_no", lambda *_a, **_k: False
|
|
)
|
|
monkeypatch.setattr(
|
|
"hermes_cli.tools_config.save_config", lambda cfg: None
|
|
)
|
|
_configure_shared_metrics_interactive(config)
|
|
shared = config["telemetry"]["shared_metrics"]
|
|
assert shared["enabled"] is False
|
|
assert shared["send"] is False
|