Files
hermes-agent/website/docs
teknium1 802e6797a0 fix(telemetry): provider allowlist comes from shipped providers only (B1)
What: shared_metrics_catalog.provider_names() no longer reads the live
PROVIDER_REGISTRY / models._KNOWN_PROVIDER_NAMES. It is now the built-in
auth rows (BUILTIN_PROVIDER_IDS), HERMES_OVERLAYS, the three static alias
tables, openrouter/custom, and the names+aliases of profiles registered
from the in-tree plugins/model-providers dir (providers._SOURCES ==
"bundled", process-wide layer, independent of the bound profile home).

Why: auth_plugin_providers mirrors every $HERMES_HOME (and pip) provider
plugin, plus its aliases, into PROVIDER_REGISTRY and the picker labels, so a
user-chosen provider name was treated as shipped and its model id was not
collapsed either. Every consumer goes through provider_metric_name, so the
one fix covers model_route and every per-model v4/v5 metric (tokens,
context_peak, friction, tool_quality, loop_guard, tool_recovery,
model_reply_issue, task_cost, wasted_tokens, cache_break, switch_after,
tool_unavailable, engagement.day), provider_setup (all surfaces, incl. the
PUT /api/env key path), the on-disk pending marker, setup.completed,
model_switch/fallback and install.snapshot main_provider. Pip-installed
provider plugins now read custom too (third-party, not shipped); doc says so.

Probe (fix-catalog/probe_b1_m12.py, user plugin acmecorp-internal alias
acme-llm):
  before: provider_metric_name('acmecorp-internal') -> 'acmecorp-internal'
          route: {'model': 'acme-secret-model-v2', 'provider': 'acmecorp-internal'}
          marker on disk: {..., "provider": "acmecorp-internal"}
  after:  provider_metric_name('acmecorp-internal') -> 'custom' ('acme-llm' too)
          route: {'model': 'custom', 'provider': 'custom'}; marker "provider": "custom"
          install main_provider custom; switch/fallback/setup custom
          shipped plugin deepinfra: {'model': 'meta-llama/...', 'provider': 'deepinfra'}

Test: test_user_provider_plugin_name_and_model_never_leave (RED on base:
marker carried "acme-llm"; GREEN after).
2026-09-28 12:43:03 -07:00
..
…