Commit Graph

3 Commits

Author SHA1 Message Date
Teknium
ac9c8b4d02 fix(telemetry): a cancelled or lapsed provider sign-in is abandoned, not failed
hermes.provider_setup.count recorded every user cancel as event=failed
failure_class=cancelled, on every surface: Esc/Ctrl-C/Back in the CLI pickers,
Cancel on a Desktop/dashboard OAuth sign-in (DELETE /sessions), a cancel that
landed before the start response, and consent declined on the provider page.
A device code left to run out was also failed/other on the Nous, xAI and
Codex dashboard pollers (only the free-tier `timeout` reason and Codex
`expired` status were abandoned). Every provider's failure rate was inflated by
walk-aways (openai-codex desktop: 64 of 75 failed rows were `cancelled`).

- provider_setup_fields: failed + cancelled/expired is recorded abandoned, so
  every emitter (CLI, web routers, pollers) gets the same answer.
- setup_failure_class: a lapsed device code (DeviceCodeExpired on the Nous poll,
  AuthError code device_code_timeout/timeout, RFC 8628 expired_token) is
  `expired`; access_denied is `cancelled`; an error whose __cause__ is a network
  error is `network`.
- Dashboard/Desktop pollers keep the exception's closed class on the session
  (note_oauth_failure) so `status=error` no longer collapses to `other`;
  free-tier anon_unreachable is network, a start-route 504 is network.
- DELETE /sessions settles the metric immediately: a Nous/xAI poll blocks for
  the code's lifetime, so the cancel row used to land minutes later or never.

Live (real dashboard server, Desktop-owned, isolated HOME, real providers):
  base: nous/openai-codex cancel -> failed/cancelled (xai cancel never
        settled); nous code left to lapse -> failed/other
  fix:  nous/xai-oauth/openai-codex cancel -> abandoned/none, each at cancel
Live (real `hermes model` in tmux, Ctrl-C at the OpenRouter key prompt):
  base failed/cancelled, fix abandoned/none.
2026-10-05 15:36:14 -07:00
teknium1
c13a4abc8d feat(telemetry): v5 signals — tool_unavailable, provider_setup, feature_adoption, feature_disabled
- hermes.tool_unavailable.count: model called a shipped built-in (BUILTIN_TOOL_NAMES) not enabled in
  the session; tool name + catalog provider/model. Unknown names stay in v4 unknown_tool quality.
- hermes.provider_setup.count: started/completed/abandoned/failed per provider and surface
  (cli_setup, cli_model, tui, desktop, dashboard) with a closed failure_class. Pending marker per flow;
  a dead or stale marker is reported abandoned at the next start (v4 process-marker pattern).
- hermes.feature_adoption.count: once per feature per install at first real use, derived from existing
  counters in the subscriber (metric -> feature table) plus Bot Mode / Projects hooks; bucketed by the
  owning profile's install age.
- hermes.feature_disabled.count: turning off a default-on toolset/skill/plugin/platform/setting (and the
  re-enable back to default), diffed at the config write chokepoints; once per (kind,name,event)/day.
  Surface comes from the user entry point; setup/migrations record nothing.
2026-09-28 12:43:03 -07:00
teknium1
23036e20a6 fix(ux): plain-language, actionable user-facing messages (core)
Squashed integration of the user-facing message audit for this surface set.
Full per-finding receipts: /tmp/ux-audit/lanes/*-receipt.md (campaign artifacts).
2026-09-15 04:12:13 -07:00