Files
hermes-agent/agent
kshitij 0038d96b78 perf(caching): narrow the widened capability lookup to the LiteLLM grant
Self-review follow-up, caught by benchmarking the previous commit.

Widening the custom-provider capability-lookup gate to `is_anthropic_wire or
_is_litellm_route(...)` made EVERY chat_completions route with a litellm-ish
provider/host enter the lookup, including non-Claude models that the grant
branch below can never match. Measured on a route with no config.yaml
(the uncached worst case) that was ~7.5us -> ~1528us per evaluation.

Narrowed the gate to the exact condition the LiteLLM branch grants on
(chat_completions + Claude + litellm route), computed once into a local and
reused by the branch itself so the predicate no longer runs twice.

Measured with a realistic config.yaml present (mtime cache warm), vs
origin/main:
  live-agent policy      20.6us -> 61.7us
  destination planning  219.3us -> 347.7us

Sub-millisecond and scoped to the routes that actually opted in. The
earlier 1.5ms figures were a tempdir artifact: load_config_readonly's
mtime cache cannot engage when no config.yaml exists, which is never true
of a real install. Non-LiteLLM and non-Claude routes are unaffected
(openrouter Claude measured flat at ~7.9us).

Tests: 82 passed across the policy and TTL-propagation modules.
2026-08-16 14:34:30 +05:30
..
2026-08-12 13:37:04 -04:00