Commit Graph

2 Commits

Author SHA1 Message Date
Teknium
1e6cfaa0d0 simplify(compat): setup/model_switch/nous_subscription — drop 34 re-exports, repoint 14 callers + 26 test files (~100 sites) 2026-09-03 13:33:10 -07:00
lepetitprince716-prog
7e439dbb1b perf: parallelize provider model-list fetches in model picker
When the 1h provider_models_cache.json TTL lapses, the model picker
serially fetches /v1/models for each authenticated provider. With 10+
providers this stacks to 15-30s of blocking before the picker renders.

Add a parallel prefetch step before the serial picker build loops:
- _collect_authed_provider_slugs(): lightweight credential pre-scan
  that mirrors sections 1/2/2b without fetching model lists
- _prefetch_provider_models_parallel(): ThreadPoolExecutor-based
  concurrent fetch of stale/missing cache entries (max 8 workers)
- update_provider_cache_entry(): thread-safe single-entry cache writer
  with threading.Lock to prevent concurrent write races

Guardrails:
- Skipped when <=3 authed providers (overhead not worth it)
- Skipped when refresh=True (serial path force-refreshes)
- Exception-isolated (falls back to serial path on any failure)
- No behavioral change (same model lists, same picker output)

Closes #80413

(cherry picked from commit 89dddd6cb5d53d73278e0518c375fb5b878e5c6b)
2026-08-15 00:34:29 -07:00