Files
hermes-agent/tests/agent/conftest.py
teknium1 df1074b4e5 fix(aux): structured-output rejection no longer kills fallback candidates or costs a doomed first request
Two open atoms of #83390 (DeepSeek "This response_format type is unavailable now"):

* `_call_fallback_candidate_sync/_async` only special-cased auth errors, so when the primary
  aux provider failed (timeout, rate limit, payment) and the fallback landed on a provider that
  rejects `json_schema`, the 400 re-raised and the whole task died — the primary-path rung from
  #89589 never applied there. Both fallback paths now retry once without `response_format`.
* Every structured aux call (titles, kanban decomposer, goal judge, plugin structured calls)
  paid a guaranteed-fail request on providers that lack `json_schema` before the retry. A
  provider profile can now declare `unsupported_response_formats` (DeepSeek: json_schema, per
  https://api-docs.deepseek.com/guides/json_mode) and the recovery ladder remembers any route
  that rejected a type once (host:port scoped), so `_build_call_kwargs` — shared by the primary
  and fallback paths — omits the field before the first request. Dropping rather than
  downgrading to json_object matches the end state the retry already produced; json_object
  needs a JSON-mentioning prompt and some relays return empty content under it.

New logic lives in agent/auxiliary_structured_output.py; the facade only gains the fallback rung
next to the predicate it uses. tests/agent/conftest.py resets the process-level memo per test.

Fixes #83390, #105191. Closes duplicates #84976, #88830, #102849, #113064.
Co-authored-by: Legion-is-life <Legion-is-life@users.noreply.github.com>
2026-09-17 09:11:38 -07:00

48 lines
2.1 KiB
Python

"""Fast-path fixtures shared across tests/agent/.
Many tests in this directory exercise the retry/backoff paths in the
agent loop. Production code uses ``jittered_backoff(base_delay=5.0)``
with a ``while time.time() < sleep_end`` loop — a single retry test
spends 5+ seconds of real wall-clock time on backoff waits.
Mocking ``jittered_backoff`` to return 0.0 collapses the while-loop
to a no-op (``time.time() < time.time() + 0`` is false immediately),
which handles the most common case without touching ``time.sleep``.
We deliberately DO NOT mock ``time.sleep`` here — some tests
(test_interrupt_propagation, test_primary_runtime_restore, etc.) use
the real ``time.sleep`` for threading coordination or assert that it
was called with specific values. Tests that want to additionally
fast-path direct ``time.sleep(N)`` calls in production code should
monkeypatch ``run_agent.time.sleep`` locally (see
``test_anthropic_error_handling.py`` for the pattern).
"""
from __future__ import annotations
import pytest
@pytest.fixture(autouse=True)
def _fresh_structured_output_memo(monkeypatch):
"""The aux client remembers routes that rejected ``response_format`` for the whole process;
a rejection recorded by one test must not strip the field from the next test's request."""
from agent import auxiliary_structured_output
monkeypatch.setattr(auxiliary_structured_output, "_REJECTED_ROUTES", set())
@pytest.fixture(autouse=True)
def _fast_retry_backoff(request, monkeypatch):
"""Short-circuit retry backoff for all tests in this directory.
Tests that assert on the real backoff value opt out with
``@pytest.mark.real_retry_backoff``.
"""
if request.node.get_closest_marker("real_retry_backoff"):
return
# The agent.turn_* retry paths import ``jittered_backoff`` lazily from
# ``agent.retry_utils``; patch it there so rate-limit / invalid-response /
# server-error retries don't burn real wall-clock seconds.
from agent import retry_utils as _retry_utils
monkeypatch.setattr(_retry_utils, "jittered_backoff", lambda *a, **k: 0.0)