fix(aux): title calls on reasoning-mandatory routes stop sending a doomed disable

The title lane asks for thinking off (reasoning_config={"enabled": False}). A route
whose model cannot disable reasoning answers 400 "Reasoning is mandatory for this
endpoint and cannot be disabled"; the ladder then retried at the floor effort and
memoised the route, but only in memory. OpenRouter and the Nous Portal already
publish reasoning.mandatory per model in /v1/models, and Hermes mirrors that
catalog to cache/reasoning_caps.json. The aux floor never read it, so an aux-only
OpenRouter route (nothing else warms that catalog) paid the failed round-trip in
every process.

known_reasoning_floor now also floors a thinking-off aux call when the route's
catalog (memory, then disk mirror; never HTTP) flags the model mandatory, and
kicks the background warm on a cold catalog so the mirror answers every later
call and process. Optional-reasoning models keep their disable.
This commit is contained in:
teknium1
2026-09-23 09:24:36 -07:00
committed by Teknium
parent a5c389e16d
commit 9135bd35c7
3 changed files with 135 additions and 2 deletions

View File

@@ -79,15 +79,45 @@ def remember_reasoning_floor(
_FLOORED_ROUTES.add((_route_key(provider, base_url), str(rejected_kwargs.get("model") or "")))
_NOUS_PROVIDERS = {"nous", "nous-portal", "nousresearch"}
def _catalog_marks_mandatory(provider: Optional[str], base_url: Optional[str], model: Optional[str]) -> bool:
"""True when the route's ``/v1/models`` catalog (OpenRouter, Nous Portal) flags *model*
``reasoning.mandatory``. Cache-only — memory, then the disk mirror — so it never blocks; a cold
catalog is warmed in the background, and the mirror it writes answers every later call and process.
Without this an aux-only OpenRouter route (nothing else warms that catalog) paid the 400 in every
process."""
provider_norm = str(provider or "").strip().lower()
host = (urlparse(base_url or "").hostname or "").lower()
from hermes_cli import models_reasoning_caps as caps_mod
if provider_norm == "openrouter" or host == "openrouter.ai" or host.endswith(".openrouter.ai"):
lookup, warm = caps_mod.openrouter_model_reasoning_capabilities, caps_mod.warm_openrouter_reasoning_caps_async
elif provider_norm in _NOUS_PROVIDERS:
lookup, warm = caps_mod.nous_model_reasoning_capabilities, caps_mod.warm_nous_reasoning_caps_async
else:
return False
try:
caps = lookup(model)
if caps is None:
warm()
except Exception:
return False
return bool(caps and caps.get("mandatory"))
def known_reasoning_floor(
reasoning_config: Any, provider: Optional[str], base_url: Optional[str], model: Optional[str],
task: Optional[str] = None,
) -> Any:
"""*reasoning_config* lifted to the floor when this route+model is known to refuse a disable; unchanged
"""*reasoning_config* lifted to the floor when this route+model is known to refuse a disable — learned
from an earlier 400 in this process, or flagged mandatory by the route's model catalog; unchanged
otherwise. Runs before the profile projection so every wire shape starts at the floor."""
if not _is_disabled(reasoning_config):
return reasoning_config
if (_route_key(provider, base_url), str(model or "")) not in _FLOORED_ROUTES:
if (_route_key(provider, base_url), str(model or "")) not in _FLOORED_ROUTES and not _catalog_marks_mandatory(
provider, base_url, model,
):
return reasoning_config
logger.info(
"Auxiliary %s: %s (%s) cannot disable reasoning; sending effort=%s up front",

View File

@@ -0,0 +1,101 @@
"""The route's model catalog decides the title lane's first request (no guaranteed 400 on mandatory routes).
OpenRouter's (and the Portal's) ``/v1/models`` flags ``reasoning.mandatory`` per model; such a route answers
a thinking-off aux call (``reasoning: {enabled: false}``) with 400 "Reasoning is mandatory for this endpoint
and cannot be disabled". The floor memo only learned that from the 400 itself, per process, so an aux-only
OpenRouter route — nothing else warms that catalog — paid the failed round-trip in every process.
"""
import json
import threading
from unittest.mock import MagicMock, patch
import pytest
import hermes_cli.models as models_mod
from agent import auxiliary_reasoning_floor
from agent.auxiliary_client import call_llm
from hermes_cli import models_reasoning_caps
_CATALOG = [
{"id": "openai/gpt-oss-20b", "supported_parameters": ["reasoning", "tools"],
"reasoning": {"mandatory": True, "supported_efforts": ["high", "medium", "low"]}},
{"id": "x-ai/grok-4.3", "supported_parameters": ["reasoning", "tools"],
"reasoning": {"mandatory": False, "supported_efforts": ["high", "medium", "low", "none"]}},
]
@pytest.fixture
def fresh_process(monkeypatch):
"""Module state of a newly started process (catalog neither in memory nor memo); disk untouched."""
def _reset():
auxiliary_reasoning_floor._FLOORED_ROUTES.clear()
for name in ("_openrouter_reasoning_caps_cache", "_openrouter_reasoning_caps_failed_at"):
monkeypatch.setattr(models_mod, name, None)
for name in ("_openrouter_caps_disk_checked", "_openrouter_caps_warm_started"):
monkeypatch.setattr(models_mod, name, False)
_reset()
yield _reset
auxiliary_reasoning_floor._FLOORED_ROUTES.clear()
def _title_request(model):
client = MagicMock()
client.base_url = "https://openrouter.ai/api/v1"
client.chat.completions.create.return_value = {"ok": True}
with (
patch("agent.auxiliary_client._resolve_task_provider_model",
return_value=("openrouter", model, None, "sk-or-x", None)),
patch("agent.auxiliary_client._get_cached_client", return_value=(client, model)),
patch("agent.auxiliary_client._validate_llm_response", side_effect=lambda resp, _task, **_kw: resp),
patch("agent.auxiliary_client._try_payment_fallback", return_value=None),
):
call_llm(task="title_generation", messages=[{"role": "user", "content": "hi"}],
reasoning_config={"enabled": False})
return client.chat.completions.create.call_args_list[0].kwargs.get("extra_body", {}).get("reasoning")
def test_catalog_mandatory_model_gets_the_floor_on_the_first_request(fresh_process):
"""A mirror left by an earlier process: the mandatory model's first thinking-off request already
carries the floor; an optional model on the same route keeps its disable."""
models_reasoning_caps._seed_reasoning_caps(models_reasoning_caps._OPENROUTER_CATALOG_URL, _CATALOG)
assert _title_request("openai/gpt-oss-20b") == {
"enabled": True, "effort": auxiliary_reasoning_floor.REASONING_FLOOR_EFFORT}
assert _title_request("x-ai/grok-4.3") == {"enabled": False}
def test_cold_catalog_is_warmed_so_the_next_process_starts_at_the_floor(fresh_process, monkeypatch):
"""No mirror yet: the first process can only learn from the 400, but its lookup warms the catalog, so
the next process never sends the disable."""
monkeypatch.delenv("PYTEST_CURRENT_TEST", raising=False) # the warmer is a no-op under pytest
warmed = []
real_thread = threading.Thread
def _joined_thread(*args, **kwargs):
thread = real_thread(*args, **kwargs)
if kwargs.get("name") == "reasoning-caps-warm":
warmed.append(thread)
return thread
class _Resp:
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def read(self):
return json.dumps({"data": _CATALOG}).encode()
monkeypatch.setattr(models_mod, "_urlopen_model_catalog_request", lambda req, timeout: _Resp())
monkeypatch.setattr(models_reasoning_caps.threading, "Thread", _joined_thread)
_title_request("openai/gpt-oss-20b")
assert warmed, "a cold catalog lookup must start the background warm"
for thread in warmed:
thread.join(timeout=10)
fresh_process()
assert _title_request("openai/gpt-oss-20b") == {
"enabled": True, "effort": auxiliary_reasoning_floor.REASONING_FLOOR_EFFORT}

View File

@@ -1474,6 +1474,8 @@ This is the per-task counterpart of the global `agent.reasoning_effort`: run com
If the endpoint rejects the reasoning field outright (a chat-only model behind an OpenAI-compatible relay answering `400 Unrecognized request argument supplied: reasoning_effort`, or the reversed wording `400 reasoning_effort 'none' unsupported; use minimal|low|medium|high|xhigh`), the auxiliary call is retried once with every reasoning field omitted, so the task (for example the session title) still completes with the endpoint's default behaviour. The main conversation applies the same recovery: when a route rejects the reasoning-off request Hermes sends for a thinking-only truncated continuation, the disable is dropped for the rest of the session and the request is retried with the route's default.
Some models cannot turn thinking off at all (`400 Reasoning is mandatory for this endpoint and cannot be disabled`). For those, a thinking-off auxiliary call (title generation, or any task set to `reasoning_effort: none`) goes out at the lowest effort (`low`) instead of the disable. Hermes knows ahead of time when the route's model catalog marks the model mandatory (OpenRouter and Nous Portal `/v1/models`, cached in `cache/reasoning_caps.json`), or when the route already answered an earlier disable that way in the same process. So the rejected request is not sent. A fresh install with no cached catalog can still see that 400 once: the lookup fetches the catalog in the background and later calls use it.
**Background review is different:** a same-model review fork always inherits the parent's reasoning effort. `auxiliary.background_review.reasoning_effort` is ignored on that path, including when the parent provider/model is explicitly selected. This preserves byte-identical reasoning settings, system prompt, full conversation snapshot, and tool definitions for prompt-cache parity; there is no independent-effort switch for same-model reviews. See [background review reasoning](./features/memory.md#same-model-review-reasoning). When the review is routed to a different provider/model, `reasoning_effort` applies to that routed fork (unset = the routed provider's default). Hermes prints a one-time warning when the key is set but the review runs on the main model.
**MoA also uses a different configuration:** reasoning depth for Mixture-of-Agents is configured **per slot** in the MoA preset (`moa.presets.<name>.reference_models[].reasoning_effort` / `aggregator.reasoning_effort`), not on the `moa_reference`/`moa_aggregator` auxiliary blocks — see [Mixture of Agents](./features/mixture-of-agents.md).