Files
hermes-agent/tests/gateway/test_model_command_reasoning_flag.py
teknium1 1a59519244 fix(gateway): a --global /model pick leaves config.yaml as the only durable model authority
A `/model <m> --global` switch (typed or picker) used to persist twice: the profile
config.yaml AND a per-session `model_override` in the session store. The session copy
has higher precedence on rehydration, so after a later global change (CLI `hermes model`,
another chat's `--global`) and a gateway restart the stale override silently won —
#100314 saw an explicit `gpt-5.6-sol-900k` resume as the base 272K `gpt-5.6-sol`.

Now `_record_model_switch` writes config.yaml FIRST and, on success, drops the redundant
session override from memory and the store. If the config write fails the switch stays a
truthful session override and the confirmation says "config.yaml not updated (...)" plus
the session-only hint instead of claiming "Saved to config.yaml"; a riding `--reasoning`
pin follows the same effective scope. `--session` and `--once` semantics are unchanged.

Tests: two invariants in tests/gateway/test_model_picker_persist.py (typed+picker clear
the durable override and a fresh SessionStore rehydrates nothing; failed config write keeps
the override and an honest reply), red on base. test_model_command_request_overrides now
points get_hermes_home at its own config so the --provider switch resolves session-scoped
as intended instead of the sandbox's fresh-install first-pick rule.

Fixes #100314
Supersedes #99825 (slim redo; the original wrapped the commit boundary through a
sys.modules-swapped mixin).

Co-authored-by: Andrex Ibiza, MBA <andrexibiza@gmail.com>
2026-09-19 09:59:32 -07:00

54 lines
2.2 KiB
Python

"""Gateway ``/model <m> --reasoning <level>``: the effort rides with the pick through the same
applier ``/reasoning`` uses (session override by default, ``agent.reasoning_effort`` on --global)."""
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
from gateway.config import Platform
from gateway.slash_commands_model import _ModelSwitchContext
def _runner():
from gateway.run import GatewayRunner
runner = object.__new__(GatewayRunner)
calls = {}
runner._switch_cached_agent_model = lambda *_a, **_k: None
runner._record_model_switch = AsyncMock(return_value=None) # None = config write succeeded
runner._model_switch_confirmation = AsyncMock(return_value="switched")
runner._apply_reasoning_selection = (
lambda session_key, platform_key, value, persist_global=False:
calls.setdefault("applied", (session_key, platform_key, value, persist_global)) and "effort set")
return runner, calls
@pytest.mark.asyncio
async def test_reasoning_flag_applies_after_the_switch_with_the_pick_scope():
runner, calls = _runner()
ctx = _ModelSwitchContext(session_key="telegram:c1", source=None, config_path=None,
persist_global=True, reasoning_effort="high")
result = SimpleNamespace(new_model="m", target_provider="nous")
source = SimpleNamespace(platform=Platform.TELEGRAM)
reply = await runner._commit_model_switch(result, ctx, source=source)
assert calls["applied"] == ("telegram:c1", "telegram", "high", True)
assert reply == "switched\neffort set"
@pytest.mark.asyncio
async def test_no_flag_and_once_leave_reasoning_untouched():
runner, calls = _runner()
source = SimpleNamespace(platform=Platform.TELEGRAM)
result = SimpleNamespace(new_model="m", target_provider="nous")
await runner._commit_model_switch(
result, _ModelSwitchContext(session_key="k", source=None, config_path=None, persist_global=False),
source=source)
await runner._commit_model_switch(
result, _ModelSwitchContext(session_key="k", source=None, config_path=None, persist_global=False,
one_turn=True, reasoning_effort="high"),
source=source)
assert "applied" not in calls