turn/start can change the model but not modelProvider, which is fixed when the thread starts. With the model now forwarded per turn, switching in place between codex's own provider and a named custom provider would send the new provider's model to the old provider's thread. Record the thread's model provider next to its prompt composition and retire the thread when either changes.
The OpenAI API-key rung reaches the app-server as provider=custom with no
codex model_providers id, and its model is not vendor-normalized at agent
init (only openai-codex is), so a configured "openai/gpt-5.5" now went to
codex verbatim where it was previously dropped for codex's default. When
codex's own provider serves the turn, normalize the slug the way
openai-codex does; a named custom provider's model ids pass through.
The provider lookup moves into _codex_model_provider and is passed to
_codex_wire_model, and the session comment now says model is sent for every
provider, modelProvider only for named custom providers.
The CLI/TUI /model switch mutates the running agent, and its codex session
(and thread) is reused, so the model fixed at thread/start stayed in force
for the rest of the session. turn/start accepts a model that codex applies
to that turn and later ones (codex 0.147 schema; live: a thread started on
codex's default ran a turn on gpt-5.5 with modelContextWindow 258400).
Every turn now sends the agent's current wire model.
With the model now forwarded, a selected -900k alias reached codex verbatim
and every turn failed: live on codex 0.147 with ChatGPT auth,
thread/start {model: "gpt-5.6-sol-900k"} -> turn 400 "The
'gpt-5.6-sol-900k' model is not supported when using Codex with a ChatGPT
account". The alias is Hermes-side only (the Responses transport already
strips it); codex applies the catalog's extended window to the base slug on
its own (thread/tokenUsage reports modelContextWindow 828400 for
gpt-5.6-sol). Strip the valid suffix before it goes on the wire.
Codex homes are service-scoped while Hermes model selection is profile and
session scoped. thread/start only carried the model for named custom
providers, so openai/openai-codex sessions on model.openai_runtime:
codex_app_server silently ran codex's own default model, whatever /model or
config.yaml selected. Forward the agent's model on every thread/start;
the custom-provider modelProvider id and the no-credential payload are
unchanged.
Salvaged from #83269 onto current main (the session already accepts model=
since #115905; the forwarding was narrowed to custom providers there).
OpenAI's Ultrafast service tier (service_tier="ultrafast") is broadly
available for GPT-6 Astra in the API and Codex (Pro 500 / Enterprise),
billed at 6x Standard. GPT-6.1 Sol Ultrafast is announced as coming soon.
- One tier table: agent.fast_mode.parse_service_tier / STATIC_TIERS now
back the CLI, gateway and TUI config loaders (three hand-copied parsers).
- resolve_fast_mode_overrides(tier="ultrafast") sends the tier only for
Ultrafast models on api.openai.com / chatgpt.com; anything else gets
nothing rather than a silent swap to another paid tier.
- /fast ultrafast on CLI, gateway (typed + picker when supported) and TUI;
static-tier attach sites pass the tier through.
- Pricing: the response's SERVED service_tier is folded into usage (Codex
stream assembler keeps it); served ultrafast prices from a published
6x row (272K whole-request tier included). A request asking for
Ultrafast but served at default bills at Standard.
- i18n: 4 new keys in all 17 locales; option lists mention ultrafast.
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule
The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:
- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
names, restored dead definitions) and the three re-export stub modules
(gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
`plugins.allow_deprecated_imports` escape hatch
An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.
hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).
In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.
* chore: retrigger CI (zero-job startup_failure phantom)
* test: drop resolution allowlist rows for the two deleted which() sites
hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
Codex app-server webSearch items fire the live tool bubble under
codex_web_search_<itemId>, but the projector had no entry for them, so the
stored transcript kept an opaque "[codex webSearch] {json}" note instead.
Any transcript refresh or reopen then replaced the search card with raw
JSON.
The projector now records webSearch as a web_search tool_call under the
same id and name the live bubble uses. The tool result is
{"provider": "codex", "status": ...}, so a later non-Codex turn can tell
Codex ran the search.
Salvaged from #66995: only the webSearch projection is kept, reworked
onto the _TOOL_PROJECTIONS table and the live web_search name. The MCP
elicitation trust change is not included.
The event bridge listed the text-delta methods twice (display handlers and
the #118410 liveness set) and duplicated _fire_delta's text extraction.
Derive both from _CODEX_TEXT_DELTA_METHODS, share _delta_text(), and hoist
the progress item-type set to a module constant (no per-event frozenset).
The attempt-local first-progress deadline added `progress_timeout` to
`_NonStreamWatchdogs`, but the SimpleNamespace stubs in
test_nonstream_wait_notice.py and test_wait_notice_cadence.py were not
updated. `_emit_wait_notice` reads `wd.progress_timeout` inside its
blanket `except Exception`, so the AttributeError silently produced no
notice and no liveness touch: 11 tests red.
The new "Codex stream opened" debug line also called `time.time()`
eagerly, consuming a tick of the 3-tick stub in
test_codex_first_event_timing.py and shifting the first-event stamp.
The first-event log a few lines below already carries the attempt's
timeline anchor, so the open marker logs without an epoch.
The per-attempt "first parsed event" / "first substantive progress" /
"physical retry" markers are what an operator needs to see where a
lifecycle-only Codex attempt stalled, so they stay at info. The
"stream opened" line fires on every attempt and carries no diagnostic
value on its own, so it goes to debug to keep the default log quiet.
Co-authored-by: JoaoMarcos44 <joaomarcosdias444@gmail.com>
A codex thread that codex hands back via thread/resume already holds the conversation, but a
thread started fresh did not: a session that ran on another provider before /model switched to
openai-codex, a stored thread codex could not resume, or a thread retired mid-session (prompt
composition change, wedged client) answered the first turn blind. The prior user/assistant text,
tool names and tool-result previews (most recent 32K chars) now ride once on
thread/start.developerInstructions after the prompt composition; thread/resume never carries them,
and the recorded composition stays the bare prompt so the seed cannot make the next turn retire the
thread.
Direction from #26081 (first-turn seeding of the Hermes transcript); redone on the extracted
agent/codex_runtime.py path with the system prompt sent once (#115759) instead of duplicated.
Completes #26035 / #74712 (closed by #115759 for the prompt half; this is the history half).
Co-authored-by: LeonSGP43 <154585401+LeonSGP43@users.noreply.github.com>
Codex app-server thread survives an API-server restart: thread id persisted per session and thread/resume'd, fail-closed to a fresh thread (#100531, salvage #103352)
After a codex app-server turn's projected rows are durable in the session DB,
store the codex thread id as ``codex_thread_id`` in the session row's
model_config (atomic merge via patch_session_model_config; never for a retired
thread). The FIRST CodexAppServerSession an AIAgent builds for that session
passes the stored id as resume_thread_id, so a rebuilt agent — the next
/api/sessions/{id}/chat request, or the first turn after the API server or
gateway restarts — resumes the model-side thread before turn/start instead of
starting an empty one while Hermes' own transcript continues.
Fail closed when codex cannot hand the thread back (rollout gone, CODEX_HOME
changed, previous app-server killed mid-write): drop the stored id, start a
fresh thread on the same client, and say so once —
"Codex thread could not be resumed; starting a new one." — through
_emit_diagnostic_status, the lifecycle status rail every surface renders (CLI
vprint, TUI/Desktop and gateway status_callback). No other lifecycle change:
a retired or prompt-recreated session in the same process keeps today's
fresh-thread behaviour and overwrites the binding once its turn commits.
Why: CodexAppServerSession kept the thread id in memory only, so every
API-server restart (and every per-request agent) silently reset the model's
memory of the conversation (#100531). Supersedes the persistence half of
#100528 (_persist_projected_messages now reports durability) and the
refuse-and-raise policy of #103352 with the maintainer-approved bounded slice.
When every pre-stream connect attempt to an endpoint fails, the user only saw
"Connection error." repeated per outer retry; the host, the attempt count and
the serialized request size lived in agent.log alone. #97548's reporter had an
~829 KB Codex Responses request fail twice before the stream opened while short
chats went through, which is the request-size-limit signature, and nothing on
screen said so.
One buffered diagnostic line now flows through the existing retry-status path
(flushed on terminal failure, dropped on recovery) on both the Codex Responses
runtime and the Chat Completions stream worker: "Could not open a stream to
<host> after N attempts (request X KB); ...". The host comes from the failed
request's URL (the endpoint actually contacted, proxies included) and the size
from the buffered httpx request body. Re-entering the stream call from the
outer retry/fallback loop does not add another copy.
Part of #97548
_on_event is only ever installed as on_event=_fenced(_on_event), and _fenced
already refuses every call once the request is retired, so the inner predicate
could never be false when reached. Comment now points at the fence.
Responses lifecycle events may arrive before answer text, but the streaming runtime never populated the existing post_api_request timing field. Record the first accepted event while preserving request-retirement fencing.\n\nCloses #105311.
On the Relay-managed path ManagedLlmStream.close() cannot reach the provider
response while the loop is still running the drain (RuntimeError, swallowed by
_close_event_stream), so the httpx response and drain thread lingered until the
socket read timeout. Capture the raw stream in _codex_stream_created and close it
too on drain timeout; the unmanaged path is unchanged (idempotent double close).
Replace the hardcoded 2.0s module constant from the salvaged commit with
``agent.stream_drain_timeout`` in DEFAULT_CONFIG (default 2.0, ``0`` skips the
drain), read through ``load_config_readonly`` at drain time. Non-secret knobs
belong in config.yaml, not a new HERMES_* env var. Documents the budget in the
timeouts table and adapts the salvaged test to the new seam.
Why: a relay that never closes the SSE socket after ``response.completed``
wedged the turn until the stale watchdog fired and discarded the already-billed
response (#103864). The drain is a courtesy to Relay's finalizer, so it must be
bounded; the bound needs a user-tunable knob for slow-but-honest relays.
#95429 acceptance criterion 3: retrying a zero-event oversized request must not
resend the same pathological payload unchanged. run_codex_stream's in-place
transport reconnect re-sent dict(api_kwargs) verbatim with no diagnostic.
When an attempt dies before any stream event was received and the serialized
input exceeds the per-turn budget, inline function_call_output strings over the
per-result threshold (results that escaped commit-time persistence) are spilled
through maybe_persist_tool_result -- bounded preview + recoverable
<persisted-output> reference -- and one warning records
serialized_input_bytes=before -> after with the attempt number. When nothing is
prunable the warning says the payload is being resent unchanged. Only the
retried wire payload changes; the caller's kwargs and history are untouched.
No new config or env vars.
Before: zero-event reconnect resent 760K-char tool output byte-identical, no log.
After: retried input < 1/10 the size, spill file holds the full text, one
"Codex zero-event retry (attempt 1/2): ... serialized_input_bytes=N -> M" line.
_record_codex_app_server_usage fed inputTokens — which the app-server protocol
reports INCLUSIVE of cachedInputTokens — into CanonicalUsage.input_tokens while
also recording cachedInputTokens as cache_read. CanonicalUsage.prompt_tokens sums
both buckets, so cached tokens were counted twice: inputTokens=80,
cachedInputTokens=20 produced prompt_tokens=100 instead of 80. The inflated value
reached session accounting, cost estimation and context_compressor.last_prompt_tokens
(premature compaction) on every cache-hit turn (#105412, #63654, #48801).
Subtract cache_read from the reported input to get the uncached remainder,
mirroring normalize_usage's codex_responses branch so both Codex transports yield
identical accounting. The transport exposes no cache-write, so only cache_read is
subtracted. totalTokens stays the provider passthrough. The existing integration
test asserted the wrong totals; it now pins prompt=80, uncached=60, cached=20.
Ported from PR #63654 onto the current adapter shape.
Fixes#105412
codex app-server emits commentary deltas + completed, then final deltas +
completed. The delta buffer accumulated "commentary + final", so the completed
final no longer prefix-matched its own deltas, was delivered with
already_streamed=False and the gateway posted a second copy next to the
streamed message (#74248 boundary 2). Drop the buffer once a completed
agentMessage has been compared against its own deltas.
writer_token now carries value/raw_stream/superseded_logged; the per-attempt reset clears all three
so the superseded raw stream is still closed (#115817) and the warn-once flag is kept (#115900).
_ensure_codex_session records the developerInstructions composition it sent; when the
TUI/Desktop gateway mutates the live agent's prompt in place (/personality, prompt
mirror) the next turn closes the stale thread and starts one carrying the new prompt,
matching what the standard loop applies on its next API call. Sessions attached
without a recorded composition are kept. Docs sentence updated.
_on_event is only ever installed as on_event=_fenced(_on_event), and _fenced
already refuses every call once the request is retired, so the inner predicate
could never be false when reached. Comment now points at the fence.
On the Relay-managed path ManagedLlmStream.close() cannot reach the provider
response while the loop is still running the drain (RuntimeError, swallowed by
_close_event_stream), so the httpx response and drain thread lingered until the
socket read timeout. Capture the raw stream in _codex_stream_created and close it
too on drain timeout; the unmanaged path is unchanged (idempotent double close).
`model.openai_runtime: codex_app_server` only ever admitted `openai` /
`openai-codex`: a named custom provider (`providers.<name>`) resolves to
provider="custom" on the named-custom ladder rung, which never ran the
runtime gate, so `/codex-runtime codex_app_server` silently left the main
turn on Hermes' chat-completions client. And even when routed, thread/start
sent only `cwd`, so codex could not know which of its own providers to use.
- `_maybe_apply_codex_app_server_runtime` takes `requested_provider` and
admits provider="custom" only when `codex_model_provider_id()` finds a
configured `providers.<name>` entry (bare `custom`, ollama/vllm aliases
and unknown names have no stable id -> ineligible, unchanged).
- The named-custom rung applies the same opt-in the pool rung already does
for openai/openai-codex.
- `CodexAppServerSession(model=, model_provider=)` -> `thread/start.model` /
`.modelProvider` (fields verified against the codex 0.147 app-server
schema). `_ensure_codex_session` fills them only for custom agents; codex
resolves base_url/env_key from its own `[model_providers.<name>]`, so the
Hermes credential never enters the JSON-RPC payload.
- `tui_gateway/server.py::_make_agent` forwards `requested_provider` so the
Desktop/TUI agent knows the provider id (it otherwise collapses to
"custom" and codex would fall back to its default provider).
- Docs: matching `[model_providers.<name>]` + `env_key` requirement and the
bare-`custom` ineligibility.
Ported and trimmed from #75191 by @cosin2077 (aux-loop `allow_codex_app_server`
plumbing, `cli-config.yaml.example` block and the integration-test suite dropped:
background_review already maps codex_app_server -> codex_responses on main).
Fixes#75186
When a newer attempt claimed the stream-writer slot, run_codex_stream's
accept_chunk hook returned False and Relay stopped consuming the Codex
Responses stream. The assembler then returned a "completed" response
holding only the text seen so far, and the gateway delivered a reply cut
mid-sentence (#69486) even with display streaming disabled.
Supersession now fences only the live callbacks (text and reasoning
deltas, commentary, first-delta) through the attempt's writer token and
consumption continues to the terminal frame, so the assembled final
response is complete. The single-writer invariant still holds: a
superseded attempt never writes into the live display. Request
retirement (watchdog kill) is unchanged and still stops the worker.
Ported from PR #69502 onto the Relay-backed stream path.
Fixes#69486
The OpenAI SDK always raises ``APIConnectionError(...) from err`` with the
httpx transport error as the direct ``__cause__``, so the chain-walking
helper from the salvaged commit is more machinery than the signal needs.
Replace it with an ``isinstance(exc.__cause__, httpx.TransportError)``
check at the single call site and document WHY the raw
``transport_errors`` branch never sees a pre-stream failure (the SDK wraps
them), which is the root cause behind #103673.
Responses lifecycle events may arrive before answer text, but the streaming runtime never populated the existing post_api_request timing field. Record the first accepted event while preserving request-retirement fencing.\n\nCloses #105311.
Replace the hardcoded 2.0s module constant from the salvaged commit with
``agent.stream_drain_timeout`` in DEFAULT_CONFIG (default 2.0, ``0`` skips the
drain), read through ``load_config_readonly`` at drain time. Non-secret knobs
belong in config.yaml, not a new HERMES_* env var. Documents the budget in the
timeouts table and adapts the salvaged test to the new seam.
Why: a relay that never closes the SSE socket after ``response.completed``
wedged the turn until the stale watchdog fired and discarded the already-billed
response (#103864). The drain is a courtesy to Relay's finalizer, so it must be
bounded; the bound needs a user-tunable knob for slow-but-honest relays.
The codex_app_server early-return forked away from the standard loop before any
prompt assembly was used: the codex thread received only cwd and the raw user
text, so SOUL.md, MEMORY.md/USER.md and channel_overrides.system_prompt were
composed, persisted to sessions.system_prompt, and then silently discarded.
_ensure_codex_session now composes the prompt exactly as turn_context does
(agent._cached_system_prompt + "\n\n" + agent.ephemeral_system_prompt) and
passes it to CodexAppServerSession(developer_instructions=...) once per thread.
Sessions are created lazily per turn only when none exists, so the prompt is
never duplicated per turn and a retired session re-sends the current one.
Hand-ported from #74726: same composition, but the session kwarg is spelled
developer_instructions -> thread/start.developerInstructions because the
`instructions` field #74726 used is accepted and ignored by codex.
Fixes#74712Fixes#26035
Replace the six parametrized subprocess-driven cases from #111616 with two
invariants: a successful marker-ending output round-trips SessionDB and
replay sanitize as data (envelope preserved), and a real interrupt or an
unknown exit status still recovers as UNKNOWN. The projector consumes the
event dict, so spawning real children added no signal.
Also re-word the live-card payload docstring in agent/codex_runtime.py: it
no longer mirrors the persisted tool-result content byte for byte.
The post-tool quiet timer no longer retires the session (it only warns), so the
codex_runtime retirement comment listed a cause that cannot happen and the
retained test's name/docstring still promised a watchdog that cannot trip.
Comment and test wording now describe the warning semantics; assertions unchanged.
Internal moves get no compat aliases (root AGENTS.md); codex_runtime and
auxiliary_client import bypass_sdk_request_transform from agent.sdk_transform_bypass.
