Follow-up to the two contributor commits for #121486. The picker, the
image plugin and the auxiliary Codex client still composed a pooled
gateway key with a base re-read from ambient state (HERMES_CODEX_BASE_URL
or the chatgpt.com default), so a model.base_url-only gateway (env unset)
still sent its key to chatgpt.com.
- auth_codex: resolve_codex_runtime_credentials reports the host a pooled
credential actually routes to (runtime_provider._pool_entry_mode_and_url:
env > model.base_url while the row is canonical > row URL) instead of the
ambient default; get_codex_auth_status carries the same bound base_url.
- picker: get_codex_model_ids(access_token, base_url=) now receives the base
resolved with the token from hermes_cli/models.py, the CLI default-model
swap (self.base_url) and the `hermes model` Codex flow.
- aux/image: _resolve_codex_credential_and_base() returns (token, base) from
one pool selection; the image plugin, _build_codex_client and the raw
Codex client use it (profile-scoped override from #121497 still wins).
- model_metadata: the non-JWT refusal now applies only when the target is
chatgpt.com; a gateway key may probe its own gateway's /models.
Adversarial regressions: model.base_url with env unset, env/route mismatch,
opaque + JWT gateway keys, pool-selected credential, pool row with its own
gateway URL, direct-ChatGPT positive control.
Addresses @andrexibiza's review on #121508.
Behind a custom Codex base URL (HERMES_CODEX_BASE_URL / model.base_url
gateway) three paths still hit the hard-coded chatgpt.com host with the
gateway's credential-pool key (#121486):
- the OAuth context-length probe (agent/model_metadata.py) and the
/model picker's live discovery (hermes_cli/codex_models.py) both GET
https://chatgpt.com/backend-api/codex/models with
Authorization: Bearer <gateway key> whenever model.context_length is
not pinned — the key is sent to a service it does not belong to and
cannot answer for;
- the openai-codex image_gen plugin posts to the same hard-coded base.
Fix, mirroring the quota probe's existing gate in auth_codex:
- both catalog sites now decline to probe non-JWT credentials (real
Codex access tokens are JWTs; a gateway key is not one) and fall
back to the static table / offline sources — same outcome as the
doomed request today, minus the credential leak;
- a JWT reached through a custom base now probes that base's own
/models instead of chatgpt.com (catalog URLs are built from the
resolved base; the per-token cache key includes the base);
- the image plugin resolves its base from HERMES_CODEX_BASE_URL the
same way the text client does.
Fast-mode host gating in the /fast picker is intentionally left
untouched: lifting it needs an explicit opt-in design decision, not a
bug fix.
(cherry picked from commit 5d76ec525674d7b103ab53955ba5279605457ca1)
[salvage: plugins/image_gen/openai-codex/__init__.py hunk dropped in favour of #121497 (first submitter, profile-scoped override + base-aware Cloudflare headers)]
Independent review of #120220 found restored tests that could pass without
reaching the code they name. Each fix below was A/B'd against a sabotaged
production line (red) and the real one (green).
- test_clarify_progress_leak: the fake agent slept 2x0.35 s as its only sync.
It now waits (<= 5 s) for the progress task's first bubble; the queue is
FIFO, so a leaked clarify bubble is what lands first. Reverting the clarify
skip in run_turn_runner goes red on the leaked JSON.
- test_desktop_lifecycle_windows_live holder-scan case: the kanban row was
rebuilt after the serve kill, re-reading the dead pid; NoSuchProcess was
swallowed into an empty scan and returned False without reaching the
classifier. Rows are now snapshotted while both processes live; a
classifier that accepts everything goes red (the old version stayed green).
- *_windows_live (control socket, desktop lifecycle, plan reconciliation,
shim fail-closed): fixed time.sleep settles become bounded waits on the
real condition (argv visible to psutil, pipe gone, handle released) and
child readline() calls get a 60 s bound.
- test_gateway_proc_fallback BSD ps case: drop the is_windows stub and mark
it linux_only like its neighbour; a revert to `ps -A eww` goes red.
- test_delivery_ledger_single_connection counts only connections to the
ledger's own database, not every sqlite3.connect in the process; pruning on
a second connection goes red.
- test_codex_models: the hardcoded gpt-5.6 list becomes a relationship to
DEFAULT_CODEX_MODELS and the forward-compat table; adding a -pro slug to
either goes red.
- group-chat-view inline-code: the renderer stub wrapped the code in .aui-md,
which the generic inline-code rule themes on its own, so the room's
data-slot rule was never tested. Without the wrapper, renaming the room's
data-slot goes red.
Change-detectors, tautologies, source-reading tests, redundant duplicates,
mock-echo tests and dead/unrunnable tests. Per-test rationale in the lane
ledger (category + reason for every removal).
The ChatGPT Codex models endpoint gates entries on client_version against each
model's minimal_client_version. Hermes sent the "0.0.0" sentinel, which used to
return the whole account catalog; since the GPT-6 Sol/Luna rollout it returns a
frozen legacy list (astra + the 5.6 trio) while any version at or above the newest
minimal_client_version (0.155.0, 1.0.0, 99.0.0 alike, live 2026-09-22) returns
everything the account is entitled to. So the picker and the context-window probe
hid gpt-6-sol / gpt-6-luna that the vendor's own client showed for the same account.
Both request sites now go through fetch_codex_catalog_entries(): try the
newest-client URL first and fall back to the "0.0.0" sentinel only when that
answer is non-200 or empty (the backend used to reject out-of-sequence versions
that way). No local ~/.codex/models_cache.json is consulted: a missing or stale cache (this
host records 0.147.0, which hides gpt-6-sol at 0.155.0) must not decide what the
account can see.
Report by @Stone441 (#119412); supersedes #119420 by @KoNit-K, whose fix read the
compatibility version from the local `~/.codex` cache.
Resolved toward the branch: PM provisions uv/python (main's install.ps1 uv-shim
salvage + its test and workflow steps dropped), the shim re-exec stays retired,
package.json carries no electron-builder block (afterExtract identity stamp wired
into electron-builder.config.cjs instead; after-pack.mjs keeps signing only),
Desktop workspace-deps helpers stay retired. Main's scratch-dir bootstrap
(export_scratch_tmp_env) is taken and re-run after profile resolution.
What: the contributor's pick removes the slug from DEFAULT_CODEX_MODELS and the
forward-compat templates. This follow-up finishes the class on the sibling
surfaces that still taught users the retired slug:
- hermes_cli/cli_model_switch_mixin.py: `/model` -> openai-codex replaced an
untouched default with the literal "gpt-5.3-codex" when live discovery failed;
it now uses DEFAULT_CODEX_MODELS[0] so the fallback can never drift from the
curated list again.
- run_agent.py: the Codex silent-hang hint no longer recommends the retired slug.
- website/docs (+ zh-Hans): the Codex OAuth vision/fallback examples used the
retired slug and claimed a default that no longer exists (openai-codex has no
implicit auxiliary model); examples now use gpt-5.4 and say to set `model`.
- tests: one invariant test pins the slug out of the curated list and every
template tuple (#52492); hint/watchdog tests assert the retired slug is absent;
catalog fixtures switched to live slugs.
Why: the ChatGPT Codex backend returns HTTP 400 "not supported when using Codex
with a ChatGPT account" for gpt-5.3-codex (#52492; second field report incl. the
official CLI on the #81558 thread, 2026-08-10). Same shape as e8955f222c, which
dropped gpt-5.2-codex / gpt-5.1-codex-max / gpt-5.1-codex-mini. Live discovery
still surfaces the slug if the backend re-enables it.
checkpoint_manager_profile_rename.py (merged from main) reaches for
_store_has_head/_ref_tip/_list_project_refs/_unlink_quiet, which main
extracted from checkpoint_manager after this branch diverged; a profile
rename with checkpoints raised AttributeError. Add the same helpers here.
Also: tests follow this branch's contracts (model_metadata_http.get instead
of a requests module, no agent.ssl_guard, backup excludes PM trees too,
_safe_restore_plan), and run_tests.sh forwards SSL_CERT_FILE/DIR so network
tests verify TLS with the pinned interpreter on hosts without a compiled-in
bundle path.
The ChatGPT Codex models endpoint interprets client_version as a Codex
CLI compatibility version and filters out any model whose
minimal_client_version is newer than the value sent. Hermes hardcoded
client_version=1.0.0 at both catalog request sites, so model visibility
was accidentally coupled to a version scheme Hermes doesn't follow —
future models gated behind a higher minimal version would silently
vanish from the account catalog.
The backend accepts the exact sentinel 0.0.0 as an ungated request
returning the complete account catalog (verified live: 0.0.0 and
current versions return identical model sets today, while omitting the
parameter is HTTP 400 and out-of-sequence values like 0.0.1 return no
models). Both request sites (hermes_cli/codex_models.py and the
context-length probe in agent/model_metadata.py) now share one
CODEX_UNGATED_CLIENT_VERSION constant.
Clean-room port of the observed behavior in zed-industries/zed#62729;
no GPL code translated.
The slug pair {"gpt-6-astra", "gpt-6-astra-900k"} was spelled out five times
(reasoning_effort, transports/codex, models.py x2, codex_models) with five hand-rolled
``.strip().lower().rsplit("/", 1)[-1]`` normalisations. ``is_astra_model`` in
agent/reasoning_effort.py (the module the other four already import) is now the only home, so a
new Astra alias is one edit.
``_finalize_codex_models(..., allow_astra=)`` was a control-coupling flag set True by the two
live callers and left False by the two static ones. The filter now lives where the untrusted
inputs are — ``_drop_undiscovered_astra`` over config.toml default + models_cache.json in
``get_codex_model_ids`` — and ``_finalize_codex_models`` is back to its one-line original.
DEFAULT_CODEX_MODELS never contains Astra, so the static catalog needs no gate.
``_openai_catalog`` appends whatever Astra ids live discovery returned instead of a name-keyed
``if "gpt-6-astra" in live_lower`` branch with a literal fallback that could never fire.
Review findings on #92797 (@100yenadmin):
- is_codex_900k_base() is now the single source of truth used by picker
synthesis, context resolution, /model validation, and wire stripping.
Eligibility is an exact table (sol/terra/luna, gpt-5.4, daybreak alias)
plus date-shaped 5.6 snapshots — family-prefix matching removed, so
non-routable -pro slugs and unknown descendants never gain variants.
- strip_codex_context_variant_suffix() strips conditionally: ineligible
aliases (gpt-5.5-900k) are returned unchanged and fail honestly at the
API instead of silently running as the base model at 272K.
- validate_requested_model() rejects ineligible *-900k aliases before the
hidden-slug soft-accept, and accepts valid variants missing from a
stale catalog without letting the typo auto-corrector eat the suffix.
- Codex context resolver drops vendor/ namespaces, so
openai/gpt-5.6-sol-900k resolves to 900K like the bare id.
- Table-driven regression covering eligible bases/snapshots/namespaced
ids and rejected -pro/-mini/5.5/unknown aliases, asserting context AND
wire model.
The Aug 16 change that auto-raised gpt-5.4/5.6 Codex OAuth context to the
live-verified 900K burned through subscription usage for users who never
asked for the larger window (bigger window = more input tokens per request).
- Base Codex slugs (gpt-5.6-sol/terra/luna, gpt-5.4) now resolve to the
advertised 272K again — the cheaper limit is the default.
- The model picker synthesizes explicit <slug>-900k variants (e.g.
gpt-5.6-sol-900k) for every live-verified slug; selecting one opts into
the 900K window. Slugs that genuinely enforce 272K (gpt-5.5,
gpt-5.4-mini) get no variant.
- The -900k suffix is Hermes-side only: stripped before the model id hits
the wire (main transport + auxiliary Responses adapter), and pricing
aliases the variants onto the base entries.
- Docs: new opt-in section in context-compression-and-caching.md.
Port the stale PR #61665 behavior to current main. The original two-file contribution is by yungchentang; this candidate preserves its scoped Codex OAuth fallback intent.
Co-authored-by: Taneli Mielikäinen <taneli.mielikainen@iki.fi>
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
Add regression tests locking in the new behavior: a JWT carrying a
chatgpt_account_id claim causes the probe to send ChatGPT-Account-Id,
while a malformed token omits the header instead of crashing.
Rebased onto current main and re-ported across the restructured
surfaces: model flows now thread confirm_provider/base_url/api_key
through hermes_cli/model_setup_flows.py, the Discord picker lives in
plugins/platforms/discord/adapter.py, and the web dashboard picker
applies chat-mode switches via config.set so the expensive-model
confirmation can ride the response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DEFAULT_CODEX_MODELS shipped three slugs that the chatgpt.com Codex
backend rejects with HTTP 400 'The <slug> model is not supported when
using Codex with a ChatGPT account.' on every account tested live:
gpt-5.2-codex
gpt-5.1-codex-max
gpt-5.1-codex-mini
Live verified against https://chatgpt.com/backend-api/codex/models
which returns gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex,
gpt-5.3-codex-spark, gpt-5.2 for ChatGPT Pro accounts.
When _fetch_models_from_api fell back to DEFAULT_CODEX_MODELS (offline
first-run, transient API failure) the picker surfaced these dead slugs
and crashed on selection. The forward-compat synthesis table chained
them downstream too.
If OpenAI re-enables them on the OAuth-backed Codex backend, live
discovery will pick them up automatically — the defaults list is only
consulted when live discovery is unavailable.
Test fixture pivoted to use gpt-5.3-codex (templated by 4 entries) as
the synthesis driver so the forward-compat test still exercises the
synthesis path.
Two follow-ups from self-review:
1. Add unit test for _fetch_models_from_api covering the live HTTP path.
The salvaged PR #19530 dropped the supported_in_api:false filter in
both _fetch_models_from_api and _read_cache_models, but only the
cache path had a regression test. This adds the symmetric live-fetch
test (mocked httpx) so a future drive-by change to the HTTP path
can't silently re-introduce the filter.
2. Pin test_codex_picker_uses_live_codex_catalog to the cache fallback.
The test wrote a fake JWT and a CODEX_HOME cache, but provider_model_ids
('openai-codex') still issued a real 10s HTTP probe to
chatgpt.com/backend-api/codex/models before falling back to the cache.
That made the test slow and non-deterministic in restricted/CI
networks. Patch _fetch_models_from_api to return [] so we go straight
to the cache path the test actually means to exercise.
* refactor: re-architect tests to mirror the codebase
* Update tests.yml
* fix: add missing tool_error imports after registry refactor
* fix(tests): replace patch.dict with monkeypatch to prevent env var leaks under xdist
patch.dict(os.environ) can leak TERMINAL_ENV across xdist workers,
causing test_code_execution tests to hit the Modal remote path.
* fix(tests): fix update_check and telegram xdist failures
- test_update_check: replace patch("hermes_cli.banner.os.getenv") with
monkeypatch.setenv("HERMES_HOME") — banner.py no longer imports os
directly, it uses get_hermes_home() from hermes_constants.
- test_telegram_conflict/approval_buttons: provide real exception classes
for telegram.error mock (NetworkError, TimedOut, BadRequest) so the
except clause in connect() doesn't fail with "catching classes that do
not inherit from BaseException" when xdist pollutes sys.modules.
* fix(tests): accept unavailable_models kwarg in _prompt_model_selection mock
