Selecting Image Generation -> OpenAI (Codex auth) in `hermes setup` / `hermes tools` on a
fresh install saved `image_gen.provider=openai-codex` and printed "no configuration
needed!" without ever signing in, so the backend was unusable until the user guessed the
auth command — and the schema hint pointed at `hermes auth codex`, which does not exist.
Root cause: the row declares `env_vars: []` and only a `post_setup_hint`, a key nothing
consumes; `_configure_provider` runs a hook only for `post_setup`.
- plugins/image_gen/openai-codex: schema declares `post_setup: "openai_codex"`; the hint
and the `auth_required` error name `hermes auth add openai-codex`.
- hermes_cli/tools_config_post_setup: `_post_setup_openai_codex` in the existing
`_POST_SETUP_HOOKS` table (sibling of the `xai_grok` credential bootstrap). With
credentials present it continues; otherwise it offers the device-code sign-in (or skip)
and saves tokens with `set_active=False`, so picking an image backend never rewrites
`model.provider` the way the model-provider login does.
- `_POST_SETUP_AUTH_READY` table replaces the `post_setup == "xai_grok"` special case in
`provider_readiness_status`, so any credential-bootstrap row reports ready/needs_auth
from the auth store.
- `_save_codex_tokens(set_active=...)` mirrors `_save_xai_oauth_tokens`.
Live probe (real `_configure_provider`, real plugin row, temp HERMES_HOME, OAuth start
stubbed with a recorder): before — post_setup=None, OAuth fired [], hint `hermes auth codex`;
after — no creds: OAuth fired once, logged in, model.provider untouched; with creds: OAuth
not fired.
Fixes#102144
Salvages #102165 (@liuhao1024) — superseded: same direction (post_setup hook), redone on
the split tools_config siblings without calling `_login_openai_codex`, which would have
switched the main model provider.
httpx 0.28 binds getproxies at import (httpx._utils), so patching
urllib.request.getproxies never reached it and the test stayed green with
_build_client neutered to a plain httpx.Client(). Patch httpx._utils.getproxies,
assert a plain httpx.Client() control DOES mount the fake system proxy, and
assert the http_client generate() hands openai.OpenAI carries no HTTPProxy
mount. Red with the neutered client, green on the branch.
Custom model ids (#97928): a value of image_gen.openai.model / OPENAI_IMAGE_MODEL outside the
gpt-image-2 / gpt-image-2.5 catalog used to fall through silently to the default tier, so a
gateway serving its own image model names always received `model: gpt-image-2` +
`quality: medium`. resolve_static_model(passthrough=True) now returns the unknown id as the
API model with quality=None, and the request omits `quality` for it — OpenAI-compatible
gateways reject enum values they do not know, and a foreign model has no OpenAI quality
tiers. Only the provider-scoped key and env var pass through; the shared top-level
image_gen.model can hold another provider's id (a FAL path) and is never forwarded.
Named custom endpoint (#83080, config-reuse half): image_gen.openai.provider names a
providers:/custom_providers: entry; its base_url and api_key/key_env fill in whatever
image_gen.openai.base_url / key_env leave unset, so a gateway already declared for chat is
not re-declared with a duplicated key. Explicit image_gen.openai values keep precedence; an
unknown name logs a warning and falls back to the OpenAI variables. The lookup goes through
hermes_cli.runtime_provider._get_named_custom_provider, the same resolver the auxiliary
clients use, so aliases and legacy list entries behave identically.
image_gen is an open config root (deliberately absent from DEFAULT_CONFIG), so
`image_gen.openai.provider` already validates as a known key; docs cover both behaviours.
Live probe (fake /v1/images/generations gateway recording the body, real provider on a temp
HERMES_HOME): before — `custom-image-model` sent as model=gpt-image-2 quality=medium; named
provider → is_available False / auth_required. After — model=custom-image-model, no quality;
named provider → request to the entry's URL with `Bearer <its key_env>`; the catalog model
still maps to gpt-image-2 + quality=medium.
What: plugins/image_gen/openai resolves its endpoint and credential through one
resolver — image_gen.openai.base_url → OPENAI_BASE_URL → SDK default, and the env
var named by image_gen.openai.key_env → OPENAI_API_KEY — shared by is_available()
and generate() so the two cannot disagree. The client is built on
build_keepalive_http_client (env-only proxy policy) and sends a blank
OpenAI-Project header.
Why: the image endpoint could only be routed via the process-wide OPENAI_BASE_URL /
OPENAI_API_KEY, so a local or third-party image gateway could not be configured
independently of the chat provider (#65309, #97928, #13798). openai.OpenAI() with
trust_env routed localhost endpoints through a macOS system proxy whose
ExceptionsList httpx never sees (#64888). An OPENAI_PROJECT_ID set for chat made
/v1/images/generations 403 model_not_found on projects with a model allow-list
even though the key already carries the project (#60748).
Slim redo of the contributor direction in #18796 (@y0shua1ee), #37208/#37209
(@charzhou), #65312/#65323/#64893 (@asdlem), #60749 (@perelin); all predate the
StaticImageGenProvider refactor and no longer apply.
OpenRouter (_generate_via_chat, _generate_via_image_api) and OpenAI gpt-image
called record_token_usage only after image extraction and save succeeded. A
token-billed 200 that returned text but no image (the "returned no image"
fallback case), an empty `data` list, or a failed save therefore consumed
billed tokens that never reached session_model_usage; on the fallback chain
only the fallback model's tokens were recorded.
Move the call to immediately after a successful post_json / SDK call, before
extraction/save, in all three paths. One call per HTTP success, so no double
count; failure responses (non-2xx, timeouts) still record nothing because
there is no body to read usage from.
Tests: the parametrized invariant in test_openrouter_compat_provider gains the
no-image-with-usage case for both surfaces (result is `empty_response`, one row
still recorded); the OpenAI test is parametrized on has_image the same way.
Widen the salvaged Image API hunk (#114340) to the whole class:
- plugins/image_gen/_common.py: record_token_usage() — one helper feeding the
aux accounting chokepoint (agent.aux_accounting.record_aux_usage) with task
"image_generation", the billing provider and the priced model id. Dict and
SDK usage objects alike; a body without tokens is a no-op, as is a call
outside a turn.
- plugins/image_gen/openrouter: the /chat/completions path (the DEFAULT model
chain — openai/gpt-5.4-image-2, google/gemini-3-pro-image — is chat-only and
token-billed) now records too; the Image API path uses the shared helper
(the contributor's local _record_image_api_usage is folded into it) and both
pass base_url so pricing resolves the route. Task renamed image_gen ->
image_generation to match the other aux task names.
- plugins/image_gen/openai: gpt-image bills per text/image token; record the
Images API usage block under the API model (gpt-image-2), not the Hermes
quality-tier label.
- FAL, xAI, Krea, DeepInfra, Meta, openai-codex return no token usage and stay
unrecorded (nothing to bill per token).
- tests: the contributor's two tests folded into one invariant parametrized
over chat / Image API / no-usage control; one OpenAI invariant.
- docs: image-generation.md "How It Works Internally" gains the accounting step.
Fixes#114324
OpenRouter token-billed image calls parsed usage into extra only, so
session_model_usage never saw them. Record prompt/completion tokens via the
ambient aux accounting on success; flat-fee responses without token usage
write nothing.
Fixes#114324.
Replace the per-site refusal tests with a single parametrized invariant
driven through a real loopback listener: each of the nine call sites
refuses a loopback target with zero connections recorded, and a
safe-looking first hop that 302s to the metadata endpoint is stopped at
the hop with nothing cached. save_url's redirect contract (caller headers
first-hop only, missing-Location 3xx fails closed) stays in
test_provider_media. Existing save_url tests keep their httpx/allow-
private adaptations; the codex edit test keeps its guarded-client stub.
Provider response URLs, model-supplied image refs, manifest-derived pet
URLs, and remote sitemap <loc> entries were fetched with raw
requests/httpx/urllib — bypassing tools/url_safety while every platform
media path already uses it. A hostile or compromised provider/manifest
endpoint could steer a server-side fetch at internal or metadata
addresses; several sites cache the body where it is deliverable back.
Apply the canonical is_safe_url + create_ssrf_safe_client pattern at
every site: per-hop revalidation at TCP connect (closing the
DNS-rebinding window), bounded redirect chains that fail closed on
missing Location, and caller headers scoped to the first hop only —
matching the openrouter provider's own documented contract that its
bearer key must never leave the operator-selected host.
Operator-configured endpoints and pinned release assets are out of
scope — those URLs are operator-selected, not remote-party-controlled.
Fixes#114468Closes#44728
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: Ray <rayjun0412@gmail.com>
Co-authored-by: zapabob <1920071390@campus.ouj.ac.jp>
The openai-codex image provider rode a Responses call with a hosted
image_generation tool on a pinned chat model (gpt-5.5). Two failure
classes came with that shape: when OpenAI withdrew gpt-5.5 from an
account cohort every image call 404'd while chat kept working
(#105398, #107076), and the host model was free to answer in text
instead of calling the tool, so we streamed SSE, kept partial frames
and retried on empty streams.
Post to chatgpt.com/backend-api/codex/images/generations and
images/edits instead - the route the official Codex client uses
(codex-rs/ext/image-generation). No host model, no SSE, no
partial-frame handling; the response is a plain JSON body with
b64_json. Remote source URLs are fetched client-side and inlined as
data URLs because the backend's own downloader 400s on ordinary
public images.
The backend treats model/quality/size as advisory (#107233), so the
result now reports reported_quality/reported_size next to the
requested values plus the x-codex-imagegen-request-id for support.
GPT Image 2.5 is deliberately not added to this catalog: the backend
accepts any model id, including nonexistent ones, and generates with
its server-managed engine (C2PA reports gpt-image 2.0), so a 2.5 tier
here would be a label with no effect (#106708).
- generate() now passes kwargs.get("model") into _resolve_model(), so the
user's hermes tools pick (forwarded by the dispatcher as top-level
image_gen.model) is honored instead of silently dropped (#55893 class;
matches xai/krea/openrouter).
- Setup schema badge "internal" -> "paid" to match every other paid
image backend in the hermes tools picker.
- Tests: caller-model precedence, unknown caller model falls through,
model kwarg reaches the API payload, badge contract.
Adds a bundled image-generation backend for the Meta Model API
(https://api.meta.ai/v1), which is OpenAI-compatible. Exposes the
muse-image-1.0 model via the standard image_generate tool. This is the
image-gen companion to the already-bundled meta-ai chat provider
(plugins/model-providers/meta-ai, PR #88565).
- plugins/image_gen/meta-ai/ — provider registered as `meta-ai`, matching
the chat provider's id. Reuses the openai SDK pointed at Meta's base URL.
- Auth mirrors the chat provider: MODEL_API_KEY (Meta's documented var),
with META_API_KEY / META_MODEL_API_KEY aliases and a META_BASE_URL
override.
- Text-to-image only for now (capabilities gated); base64 (WebP) and URL
responses both handled and saved under $HERMES_HOME/cache/images/.
- Auto-loads as `kind: backend` and appears in `hermes tools` with no
central list edits, matching the other bundled providers.
- tests/plugins/image_gen/test_meta_ai_provider.py — 27 tests (metadata,
auth-alias resolution, base-url override, model resolution, generate
paths incl. b64 save, aspect mapping, URL caching, error handling).
- docs: image-generation feature page + provider-plugin built-in list.
Codex Responses streams can emit partial_image_b64 previews without a final
image_generation_call.result. The provider treated any b64 as success and could
let a partial overwrite a coexisting final in the same payload, delivering
smeared intermediates as finished GPT Image 2 outputs.
Request partial_images=0, prefer final over partial in extraction, fail closed
(with one content-agnostic retry) unless source=final, and surface image_source
plus pixel_size for QA.
Follow-ups on top of the salvaged #82631 surface:
- _select_surface: an unknown model id found in the live /images/models
catalog now ROUTES to the dedicated Image API instead of only logging a
hint — without this, a model picked from the live picker that postdates
the curated snapshot would fall onto chat-completions and fail. Curated
defaults stay pinned to chat (no behaviour change for existing setups);
offline probes still fall back to chat. _HINTED_MODELS removed.
- list_models (OpenRouter): union of the live GET /images/models catalog
(43 models today) and the chat-completions image models, deduped,
defaults first; curated metadata wins for known ids, API names for the
rest. Nous Portal (no /images route) keeps its chat-only catalog.
Offline fallback: static chain + curated Image API snapshot.
- Tests updated/added: unknown-id routing (flipped from the hint-only
pinning test), non-catalog id stays on chat, merged-picker union/dedupe/
order, Nous exclusion.
- Docs: image-generation.md gains the OpenRouter Image API section and an
editing-support row.
Live-verified: picker lists 43 models; generation succeeded through the
dedicated API on google/gemini-3.1-flash-lite-image and on the previously
unreachable black-forest-labs/flux.2-klein-4b (config-selected, no kwarg).
- plugins/image_gen/openrouter: list_models() now queries the endpoint's
/models catalog filtered to output_modalities containing "image"
(per-backend 5-min cache, 10s timeout, static 2-model chain as offline
fallback; openrouter/auto* router pseudo-models excluded). Every image
model OpenRouter serves — including future releases — is selectable in
`hermes tools` with no code change. Applies to Nous Portal too via the
shared provider class.
- plugins/image_gen/xai: forward the dispatched model kwarg into
_resolve_edit_model() so an explicitly selected edit-capable model is
honored on /images/edits (extends the salvaged #55893 fix to the edit
path; text-only models still fall back to quality).
- Tests: OpenRouter live-catalog filtering/exclusions/order, offline
fallback, cache single-fetch; xAI edit-kwarg forwarding incl. the
text-only-hijack negative case.
Live-verified against openrouter.ai: 9 image-output models returned and
rendered, matching the public models?output_modalities=image listing.
- plugins/image_gen/xai: merge the live /v1/image-generation-models catalog
(5-min cache, 10s timeout, static-table fallback when offline/unauth)
into the picker so new xAI Imagine models appear automatically the day
they launch, with generic metadata until curated text is added.
- Add grok-imagine-image-2.0 to the curated static table (typography/
layout-aware model, API-available since Aug 8 2026).
- Edits honor an explicitly selected image-input-capable model
(e.g. grok-imagine-image-2.0) instead of always forcing
grok-imagine-image-quality; quality remains the default edit baseline.
- Tests: hermetic autouse fixture keeps unit runs offline; new coverage
for live-merge, unknown-future-model selection, offline fallback, and
edit-model resolution. Docs model table updated (en + zh-Hans).
Live-verified: /image-generation-models returns grok-imagine-image,
grok-imagine-image-2.0, grok-imagine-image-quality; real generation with
2.0 succeeded end to end.
The Aug 8 default-on upscaling policy (66ea4e686) chained the Clarity
Upscaler after every sub-2MP generation. Clarity is an SD1.5 creative
tile-diffusion enhancer (creativity 0.35, "masterpiece" prompt prefix) —
it redraws content, which degraded output on 100% of generations for
models like GPT Image 2 and Ideogram whose value is precise text
rendering, CJK, and photorealistic detail.
Policy now: no model upscales by default, on FAL or Krea. The `upscale`
tool param remains as a per-call opt-in (`upscale: true`); explicit
requests still chain Clarity (FAL) / Krea Enhance as before.
- FAL catalog: all 17 default-on entries flipped to upscale=False
- Krea plugin: medium + medium-turbo per-model defaults flipped off
- Tool schema: upscale param described as opt-in with a fidelity warning
- Tests updated: catalog invariant now pins all-off; default-on cases
now assert no upscaler call
- Docs (en + zh) updated to the opt-in policy
Per review: upscaling should be the default behavior (like the original
flux-2-pro chain), not agent opt-in. Policy: every image model whose
native output is below ~2MP now sets upscale=True in its catalog —
users never silently get low-res images. Native hi-res models
(Seedream 5 Pro/Lite, Krea 2 Large) stay off to avoid paying to
upscale already-large output.
- FAL catalog: 16 models flipped to upscale=True (klein, z-image,
nano-banana pro/2/2-lite, gpt-image 1.5/2, ideogram v3/v4, recraft
v4/v4.1, qwen image/3, krea-2 medium on FAL, MAI 2.5 pro).
- Krea plugin: per-model upscale defaults (medium + medium-turbo ON at
1.5K native; large OFF at 2K native), precedence explicit kwarg >
image_gen.krea.upscale config > catalog default.
- The 'upscale' tool param remains as a per-call override in both
directions (false = fast draft, true = force on hi-res/edits).
- Video unchanged: opt-in only (default-on would double every video's
cost and latency).
- Sibling tests updated: routing/payload tests pass upscale=False where
the assertion targets the generation submit; catalog test now pins
the native-resolution policy instead of the flux-2-pro snapshot.
The generated-media surface previously had almost no upscaler coverage:
only fal-ai/flux-2-pro chained Clarity Upscaler (hardcoded catalog
default), every other image model returned ~1MP output with no high-res
path, and video had no upscaler at all. Krea's API treats the enhancer
as a standard second pass; this brings the same shape to Hermes.
- image_generate: new optional 'upscale' boolean in the tool schema.
Explicit true chains the backend upscaler on ANY model (including
edits); explicit false disables flux-2-pro's automatic default;
omitted keeps per-model catalog behavior. Response now reports
'upscaled' so the agent knows which resolution it got.
- FAL image path: explicit flag overrides the catalog 'upscale' default
(Clarity Upscaler, 2x). Failure falls back to the native image.
- Krea plugin: upscale=true chains Krea Enhance
(/generate/enhance/krea/enhance, 2x, prompt-guided) through the same
BYO/managed base URL + auth as generation, with a best-effort poll
loop that never fails a successful generation.
- video_generate: new optional 'upscale' boolean; FAL video plugin
chains ByteDance SeedVR2 (fal-ai/seedvr/upscale/video, 2x factor
mode). Providers without upscalers ignore the kwarg per the ABC
contract (documented in both ABCs).
Validation: targeted suites green (123 tests across 6 files, including
new coverage for override-wins/default-kept/failure-fallback on all
three paths); live E2E on direct FAL verified both chains end-to-end
(klein 9b + Clarity upscaled image; pixverse-v6 1s 360p + SeedVR2
upscaled video).
The Codex image backend rejected our own request shape for every account, and
we then translated that rejection into "Image generation is not enabled for the
current Codex account. Switch the image provider to OpenAI API key, FAL, or
xAI." — telling every affected user to abandon a provider that had never
actually been tried. That message is why this reads as a setup failure rather
than a bug: the wire error was replaced with a confident, wrong diagnosis.
Removes the classifier and its exception, so any HTTP failure surfaces
verbatim. The paired request-shape fix (previous commit) is what makes the
400 stop happening; this commit makes the next one diagnosable.
Also fixes error-body truncation: bodies were head-truncated at 500 chars, and
Codex error payloads can carry hundreds of bytes of leading metadata, so the
user got a wall of padding and no message. _summarize_error_body() prefers the
parsed error.message and falls back to a truncated raw body.
Docs: drop the unqualified image-to-image claim for the Codex backend and note
that the hosted tool call cannot be forced, so it is best-effort.
Verified E2E against a local fake Codex backend: success path writes a real PNG
with no tool_choice on the wire; the 400 path now returns api_error carrying
"Tool choice 'image_generation' not found in 'tools' parameter" (148 chars)
instead of the entitlement message. Sabotage run confirms all 4 regression
tests fail when the old behavior is restored.
Refs #19505, #49008, #31335.
The chatgpt.com/backend-api/codex backend 400s on every tool_choice
shape for the hosted image_generation tool — it looks up tool_choice as
a function name and never recognizes hosted-tool entries. Removing the
field from _build_responses_payload() lets the host model decide; the
instructions field nudges it toward the tool.
Salvaged from PR #19979 (originally targeted the old
client.responses.stream call, which no longer exists on upstream/main;
the live request now flows through _build_responses_payload + httpx in
_collect_image_b64).
Follow-up to the per-provider guards. Three improvements from review:
1. Extract agent.file_safety.raise_if_read_blocked() as a single shared
chokepoint and route the OpenAI, OpenRouter, and (newly) xAI image
providers through it, replacing the 3x-duplicated inline try/except.
Fixes the whole bug class: xai/_xai_image_field read a model-supplied
local path via open() with no guard — the same vulnerability the PR
fixed for OpenAI/OpenRouter, in a sibling provider it missed.
2. Strengthen the regression tests from pass-on-any-ValueError to true
security invariants: spy open()/read_bytes() and assert the blocked
credential is NEVER read; add negative controls (legit local image
still loads; remote/data: URIs pass through unguarded) so a
block-everything regression can't pass.
3. Guard is best-effort by design (defense-in-depth, not a security
boundary) — documented on the shared helper.
- agent/file_safety.py: raise_if_read_blocked()
- plugins/image_gen/{openai,openrouter,xai}: route through helper
- tests: no-read spies + negative controls across all three providers
Replace the plugin-local _IMAGE_MAGIC_MIME table + _sniff_image_mime
body with a delegation to agent.image_routing._sniff_mime_from_bytes,
the canonical magic-byte sniffer already used across the codebase, then
gate its result to the raster formats gpt-image-2's Responses
input_image actually accepts (png/jpeg/gif/webp).
The shared sniffer also recognizes SVG/TIFF/ICO; without the allowlist
those would pass local validation and be rejected server-side with an
opaque HTTP 400. Gating locally fails them cleanly as invalid_image_input.
Adds a regression test for SVG rejection.
Follow-up on top of @CrazyBoyM's #55828.
Add durable public-URL output and URL-based chaining to xAI Grok Imagine:
- Store generated media on files-cdn with permanent public HTTPS URLs
(public_url: true, no expiry by default).
- Chain by URL: generate -> edit -> extend each take a prior result's
public HTTPS URL (or a data URI / local file for inputs).
- Add provider-specific xai_video_edit and xai_video_extend tools.
- Image generation: public-URL/storage output, multi-reference edits,
and ~/ local-path support for image edits.
Credentials use xAI Grok device-code OAuth (separate PR).
OpenRouter/Nous image gen now runs a quality-first model chain by default:
attempt the highest-fidelity OpenAI image model first, then fall back to
Gemini 3 Pro Image when it's access-gated/unavailable/times out. An explicit
OPENROUTER_IMAGE_MODEL / config model override pins one model with no fallback.
Atlas validation rejects malformed model output instead of shipping it: adds a
per-state collapse guard (a single sliver/fragment row no longer passes because
other rows are healthy), on top of the existing postage-stamp + multi-pose
checks.
Desktop: pet-gen native notifications are now "global" (not tied to a chat
session), so a background generation started from the command center fires an
OS notification when the user is away even with no active session. Adds a
neutral "This can take up to 5 minutes." banner on step 1, and lets the
provider picker auto-size.
Tests updated/added for the OpenRouter fallback chain, the collapse guard, and
the global notification path.
Reference-grounded image provider over the OpenRouter-compatible
chat-completions image protocol (Gemini Flash Image et al.). Nous Portal
proxies OpenRouter, so one provider serves both — giving pet generation a
reference-capable backend beyond OpenAI gpt-image.
Remove unused imports (F401) and duplicate/shadowed import
redefinitions (F811) across the codebase using ruff's safe
autofixes. No behavioral changes -- imports only.
- ~1400 safe autofixes applied across 644 files (net -1072 lines)
- __init__.py re-exports preserved (excluded from F401 removal so
public re-export surfaces stay intact)
- Re-exports that are imported or monkeypatched by tests but look
unused in their defining module are kept with explicit # noqa:
F401 (gateway/run.py load_dotenv; run_agent re-exports from
agent.message_sanitization, agent.context_compressor,
agent.retry_utils, agent.prompt_builder, agent.process_bootstrap,
agent.codex_responses_adapter)
- Unsafe F841 (unused-variable) fixes deliberately skipped -- those
can change behavior when the RHS has side effects
- ruff lints remain disabled in pyproject.toml (only PLW1514 is
selected); this is a one-time cleanup, not a config change
Verification:
- python -m compileall: clean
- pytest --collect-only: all 27161 tests collect (zero import errors)
- core entry points import clean (run_agent, model_tools, cli,
toolsets, hermes_state, batch_runner, gateway)
- static scan: every name any test imports directly from an edited
module still resolves
* feat(image_gen): add Krea provider plugin (Krea 2 Medium + Large)
New built-in image_gen backend wrapping Krea's Krea 2 foundation
image model family. Auto-discovered like the other image_gen plugins
and appears in 'hermes tools' → Image Generation → Krea.
Krea's API is asynchronous — submit returns a job_id, poll /jobs/{id}
until terminal. The provider hides that behind the synchronous
ImageGenProvider.generate() contract: submit, poll every 2s with
light backoff (max 5s), 3-minute ceiling matching Krea's hosted-tool
timeout. Result URL is materialised to $HERMES_HOME/cache/images/
to avoid CDN-expiry 404s downstream (same fix as xAI #26942).
Models:
- krea-2-medium (default — Krea's 'start here' recommendation)
- krea-2-large
Aspect ratios map landscape→16:9, square→1:1, portrait→9:16.
Resolution: 1K (Krea's only current option).
Kwarg passthrough: seed, creativity (raw/low/medium/high), styles,
image_style_references (capped 10), moodboards (capped 1) — matches
Krea's per-request limits. Unknown kwargs are ignored.
Config knobs (config.yaml):
image_gen.provider: krea
image_gen.krea.model: krea-2-medium | krea-2-large
image_gen.krea.creativity: raw | low | medium | high
Env overrides: KREA_API_KEY (required), KREA_IMAGE_MODEL.
KREA_API_KEY is registered in OPTIONAL_ENV_VARS so 'hermes setup'
prompts for it.
31 new tests; image_gen suite + picker + tools_config: 211/211.
* fix(image_gen/krea): address review feedback
- Update KREA_API_KEY setup URL to the canonical token-creation page
(https://www.krea.ai/app/api/tokens). The previous URL returned 404.
- Fail fast on non-retryable HTTP statuses during poll. The previous
loop retried every HTTPError for the full 180s deadline, so an auth
(401), billing (402), forbidden (403), or not-found (404) response
would make image_generate hang for three minutes. Only retry
transient statuses (408/409/425/429/5xx); surface everything else
immediately.
- Add 5 tests covering fail-fast on 401/403/404 and retry on 429/503.
* fix(krea): point users at the real API token dashboard URL
Three call sites linked users to dashboard pages that don't exist:
- hermes_cli/config.py: https://www.krea.ai/app/api/tokens
- plugins/image_gen/krea/__init__.py get_setup_schema: https://www.krea.ai/api-keys
- plugins/image_gen/krea/__init__.py auth_required error: https://www.krea.ai/api-keys
Per Krea's own docs (https://docs.krea.ai/developers/api-keys-and-billing),
the real dashboard URL is https://www.krea.ai/settings/api-tokens. All three
sites now point there.
xAI's grok-imagine-image API returns ephemeral imgen.x.ai/xai-tmp-* URLs
that 404 within minutes — long before downstream consumers (Telegram
send_photo, browser preview, multi-tier delivery fallback) get a chance
to fetch them. The xAI image_gen provider was passing those URLs
through unchanged on the elif url: branch; b64 responses were already
cached locally via save_b64_image. Result: every image_generate call
on a Telegram-routed xai-oauth profile delivered no image, falling
through to text-only.
Adds agent.image_gen_provider.save_url_image() — a sibling helper to
save_b64_image that downloads URL bytes to $HERMES_HOME/cache/images/.
Content-type-aware extension inference with URL-suffix fallback;
oversize cap (25MB default) with partial-write cleanup; empty-body
refusal. Mirrors the audio_cache pattern used by text_to_speech.
Wires save_url_image into both the xAI and OpenAI providers' URL
branches. When the download fails (network blip, 404 in-flight) we
log a warning and fall back to the bare URL rather than turning the
tool call into a hard error — the gateway's existing URL-send fallback
then gets a chance to surface the original error legibly.
Test plan:
- tests/agent/test_save_url_image.py — 8 direct tests against a real
in-process HTTP server: bytes round-trip, content-type → extension,
URL-suffix fallback, default-to-png, 404 propagation, empty-body
refusal, oversize cap + cleanup, filename uniqueness.
- tests/plugins/image_gen/test_xai_provider.py — flip
test_successful_url_response (was asserting the bug), add
test_url_response_falls_back_to_bare_url_when_download_fails.
- tests/plugins/image_gen/test_openai_provider.py — symmetric pair.
160/160 in the broader image_gen test surface.
Mirrors the architecture established by the web (#25182), browser
(#25214), and video_gen (#25126) plugin migrations:
* `tools/fal_common.py` — stateless atoms shared by both FAL-backed
plugins (image_gen + video_gen). Holds the lazy `fal_client` import
helper, `_ManagedFalSyncClient`, `_normalize_fal_queue_url_format`,
`_extract_http_status`. Stateful pieces (`fal_client` module global,
`_managed_fal_client*` cache, `_submit_fal_request`,
`_resolve_managed_fal_gateway`, `_get_managed_fal_client`)
intentionally stay on `tools.image_generation_tool` so the existing
`monkeypatch.setattr(image_tool, ...)` patch sites keep working
unchanged.
* `plugins/video_gen/fal/__init__.py` — drops its inline
`_load_fal_client` duplicate; consumes `tools.fal_common.import_fal_client`.
* `plugins/image_gen/fal/{plugin.yaml,__init__.py}` — new plugin.
`FalImageGenProvider` is a thin registration adapter that resolves
the legacy module via `import tools.image_generation_tool as _it`
and calls `_it.image_generate_tool` + `_it._resolve_fal_model` at
call time. The 18-model catalog, `_build_fal_payload`, managed-
gateway selection, and Clarity Upscaler chaining all remain in
`tools.image_generation_tool` as the single source of truth —
the plugin is a registration adapter, not a parallel implementation.
* `tools/image_generation_tool.py::_dispatch_to_plugin_provider` —
drops the `configured == "fal"` skip. Setting `image_gen.provider:
fal` now routes through the registry like any other provider; the
plugin re-enters this module's pipeline so behavior is identical.
Unset `image_gen.provider` still falls through to the in-tree
pipeline (preserves no-config-with-FAL_KEY UX from #15696).
* `hermes_cli/tools_config.py` — drops the hardcoded "FAL.ai" row from
`TOOL_CATEGORIES["image_gen"]["providers"]` (now injected by
`_plugin_image_gen_providers` like every other backend) and the
`getattr(provider, "name") == "fal"` skip that protected against
duplication with the hardcoded row. The "Nous Subscription" row
stays as a setup-flow entry — same shape browser kept "Nous
Subscription (Browser Use cloud)" after #25214.
* `tests/plugins/image_gen/test_fal_provider.py` — 14 cases covering
the ABC surface, call-time indirection (verifying
`monkeypatch.setattr(image_tool, "image_generate_tool", ...)` takes
effect through the plugin), response-shape stamping, exception
handling, and registry wiring.
* `tests/plugins/image_gen/check_parity_vs_main.py` — subprocess
harness mirroring `tests/plugins/browser/check_parity_vs_main.py`.
Pins one path to origin/main, one to the worktree; runs six
scenarios (unset, explicit-fal-no-creds, explicit-fal-with-creds,
explicit-fal-with-model, typo provider, managed-gateway-only) and
diffs the reduced shape `{dispatch_kind, provider_name, model}`
per scenario. The only acceptable diff is "legacy_fal → plugin
(fal)" for explicit-FAL paths — every other delta is flagged as
a regression.
* `tests/hermes_cli/test_image_gen_picker.py::test_fal_surfaced_alongside_other_plugins`
— flips the previous `test_fal_skipped_to_avoid_duplicate` to
match the new shape (FAL is a plugin now, no dedup needed).
Verified: 195/195 tests across
`tests/{tools/test_image_generation*,tools/test_managed_media_gateways,plugins/image_gen,plugins/video_gen,hermes_cli/test_image_gen_picker}.py`
pass on this branch with no test patches modified outside the picker
test that asserted the old skip behaviour.
Fixes#26241
Adds a new authentication provider that lets SuperGrok subscribers sign
in to Hermes with their xAI account via the standard OAuth 2.0 PKCE
loopback flow, instead of pasting a raw API key from console.x.ai.
Highlights
----------
* OAuth 2.0 PKCE loopback login against accounts.x.ai with discovery,
state/nonce, and a strict CORS-origin allowlist on the callback.
* Authorize URL carries `plan=generic` (required for non-allowlisted
loopback clients) and `referrer=hermes-agent` for best-effort
attribution in xAI's OAuth server logs.
* Token storage in `auth.json` with file-locked atomic writes; JWT
`exp`-based expiry detection with skew; refresh-token rotation
synced both ways between the singleton store and the credential
pool so multi-process / multi-profile setups don't tear each other's
refresh tokens.
* Reactive 401 retry: on a 401 from the xAI Responses API, the agent
refreshes the token, swaps it back into `self.api_key`, and retries
the call once. Guarded against silent account swaps when the active
key was sourced from a different (manual) pool entry.
* Auxiliary tasks (curator, vision, embeddings, etc.) route through a
dedicated xAI Responses-mode auxiliary client instead of falling back
to OpenRouter billing.
* Direct HTTP tools (`tools/xai_http.py`, transcription, TTS, image-gen
plugin) resolve credentials through a unified runtime → singleton →
env-var fallback chain so xai-oauth users get them for free.
* `hermes auth add xai-oauth` and `hermes auth remove xai-oauth N` are
wired through the standard auth-commands surface; remove cleans up
the singleton loopback_pkce entry so it doesn't silently reinstate.
* `hermes model` provider picker shows
"xAI Grok OAuth (SuperGrok Subscription)" and the model-flow falls
back to pool credentials when the singleton is missing.
Hardening
---------
* Discovery and refresh responses validate the returned
`token_endpoint` host against the same `*.x.ai` allowlist as the
authorization endpoint, blocking MITM persistence of a hostile
endpoint.
* Discovery / refresh / token-exchange `response.json()` calls are
wrapped to raise typed `AuthError` on malformed bodies (captive
portals, proxy error pages) instead of leaking JSONDecodeError
tracebacks.
* `prompt_cache_key` is routed through `extra_body` on the codex
transport (sending it as a top-level kwarg trips xAI's SDK with a
TypeError).
* Credential-pool sync-back preserves `active_provider` so refreshing
an OAuth entry doesn't silently flip the active provider out from
under the running agent.
Testing
-------
* New `tests/hermes_cli/test_auth_xai_oauth_provider.py` (~63 tests)
covers JWT expiry, OAuth URL params (plan + referrer), CORS origins,
redirect URI validation, singleton↔pool sync, concurrency races,
refresh error paths, runtime resolution, and malformed-JSON guards.
* Extended `test_credential_pool.py`, `test_codex_transport.py`, and
`test_run_agent_codex_responses.py` cover the pool sync-back,
`extra_body` routing, and 401 reactive refresh paths.
* 165 tests passing on this branch via `scripts/run_tests.sh`.
