plugin-catalog/supermemory.yaml already points at supermemoryai/hermes-supermemory,
the same provider maintained by Supermemory. The catalog entry stays dormant while a
bundled copy exists (bundled providers win in find_provider_dir), so deleting the
in-tree copy is what lets hermes_cli/memory_provider_migration.py install it for
every home whose memory.provider is "supermemory".
retaindb, byterover, holographic, mem0, openviking and honcho stay bundled.
Prose that lists the bundled set (CONTRIBUTING en/es, plugins/AGENTS.md, the
memory.provider default comment, `hermes memory --help`) now names supermemory as
a catalog provider. The SUPERMEMORY_* env hints and redaction stay: the catalog
copy still reads them.
Submitters only saw a stale 5-point summary on the catalog page (it even listed
a 'Released' requirement nothing enforces), while reviewers applied ten README
rules plus criteria written nowhere public (credentials, approvals, disclosure,
compatibility metadata, forks of bundled plugins).
- plugin-catalog/README.md: rules 11-15 write down those review criteria; the
rule list sits between admission-rules markers.
- New developer-guide page 'Submitting to the plugin catalog': before you
submit, the PR flow, the full rule block (verbatim mirror), pin updates,
delisting vs removal, asking for a hook.
- The /plugins 'Submit' links and the user-guide section point at it.
- tests/website/test_catalog_rules_mirror.py fails when the two rule blocks
differ; classify_changes makes a README-only edit run it.
- AGENTS.md, plugins/AGENTS.md and CONTRIBUTING.md reference both files.
SMS and both WeCom adapters now split long replies, and their chunk loops
returned a plain failure when a later chunk was rejected. The base send
path cannot tell that from a total failure, so _send_with_retry retried the
whole payload or sent the 3500-char plain-text fallback and reported
success: the recipient got the head twice and never got the tail.
The three loops now share gateway.platforms.helpers.send_chunks(), which
marks a failure after delivered chunks with the existing partial_overflow
contract (the one Telegram and Discord set and _is_partial_delivery reads).
The retry and plain-text fallback already refuse to re-send a partial
delivery, so the caller gets an honest failure and nothing is duplicated.
Resuming the undelivered tail is #124295's scope.
Both WeCom adapters dropped everything past the platform limit with no
error: the AI Bot adapter sliced markdown content at 4000 chars, and the
callback (self-built app) adapter sliced text at 2048 *characters* while
message/send keeps only the first 2048 *bytes* (so CJK replies lost
everything past about 680 characters). Cron output was additionally cut
to 4000 chars with a gateway-host file pointer.
AI Bot: send() queues one markdown message per MAX_MESSAGE_LENGTH chunk,
so every chunk draws its own token from the 30 msgs/min per-chat bucket
(errcode 846607), and each goes out on the cached reply req_id like any
other send in that chat (group chats cannot send proactively). Callback: chunk by UTF-8 bytes at 2048.
Both declare splits_long_messages so cron delivery hands over the full
payload. Each stops at the first failed chunk and returns its error.
Fixes#19689Fixes#107443
Co-authored-by: luyao618 <364939526@qq.com>
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: Andy <sz.private@gmail.com>
The delivery router truncated cron output above 4000 chars and appended
a footer pointing at a file on the gateway host, which an email
recipient cannot open. One email carries the whole body, so declare
splits_long_messages and let the router hand over the full payload.
Fixes#61990
Co-authored-by: Petra <petra.kersten@agentmail.to>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
send() called truncate_message() without a limit, so it chunked at the
4096 default; Twilio rejects any Body over 1600 characters (code 21617),
so every reply longer than 1600 chars failed outright. Oversized cron
output was first cut to 4000 chars by the delivery router (SMS did not
declare splits_long_messages) and then failed the same way.
Chunk at MAX_MESSAGE_LENGTH (1600) and declare splits_long_messages so
the router hands send() the full payload.
Fixes#62751
Co-authored-by: Philip D'Souza <9472774+PhilipAD@users.noreply.github.com>
Co-authored-by: JackJin <1037461232@qq.com>
Co-authored-by: euisuh <euisuh.jeong@gmail.com>
Co-authored-by: Froraut <79215537+Froraut@users.noreply.github.com>
The hardcoded "group" duplicated the registered callback's own
chat_type default and implied a DM/group decision that was never made.
Passing chat_id only yields identical verdicts.
Generated image/video output now writes under cache/generated/images|videos
instead of the swept inbound caches (cache/images, cache/videos). The hourly
gateway housekeeping deletes 24h-old files in those dirs; for base64-returning
providers the swept file was the only copy, breaking transcript references,
the Artifacts panel and later edits.
Rebased onto current main, where the writers delegate to agent/provider_media,
so the media kind is now the generated/ subpath. cache/generated is added to
tools/credential_files._CACHE_DIRS: without a mount/sync entry the Docker,
SSH and Modal backends never receive the bytes and the provider result loses
agent_visible_image. All callers consume the returned absolute paths, so no
call-site changes were needed; inbound download paths (get_image_cache_dir /
get_video_cache_dir) and the sweeper are untouched.
RED: tests/agent/test_generated_media_durable.py (9 failed on main) — 25h-backdated
generated files deleted by the real cleanup_image_cache / cleanup_video_cache, and
no mount mapping for cache/generated. tests/tools/test_image_generation_artifacts.py
::test_real_generated_image_is_agent_visible_under_ssh_env covers the sandbox
regression with the real writer. Green: 533 passed, 19 skipped.
Co-authored-by: Froraut <79215537+Froraut@users.noreply.github.com>
(cherry picked from commit fd23181f22232f66aec7076d6bf0868ed45661f2)
The plugin now registers a pre_tool_call hook, but plugin.yaml still listed
only post_tool_call/on_session_end, so `plugin doctor` warned "registration
adds hook 'pre_tool_call' not listed in provides_hooks" and catalog/capability
listings under-reported what the plugin hooks.
Also: README notes calls without a tool_call_id (execute_code RPC) are not
tracked; a single _owner() helper keys both _pre_call and
_recent_test_tracks so the two can't drift; drop the unused `result`
parameter from the post hook (still absorbed by **_); trim the _pre_call
comment.
The pre/post snapshot handshake had two ways to mark a user's existing
file as created, which queues it for deletion when the turn ends:
- Snapshots were keyed by tool_call_id alone. llama.cpp sends one
constant id for every call. When session B's pre hook ran between
session A's pre and post, A consumed B's snapshot. A's
already-existing test_user.py then looked "not in existing", and A
tracked it.
- Every pre_tool_call hook sees the original args. Another plugin's
`modify` directive is merged afterwards, and post_tool_call sees the
final args. A path swapped in by that hook was never snapshotted, so
"not in existing" held for it too. The file was tracked even when the
write was refused.
Snapshots are now keyed by (task_id or session_id, tool_call_id), the
identity both hooks receive from the dispatcher. They store the paths the
call named that did not exist yet, instead of the ones that did. The post
hook tracks a path only if its own snapshot saw it absent and it exists
now. Any path the snapshot did not cover fails closed.
I reproduced both end to end through model_tools.handle_function_call
with a second plugin's modify hook and two interleaved sessions. The
user's file was deleted before this fix and survives after it, and a
file the call really created is still cleaned up. The README hook
contract is updated to match.
(cherry picked from commit 8d0b853981f1ab0775549f4831a3d4f733a46106)
post_tool_call sent write_file/patch paths, every absolute path in a
terminal command AND every path in its output to dg.track(), gated only
on `p.exists()` plus the test_*/tmp_* name rule. Category 'test' is
deleted at age 0 by quick(), which on_session_end runs after every turn,
and directories are rmtree'd. So a user's cron script
$HERMES_HOME/scripts/test_uptime.py the agent merely patched, or a hook
dir $HERMES_HOME/hooks/test_notify it listed (`ls -la`, `find ... -name
'test_*'`), was irreversibly deleted when the turn ended — then the
emptied scripts/ and hooks/ dirs too. The plugin's contract is files
"created during Hermes sessions".
A pre_tool_call hook now snapshots which of the call's argument paths
already exist (keyed by tool_call_id; it never raises, since pre hooks
fail closed; entries expire after an hour if the post hook never
fires). post_tool_call tracks a path only when the call named it, it
did not exist before and exists now. Paths seen only in terminal output
carry no such proof and are no longer tracked; no snapshot means
nothing is tracked. Existing tests now drive the real hook order.
(cherry picked from commit e0e0e1f7450df0c75a8499fe57b63dc46c6d2af2)
connect() and is_trusted_peer() each spelled out the same
"network-exposed token bind with no allow-list" condition; drift between
the two would let the startup warning and the actual refusal disagree.
Extract A2ASecurityContext.dispatch_fails_closed() and use it in both
places (truth table unchanged). Also hoist the is_loopback_host import to
module top (no cycle) and drop the restating docstring.
The fail-closed change dropped the localhost_only() short-circuit from
is_trusted_peer. In no-token mode authenticate() always yields ip:<addr>,
so any install with A2A_TRUSTED_PEERS / a2a.trusted_peers configured
started 403-ing every local dispatch. Restore the short-circuit and pin it
in test_localhost_trusts_all.
Also:
- log the non-loopback/no-allow-list misconfiguration once in
A2AAdapter.connect() instead of an ERROR on every refused request;
- is_loopback_bind reuses agent.proxy_bypass.is_loopback_host instead of a
hand-rolled loopback check;
- fold the loopback backward-compat test into
test_no_allowlist_trusts_authenticated (now explicit A2A_HOST=127.0.0.1)
and drop implementation asserts on is_loopback_bind().
Operators exposing the adapter with a token now get dispatch refused
without an allow-list (#126756); say so in the env table.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
The loopback branch repeated the final `identity in self.trusted_peers`
membership check. Only the empty-allowlist case depends on the bind, so
branch on that alone: loopback stays open, non-loopback logs and fails
closed. Behaviour is unchanged (probe: 0 fails).
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Co-authored-by: Sahil Vishnalya <sahilvishnaliya250@gmail.com>
The refusal named a cause that cannot happen. `_vendor_gateway` never contacts the
network, so "unreachable" was wrong for the no-identity, failed-lookup and
not-entitled cases alike — and a guest was told it had no Nous identity when it
holds a perfectly good one.
Root of the guest case: `_error_info` dropped `account_tier`, so a guest whose
account lookup failed — precisely the state where the tier can no longer be read
from the JWT, because the token is inside its refresh window — came back looking
like a registered identity. Carry the stored tier through the error snapshot, and
both this refusal and every other tier-aware surface see it.
Pick the reason in priority order (guest / no usable identity / unreachable) in
one helper beside `fast_search_entitled`, so the predicate and its explanation
cannot drift, and phrase it as a clause for `selection_error`'s "but …" template —
the canonical `FREE_TIER_NEEDS_ACCOUNT` strings are standalone sentences and read
wrong embedded mid-sentence.
Also record why the `error` clause in `fast_search_entitled` is load-bearing
rather than defensive, so it is not removed as belt-and-braces.
The test helper now mirrors what anon_auth actually persists (the tier alongside
the token), which is what makes the failed-lookup case reproducible at all.
Two review findings on the free fast-search route.
A guest with a stored `web.backend: nous` holds a real Nous identity and a
usable token; the route excludes it by tier, not by absence. The refusal said
"no Nous identity or unreachable", which is wrong for exactly the cohort this
route was gated against. Reuse the canonical FREE_TIER_NEEDS_ACCOUNT text for
the anonymous tier and keep the unreachable wording for everyone else.
search() then called _managed_web_search() a third time just to choose that
message, after _managed_gateway() had already resolved the route — a full
autodetect ladder plus a Portal account read and token peek. Let _managed_gateway
accept the answer its caller already computed so a dispatch walks the ladder
once.
Docs: the autodetect row implies the keyless switch governs a registered
identity's web_search. It does not.
The Tool Gateway serves Perplexity POST /search with search_type: "fast"
to any Nous identity without funding checks, but Hermes gated the route
locally behind managed_nous_tools_enabled() (paid access or a live tool
pool), and autodetect only selected managed Perplexity after the gated
Firecrawl probe passed. Anonymous and credit-exhausted users fell through
to the keyless ring without trying it.
Resolve the Perplexity search route on identity alone
(resolve_free_search_gateway: usable Nous token, same guest-disabled and
refresh rules), and let autodetect pick it when nothing explicit is
configured. Every other vendor keeps the entitlement gate, and the billed
managed-Firecrawl fallback now requires it too, so a free-search-only
caller whose call fails goes to the keyless rescue ring as before.
Explicit selections and direct keys keep precedence.
managed_image_partner inlined the dispatcher's strip-or-None reading of
image_gen.provider/model; managed_route now normalises raw values itself,
so both callers pass config values straight through and cannot drift.
_managed_model_plugin's docstring points at managed_route instead of
restating the rule, and NOUS_MANAGED_PROVIDER is a top-level import (no
cycle). A Krea style reference that disappears between is_file() and open()
now returns invalid_image_url instead of raising. Test comment corrected:
only managed-model routing ignores legacy use_gateway.
The inlining step called ref.get() on every reference, so a legacy
image_style_references entry that is neither a string nor a dict raised
AttributeError out of generate() (main forwarded it to Krea). It now passes
through untouched.
The 3 MB budget was checked against stat() and the file read separately,
so a file that grew in between was sent in full. Each file is now read once,
at most one byte past the remaining budget, and the cap applies to those
bytes. The refusal test states each expected error_type and message
directly.
The salvaged inlining checked is_file() and summed st_size before
raise_if_read_blocked(), so a denied path (e.g. HERMES_HOME/.env) answered
"not found" or "over 3 MB" and revealed whether it exists and how big it
is. The guard now runs first, and its denial comes back as the plugin's
normal error dict instead of an exception escaping generate().
Local files are inlined after the managed-gateway LoRA/moodboard fail-fast,
so a request about to be refused no longer reads and base64s up to 3 MB.
The size message is built from the constant. The four salvaged tests
become one embed test and one parametrized refusal test (missing file,
total over the cap, guard-denied path).
OpenAI released GPT-6.1 Sol on 2026-09-29. It is live on OpenRouter
(openai/gpt-6.1-sol, -pro) and the direct API; not yet on Nous Portal
or on Codex OAuth for our test account.
- Catalog: OPENROUTER_MODELS + openai-api picker; kept off the nous
list via _OPENROUTER_ONLY until the Portal serves it. Manifest regenerated.
- Context: 1.05M direct API, 272K Codex fallback; 272K compaction
auto-raise on the Codex route.
- Pricing: $2/$10, cached input $0.10 (5%, not 6 Sol's 10%), cache write
$2.50, 2x/1.5x above 272K; -pro aliased to the base row.
- Reasoning: none/minimal return 400 on 6.1 Sol (unlike 6 Sol), so it uses
Astra's low..max ladder without Astra's account gating. OpenRouter's
catalog wrongly lists 'none' for it, so the OpenRouter profile omits a
disable when the OpenAI ladder has no 'none'.
The #127047 comment said a supervisor's stop signal reaches the bridge
through the shared process group. The bridge is spawned with
start_new_session=True; what reaches it is a container-wide broadcast
such as s6-overlay's stage-3 SIGTERM.
Same race as the WhatsApp bridge (#127047): a container/supervisor stop
signals the whole process tree, so the Photon sidecar (its own session)
can die before the gateway's stop flow reaches disconnect(), while
_inbound_running is still True. _supervise_sidecar() then reported
SIDECAR_CRASHED and queued a reconnect during shutdown.
Consult the runner's _stop_requested_by_signal, which the signal handler
sets before any stop work runs. A sidecar exit while the gateway keeps
running is unchanged: still a retryable fatal error.
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
A supervisor's stop signal (docker stop, s6-supervise, systemd) reaches the
bridge child in the same process group before the gateway's stop flow reaches
WhatsAppAdapter.disconnect(), so _shutting_down is still False when the poll
loop observes the -15 exit. _check_managed_bridge_exit() then reports a
retryable fatal adapter error, the stranded-platform check exits the gateway
with failure, and the supervisor restarts it into a crash loop (#127047).
Also consult the runner's _stop_requested_by_signal, which the signal handler
flips before any stop work runs. A bridge SIGTERM/SIGINT while the gateway
keeps running is unchanged: still a retryable fatal error so the reconnect
watcher revives it.
(cherry picked from commit 252ae311f06e402f9a8ef1482c87aaefe8d9ac39)
* feat(i18n): layered catalogs — plugin packs and user overlay over bundled locales
* feat(tui): i18n layer — en catalog, nanostore runtime, RPC pack loader, _keys.tui.json emitter
ui-tui/src/i18n/: en.ts (facade over topical siblings under en/), types.ts
(Translations + dotted TranslationKey derived from en), runtime.ts ($locale/
$catalog atoms, translateFrom active→en→key, pack merge with string→fn
wrapping for {0}/{1} placeholders), loader.ts (display.language →
i18n.catalog {lang, surface:'tui'}, English when the method is missing),
useT()/useLocale() hooks, t() for non-React code. useConfigSync feeds the
loader from the existing config.get full hydration. `npm run i18n:keys`
writes locales/_keys.tui.json (sorted flat key list) and runs before build.
* feat(plugins): provides_locales manifest field, ctx.register_locale/register_locale_dir, manifest-only language packs
* chore(tui): split en catalog siblings by lane (slash sibling)
* feat(plugins): validate language packs — parse, text-only, key-subset WARN against en / _keys exports
* feat(tui_gateway): i18n.languages / i18n.catalog RPC + regenerated contracts
* feat(config): display.language accepts any supported_languages() id, refuses unknown ids with the list
* docs(i18n): language packs user guide, pluggable display.language, plugin developer section, AGENTS notes
* feat(plugins): report language-pack layers in the mid-run activation summary
* feat(tui): wire status bar, composer placeholders, hotkey help and approval/clarify/confirm prompts through i18n
StatusRule maps compared state values (ready/running…/summoning) to catalog
text at render via displayStatus(); hotkeys()/placeholder() resolve lazily so
a pack that arrives after boot applies. Catalog grows to 81 keys.
* feat(desktop): pluggable app locales — registry, host.i18n.registerAppLocale, backend packs, keys emitter
- Locale widens to string (BundledLocale keeps the union); TRANSLATIONS stays
the bundled record and every consumer resolves through the registry.
- src/i18n/registry.ts: registerAppLocale(id, {endonym, rtl, translations})
layers partial packs (nested or flat dotted) over bundled/en via
mergeTranslations; a string over a function-valued en entry becomes a
positional {0}/{1} formatter; $appLocaleVersion bumps so translators
re-render; per-source disposers + replaceAppLocaleSource for atomic swaps.
- Backend packs: i18n.languages + i18n.catalog {surface:'desktop'} feed the
registry as source 'backend' (method-not-found is silent); re-synced on
socket open, display.language change and profile switch. A saved pack-only
language is promoted once its pack registers.
- SDK: host.i18n.registerAppLocale / languageOptions; ctx.i18n.registerAppLocale
tracked for unload. Docs in the desktop plugin SDK guide + skill reference.
- Language switcher lists bundled ∪ registered ∪ backend, endonym-only; RTL
from the registry (applyDocumentLocale takes rtl).
- npm run i18n:keys emits locales/_keys.desktop.json (wired into build).
* i18n(cli): route /topup + /subscription copy through t() (cli.billing.*, cli.subscription.*)
Module-level copy tables and modal choice tuples in cli_billing_mixin.py froze
English at import, before display.language was known. They are now key tables /
builder functions evaluated at call time; every user-facing line in the /usage
balance block, /subscription and the five /topup screens reads the catalog.
Choice VALUES stay English identifiers. Fragment-assembled status lines
(Plan: … → cancels · $x left · renews …) become full templates.
* i18n(gateway): exec-approval card contract + base/run/run_busy/run_inbound replies through t()
- base_exec_approval: EA_* English constants stay; add ea_header_text()/ea_reason_label_text()/
ea_smart_deny_line_text()/ea_default_reason_text()/ea_action_labels()/approval_timed_out_notice()
accessors; deadline + timed-out notice resolve via gateway.exec_approval.*
- BasePlatformAdapter._EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/_EA_ACTION_LABELS become
properties (adapters still shadow them with markup class attrs)
- run.py: provider error replies table holds catalog keys; _CONTEXT_OVERFLOW_REPLY -> _context_overflow_reply()
- run_busy/run_inbound: typed approval + slash-confirm matchers accept English ∪ approval.inputs.* (t())
- locales/en.yaml: gateway.exec_approval/busy/errors/... namespaces
* i18n(cli): wire modal, loops, agent-setup mixins through t() (cli.* keys)
* i18n(platforms): route Slack, Matrix and Feishu user-facing text through t()
Exec-approval markup overrides (_EA_HEADER/_EA_REASON_LABEL/_EA_SMART_DENY_LINE/
_EA_ACTION_LABELS) become per-call properties over the shared
gateway.exec_approval.* contract keys, so Slack's 3000-char section budget
measures the resolved template. Slack _APPROVAL_DECISIONS/_CONFIRM_DECISIONS,
Feishu _APPROVAL_LABEL_MAP and Matrix _EA_LEGEND/_EA_TYPED_HINT turn into
key tables resolved at click time; the Matrix typed hints become whole
sentences per offered tier instead of spliced fragments. Slack button labels
are cut to 75 chars and select placeholders to 150 after translation; the
model-facing clarify fallback answer ('choice N') stays English while the
card copy localizes.
locales/en.yaml gains the gateway.exec_approval.* contract keys plus the
platform.shared.* / platform.slack.* / platform.matrix.* / platform.feishu.*
namespaces (and the keys for the other adapters wired in follow-up commits).
* i18n(gateway): run_turn / run_turn_runner / approval-settle copy through t()
- status hints, proxy errors, background task notices, progress heartbeats, session info lines
- tool progress chrome (tool_head/tool_pending/tool_preview/tool_verbose) shared by base.format_tool_event
- run_turn_runner:1406 Chinese clarify placeholder -> gateway.clarify.native_stream_placeholder (zh text kept in zh.yaml)
- _UNEXPECTED_SILENCE_REPLY/_CLARIFY_EXPIRED_NOTICE -> accessor functions
* i18n(platforms): route Google Chat and Teams user-facing text through t()
Google Chat clarify card, typing placeholder, orphan-card labels and the whole
/setup-files reply set (module constants become platform.google_chat.setup_files.*
keys resolved at reply time). The attachment-fallback notice that shipped
hardcoded in Spanish is keyed with an English en value; es.yaml carries the
original Spanish text for those four keys.
Teams approval card header/reason use the gateway.exec_approval.* contract,
_APPROVAL_LABELS becomes a key table resolved at click time, and the meeting
summary writer resolves its section headings/fallbacks per render.
* i18n(platforms): route LINE, WeCom, email, DingTalk, IRC and Home Assistant text through t()
LINE default copy constants become catalog keys resolved in __init__ (the
LINE_*_TEXT / extra.* operator overrides still win); the busy-ack bypass
matcher keys on the leading emoji marker only, so it keeps firing once the
gateway busy heads are localized. WeCom media size/format notices that shipped
hardcoded in Chinese are keyed with English en values and zh.yaml carries the
original Chinese text. DingTalk emotion bubbles resolve per send.
* i18n(cli): route /model switch output and -q status lines through t() (cli.model.*, cli.single_query.*)
Switch-summary labels shared with the gateway reuse gateway.model.* keys
(provider/context/max-output/capabilities/prompt-caching); CLI-only variants
(glyph or no-backtick forms) live under cli.model.*. The hand-padded /model usage
block becomes a (form, description-key) table padded at render time so the
command syntax stays fixed while descriptions translate. -q 'Error:' reuses
gateway.model.error_prefix.
* i18n(cli): route TUI panel/hint/placeholder copy through t() (cli.tui.*)
_APPROVAL_CHOICE_LABELS and _TUI_MODAL_HINTS become key tables resolved at
render time; vault/sudo panel bodies are one catalog value per panel split on
newline; inline plurals use <key>_one/<key>_other. Adds the cli.* namespace
(shared/tui/voice/render/subagents/dock) to locales/en.yaml.
* i18n(cli): voice/wake-word CLI copy through t() (cli.voice.*)
RuntimeError texts raised in _voice_start_recording are human copy (callers
print {e}) and are keyed; the Termux requirement-check match stays English.
Wake state ids stay internal, only their labels localize.
* i18n(cli): live-work dock, subagent monitor and render copy through t()
cli.subagents.* / cli.dock.* / cli.render.*; count fragments pluralize via
_one/_other keys, verdict table holds keys resolved at paint time so width
clipping measures the translated text.
* i18n(gateway): unauthorized/pairing, voice, topics, shutdown, startup, notifications, kanban pings through t()
* i18n(cli): move tips + composer placeholders into the catalog (tips.tNNN / tips.placeholder.pNN)
get_random_tip()/get_random_composer_placeholder() pick a key from the English catalog
(the parity baseline, probed once per process) and resolve it through t() for the active
language, so language packs translate tips like any other string. Also lands the cli.*
en.yaml namespace consumed by the CLI info/help/error-copy wiring in the next commit.
* i18n(cli): wire chat-turn + session mixins through t(); kanban log trimmer matches t() output
* test(cli): assert TUI/dock/voice copy via t(key); prove labels resolve at render time
Pinned-English assertions in the approval-UI, live-work dock and voice tests now
go through the catalog. New test swaps the catalog after import and checks the
approval panel + hint row follow it (the reason _APPROVAL_CHOICE_LABELS and
_TUI_MODAL_HINTS became key tables).
* i18n(cli): route CLI info/help/error copy through t() (cli.* namespace)
cli_info_mixin: /help consumes CommandDef.describe() (added to commands.py: slash.<name>.description
with fallback to .description), section titles/skill/quick-command headers, /tools, /toolsets,
/usage labels, /context, /whoami, /insights, /gateway status, tool-progress labels, bang-shell
denials, MCP config-watch + /reload-mcp confirm/reload lines, /reload-skills, and the session-store
warning all read the catalog at call time (module-level label tables became functions so the
active language is honoured after startup). cli.py: worktree cleanup, tirith warning, show_config
(labels re-padded at print time), quick/plugin/skill slash-command errors, ambiguous-command hint,
stdin error, gateway start, profile warning. cli_chat_error_copy / cli_unknown_command /
cli_output / cli_init_mixin: chat error panel copy, did-you-mean lines, n-more / yes-no prompt
(localized affirmative initial alongside 'y'), unknown-toolsets warning.
* i18n(w1a): wire agent display/explainers/approval + slash registry/help through t()
- hermes_cli/commands.py: CommandDef.describe() resolves slash.<name>.description
at call time; category labels via slash.category.*; help/alias/usage suffixes
via slash.shared.*; gateway_help_lines and commands_platforms/slash_exec use them.
- agent/display.py: display.verb.* resolved at call time via get_tool_verb();
bridge/spinner/thinking-verb/cute-row/failure/preview/diff text via display.*.
- agent/turn_explainers.py: exit-reason / persistence-cause tables become call-time
lookups (explainer.exit.*, explainer.persistence.*, explainer.file_mutation.*).
- agent/background_review.py, session_activity.py, context_breakdown.py,
status_output.py: review summaries, iteration progress, context notices.
- tools/approval.py, approval_context.py: approval.summary.*, approval.noun.*,
approval.window.* pluralized keys.
- gateway/slash_commands*.py: remaining raw strings (busy, whoami, platform,
bundles, memory, skills, approvals, set_home, diff, update, debug, profile,
heartbeat, refine, review, subgoal, loop, retry, compress codex path, save,
sessions, model guard/errors, agents rows, topup, login). HISTORY_UNREADABLE
keeps its English constant; callers use history_unreadable() ->
gateway.shared.history_unreadable.
- locales/en.yaml: new approval/display/explainer/slash blocks + gateway leftovers.
* i18n(telegram): route adapter chat copy through t()
Approval card (header/reason/smart-deny as HTML-escaped properties), inline
button labels, callback toasts (cut at Telegram's 200-char cap), model/choice
pickers, clarify/update/slash-confirm prompts, gmail-triage labels and the
inbound-media failure notice now come from the catalog. _UNAUTHORIZED is a
lazy _unauthorized() so the import no longer binds a language. The command
menu carries a language+payload fingerprint (forum scopes re-register on
change) and BotCommand descriptions are cut at 256.
Adds gateway.exec_approval.* (WAVE2 contract), platform.telegram.*,
platform.discord.* and the slash.*.description keys the Discord table shares
with the CLI registry to locales/en.yaml.
* i18n(gateway/platforms): whatsapp_cloud, yuanbao, weixin, signal, api_server copy through t()
- whatsapp_cloud: clarify list/buttons, approve/deny + slash-confirm labels via platform.whatsapp.* (t()-then-truncate at 20/24/72 caps); _EA_HEADER becomes a property wrapping ea_header_text()
- yuanbao: SLOW_RESPONSE_MESSAGE -> slow_response_message() (platform.yuanbao.slow_response_notice; zh keeps the original text); cron-wrapper markers centralized as module constants for strip_cron_wrapper
- api_server: PROVIDER_AUTH_FAILED_LABEL/PROVIDER_RATE_LIMITED_LABEL stay English for run.py matchers; user_text() renders via t()
- signal/_format_wait, weixin voice caption, openai_routes transformed notice
- run_turn: second _UNEXPECTED_SILENCE_REPLY consumer -> accessor
* i18n(discord): route adapter chat copy through t()
Native slash-command table becomes _NATIVE_SLASH_COMMAND_SPECS holding catalog
keys; _native_slash_commands() resolves descriptions, parameter descriptions
and Choice names for the active language, each cut at Discord's 100-char cap,
and the app-command sync fingerprint now includes get_language() so a
display.language change re-syncs. Exec-approval card (gateway.exec_approval.*
contract), slash-confirm / clarify / update views, model+choice pickers,
thread creation, forum titles, voice acks, the response-truncation notice,
the unauthorized-slash security alert and the media upload-size notices all
read from platform.discord.*. Decorator-declared button labels are relabelled
in __init__ (80-char cap); embed titles cut at 256, select placeholders at
150, option label/description at 100. _UNAUTHORIZED is a lazy _unauthorized().
* i18n(cli): wire status-bar, stream, terminal mixins + terminal_input through t(); rename kwargs that shadow t(key)
* i18n: wire hermes_cli/cli_commands_mixin.py slash-command copy through t()
- 431 new leaves under cli.commands.<cmd>.* in locales/en.yaml; 12 rows reuse
existing gateway.* keys (rollback, diff, resume, branch, btw, model, reasoning)
via a _gt() helper so CLI and gateway replies stay identical.
- Module-level English tables (_BUSY_MODE_*, _REASONING_TOGGLES, _HATCH_PROGRESS,
_DIFF_LABELS, _LOCAL_ENGINE_LINES) become call-time catalog lookups keyed by id.
- Verb tables (Enabling/Disabling, Paused/Resumed/Triggered, planned/done,
Updating/Generating) are one full template per variant; plurals use
<key>_one/<key>_other via _tn(); hand-padded column labels (/snapshot list)
translate the value and re-pad at the call site.
- Multi-line usage blocks are single catalog values split with _lines().
- Model-facing system notes and DB-stored reasons stay English (EXCLUDED).
* tests: assert /handoff, /worktree, /login CLI copy via t(key) instead of pinned English
* test(i18n): pin Telegram/Discord adapter catalog wiring
Lazy unauthorized notice, exec-approval contract keys, HTML escaping before
Telegram <b> wrapping, 200-char toast / 256-char BotCommand caps, Discord
100-char app-command text and 80-char button caps, and language-bearing
command-menu fingerprints on both platforms.
* i18n: reconcile cli.shared on/off vs enabled/disabled after lane merge
* i18n: describe() in TUI-gateway slash listings; localize TUI exit resume hint
* i18n(tr): translate bundled catalog + tui pack
* i18n(ja): translate bundled catalog + tui pack
* i18n(ko): translate bundled catalog + tui pack
* i18n(zh): translate bundled catalog + tui pack
* i18n(fr): translate bundled catalog + tui pack
* i18n(af): translate bundled catalog + tui pack
* i18n(uk): translate bundled catalog + tui pack
* i18n(ar): translate bundled catalog + tui pack
* i18n(pt): translate bundled catalog + tui pack
* i18n(it): translate bundled catalog + tui pack
* i18n(es): translate bundled catalog + tui pack
* i18n(zh-hant): translate bundled catalog + tui pack
* i18n(ru): translate bundled catalog + tui pack
* i18n(hu): translate bundled catalog + tui pack
* i18n(hu): translate pre-existing English-valued leftovers (kanban wake, /context, /status, fast labels)
* i18n(de): translate bundled catalog + tui pack
* i18n(ga): translate bundled catalog + tui pack
* test(i18n): fixture matches _normalize_lang(lang, home) signature
* i18n(tui): scaffold userMessages/slashCmd en siblings
* i18n(tui): wire secure prompts + content tables
* feat(tui): i18n — wire billing, subscription, connection-setup and journey overlays
Adds en siblings billing.ts / subscription.ts / connection.ts (namespaces
billing, subscription, connection, journey) and routes every user-facing
literal in billingOverlay, subscriptionOverlay, connectionSetupOverlay and
journey through useT()/messages(). Module-level label tables became lazy
(scopeStillDeniedResult(), verbOf(T, action)); auto-reload rows dispatch on
stable ids instead of label text. Regenerates locales/_keys.tui.json.
* i18n(tui): wire slash ops/wake replies
* i18n(tui): wire pickers (modelPicker, activeSessionSwitcher, petPicker)
* i18n(tui): wire slash core/debug/setup replies
* i18n(tui): wire hubs (agents overlay/panel/controls, skills, plugins)
* i18n(tui): wire slash session/topup/subscription replies
* i18n(tui): wire chat bits (branding, thinking, messageLine, loaders, todo, queued, banner, entry)
* i18n(tui): register t3 siblings (pickers, hubs, secure, content, chatBits) and regenerate keys
* i18n(tui): wire userMessages copy through the userMessages namespace
* i18n(tui): lazy-copy test for userMessages, regenerate _keys.tui.json
* i18n(tui): wire session/gateway/lib text through the TUI catalog (lane t2)
Adds en siblings session.ts, gatewayMsg.ts, libText.ts (namespaces session,
gatewayMsg, libText) and routes user-facing literals in app/{useMainApp,
useSessionLifecycle,useInputHandlers,turnController,createServerRequestHandler,
setupHandoff,createGatewayEventHandler}.ts, gatewayClient displayed reasons,
lib/*, domain/*, hooks/* through t()/messages(). Status-bar state values that
code compares against, backend-matched strings, log lines, model-bound text,
and machine 'error:' prefixes stay literal. Regenerates locales/_keys.tui.json
(232 keys).
* i18n: serve bundled locales/<lang>.tui.yaml under overlay/packs; TUI pack parity test; regen _keys.tui.json (1250)
* i18n: translate pre-existing English stubs in bundled locales (424 leaves, 14 locales)
* tui: i18n-export-en script (English templates for pack translators)
* docs(i18n): bundled TUI packs are the bottom layer of the tui surface
* i18n(ru): translate TUI pack
* i18n(ar): translate TUI pack
* i18n(es): translate TUI pack
* i18n(pt): translate TUI pack
* i18n(ko): translate TUI pack
* i18n(de): translate TUI pack
* i18n(ja): translate TUI pack
* i18n(fr): translate TUI pack
* i18n(tr): translate TUI pack
* i18n(it): translate TUI pack
* i18n(zh): translate TUI pack
* i18n(zh-hant): translate TUI pack
* i18n(hu): translate TUI pack
* i18n(uk): translate TUI pack
1,169 missing keys translated; 81 pre-existing kept byte-identical. Parity OK missing=0 extra=0 placeholder_mismatch=0 empty=0.
Deliberately identical to en: chatBits.branding.mcpSummary ({0} MCP), chatBits.thinking.agentsHint ((/agents)), session.main.voiceStt (◉ STT), session.main.voiceTtsSuffix ( [tts]), slashCmd.core.help.tuiSection (TUI), slashCmd.core.history.hermesTag (Hermes #{0}), slashCmd.debug.heapdump.heapPath (heapdump: {0}), slashCmd.debug.mem.rss (rss), subscription.stepUp.title (Remote Spending — product feature name, as in core catalog), content.faces.* (glyph-only kaomoji).
* i18n(ga): translate TUI pack
* i18n(af): translate TUI pack
* plugin_guard: locale catalogs in language packs step down the agent-config family
A translated status line such as "Updating AGENTS.md" in locales/<lang>.yaml is UI text the loader
reads as a string leaf; it cannot edit a file. The bundled en.yaml itself tripped agent_config_mod
at critical, making any faithful language pack uninstallable. Injection shapes keep full severity.
* plugin_validate_locales: read key exports with utf-8-sig (Windows footgun lint)
* i18n(relay): route relay adapter prompt copy through t(); drop dead import-bound approval header
Adds platform.relay.* (5 keys) to en and all 16 bundled locales, reusing the sibling platform
translations for the confirm buttons and the Other option.
* ci: fix TUI import order, MDX table pipe, main's overflow-warning wording in all locales; fresh-install fixture carries the i18n kernel
- ui-tui/src/i18n/en.ts: perfectionist/sort-imports (slash before slashCmd)
- docs plugins/index.md: escape the | inside the provides_locales table cell (MDX parsed <id> as JSX)
- display.notice.uncompressed_context_overflow: adopt main's wording (names compression.enabled: false
and /compact) in en + 16 locales; the guardrail test pins that phrase
- tests/scripts/test_fresh_source_install.py: the installer tail now resolves CLI text through
agent.i18n, so the fixture tree carries the i18n kernel + en.yaml (not the agent runtime)
* docs(desktop-plugin-sdk): double-backtick the template-literal example (MDX evaluated ${n})
* test(e2e): display.language is validated against the live language set; exclude it from the arbitrary-string set property
* commands: keep the localized COMMANDS/COMMANDS_BY_CATEGORY module __getattr__ after the compat block removal
* build: never write locales/_keys.*.json from the desktop/TUI builds; regenerate the committed desktop key export
The desktop build regenerated locales/_keys.desktop.json in the checkout, so a
hermes update that rebuilt the app left the tree dirty (Desktop update E2E:
'M locales/_keys.desktop.json'). The key exports are committed artifacts pinned
to en.ts by apps/desktop/scripts/i18n-keys.test.mjs and ui-tui i18n:keys:check;
builds read them, never write them. Regenerated after main's new desktop strings.
* test: unbreak two main-red timing tests the PR merge-ref inherits
- test_local_runtime racing fake publishes the modern state record (legacy pid-only
records are rejected since 65ff3ad353; main has been red on this test since)
- test_run_progress_topics ManyProgressLinesAgent waits for the first bubble instead of
a fixed 0.35s, which a loaded CI runner does not always meet
* chore(i18n): regenerate desktop key catalog for main's new strings (model pricing, copy changelog)
* test(e2e): torture-chamber fd monitor confirms a deleted sidecar is still held before calling it a leak
SQLite's WAL last-close unlinks -shm before closing its descriptor (unixShmUnmap, then
unixShmPurge), so a healthy close shows a (deleted) -shm for microseconds; the 20ms poll
occasionally caught that window on the short-lived opener and failed the episode.
* chore(i18n): regenerate desktop key catalog for main's telemetry/consent strings
* chore(i18n): regenerate desktop key catalog after main sync
---------
Co-authored-by: Teknium <teknium@nousresearch.com>
Phase 2 review follow-ups:
- OpenViking overlaid the bound profile's whole .env after the scrub,
so that profile's bot, dashboard and relay tokens reached the server
again. The overlaid env is now scrubbed a second time for Tier 1.
Provider keys still pass, and they still come from the bound profile.
- The strict per-home manifest read raised on any unreadable entry
under <home>/plugins, which failed every child spawn for that
profile. It is now strict only where the manifest is known to be a
platform's: bundled or plugins/platforms/. Dunder and dot
directories, unsearchable plugin directories and unreadable flat
plugins/* manifests are skipped with a warning, because they can't
load as plugins either.
- The per-home cache is keyed on hermes_home_key(), so a symlinked
alias no longer gets a second entry.
- The modal snapshot test swapped hermes_cli.config for a stub, which
the policy import can no longer use. HERMES_HOME already points the
real module at the test home, so the stub is gone.
- The env_passthrough docstring now describes declared names rather
than the dropped prefix rule.
Review follow-ups on the declared-secret policy:
- A profile's user-installed platform plugins declared secrets into
one process-wide set, read from the launch home at import. Under
multiplex that stripped profile B's same-named user variable and
missed profile A's own declarations. Bundled manifests stay
process-wide; user manifests are now read per bound home, cached
until that home's plugin dirs change, and are Tier 1 for that
profile only.
- The declaration scans no longer fail open. A registry error is no
longer swallowed into an empty set, non-string required_env entries
are skipped rather than breaking the set, and a manifest that
cannot be read raises instead of vanishing from the policy. A
malformed manifest still declares nothing.
- A plugin manifest can no longer reclassify a core-declared name
such as OPENAI_API_KEY; the same rule the config form applies.
- The dashboard basic-auth password and signing secret, the OIDC
client secret and the drain bearer move to Tier 1. Credentialed
CLIs (claude, codex) no longer receive them.
- openviking-server starts from served_profile_child_env, so it gets
the bound profile's provider keys, never the launch profile's.
With no bound profile under multiplex the start is refused.
- NOUS_API_KEY and QWEN_API_KEY are listed statically. Discovering
provider plugins while the policy module imported re-mirrored them
over a plugin's own auth registry entry
(tests/providers/test_auth_registry_import_order.py).
Two more spawns inherited the gateway's full environment. The bot
desktop launcher copied os.environ minus the display variables; the
agent drives that desktop and its dock opens xfce4-terminal, so every
provider key and bot token was one click away. The Telegram gmail-triage
buttons ran HERMES_HOME/scripts/gmail-triage/*.sh with no env at all, a
path the agent can write. The desktop now builds from
served_profile_child_env() like the agent browser (keeping the user's
HOME); the triage scripts use build_subprocess_env like cron and
webhook-filter scripts.
The scrubbed builder rewrites HOME to the profile home in containers and
profile home mode (the official image does this by default). These are
third-party programs whose own config and logins live under the user's
HOME: ~/.openviking, the raft CLI profile, ~/.config/buzz. The openviking
change copied the parent's HOME back, which is itself a profile home
when Hermes runs under one; use HERMES_REAL_HOME, which the builder
already computes, for all three.
The shape rule (a platform prefix plus _TOKEN/_SECRET/_PASSWORD/_KEY) also
matched variables Hermes never reads: the platform list holds plain words
(LOCAL, GATEWAY, WEBHOOK, SLACK), so a user's SLACK_USER_TOKEN,
LOCAL_LLM_API_KEY or GATEWAY_API_KEY vanished from the terminal and
terminal.env_passthrough could not bring them back. The prefix census it
leaned on also failed open (an unreadable plugins dir cached an empty set).
Adapter secrets now come from what adapters declare: password entries of
the messaging OPTIONAL_ENV_VARS (built-ins plus every platform plugin
manifest) and secret-named keys of the gateway env-override table, both
Tier 1 and refused by passthrough; plus the secret-named required_env of
adapters registered in the current profile scope, read per spawn without
loading deferred adapters, Tier 2 only because required_env is an
unchecked setup list. Secrets nothing declared get a manifest entry
(TELEGRAM_WEBHOOK_SECRET, PHOTON_SIDECAR_TOKEN, A2A_PUSH_SECRET,
TEAMS_GRAPH_ACCESS_TOKEN, TEAMS_INCOMING_WEBHOOK_URL) or join the
policy's read-in-code list (QQ_STT_API_KEY, the two MSGRAPH names).
The autostarted server copied the whole gateway environment, so bot, gateway and relay tokens reached it along with the provider keys its embedding/VLM models use. Build it with hermes_subprocess_env(inherit_credentials=True), the compute host's allowance, keep the user's HOME for ov.conf and keep stripping PYTHONPATH (#78153). Drop its exemption from the raw spawn-env guard and tighten the remaining entries' reasons.
_exec_buzz built the CLI env from os.environ.copy(), so every buzz call
carried Hermes' gateway tokens and provider keys. It now starts from
hermes_subprocess_env() and adds only the relay URL, key and auth tag.
The raft bridge was spawned with {**os.environ, RAFT_CHANNEL_TOKEN},
handing the raft CLI every Hermes credential. It now gets
hermes_subprocess_env() plus RAFT_PROFILE (the profile-resolved value,
not the launch env's) and the bridge token.
Ported from the raft half of #56245 onto current main.
* refactor(plugins): remove the Sep 2026 decomposition compat layer on schedule
The PLUGIN-COMPAT layer (2776813df3 + d63e380324 + 0a5164cebe) kept pre-#102117 import paths
alive for external plugins until 2026-09-14. That window closed two weeks ago; since then the loader
has already been skipping plugins that use the old paths. This removes the layer itself:
- 328 appended `PLUGIN-COMPAT` blocks (lazy `__getattr__` pointer tables, re-exported third-party
names, restored dead definitions) and the three re-export stub modules
(gateway/startup_watchdog, hermes_cli/observability/relay_runtime, tools/environments/modal_utils)
- COMPAT_MANIFEST.md, compat_manifest.json, scripts/check_compat_pointers.py and its lint step
- the reporting surfaces: CLI banner notice, `hermes plugins compat`, the `hermes doctor` section,
the post-update notice, the Desktop one-time dialog, the loader's pre-import skip and the
`plugins.allow_deprecated_imports` escape hatch
An external plugin that still imports an old path now fails to load with its ImportError as the
reason in `hermes plugins list`, the same path as any broken plugin.
hermes_cli/plugin_compat.py stays as three inert stubs (compat_report, removal_in_effect,
summary_lines): an already-running pre-removal `hermes update` lazy-imports them after the checkout
swap (tests/compat/old_updater_surface.json).
In-tree fallout, both already dead: hermes_cli/setup.py::_check_espeak_ng (no callers; its
`shutil` came from a compat block) and gateway/config.py::SessionResetPolicy ("retained solely for
the scheduled plugin-compat window"). Two test_run_agent patches targeted the removed
`run_agent.handle_function_call` pointer; they now patch `model_tools.handle_function_call`, the
seam production reads, like every sibling test in that file.
* chore: retrigger CI (zero-job startup_failure phantom)
* test: drop resolution allowlist rows for the two deleted which() sites
hermes_cli/setup.py::_check_espeak_ng (dead) and tools/skillevaluator_scan.py::scanner_available
(a restored definition inside a PLUGIN-COMPAT block) no longer exist; the stale-row gate requires
their allowlist entries go with them.
Inside a multiplexed secondary profile scope, `_profile_buzz_extra` read the buzz block only from
`gateway.platforms.buzz`, while the runtime loader (`gateway/config_loader.py::platform_section`,
the same seam that feeds this plugin's `apply_yaml_config_fn`) also resolves a top-level `buzz:`
block and the documented `platforms.buzz` shape. A fully configured secondary profile therefore
failed `check_requirements` closed on every boot ("Platform 'Buzz' requirements not met").
Resolve the section through `platform_section` so the gate sees exactly what the loader sees.
Row L1 (profile-scoped gate reads config), topology T2 (multiplexed secondary profile).
Salvaged from #126060 by @liuhao1024 (authorship kept); the test is trimmed to one A->B->A
invariant across two profile homes. Supersedes #126067 (same mechanism, later).
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Follow-up trim to @JE4NVRG's salvaged commit (is_connected → env_is_connected("A2A_PORT")):
- plugins/platforms/a2a/tools.py::_a2a_tools_available read os.getenv("A2A_PORT") too — the
same unscoped gate in the same plugin, so every secondary profile also paid for the a2a
toolset whenever the launch profile set A2A_PORT. Now get_scoped_secret, same policy as
the adapter's port read (scoped miss fails closed; default/T1 reads its own environ).
- Tests trimmed to two invariants: A→B→A over two temp profile homes under
set_multiplex_active(True) with the launch value in os.environ (profile B with no
A2A_PORT of its own is NOT connected and gets no tools; red on origin/main at index 1),
and the standalone T1 case where os.environ IS the profile.
Row L1 (boot-time platform enablement probe), topologies T2/T3 (multiplexed host serving
secondary profiles). No other plugins/platforms/*/__init__.py is_connected/check_requirements
does an unscoped env read (buzz #125985 is handled separately).
Co-authored-by: JE4NVRG <jean.v1803@gmail.com>
is_connected() gated on a raw os.getenv("A2A_PORT"), which under multiplexing
holds the launch (default) profile's value. Every secondary profile therefore
instantiated the inbound A2A server, all of them fell back to the module
default port 9900, and all but one died with "could not bind ... Address
already in use" (fatal / bind_failed in gateway_state.json) on every boot.
Resolve the gate through gateway.platforms._shared.env_is_connected, the same
scope-aware idiom the sibling adapters use -- the adapter's own port read
already goes through _get_scoped_secret for exactly this reason.
Tests: tests/plugins/test_a2a_plugin.py pins the behavior. A profile with no
A2A_PORT in its own scope is not connected and never reads os.environ (a raw
os.getenv fails the test), a scoped port connects, and extra.enabled still
short-circuits.
Fixes#122126
The r3 fold scoped only the dmarc verdict to its clause. SPF and DKIM
still came from a whole-string, last-match-wins regex scan, so the same
quoted/comment smuggle closed for dmarc still authenticated a spoofed
From (GHSA-rxqh-5572-8m77), e.g.
spf=fail smtp.mailfrom="x spf=pass smtp.mailfrom=example.com "@evil.test
spf=fail (spf=pass) smtp.mailfrom=a@example.com
spf=fail smtp.mailfrom=a.spf=pass@example.com
dkim=pass header.d=evil.test header.i="x header.d=example.com y"@evil.test
Every verdict now comes from the leading method=result token of its
own clause (from _ar_clauses, comments dropped), and its domains only
from that clause. Properties are read by a token scanner that consumes
quoted-strings and other key=value tokens whole, so quoted contents are
never read as properties while a quoted value still is
(header.from="example.com"). SPF fails closed on more than one spf
clause; DKIM accepts any single dkim=pass clause whose own header.d
aligns (multi-signature mail is normal), never mixing clauses. The
whole-string methods/props (methods["dmarc"] was dead) are gone.
Also: a stray ')' at depth 0 is now unbalanced (it split header.from
out of the dmarc clause); a From with more than 64 '(' takes the silent
empty-sender drop instead of a parseaddr RecursionError logged as an
error; the cap test asserts the cap directly instead of wall-clock
timing; the empty-From drop assertion moves next to the other
_extract_email_address rejects; and the untested >1-dmarc, unbalanced
and backslash-escape rules get reject strings.
Round-3 review of the #124322 salvage:
- stdlib parseaddr is pure Python and superlinear on hostile input: a
100KB `From: <a@a@...` held the GIL ~1s per message (main ~0), and any
remote sender reaches it before auth. Refuse values over _MAX_FROM_LEN
(2048; RFC 5322 lines cap at 998) right after unfolding so they take the
existing empty-sender drop. The timing assertion now times
_extract_email_address itself instead of a private regex.
- The dmarc clause split ignored quoted-strings and stripped comments one
level only, so an attacker-controlled SPF-passing envelope sender like
smtp.mailfrom="x;dmarc=pass header.from=example.com x"@evil.test planted
a fake dmarc=pass ahead of the real dmarc=fail and authenticated
admin@example.com. Split clauses with a quote- and nested-comment-aware
scanner (comments dropped, quoted values kept so header.from="x" is still
read and unquoted), and fail closed on an unbalanced value or when more
than one clause starts with dmarc=. Tests pin the smuggled clause,
reason="a;b", the nested comment, a quoted aligned header.from, and that
every header.from in the dmarc clause must align (all->any goes red).
- `Doe, John (CEO) <j@x>` was dropped because the fallback refused any
paren. Strip (comments) from the display part first; `attacker@evil.test
(c) <victim>` stays rejected by the '@' rule.
- Results without '@' (`John` -> `john`, `a\"b <v@x>` -> `a\`) were
dispatched as sender ids; return "" so they take the drop path.
The r1 malformed-From fallback regex ([^<>\s]+@[^<>\s]+) backtracked
catastrophically on `From: <a@a@a@...`: 32KB held the GIL ~23s, and any
remote sender reaches it before auth. Capture <([^<>\s]+)> instead and
check the '@' in Python (same accepted set, 100KB in ~3ms).
The fallback also mapped multi-mailbox / group / comment-prefixed From
values (`attacker@evil.test, <victim@x>`, `Grp: a@evil; <victim@x>`,
`attacker@evil.test (c) <victim@x>`) to the bracketed victim, which a
dmarc=pass without header.from then authenticated. Only fall back when the
display part has no ; : ( ) and no '@' unless it is exactly the bracketed
address; `Doe, John <j@x>` and `j@x <j@x>` still resolve. The rule now lives
only in the docstring (the old inline comment was wrong).
dmarc: strip (comments) before splitting clauses, and take the verdict and
header.from from the same first clause that starts with dmarc=, so
`dmarc=pass (p=none; sp=none) header.from=evil.test`, a later dmarc=pass
clause after dmarc=fail, and duplicate misaligned header.from are rejected,
while `arc=pass (dmarc=fail ...); dmarc=pass header.from=<ours>` passes.
Property clean-up is shared via _auth_props.
The empty-sender drop now runs right after address parsing and gets an
assertion (it was untested). Tests extend existing ones (no new tests).
Strict parseaddr returns '' for common RFC-invalid From values that the
old regex resolved: an unquoted address as display name
(john@example.com <john@example.com>), an unquoted comma (Doe, John
<j@example.com>), or a@x.test <b@y.test>. Allowlisted senders using such
clients were silently dropped, and under open access every one of them
shared an empty chat_id. Fall back to the bracketed address only for the
unambiguous shape (no quotes, exactly one <...> pair), so the quoted
display-name spoof still resolves to the attacker. _parse_fetched_message
now drops messages whose From yields no address instead of dispatching
an empty identity.
The dmarc header.from alignment check read header.from from props merged
across all clauses, so a later dkim clause's header.from could override
the dmarc clause's value. Read it from the dmarc clause only, and pin the
misaligned dmarc=pass rejection in the existing dmarc test (previously
no test covered it). Also drop a redundant _domain_of ternary, trim the
_extract_email_address docstring and a duplicate test assertion.
_verify_sender_authentication accepted any dmarc=pass verdict, even one
issued for a different domain than the From we parsed. That is what let
the quoted-display-name spoof ride on the attacker's own truthful DMARC
pass. When the trusted Authentication-Results names header.from, require
it to align with the From domain; otherwise fall through to the aligned
SPF/DKIM checks. Defense in depth for the #124322 parser fix.
