plugins.isolation: host imports every non-bundled plugin in a host process started
on demand per profile with only that profile's environment, and hands plugins a
RemotePluginContext; Hermes registers proxies for tools, hooks, commands, skills and
provider objects (subclasses of the ABC it checks). Re-entrant: handlers may call
ctx back into Hermes while Hermes waits on them. Every user-code import path routes
to the host (general, memory/context-engine/cron, model-provider profiles via a
cached credential-free extraction, dashboard APIs via an ASGI bridge) or refuses.
One table names what cannot cross (live gateway objects); the static audit shown
by `hermes plugins validate/show` reads it. A host that dies is restarted with its
plugins (bounded). Default stays in_process: nothing changes unless opted in.
Why: shared-pool deployments must never run one tenant's plugin Python in an
interpreter holding another tenant's data, without plugins becoming tenant-aware.
The HEADER.FIELDS list hand-copied the _AUTOMATED_HEADERS keys, so a key
added there would silently miss the preflight and the sender's full body
would be fetched before being dropped. Build it from the registry (the
IMAP command is byte-identical today). The preflight also repeated the
RFC822 payload unwrap and had three identical warn/consume/continue
branches: one _imap_payload helper now serves both unwraps and the
rejections collapse to one branch (gate reuse + quality Low, convergent).
preauthorize=None kept an unsafe path on the shipped adapter: any caller
that left the gate out still RFC822-fetched unauthorized mail. The only
production caller (_check_inbox) always passes it; the None default served
only direct test calls (gate quality Medium + 2c Low, convergent). Make it
required, drop the guard, and have those tests pass an allow-all gate
(their IMAP mocks answer the BODY.PEEK preflight where needed).
The gate hand-rolled run_coroutine_threadsafe(...).result(). On a closed
loop that raised and leaked a never-awaited coroutine (gate reuse Low
finding). The shared helper closes the coroutine and returns None, which
the gate now treats as a rejection, so it still fails closed.
Every sibling signature in the adapter spells the type through the
existing email_lib import; the bare Message import served one annotation
(gate quality Low finding).
The adapter fetched the full RFC822 payload (and decoded attachments) for
every unseen UID before deciding whether the sender was allowed, and
_verify_sender_authentication searched every Authentication-Results field
for one whose authserv-id matched (or was a subdomain of) the pin. A
forged lower field or an attacker-chosen child.<authserv_id> therefore
authenticated a spoofed From:.
- Only the topmost Authentication-Results field is authoritative; a set
authserv_id must match it exactly (RFC 8601 CFWS comments allowed via
_ar_clauses). Main's _NO_AUTH_RESULTS_REASON/_UNTRUSTED_AUTHSERV_REASON
constants are kept so _DROP_HINTS still resolves.
- _check_inbox fetches a bounded BODY.PEEK[HEADER.FIELDS ...] preflight
and runs _sender_accepted on the event loop before requesting RFC822;
rejected UIDs are marked \Seen without downloading the body.
Salvaged from #129772 and rebuilt on main: the PR skipped the preflight
under open access via self._open_access(), which main removed in
c2f595d6af (open access now also authenticates From:), so the gate always
runs here. Poll-loop test mocks now authorize their unauthenticated test
senders instead of forcing open access.
Gate BR3 r1 reuse (Low): build_update_processor hand-rolled the int-or-
default parse that gateway.platforms._shared.coerce_port already does.
Neither caught OverflowError, so `max_concurrent_updates: .inf` in YAML
aborted connect(). coerce_port now treats non-finite values as unparseable
(as gateway.config._coerce_num already does), which also hardens its other
port/int callers; the telegram setting warns and falls back to 32.
Gate BR3 r1 quality (Low): PTB's SimpleUpdateProcessor already provides the
no-op initialize/shutdown the class copied, and config.extra is always a
dict, so the `(extra or {})` guard was dead. The wiring test only compared
PTB's concurrent_updates property with itself (true for any processor);
assert the PerChatUpdateProcessor type and DEFAULT_MAX_CONCURRENT_UPDATES
instead - now red when a plain SimpleUpdateProcessor(32) is wired in.
PerChatUpdateProcessor awaited the predecessor's tail future directly, so
cancelling a waiting same-chat update also cancelled that future (gate
BR3 r1 2c + quality, Medium). The running predecessor then raised
InvalidStateError from done.set_result() after its handler succeeded, and
the next waiter started while the predecessor still ran (probe order:
start A, start C). Shield the await and, for a cancelled waiter, release
its turn only once its predecessor finishes.
Plain PTB concurrency (concurrent_updates=32) unblocks other chats while one
chat's handler is slow, but also lets that chat's NEXT update dispatch and
deliver before the first one finishes (download/vision/callback work happens
inline, before text batching can merge them).
PerChatUpdateProcessor (update_admission.py, not the 7k-line adapter) chains
each update behind the previous update of the same chat while keeping the
bounded cross-chat pool. The builder keeps the instance, so the connect-retry
rebuild in _initialize_app_with_retries reuses it. An invalid
max_concurrent_updates now logs a warning instead of silently falling back
(as #74652 did), and the dispatch-stall watchdog comment no longer claims PTB
dispatches sequentially.
Co-authored-by: fortona-space <299084105+fortona-space@users.noreply.github.com>
Gate finding (quality, Low): _IngressDispatchStallError had no consumer -
every branch checks isinstance(error, _PollingStallError) - so the subclass
and its docstring were dead weight. One stall was also announced three times
(the "healthy but deaf" pre-log, the handoff warning, the fatal error line);
the sibling _check_polling_stall deliberately skips a pre-log. Drop the
subclass, drop the pre-log and carry the "healthy but deaf" counts in the
error text instead, and read the polling generation once.
Tests assert on the error handed to the stubbed handoff rather than caplog.
Gate finding (2c, Low): PTB dispatches updates sequentially (no
concurrent_updates) and counts a dispatch when the handler starts, so one
slow-but-bounded handler (sticker vision_analyze at 120s+, an awaited
slash-confirm callback) plus a queued update looked like "no dispatch
progress" after two 90s heartbeats and triggered a fatal rebuild that drops
already-acknowledged updates.
Raise _INGRESS_DISPATCH_STALL_HEARTBEATS to 4 (360s) so the window exceeds
_POLLING_ERROR_TASK_STUCK_TIMEOUT (300s) with margin. Deliberately not gated
on _inflight_update_ids: that would hide a handler that is itself wedged.
Tests drive heartbeats by count; counts updated, semantics unchanged, plus a
270s-no-escalation assertion.
The confirmed-stall fatal branch hardcoded "getUpdates made no progress",
which is wrong for _IngressDispatchStallError: there getUpdates IS
progressing and PTB's dispatcher is the wedged part. Operators reading
gateway_state.json / logs were pointed at the transport. Interpolate the
(redacted) error text instead; the existing polling-stall errors already
say "getUpdates made no progress ...", so their message is unchanged in
substance.
_check_ingress_dispatch_stall is only called from the async polling
heartbeat, where a running loop always exists; the try/except RuntimeError
and the debounce reset that made its deferred retry work existed only for a
sync unit-test call. Match the sibling _check_polling_stall (no guard) and
delete the no-loop test. Without the fallback the counter stays at the
threshold until dispatch progresses, so a stall is escalated once.
The ingress dispatch watchdog logged 'healthy but deaf' but never drove
recovery, leaving the gateway reporting connected while PTB's dispatcher
stopped draining its queue (#130407 §1).
Hand the stalled adapter to the supervisor for a rebuild via a new
_IngressDispatchStallError (_PollingStallError subclass, so the ladder
skips the in-place Updater restart that keeps the wedged dispatcher and
goes retryable-fatal). _schedule_polling_recovery also marks the adapter
degraded so gateway_state.json stops saying connected. Guarded by the
recovery-in-flight check like the polling stall watchdog; no-loop direct
calls keep the warning as the diagnostic.
Tests: new test_telegram_ingress_dispatch_recovery.py (8 tests: debounce,
scheduling, degraded+fatal handoff, in-flight/webhook/teardown guards,
re-arm, no-loop fallback); existing delivery-gap counting tests stub
recovery to isolate the log contract.
(cherry picked from commit 7767ed38e4f4508b67fc4e2bf3dc726095243732)
Quality-gate Lows (BS03 r1): resolved_count always equalled
len(self._username_resolved_ids) (each name is discarded from to_resolve on
its first match, so it adds exactly one ID), and resolved_allowlist_user_ids
filtered isdigit()/"*" out of a set that only ever holds str(member.id) and is
always initialised in __init__ and reset by the resolver. Drop the counter, the
filter, the getattr fallback (no test builds the adapter without the resolver
and then calls the accessor) and the stale "Numeric only" docstring line.
Behaviour unchanged (probe-BS03-1 output identical before and after).
_resolve_allowed_usernames reset _username_resolved_ids only after the
early return for an empty allowlist / missing client, so a reconnect
after the allowlist was emptied kept the previous connect's resolved IDs
in the set the gateway authz union reads. Reset first so the set always
reflects the current connect.
resolved_allowlist_user_ids() returned every numeric ID in the adapter's
connect-time _allowed_user_ids, and the gateway authz layer unions that set
into the reloaded DISCORD_ALLOWED_USERS. So an ID removed from the allowlist
after connect (hermes pairing revoke from the CLI or dashboard, or a hand
edit) kept authorizing until the gateway restarted. #130920's in-place purge
only reaches adapters in the same process, and those commands run outside it.
Track the IDs _resolve_allowed_usernames actually resolved from username
entries and return only those: they are what the per-turn .env reload loses
(a0a63a1bc2); numeric entries are already read live from the env.
(cherry picked from commit 90451505545745dbb27ecad28f2c431ad5afd301)
_resolve_allowed_usernames resolved a DISCORD_ALLOWED_USERS name through member.name,
member.display_name or member.global_name, and the first match took the slot. Any member of a
shared server could set a nickname or display name to an allowlisted username, get the
resolved ID (DMs, tools, approval buttons) and lock the real user out.
Match the unique account username only, as #125034 did for SimpleX display names. An entry
that matches only a display name stays unresolved and gets a connect-time hint. Tests cover
a copied nickname and a copied display name seen before the real user, and the hint; the
Discord docs now say display names and nicknames are never matched.
(cherry picked from commit 252f0bccab39afe94f0f71254e47a8f785ec1fb0)
The channel gate ran inside _handle_slash_command, after the bolt listener had
already acked with an ephemeral "Running /x". In an ignored or non-allowed
channel the invoker was promised a run that never came, and an ignored channel
("never touched") still got a bot reply. Move both checks into
_slash_channel_gated() and consult it before the ack: gated -> bare ack().
_handle_slash_command keeps calling the same predicate for direct callers.
The listener becomes the _handle_hermes_command method so the existing
parametrized gate test drives the real ack path (BN11 gate r1 2c Low finding).
The allowed_channels guard recomputed str(channel_id).startswith("D") inline and
the next line computed it again as is_dm. Hoist is_dm above the gates and reuse it.
Same behaviour, one DM classification (BN11 gate r1 quality/reuse Low finding).
The message path drops a message from an ignored channel (_prefilter_inbound)
and, when allowed_channels is set, from any non-1:1 conversation outside it
(_channel_gate_allows). _handle_slash_command applied neither, so a slash
command ran in a channel the operator had excluded. Apply both gates by
channel id alone, before any lookup, so a failed conversations.info can never
drop a control command.
Rebuilt from 2b7cde506c (#128767). Fixes#130358.
The standalone payload builder imported markdown inside the per-chunk loop
under suppress(ImportError), which read as though the import could fail per
chunk. Try it once before the loop and branch on the result.
_standalone_send seeded `result = None` and returned it after the loop,
suggesting a possible None return. truncate_message always yields at least one
chunk, so the loop always runs; return the failing result directly and the
last result otherwise. No behaviour change.
An edit carries m.mentions twice: once inside m.new_content and once at the
top level. When a budget-sized edit names many distinct Matrix IDs, that
fixed part alone could pass the content budget. Shortening the fallback body
then had no room to work with, and the edit still went out at about 54 KB,
which an encrypted room would reject with M_TOO_LARGE.
When the edit is still over budget with an empty fallback, it now drops the
top-level mentions copy. That copy only drives the edit's own notifications,
and m.new_content keeps the full list.
The event-size tests move to tests/plugins/platforms/matrix/test_event_size.py,
mirroring the plugin source tree, and gain a mention-heavy text. The
tests/gateway/test_matrix_message_length.py file is back to main apart from the
env-override value that the new ceiling requires.
(cherry picked from commit 4b06bcb7dbffb732bef6e877cb66dea2fbff8c73)
The homeserver rejects any event whose canonical JSON is over 65,536 bytes
(M_TOO_LARGE), but Matrix chunked outbound text on characters, 16,000 by
default with a 65,535 ceiling. Each message carries its text twice (body plus
formatted_body HTML), and an edit carries it four times (the "* " fallback
plus m.new_content).
On the default config this caused three failures:
- A formatted chunk of CJK text came to about 95 KB and was rejected, so
_send_with_retry fell back to a plain 3,500-char truncation.
- A streamed edit near the limit came to about 80 KB even for English, so
edits failed mid-stream.
- The standalone sender (cron without a running gateway) posted each
16,000-char chunk as one event, and hit the same failure on non-Latin text.
The fix measures Matrix text in UTF-8 bytes, the unit of the cap:
- The adapter's message_len_fn now returns UTF-8 bytes, so send() and the
stream consumer's split and progress-edit budgets count the same unit.
- A chunk gets a third of a 45,000-byte content budget, which leaves room for
the event envelope and E2EE's 4/3 base64 inflation. That makes 15,000 the
default and the ceiling for max_message_length, so English chunks stay
about the same size.
- _build_text_message_content drops the HTML when escape-heavy text would
push the content past the budget.
- An edit that would still be too large keeps its full m.new_content and
shortens only the top-level fallback, which edit-aware clients never
render.
- The standalone sender re-chunks on UTF-8 bytes and applies the same HTML
guard.
(cherry picked from commit cab00c94175955a68ecd4d38c5748d63148ca6be)
r2 review: moving explicit failing verdicts to debug hid a listed contact whose
own domain breaks DKIM or whose forwarder breaks alignment. Those drops warn
again, with the verdict and no opt-out hint; forged stranger mail under open
access stays at debug. The two operator hints now live in one reason->hint
table behind a single warning call.
The previous commit moved every non-missing-header auth failure to debug,
which made a mis-pinned authserv_id (the only stamp comes from another
authserv-id) drop allowlisted mail silently at default log levels. That is
an operator misconfiguration, not routine forgery, so it warns again, with
a hint pointing at platforms.email.authserv_id instead of the opt-out.
Under open access every stranger is granted, so each forged or DMARC-failing
message logged a WARNING telling the operator to disable sender
authentication. Keep that warning (and hint) for the one verdict that points
at a server that never stamps the header; log every other unauthenticated
drop at debug, including an explicit failing verdict for an allowlisted
address. The reason string is lifted to _NO_AUTH_RESULTS_REASON so the
parser and the check share it.
The anchored fence regex from the upstream commits (kept here via
cherry-pick) fixed#85381 but required the opening ``` to sit at a
line start with <=3 spaces of indent, silently downgrading two shapes
that main renders as <pre> into escaped literal prose:
- mid-line-opened fences ("Here is the code: ```python")
- fences indented by 4+ spaces (list/blockquote-nested code)
Widen the anchors while keeping the inline-triple fix:
- the opening line may now carry any leading whitespace or lead-in
prose, as long as the opening ``` is the first triple-backtick run
on the line and the rest of the line is backtick-free (so an inline
pair like ```x``` can never end a line and open a fence);
- the closing fence may sit at any indent, not just 0-3 spaces.
The tempered prefix cannot skip past an earlier triple-backtick run,
so an inline pair followed later by a real fence no longer swallows
it (both main and the anchored regex mangled that mix in different
ways). The scar conformance vector reverts to main's committed value:
the mid-line-opened fence it encodes is protected again.
The anchored fenced-code regex ended the closing fence with [ \t]*$, so a
close line terminated by CRLF (\r\n) — Windows-authored content — failed to
match: the \r sits between the fence and the $ position. That left the whole
block unprotected, escaping its backticks into literal ` runs and dropping
the code formatting. Allow an optional trailing \r before the line end.
Adds a CRLF-fence regression (fails on the old anchor).
The MarkdownV2 fenced-code protection regex
r'(```(?:[^\n]*\n)?[\s\S]*?```)' made the newline after the opening
fence optional, so it also matched *inline* triple-backtick spans on a
single line (e.g. "the syntax is ```x``` inline"). _protect_fenced
then treated the inline span as a pre block, escaping backticks into an
unbalanced entity; Telegram rejected the whole message with 'can't find
end of pre entity' and Hermes fell back to plain text, dropping all rich
formatting.
Anchor the pattern to line starts (0-3 spaces indent) so only standalone
fences match, and rewrite _protect_fenced to use explicit capture groups
(opening line / body / closing fence) instead of naive [:-3] slicing,
which also fixes a doubled closing fence when the close line had trailing
whitespace.
Fixes#85381
#131868 passes timeout=60_000 to the Firecrawl client's scrape() so the
server-side deadline matches our 60 s wait. The SDK accepts it, but the
keyless REST client's scrape() took only url/formats, so every extract
raised TypeError for keyless users with the keyless ring disabled or the
tier pinned to paid. Accept the kwarg and forward it in the v2 payload,
which takes the same field.
A 429 between the recreated path's delete and upsert strands the command
deleted until a fully-clean later sync; on reconnect-heavy deployments
that never happens, so commands like /model silently vanish from the
slash picker. Discord's create endpoint overwrites same-name commands,
so the lone upsert recreates while keeping the command available.
The Firecrawl scrape() call was missing the `timeout` parameter, so the
server-side used its 30s default. The client-side asyncio.wait_for
expects 60s, causing SCRAPE_TIMEOUT on slow pages before the Python
deadline expires.
Fix: pass timeout=60000 (ms) to scrape() so both deadlines align.
Fixes#43272
plugin-catalog/supermemory.yaml already points at supermemoryai/hermes-supermemory,
the same provider maintained by Supermemory. The catalog entry stays dormant while a
bundled copy exists (bundled providers win in find_provider_dir), so deleting the
in-tree copy is what lets hermes_cli/memory_provider_migration.py install it for
every home whose memory.provider is "supermemory".
retaindb, byterover, holographic, mem0, openviking and honcho stay bundled.
Prose that lists the bundled set (CONTRIBUTING en/es, plugins/AGENTS.md, the
memory.provider default comment, `hermes memory --help`) now names supermemory as
a catalog provider. The SUPERMEMORY_* env hints and redaction stay: the catalog
copy still reads them.
Submitters only saw a stale 5-point summary on the catalog page (it even listed
a 'Released' requirement nothing enforces), while reviewers applied ten README
rules plus criteria written nowhere public (credentials, approvals, disclosure,
compatibility metadata, forks of bundled plugins).
- plugin-catalog/README.md: rules 11-15 write down those review criteria; the
rule list sits between admission-rules markers.
- New developer-guide page 'Submitting to the plugin catalog': before you
submit, the PR flow, the full rule block (verbatim mirror), pin updates,
delisting vs removal, asking for a hook.
- The /plugins 'Submit' links and the user-guide section point at it.
- tests/website/test_catalog_rules_mirror.py fails when the two rule blocks
differ; classify_changes makes a README-only edit run it.
- AGENTS.md, plugins/AGENTS.md and CONTRIBUTING.md reference both files.
SMS and both WeCom adapters now split long replies, and their chunk loops
returned a plain failure when a later chunk was rejected. The base send
path cannot tell that from a total failure, so _send_with_retry retried the
whole payload or sent the 3500-char plain-text fallback and reported
success: the recipient got the head twice and never got the tail.
The three loops now share gateway.platforms.helpers.send_chunks(), which
marks a failure after delivered chunks with the existing partial_overflow
contract (the one Telegram and Discord set and _is_partial_delivery reads).
The retry and plain-text fallback already refuse to re-send a partial
delivery, so the caller gets an honest failure and nothing is duplicated.
Resuming the undelivered tail is #124295's scope.
Both WeCom adapters dropped everything past the platform limit with no
error: the AI Bot adapter sliced markdown content at 4000 chars, and the
callback (self-built app) adapter sliced text at 2048 *characters* while
message/send keeps only the first 2048 *bytes* (so CJK replies lost
everything past about 680 characters). Cron output was additionally cut
to 4000 chars with a gateway-host file pointer.
AI Bot: send() queues one markdown message per MAX_MESSAGE_LENGTH chunk,
so every chunk draws its own token from the 30 msgs/min per-chat bucket
(errcode 846607), and each goes out on the cached reply req_id like any
other send in that chat (group chats cannot send proactively). Callback: chunk by UTF-8 bytes at 2048.
Both declare splits_long_messages so cron delivery hands over the full
payload. Each stops at the first failed chunk and returns its error.
Fixes#19689Fixes#107443
Co-authored-by: luyao618 <364939526@qq.com>
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: Andy <sz.private@gmail.com>
The delivery router truncated cron output above 4000 chars and appended
a footer pointing at a file on the gateway host, which an email
recipient cannot open. One email carries the whole body, so declare
splits_long_messages and let the router hand over the full payload.
Fixes#61990
Co-authored-by: Petra <petra.kersten@agentmail.to>
Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
send() called truncate_message() without a limit, so it chunked at the
4096 default; Twilio rejects any Body over 1600 characters (code 21617),
so every reply longer than 1600 chars failed outright. Oversized cron
output was first cut to 4000 chars by the delivery router (SMS did not
declare splits_long_messages) and then failed the same way.
Chunk at MAX_MESSAGE_LENGTH (1600) and declare splits_long_messages so
the router hands send() the full payload.
Fixes#62751
Co-authored-by: Philip D'Souza <9472774+PhilipAD@users.noreply.github.com>
Co-authored-by: JackJin <1037461232@qq.com>
Co-authored-by: euisuh <euisuh.jeong@gmail.com>
Co-authored-by: Froraut <79215537+Froraut@users.noreply.github.com>
The hardcoded "group" duplicated the registered callback's own
chat_type default and implied a DM/group decision that was never made.
Passing chat_id only yields identical verdicts.
