The stack inserted the errcode table directly after _strip_reply_fallback's
return with no blank lines, which reads as if the constant belongs to the
function body and trips E305. Two blank lines restore the module-level
boundary; no behaviour change.
The inherited fixture coordinate "40.4302" does not contain the substring
"403" (the dot splits it), so the old substring classifier also passed on
it and the case proved nothing. Use a coordinate that genuinely embeds the
digits so the test is red on the pre-fix classifier.
The pinned mautrix 0.21.1 raises MatrixRequestError (carrying errcode and
http_status) from HTTPAPI._send on every non-2xx and sync() returns only
the parsed JSON dict, so the result-object auth branch in _sync_loop was
unreachable; it dated from the nio client whose SyncError objects were
real. Drop it together with the nio-mock test that pinned it.
With structured attributes guaranteed, the leading-status regex and the
bounded keyword scan over the message text were the only remaining ways
for body digits or HTML words to leak into the verdict, so drop them too:
no errcode/http_status auth signal means retry. Trim the contributor's
17 tests to the two loop-level invariants: both production repros (502
HTML body embedding "403" via an SVG coordinate; timeout echoing a since
token embedding "401") keep looping, and a 401/M_UNKNOWN_TOKEN stops.
The comment above the result-object branch in _sync_loop claimed mautrix's
Client.sync() returns an object carrying a message string for auth failures.
That is wrong. In the pinned mautrix 0.21.0, HTTPAPI._send raises
make_request_error() for any non-2xx and otherwise returns parsed JSON, so a
real M_FORBIDDEN arrives as an exception and is handled by the except branch.
The claim was introduced by this PR, which rewrote an accurate comment about
the earlier matrix-nio client (whose SyncError result objects were genuine).
The branch itself is kept as defense in depth against a future client swap,
but it now classifies with the same errcode/http_status logic as the
exception path instead of a lone "unknown_token" substring test, which
silently missed M_MISSING_TOKEN and M_FORBIDDEN and resynced forever
against a credential that can never succeed.
A structured errcode/http_status is authoritative; the message text is only
consulted when the object exposes neither, since str(object) is an opaque
repr. The text scan deliberately cannot override a structured verdict, so a
transient 502 whose HTML body contains "Forbidden" is still retried.
Adds four tests. Three are discriminating RED/GREEN cases that fail against
the old substring branch (M_MISSING_TOKEN errcode, http_status=401 with no
keyword in the message, and an unstructured object whose only signal is
.message). The fourth pins the precedence rule and passes either way.
Verified: 136 passed / 1 failed in tests/gateway/test_matrix.py; the single
failure (test_password_login_uses_device_id) fails identically at the
pristine PR head and is unrelated.
(cherry picked from commit bc9e6a8dafcf349a4e6b20a261fb2449603c0239)
The independent-verifier caught that my first loop-level test did not
actually prove anything. The 502/SVG coordinate fixture I reused from
gmoranxyz's unit-level test does not contain the substring 403 once
case-folded, so the old naive substring classifier already treated it
as transient. A test that passes under both the buggy code and the
fix proves nothing about the fix.
I replaced the fixture with a plain connection timeout whose message
wraps the real Matrix sync pagination token, an arbitrary digit
string that happens to contain 401. I verified this directly: with
the pre-fix classifier restored, the retry test now fails (the old
code stops the loop on this fixture), and with the fix in place it
passes (the loop retries as it should). That is the RED/GREEN proof
the maintainer originally asked for.
I also documented in the stop test's docstring that it does not
discriminate old from new, since the word forbidden in its message
trips the old naive check too. It is still worth keeping as a
regression test proving genuine auth errors stop the loop, just not
as proof of this specific fix.
While I was in there I also fixed a stale comment above the
M_UNKNOWN_TOKEN sync-object pre-check. It said nio returns SyncError
objects, but the dependency here is mautrix, not matrix-nio, and
importing nio raises ModuleNotFoundError in this codebase. The
pre-check logic itself was already correct and untouched.
Co-authored-by: gmoranxyz <gmoranxyz@users.noreply.github.com>
(cherry picked from commit ad3aad579a675a5aae544a50f88a82717c0ac3b6)
I added two more classifier unit tests for the attribute narrowing:
a bare .code attribute that happens to be 401, and a bare .status
attribute that happens to be 403, both must stay classified as
transient since only .http_status is trustworthy. I also added a
parametrized test for the five transient exception types the sync
loop now short-circuits on.
On top of that I added two tests that exercise _sync_loop directly
instead of just the classifier function in isolation. One replays the
real 502 Umbrel repro string through a mocked client.sync and confirms
the loop retries with the 5s backoff. The other raises a genuine
M_FORBIDDEN error and confirms the loop stops on the first call with
no retry sleep. These catch a regression in how the loop wires the
classifier in, not just a regression in the classifier itself.
(cherry picked from commit f747bb4b5a6e6da1bb9136168f08d6e7af5ea64b)
I hit a bug where the Matrix sync loop treated a passing 502 from
Umbrel's app proxy as a permanent auth failure and stopped syncing for
good. The old check did a naive "403" in str(exc) substring match, and
the 502 HTML error body embedded an SVG path with the coordinate
40.4302, which contains the digit sequence 403.
I replaced the substring check with a layered classifier. Transport
exceptions like TimeoutError, ConnectionError, and OSError are always
treated as transient regardless of their message text. Structured
signals take priority next: the errcode attribute against a known set
of permanent Matrix error codes, then the http_status attribute
against 401/403 specifically (not status, status_code, or code, which
belong to unrelated exception shapes and risk coincidental integer
matches). Only when none of those are present does it fall back to a
bounded, word-boundary-safe text scan on the first 200 characters.
Added tests covering the attribute narrowing, the transient exception
types, and two loop-level tests exercising _sync_loop directly to
confirm it retries on a transient error and stops on a genuine 401/403.
(cherry picked from commit 96d3363e45a63e08d9f07518ed949df334a33b3c)
The zero-knob test only asserted that ack_stale still trips with the knob
at 0. Because _read_websocket_health evaluates ack age before the
event-silence dimension, the test stayed green even with the
`_event_max_silence_seconds > 0` guard deleted: it never observed the
stale stamp being ignored. Assert (True, "healthy") with knob=0, a stale
stamp and a green transport first, then make the ACK stale and keep the
ack_stale assertion. Dropping the guard now fails this test.
Also correct the on_socket_event_type comment: discord.py dispatches
socket_event_type before the op-code switch but only for a non-null `t`;
heartbeat ACK frames carry `t: null`, they do not "return before" it.
The #109521 explanation (socket_event_type fires for every parsed
DISPATCH frame and is not debug-gated unlike on_socket_raw_receive;
op-11 ACKs carry no event type) was written out four times: knob init,
stamp init, the on_socket_event_type handler, and _read_websocket_health.
Keep the authoritative paragraph on the handler that actually stamps, and
point the other three sites at it so a future edit has one place to go.
Drop the math.isfinite(event_silence) half of the silence guard. Both
operands are our own perf_counter floats, so their difference cannot be
non-finite; the ack_age guard is different because _last_ack comes from
discord.py. math stays imported for the remaining finiteness checks.
`_warn_liveness_config_disabled` told operators an unusable value turns
off "the websocket liveness probe". That is true for the interval,
threshold, ack-age and latency knobs, which sit in the probe's startup
guard, but `websocket_event_max_silence_seconds` is only checked inside
`_read_websocket_health`, so ack-age/latency keep guarding. Say so, or
an operator reading the log would believe the whole watchdog is down.
Incident 2 of #109521: a Gateway socket can stay ESTABLISHED and keep
ACKing heartbeats while zero DISPATCH events are parsed, so every
transport-side liveness sample (ready/open/ack-age/latency) reads
healthy for hours. The merged #109963 deliberately dropped the
event_silence dimension: a raw-frame stamp is debug-gated
(on_socket_raw_receive needs enable_debug_events) and, since heartbeat
ACKs are frames, ack_stale always fires first by construction.
This adds the dispatch-side signal that was requested instead:
- stamp on on_socket_event_type, which discord.py 2.7.1 dispatches for
every parsed DISPATCH frame with no debug gate (verified live against
the real received_message path: 4/4 frames fired with
enable_debug_events=False, on_socket_raw_receive 0/4)
- new knob websocket_event_max_silence_seconds (default 4h, the
incident report's field-proven operator bound); 0 opts out of this
dimension ONLY — the #109782 review failure put the knob in
_start_liveness_probe's all-or-nothing guard, killing the whole
watchdog; it is gated strictly inside _read_websocket_health here
- the stamp resets per connection (connect() clears it), and a None
stamp (no event parsed yet on this connection) is not silence
- docs (en + zh-Hans) cover the new knob and the per-dimension opt-out
Fixes#109521
(cherry picked from commit b4baa97fc45794209711a45e052111d7d44d5f90)
The rebase moved the inbound attachment loop into SlackAdapter._append_link_unfurls,
so the nested-table hunk now lives there and is asserted directly. Drop the source-
provenance references and duplicate cell-level cases; one ragged/malformed-row test
covers raw_text, rich_text, None and unknown cell types.
Port from qwibitai/nanoclaw#3666: Slack represents a pasted table as
'table' blocks — usually nested in attachments[].blocks[], sometimes
top-level. They appear in neither the message text nor the file list,
so the agent received the sentence before the table and nothing else.
- _render_slack_table_block(): projects rows as 'cell | cell' lines,
collecting text leaves from raw_text/rich_text cell subtrees; capped
at 20k chars with a visible '[table truncated]' marker.
- Wired into all three ingestion paths: _extract_text_from_slack_blocks
(thread history + attachment-nested blocks), the live inbound
attachment loop, and _extract_additional_text_from_slack_blocks
(top-level blocks on live messages).
- _serialize_slack_blocks_for_agent skips 'table' blocks — the
allowlist drops 'rows', so it only emitted an empty husk.
send_voice built its own discord.File from the audio bytes and never ran
the size preflight, so an oversized audio attachment still burned the
doomed 413 round-trip that #50846 is about. Route it through the same
_reject_oversized_upload helper as _send_file_attachment.
The limit constant and _discord_upload_limit_bytes lived on the adapter
facade while every consumer is in adapter_media.py; the facade+sibling
layout puts topic code in the sibling, so they move there.
Discord raised the default file upload limit from 10 MiB to 20 MiB for
users, bots, webhooks and interaction responses (developer changelog,
Sep 3 2026). The 25 MiB constant here predates the preflight salvage and
never matched the platform; more importantly discord.py 2.7.1 still
reports 10 MiB via guild.filesize_limit for unboosted guilds, so the
guild-aware path under-reported the cap and rejected 10-20 MiB files
Discord now accepts. Floor the guild value at the platform default so a
stale library constant can only widen, never shrink, the preflight.
The adapter facade is ~6.6k lines; new behaviour belongs in a topical sibling per the
facade+siblings layout. expand_link_entities() now lives in telegram_entities.py and
reuses the encode/decode UTF-16 slicing the adapter already uses for entity spans.
Also: skip inlining when the anchor text already is the URL (no 'url (url)' duplication),
trim the test file to the invariants and point it at the sibling.
Image.open() was never closed; convert()/resize() return new images so the
source file object lingered until GC (a real leak on Windows where the open
handle blocks later deletion of the original). Use the context manager.
- Keep main's media_write_timeout=60s (PR's HERMES_* env var dropped per
.env-is-secrets-only policy; main already fixed the timeout half).
- Replace stdlib imghdr (removed in Python 3.13) with a magic-byte sniff.
- Exclude GIFs: JPEG conversion flattens animations to one frame.
- Fix transparent-PNG handling: RGBA hit the len(getbands())==4 branch
before the white-background composite, rendering transparency black.
- Clean up temp JPEGs after send (both single and media-group paths);
the docstring promised caller cleanup that neither call site did.
- Write temp files via tempfile default dir instead of an undefined
DEFAULT_OUTPUT_DIR (NameError at runtime in the original PR).
- Add real-Pillow regression tests incl. a sabotage-verified
white-background test.
Behind an HTTP proxy (e.g. tgapi.indevs.in) the PTB
media_write_timeout (~20s) is exceeded by raw PNGs > 1-2MB, causing
TimedOut errors on both send_photo and the send_document fallback.
Add TelegramAdapter._compress_image_to_jpeg() which converts large
PNG/raster images (>1MB) to progressive JPEG at 85% quality, with
optional resize above 1600px. Applied in send_image_file() and in the
media-group path of send_multiple_images(). Compression is a no-op for
JPEGs, small files, and non-raster formats, and falls back gracefully
if Pillow is unavailable.
Co-authored-by: user
Swallowing FeatureUnavailable returned a bare False, so a hosted operator
saw the same generic "requirements not met / run hermes setup" line the
original report started with. The registry's _probe already logs a raised
exception with its message, so propagating the error is what puts
"target not writable" / "quarantine 404" in gateway.log.
Hosted/Docker images lock /opt/hermes/.venv, so --install-deps writing
site-packages fails with Permission denied and the adapter never starts.
Route Google Chat through lazy_deps (HERMES_LAZY_INSTALL_TARGET) and bake
the extra into the published image so a configured gateway can connect.
Rebase reconciliation with main's JSON-allowlist decoding (#109423): the two
remaining readers that consulted config.extra before the env var now follow the
per-profile precedence rule (explicit scoped env → the profile's YAML → default),
and ignored_threads still decodes a JSON-string list after the read.
The Matrix blank-YAML test asserted YAML-over-env, the old precedence #108440's
review flagged; it now pins the contract: explicit env beats YAML, a blank env
value is unset (YAML applies), YAML beats the default, and an explicit empty
list is a real "no rooms" value.
_bridge_env copied os.environ (the default profile's WHATSAPP_* values under
multiplex) and only overlaid scoped hits, so a secondary with YAML
`dm_policy: pairing` launched its Node bridge under the default profile's
`allowlist` policy and the bridge rejected valid pairing DMs before Python saw
them. The child env now carries the values the adapter resolved (scoped env →
own YAML → default); a scoped miss removes the key rather than inheriting it.
Refs #108440 (ehz0ah inline, plugins/platforms/whatsapp/adapter.py)
545e74d0ea correctly stopped writing telegram.proxy_url into TELEGRAM_PROXY
for a multiplexed secondary, but _build_ptb_requests still resolved the proxy
only from that env var, so the secondary silently connected direct (or via the
default's proxy). #100448 had deliberately left this bridge unscoped for that
reason; this finishes the consumer migration instead.
_apply_yaml_config seeds proxy_url into extra and resolve_proxy_url gains a
`configured` rung: scoped TELEGRAM_PROXY → the profile's YAML → HTTPS_PROXY/
HTTP_PROXY/ALL_PROXY (trust_env) → macOS system proxy, with NO_PROXY semantics
unchanged.
Refs #108440 (finding 6)
One reader (gateway.platforms._shared.extra_or_secret) now implements the
precedence every per-profile setting follows for the OWNING profile:
explicit scoped env/.env → that profile's config.yaml (PlatformConfig.extra)
→ the adapter's default. A scoped miss returns the default, never the launch
process's os.environ; single-profile / default-profile installs keep the
documented env-over-YAML contract.
Why: 545e74d0ea (#108705) stopped bridging a secondary's YAML into the
process env and moved readers to config.extra, but the shared reader and the
hand-rolled helpers in Discord/Slack/Matrix/Telegram consulted YAML FIRST and
then fell back to a scoped env read. Two bug classes followed (#108440
post-merge review by andrexibiza, #109032):
- an explicit env value could no longer beat YAML for the owning profile
(DISCORD_ALLOW_MENTION_EVERYONE=false lost to allow_mentions.everyone: true;
TELEGRAM_REACTIONS=true lost to the stock reactions: false);
- a secondary that OMITTED a key inherited the launch profile's bridged env
through the fallback (Matrix process_notices/session_scope, Discord
auto_thread/reactions/mentions, Slack reactions/ignored_channels).
Consumers migrated to the shared reader: Discord _build_allowed_mentions and
_extra_or_env_flag; Slack _slack_allow_bots, _reactions_enabled (the
_extra_or_env_* getters already used it); Matrix _extra_truthy, _extra_csv_set,
session_scope, reactions, require_mention parsers, and — new — the
allowed_users / ignore_user_patterns consumers that never read the seeded YAML
lists; Telegram _extra_bool, _extra_str_set, _reactions_enabled; Feishu
allow_bots; WhatsApp dm_policy/group_policy.
Refs #108440, #109032
A2A_PORT and A2A_ADVERTISED_TOOLSETS are already captured at
construction time (inside _profile_runtime_scope) via
_get_scoped_secret(), but A2A_PUBLIC_URL was still read with a bare
os.getenv() inside A2ARequestHandler._request_public_url() - which
runs on ThreadingHTTPServer's per-connection OS thread, not the
constructing thread.
Raw threading.Thread never inherits contextvars, so even swapping the
reader to _get_scoped_secret() at that call site would not help: the
request thread has no scope, secret_scope falls back to os.environ
either way. The value must be captured once at construction time
(which does run in profile scope) and threaded through as instance
state instead - same fix shape as A2A_PORT above.
A secondary multiplex profile without its own A2A_PUBLIC_URL now
falls back to the X-Forwarded-Host/Host-derived URL (or the bind
host) instead of silently advertising the default profile's public
URL in its Agent Card / discovery response.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 0c36aca5de53d88bbbc0b4cfceaed8307c744f7a)
#69090 scoped MATRIX_RECOVERY_KEY itself (via _scoped_recovery_key())
so a secondary profile resolves its own recovery key under multiplex,
but left its sibling, MATRIX_RECOVERY_KEY_OUTPUT_FILE, on a bare
os.getenv(). _recovery_key_output_path() is called from inside
_verify_or_bootstrap_cross_signing(), which runs fully inside
_profile_runtime_scope for a secondary profile: when that profile
bootstraps a new recovery key, it either doesn't get written to a
file at all, or gets written to the default profile's configured
path, depending on which one has the env var set.
Route it through the same _get_scoped_secret() helper _scoped_recovery_key()
already uses.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit fb765ee49b2f1a1853e52fb901d25f767180a2fc)
545e74d0ea made _reactions_enabled consult extra.reactions before the env
var, and _apply_yaml_config seeds extra["reactions"] whenever the YAML key
is present — including the stock reactions: false every install
materializes. The documented TELEGRAM_REACTIONS=true switch therefore
became a silent no-op after the 0.21.2 update (#109032), contradicting
yaml_env_setter's "explicit env wins over YAML" contract.
Read the scoped env first and fall back to the profile's own YAML: under
multiplex a scoped miss returns the default instead of another profile's
process-env value (#72348), so only a scoped/env hit counts as explicit
and per-profile isolation is unchanged.
Fixes#109032
(cherry picked from commit 2bd5a0a5c0a5f9630fd82f133def65f225f653a3)
`_orphan_timeout()` was `max(300, A2A_REPLY_TIMEOUT)` with no ceiling, so
an absurd value (1e18) meant the watchdog sweep could never fail an
orphan — the reply window is a floor for the grace, not a licence to
disable the sweep. Cap it at 86400s.
`disconnect()` failed and cleared `_pending`/`_pending_order` but left
`_active_tasks` populated, so a reconnected adapter would keep excluding
dead task ids from the orphan sweep forever. Clear it in the same locked
block.
The troubleshooting entry told users to raise A2A_REPLY_TIMEOUT for long tasks,
which did nothing against the hardcoded 300s orphan sweep (#106972). Now that the
sweep derives its grace from the reply window and skips tasks with a live waiter,
state that contract next to the variable.
545e74d0 (post-0.21.2) made the Matrix YAML bridge seed its values into
PlatformConfig.extra so secondary multiplex profiles read their own config. The
"csv" bridge kind seeds any non-None value, so `free_response_rooms: ''` now
reaches extra as '' — and the readers' `if raw is None` fallback no longer fires,
so MATRIX_FREE_RESPONSE_ROOMS is ignored and require_mention drops every
un-mentioned message. Before that commit the bridge only wrote env and the key
was absent from extra, so the env value applied.
Route the three identity-check readers (_extra_csv_set, _extra_truthy,
_resolve_max_message_length — the last a three-tier chain where '' also
short-circuited the plugin-registry default) through the shared
gateway.platforms._shared.extra_or_secret, whose default already treats a blank
string as unset (the idiom mattermost/dingtalk/slack readers use). Explicit
scalars, bools and lists (including []) stay authoritative.
Two invariant tests replace the salvaged suite (moved to
tests/plugins/platforms/matrix/ to mirror the source path): blank falls through
for all three readers; explicit values still beat env.
Fixes#109358
Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
The adapter fix decoded `'["-100","-200"]'` before comma-splitting, but
the runner's central gate in gateway/authz_mixin.py::_coerce_allow_set
reads the same YAML-bridged env chain (TELEGRAM_GROUP_ALLOWED_CHATS,
TELEGRAM_ALLOWED_USERS via _auth_env) and still produced
{'["1"', '"2"]'}, so a group message admitted by the adapter could
still be rejected upstream.
Move the decoder to gateway/platforms/_shared.py, which both the adapter
and authz_mixin already import from (no plugin -> gateway cycle), and
route _coerce_allow_set through it. One invariant test on the runner
side, red before this change.
`_finite_positive_config_float` and `_config_int` were the same shape with the warn call
pasted six times and asymmetric sign checks. Collapse both onto `_liveness_knob(key, default,
cast)`: usable iff finite, >= 0 and exact for the cast; else warn and return 0; explicit 0
stays silent.
Two int-path holes closed on the way: `websocket_liveness_failure_threshold: .inf` raised
OverflowError inside `DiscordAdapter.__init__`, and `0.5` truncated to 0 and disabled the
probe silently — the bug class this PR exists to remove. `2.0` / `"2"` still resolve to 2.
The cherry-picked commit added an `event_silence` probe dimension stamped from
`on_socket_raw_receive`. Two verified problems make it a regression rather than a fix:
- discord.py 2.7.1 dispatches `socket_raw_receive` only when the client is built with
`enable_debug_events=True` (client.py:330, gateway.py:410-412; the default
`log_receive` is a no-op). The adapter never sets it, so the stamp only ever moves at
`on_ready` and every healthy connection reads `event_silence` 300s later — a forced
reconnect every ~5 min. Live-verified against a real `commands.Bot` +
`DiscordWebSocket.received_message`: 6 frames delivered, stamp unchanged, probe unhealthy.
- discord.py already keeps a per-frame clock (`KeepAliveHandler._last_recv`) and closes the
socket itself after `heartbeat_timeout` without frames; and because ACKs are frames,
`ack_stale` (60s) always trips before `event_silence` (300s). A raw-frame stamp cannot
detect the "ESTAB + ACKing + zero events" incident by construction.
Kept and tightened the warning half: bool values (`float(True) == 1.0` silently enabled a
knob at 1s), negative ints, and unparsable strings now warn; an explicit `0` is the documented
opt-out and stays silent. Tests trimmed to the two invariant contracts (warn / don't warn),
proven red on origin/main. Docs updated to match.
The Gateway WS health probe sampled only transport state — ready, open,
heartbeat-ACK age, latency. A socket that stays ESTAB and keeps ACKing while
zero gateway frames arrive (the #109521 "connected-but-deaf" incident) read
healthy indefinitely, and the adapter went silent for hours with no log line
and no watchdog firing.
Two defects fixed:
1. Dispatch-side dimension. `on_socket_raw_receive` now stamps
`_last_gateway_frame_at` for every inbound raw gateway frame — heartbeats
and ACKs included, so a legitimately quiet server is not flagged. The
health check gains an `event_silence` reason with its own bound,
`websocket_event_max_silence_seconds` (default 300s, 0 disables the
dimension alone). The stamp resets on `on_ready` so a reconnect never
inherits pre-restart silence. Trip path is unchanged: consecutive
failures -> retryable `discord_websocket_health_stale` -> the existing
reconnect watcher builds a fresh adapter.
2. Silent probe disable. `_finite_positive_config_float` / `_config_int`
mapped anything `float()` rejects ("15s", "nan", "true") to 0.0 with no
log line, permanently disabling the watchdog invisibly. Unparsable and
non-positive values now log one WARNING naming the knob and raw value.
The new key rides the existing `_YAML_WEBSOCKET_LIVENESS_KEYS` seeding and is
documented in the Discord guide's liveness section.
- Correct the per-generation reset comment: an in-place updater restart keeps
PTB's update_queue, so old-generation dispatches can briefly exceed
received; the check already treats that as no backlog.
- Make the once-per-stall gate explicit (cap the heartbeat count) instead of
relying on `!=`.
- Drop the dead getattr in _record_updates_received (only reachable after
_record_polling_progress dereferenced the same instance state); keep the
fallbacks in the heartbeat check and group-99 handler, which sibling
watchdogs share because object.__new__ adapter doubles exist in tests.
- Split the single invariant test so a failure names the broken guard:
stall-once, re-arm-on-progress, generation-reset; drop the unused _app mock.
Follow-up to the cherry-picked #102383 commit. The check as written was neither
sensitive nor specific:
- It aged the newest received update, so a wedged PTB dispatcher was never
reported while new updates kept arriving more often than every 300s (probe:
1 update/250s for an hour -> 0 reports).
- `delivered` counted only MessageEvents reaching the gateway handler, while
`received`/`dispatched` counted every Update; a single handled callback_query,
reaction, unauthorized user or unmentioned group message produced a false
ERROR after 300s of quiet.
- `_record_updates_received` skipped the generation/teardown guard
`_record_polling_progress` applies, and the counters never reset across
polling generations, so a late response from a fenced poll or a reconnect
inflated the backlog.
Now `received` and `dispatched` count the same population (every fetched
update reaches the group-99 catch-all) and the report fires when a backlog
persists with no dispatch progress across two 90s heartbeats, once per stall,
re-armed on progress, reset per generation, at WARNING (diagnostic only;
#71240 owns recovery). The delivered counter and the `note_inbound_delivered`
facade method are dropped; the once-per-adapter "no message handler" error on
BasePlatformAdapter.handle_message stays. `_record_polling_progress` returns
whether the round-trip was accepted so the received stamp reuses its gate.
Tests trimmed to two invariants; every guard proven red by mutation.
Refs #102260
Every Telegram health probe measures the transport. A getUpdates round-trip
that returns 200 proves bytes are moving and nothing else: the stall watchdog
(#92991), the pending-update probe (#42909/#55769), the get_me() heartbeat
(#66377) and the polling-progress instrumentation all stay green while updates
arrive and then die downstream. The adapter then publishes "connected", logs
nothing at all, and is indistinguishable from a bot nobody has messaged.
That is #102260: three weeks of telegram.state "connected" plus "polling
confirmed healthy: getUpdates progressing (generation 1)" with zero inbound
reaching the agent, surviving every restart. Two of the issue's three
hypotheses do not hold on this code — _record_polling_progress fires on every
round-trip (not only at start_polling), and _send_path_degraded is cleared on
the first confirmed round-trip — and the reporter's own observation that fresh
messages are received but not processed places the failure downstream of the
transport, in the one stretch with no instrumentation at all.
Add the missing delivered side of the accounting:
- received: updates Telegram handed the process, read from the getUpdates
envelope the adapter already parses (an empty result proves the transport,
not arrival, so only non-empty results count).
- dispatched: updates PTB's dispatcher carried through the whole handler
chain, stamped in the existing group-99 catch-all before its early returns.
- delivered: inbound events that reached the gateway's message handler,
stamped in BasePlatformAdapter.handle_message for every platform.
_check_ingress_delivery_gap runs on the existing heartbeat and, when updates
arrived but nothing was delivered for 300s, names the broken hop: received >
dispatched means the dispatcher is not draining, dispatched > delivered means
Hermes is dropping what arrives. Diagnostic only — a received update
legitimately reaches no gateway turn, and reconnecting a healthy transport
cannot repair a dropped update, so this never drives recovery.
Also make the silent discard on the shared funnel speak: handle_message
returned with no log when no message handler was installed, so a mis-wired
adapter discarded 100% of inbound while connected and able to send.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018pT5hFJBRfLj8KMqFhm3qz
dingtalk _extra_get, mattermost _extra_or_env and slack _extra_or_env_flag/_channel_set fell
through to env only on None, so `allowed_channels: ""` / `free_response_channels: ""` meant
"no whitelist" rather than "use the env CSV". The shared reader treated blank as unset and
silently widened those to the env value. New `blank_is_unset=False` knob restores the old
semantics at those seven call sites; the default (blank = unset) stays for the readers whose
old body was `extra.get(k) or env`.
WeComAdapter mixed in OwnAccessPolicyMixin without ALLOW_ALL_ENV_PREFIX, so
_allow_all_env_names() read "_ALLOW_ALL_USERS" and every open-DM WeCom deployment
(setup still writes WECOM_ALLOW_ALL_USERS) silently denied all DMs.
- WeComAdapter.ALLOW_ALL_ENV_PREFIX = "WECOM"
- OwnAccessPolicyMixin.__init_subclass__ raises TypeError on an empty prefix so the
omission cannot ship again (every other host already sets one).
- Parity test now runs a third matrix row that sets each host's own
<PREFIX>_ALLOW_ALL_USERS; the GATEWAY-only row is why this slipped through.
Sabotage: prefix removed -> [platform] row fails even with the guard reverted.
