fold2 finding 1 (2c, Medium): the connect-time missing-pin warning was gated on
`not is_reconnect`. An account whose FIRST startup connect fails (transient
IMAP/SMTP error) is brought up by the reconnect watcher with is_reconnect=True,
so it never warned while silently dropping every sender's mail.
Dedupe per address instead: a class-level _missing_pin_warned set (the watcher
builds a fresh adapter per retry, like _seen_uids_snapshot), checked after both
IMAP and SMTP probes succeed. The existing connect test now covers a failed
first connect followed by a reconnect (one warning), a second reconnect (none),
two accounts (one each) and pinned/opted-out accounts (silent).
fold2 finding 4 (quality, Low): _verify_sender_authentication defaulted
authserv_id to "" (a silent fail-closed for any caller that forgot it) and
re-stripped/lowercased a value EmailAdapter.__init__ already normalises. Make it
a required keyword, test it with `if not authserv_id:`, and compare directly.
All callers (the adapter and the test helper) already pass normalised ids by
keyword, so no call site changes.
fold2 finding 2 (reuse+quality, Low): the per-message no-header drop hint and
the connect-time missing-authserv_id hint each spelled out the same
require_authenticated_sender / EMAIL_TRUST_FROM_HEADER opt-out sentence. Build
both from one _OPT_OUT_HINT so they cannot drift. Rendered text is byte-for-byte
unchanged (printed before/after).
fold2 finding 3 (quality, Low): the __init__ comment still said an absent pin
fails closed only "when an allowlist grants access". With sender auth on, an
unset authserv_id rejects every sender's Authentication-Results, and connect()
names the fix once per account. Comment only.
The _verify helper derived the expected pin from prod's own
_ar_clauses(...)[0], so every row was pinned to whatever its topmost
header claimed and an authserv-id parsing regression could not go red
(r1 2c Low). Default the helper to the literal "mx.ourserver.com" and
stamp that id on the rows; the forged/exact-pin rows keep their own ids
and are still refused, and the unpinned row still fails closed.
The missing-pin warning lived in the per-message drop path behind a
process-wide class bool. That hid it twice over (r1 2ab/2c Medium):
- a second unpinned mailbox in the same process (multiplex) never
warned once the first had tripped the flag;
- pairing/decline-only deployments never warned at all, because a
first-contact sender is not granted and its drop logs at debug.
A missing pin is account config, not something one sender did, so
connect() now names the fix once per account after the IMAP/SMTP
probes succeed, skipping reconnect-watcher retries (is_reconnect).
Pinned or opted-out accounts stay quiet. The per-message drop path is
back to main's rule: a listed sender's drop warns (with the reason, no
hint), an unlisted one logs at debug, so it is never noisier than main.
The guide still told users to "leave authserv_id unset" when their
server's host varies, which now drops every message that needs an
authenticated From:. State that the pin is required, how to find it
(the token before the first ';' of the topmost Authentication-Results
header your server writes), what to do when it seems to vary (RFC 8601
authserv-id names the administrative domain, so configure a stable one
and pin it), and that the only alternative is the explicit
require_authenticated_sender: false / EMAIL_TRUST_FROM_HEADER=true
opt-out. Mirror the reference tables, plugin.yaml prompt and zh-Hans
translation.
Co-authored-by: JiuYue0820 <287222957+FirmaSpring@users.noreply.github.com>
The salvaged #88560 change makes an unpinned deployment refuse every
Authentication-Results header, so each message from an allowlisted
sender is dropped. Logged per message, that warning would flood the gateway log.
The salvage gave the reason its own constant and drop hint (set
EMAIL_AUTHSERV_ID / platforms.email.authserv_id, or opt out with
require_authenticated_sender: false / EMAIL_TRUST_FROM_HEADER=true);
this logs it once per process, and later drops stay at debug.
Tests: the verifier helper now pins the topmost header's own id by
default (a correctly configured receiver), the topmost/exact-pin test
gains the unpinned forged-header row, and the dispatch warning test
checks the once-per-process warning and its text.
Co-authored-by: JiuYue0820 <287222957+FirmaSpring@users.noreply.github.com>
Trust only the topmost Authentication-Results field from the explicitly configured receiving MTA. Fail closed for allowlisted senders when no exact pin is configured, preventing injected or lower matching headers from authorizing a sender.
(cherry picked from commit e013fda3facc005e8f4d5f81e20c2d21aee698df)
Fold2 PROFX finding 2 (quality): `_DOT_DIRS` hand-copied six of the eight
HOME_CREDENTIAL_DIRS entries. Build it from the shared constant so a dir
added there is seeded as a directory store without editing the test.
`.config/gh` and `.config/gcloud` were already seeded as directories by the
dotless-name rule, so the seeded tree is unchanged.
Fold2 PROFX finding 1 (reuse/quality): `_SENSITIVE_HOME_DIRS` had become a
bare alias of agent.file_safety.HOME_CREDENTIAL_DIRS with one reader. Drop
the alias and use the shared constant at that reader so there is one name
for the list. No other references exist in the repo (tests/ and plugins/
included).
Gate r1 (2c note): agent/context_references._SENSITIVE_HOME_DIRS was a hand
copy of the home credential dirs and missed .config/gcloud, so
`@file:~/.config/gcloud/application_default_credentials.json` expanded the
gcloud refresh token into context; the canonical read deny-list does not
cover it either. The gateway feeds untrusted remote text through this
expansion, so this is the same leak class the profile-export fix closes.
_SENSITIVE_HOME_DIRS now is agent.file_safety.HOME_CREDENTIAL_DIRS, the
single list the write denylist and profile export already derive from.
file_safety imports stdlib only, so there is no import cycle.
The existing canonical-denylist test gains a .config/gcloud ADC row.
Co-authored-by: Keiran Haax <196126709+keiranhaax@users.noreply.github.com>
#132201 stopped profile export shipping .ssh/.aws/.gnupg/.kube/.envrc, but
the other credential dirs the file tools already write-deny still shipped:
a skill dir copied from a home carries .docker/config.json (registry auth),
.azure/accessTokens.json, .config/gh/hosts.yml (GitHub token) and
.config/gcloud/ (ADC / credentials.db), and the default-profile export plus
named exports archived them verbatim (binary DBs skip the redact pass).
The home-relative credential dirs now live once in
agent.file_safety.HOME_CREDENTIAL_DIRS; build_write_denied_prefixes and the
profile export (_OS_CREDENTIAL_STORES) both derive from it, so the write
denylist and the export filter cannot drift. Nested stores (.config/gh) are
matched on trailing path components, so only .config/gh and .config/gcloud
are dropped and the rest of .config still ships. .config/gcloud is included
for parity with file_safety. Distribution installs treat the same paths as
installer-owned via PROFILE_CREDENTIAL_PATHS.
Co-authored-by: Keiran Haax <196126709+keiranhaax@users.noreply.github.com>
2.0.1 stays inactive on a Hermes core that still bundles Home Assistant (no duplicate ha_* tools
or adapter override) and carries the multiplex endpoint-scope tests ported from core.
The Home Assistant gateway platform and ha_* tools as a standalone catalog plugin
(NousResearch/hermes-homeassistant). Core still bundles Home Assistant until #132469 lands; that
PR removes the bundled copy and migrates every profile that used it onto this entry.
The wizard's invite integer and the Discord docs' Recommended row carried
Use External Emojis (1<<18), a permission the adapter never exercises (it
only reacts with unicode emoji) and that no permission list in the docs
names. The voice-mode page and the zh-Hans Discord page already used the
correct integers, so the pages disagreed with each other.
- onboarding.INVITE_PERMISSION_BITS: drop Use External Emojis
(wizard invite 309241171008 -> 309240908864, matching voice-mode.md)
- discord.md: Recommended 309238025280 -> 309237763136; the Minimal row
names Embed Links (117760 already includes it)
- .env.example and the zh-Hans table follow
Co-authored-by: Emil <etoegersen@gmail.com>
With the homeassistant plugin missing, send_message to a `homeassistant:` target failed with
"Unknown or unregistered plugin platform: homeassistant" and no way forward.
Both unknown-platform errors (target resolution and platform config) now go through one helper
that appends, for a platform that left core (a LEFT_CORE row's platform), the exact install
command for the active home (`hermes [-p <profile>] plugins install homeassistant`). Other unknown
names keep the bare message.
While the plugin was missing and the catalog or GitHub unreachable, every `hermes chat` process
paid a live-catalog fetch (+5.9 s on a blackholed network) and printed the raw multi-line git
stderr of the failed clone.
A failed automatic install (catalog miss, failed install) touches
<home>/cache/left-core-<plugin>.failed; agent and gateway start skip the row, before any network,
while that stamp is under an hour old. `hermes update` ignores the stamp and always retries. The
failure notice is one line: git's own fatal:/error: line when the error carries raw git output,
else its first line.
The catalog module already serves a fresh cache (6 h) and falls back to its stale/in-tree copy on
failure; pre-resolving offline would not save the round trip, because the install resolves the
entry through the live catalog again, so the gate sits before both.
The left-core migration's only gate was "plugin dir missing + Home Assistant in use", so after
`hermes plugins remove homeassistant` every agent start, gateway start and `hermes update`
reinstalled and re-enabled it (~7.5 s each).
The automatic install now happens at most once per home and row: `_left_core_installed` in
config.yaml is written (in the same write as the toolset-scope marker when the plugin was already
there, right after a successful install otherwise) and only once the plugin dir exists in that
home. A marked row is never installed again; a catalog miss, failed install or lazy-installs-off
start writes nothing, so those homes keep retrying. Profile clones copy plugins/ with config.yaml,
so a clone's marker always matches what its source had.
Profile cloning learns which env keys and config blocks belong to a messaging channel from core's
platform registry. Moving Home Assistant into a plugin left that ownership only on the loaded
plugin, so while the migration is pending (catalog miss, offline, lazy installs off, declined),
`hermes profile create X --clone` kept HASS_TOKEN/HASS_URL and the gateway.homeassistant block in
the clone while printing "Messaging channels were NOT cloned (homeassistant)". The copied token
then reads as Home Assistant in use, so the clone would migrate onto the source's HA identity.
LEFT_CORE rows now declare the channel ownership core had for their platform: enable_env (the env
that enabled the adapter) and channel_env_prefixes (prefixes shared with the plugin's tools).
profile_channels includes those platforms in platform_ids, the shared-prefix rule, the enable-gate
and the duplicate-credential inventory regardless of plugin installation, so the source-runs-the-
adapter rule is exactly core's: enabled (config or HASS_TOKEN) -> stripped; explicitly disabled,
tool-only -> HASS_* kept as a tool credential.
The migration converts core's per-platform Home Assistant tool scope (off on telegram: [web], acp,
webhook) into known_plugin_toolsets entries, but only on the install branch: a home whose plugin
directory already existed returned before the conversion. The official plugin stays inert on a core
that still bundles HA, so a user who installed it before updating (or a partial earlier run) kept no
exclusions, and after the update the plugin toolset came on for every platform.
Installation and the one-time policy conversion are now separate: every home that uses the feature
gets the conversion once per row, plugin present or not, and the completion is recorded under
_left_core_scoped in the same config.yaml write. A later run (update, agent/gateway start) skips
it, so a choice the user makes afterwards (`hermes tools` enabling HA on telegram) is never undone.
The left-core migration decided "this home used Home Assistant" with its own config reader: the
first platforms.homeassistant block in config.yaml. The pre-extraction gateway enabled the adapter
from more places: the legacy gateway.json base layer, the gateway.homeassistant shorthand, and
nested gateway.platforms + top-level platforms blocks merged key by key. Those homes got no plugin
and no notice, and lost Home Assistant on update.
platform_configured_on() now asks the gateway loader itself: load_legacy_gateway_json +
read_yaml_layers (managed overlay, ${VAR} expansion) + merge_platform_sections, then
PlatformConfig.from_dict for the enabled flag, so the last explicit `enabled` keeps winning
(config.yaml enabled: false over a gateway.json enable stays not-in-use).
merge_platform_sections takes `also=` so the gateway.<platform> shorthand of a platform that
left core counts before its plugin registers (Platform() no longer knows the name).
With the try/except gone, stub agents (SimpleNamespace in tests/agent/test_system_prompt.py)
raised AttributeError on the say= argument before recover_at_startup ran.
Gateway start ran the left-core migration only for the launch home, so a secondary profile that
used Home Assistant as its event platform started without the plugin and without its adapter (and a
named launcher's default-root primary was never migrated). Each secondary now runs the migration in
its own runtime scope before its plugins are discovered, and gateway start covers the launch home and
the default root (the multiplex primary). The gateway-start helpers live in gateway/run_startup.py;
run.py and agent_init.py lose the try/except wrappers, since recover_at_startup never raises.
An agent-start install of a row that ships a platform now tells the user to restart the gateway
when one serves the home: the running gateway loaded its platforms before the plugin existed.
PluginContext.register_platform is last-writer-wins, so a user plugin could re-register 'telegram'
with trusted_inbound=True and every sender would skip the allowlist and pairing. The flag is now
refused for a platform core ships (a static Platform member or a bundled plugins/platforms/ plugin)
unless the registering plugin is bundled. Home Assistant is no longer either, so its catalog plugin
keeps it. `hermes plugins validate` prints a line for every trusted_inbound platform (and fails one
naming a core platform).
Also: the MCP collision test docstring no longer refers to the toolset it used to collide with.
Core's homeassistant toolset was on for a platform only when its saved platform_toolsets list (or
its default composite) carried it. As a plugin toolset it is on for every platform whose
known_plugin_toolsets does not name it, so a user who scoped HA out of Telegram with
`platform_toolsets.telegram: [web, terminal, file]` got device control there after the migration;
webhook and acp, whose default composites never carried it, gained it too.
Left-core rows now name their toolsets and the platforms whose default composite lacked them. Before
any install (automatic or the user's own) the migration records the toolsets in
known_plugin_toolsets[platform] wherever core resolved them off; platforms core had them on for are
left alone. Per-platform parity with origin/main verified on 5 configs x 8 platforms.
Moving HA out of core dropped HASS_TOKEN/HASS_URL from the static provider blocklist and HASS_TOKEN
from the Tier-1 always-strip set. Once installed, the plugin's manifest declares them, but with the
plugin absent (migration pending, declined or failed) the token reached every terminal/subprocess
child. Each left-core row now names the env core stripped while it shipped the feature
(secret_env -> Tier 1, private_env -> provider blocklist) and local_env_policy keeps stripping it.
The homeassistant catalog entry lands in its own catalog-only PR, so these tests cannot rely on
plugin-catalog/homeassistant.yaml existing in this tree. The catalog lookup is a stand-in that lists
every left-core plugin (as the memory migration tests do); the in-tree entry change-detector is gone.
Catalog entry PRs may only touch plugin-catalog/** (pinned-source-validate), so the entry moves to a
separate PR that lands first; the migration installs it from the live catalog once both are in.
The Home Assistant gateway platform (plugins/platforms/homeassistant) and the
homeassistant toolset (tools/homeassistant_tool.py) now ship as the official
catalog plugin `homeassistant` (NousResearch/hermes-homeassistant), with the
same platform name, toolset/tool names, config keys and env vars.
Migration (hermes_cli/left_core_migration.py, a generic "left core" table
reusing the memory-provider migration's helpers): every profile home that
used Home Assistant (HASS_TOKEN in its .env, platforms.homeassistant enabled
or holding a token, or the homeassistant toolset selected) gets the plugin
installed from the catalog at its reviewed pin by `hermes update`, and lazily
once per process at agent start / gateway start (honours
security.allow_lazy_installs). Every outcome reaches terminal, Desktop and
chat; a gateway-start outcome is handed to that home's first agent.
Core HA special cases become generic PlatformEntry seams the plugin uses:
trusted_inbound (allowlist bypass for event-bus platforms), display_tier
(built-in display defaults) and shared_env_prefixes (profile-clone ownership
rule for env shared with the plugin's tools). Cron delivery uses the existing
cron_deliver_env_var seam; HASS_TOKEN stays stripped from subprocesses via the
installed plugin's manifest secrets.
Both primary recovery paths fenced the route lookup but not the WebSocket
ticket mint that follows it. When a connection apply (boot hook) or a
foreground activation (request recovery) landed while the ticket was in
flight and the newer socket had already dropped, the stale attempt dialed the
old primary route. connect() no-ops on an open socket, so the gap only shows
once the applied socket is closed, but then it re-attached the window to the
gateway the user had just left.
The boot reconnect now compares the route revision it recorded against the
current one again before dialing; request recovery re-checks ownsForeground().
useGatewayRequest's primary recovery resolves the connection profile-less, so
main answers through the window's CURRENT route. When the user activated
another source while that lookup was in flight, the late answer was published
with setConnection() over the newer foreground descriptor (and a failed
lookup cleared it), so REST and filesystem helpers routed to the old backend
while the newer source was in front.
Recovery now snapshots the gateway activation epoch and publishes, clears, or
dials only while the primary still owns the foreground. The boot hook's
own-route reconnect loop keeps restoring the primary socket, and the replay
guard already refuses to resend the original request.
Reported-by: dokterdok
Address review on #132237: an in-flight primary reconnect whose route lookup
resolved after a connection apply recorded its old descriptor over the newly
applied primary. The next socket drop then dialed the old gateway by its
explicit route, where the profile-less lookup on base followed the applied
window route.
Every primary route record now bumps a revision; a reconnect snapshots it
before the lookup and drops the result if a boot or apply recorded a newer
route meanwhile, so it no longer records, publishes or dials the old route.
The source-qualified primary lookup is unchanged.
Address the ordinary-switch draft-intent race from review on #132237: a
draft started on another gateway while a source switch was still doing its
bookkeeping (rememberConnection) was cleared by the switch's late re-home,
so the next session.create went to the switched-to source instead.
Every new-chat owner intent goes through captureNewChatSource, which now
bumps an intent revision. selectConnection snapshots it when the switch
starts and only re-homes the draft if no newer intent arrived, so a newer
intent wins even when it repeats the same route values. Completed switches
with no newer intent re-home as before.
Address review on #132237:
- Project-tree rows go through the canonical stampRowsWithOwningConnection:
a row that already names an owner is never re-owned, and local stays bare
on the primary path so the legacy profile door (#94166) is unchanged. local
is stamped only when it is an active registry secondary.
- Resume remembers an ambient owner hint only for a row that actually
resolved; an unresolved id no longer gets pinned to the foreground source.
- The silent boot restore keeps an explicit new-chat route pinned while it
was in flight; the route clear lives in one helper.
- Drop the untested pre-recovery route check in useGatewayRequest (the
post-recovery check covers it) and note why the two primary reconnect
paths resolve differently.
- Tests: move the primary-reconnect cases into the reconnect describe with an
inline resolver fake, assert the explicit primary profile exactly, and drop
an overlapping parametrized case.
Primary reconnect called profile-less getConnection(), which main resolves from
the window's foreground route. A drop while a secondary was foregrounded
re-dialed the primary socket to that secondary's backend. Reconnect by the
primary's recorded registry route (or legacy profile); boot and connection
apply keep following the foreground.
useGatewayRequest recovered a transport failure through the current active
route and retried the original payload there. A gateway switch between
dispatch and the delayed rejection replayed the request on the other source.
Retry only while the dispatch route is still active; otherwise surface the
original error.
Resume kept the ambient connection only for remote mode, so an untagged row
while This device was an active registry secondary reduced to a bare profile
and routed to the window primary. Keep the legacy primary-local profile door
(#94166) unchanged, and remember the resolved owner for later prompts.
