Provider shutdown handed the manager a remaining budget, but flush_all ran first with no deadline and blocked on each session's flush lock. An async upload still in flight held that lock, so shutdown waited the full HTTP timeout past its declared budget. flush_all and the queue drain now take the deadline, skip a session whose lock or budget is gone, and log one warning with the count of messages that stayed unsynced.
With recallSync on, prefetch popped only the auth notice and returned without writing the injection log. A session whose init failed for a missing user peer never told the model that memory was off, and the audit file stayed empty for every turn. The recall sync branch now pops the peer notice the way the async branch does and records each turn as injected or recall-sync-empty.
Parametrize the sessionStart, injection-log, dashboard user_id, unresolved-peer
and deferred-save tests that differed only in their inputs. Share the blocking
remote in the concurrent flush tests. Fold _as_flag onto a word table. Cut the
added docstrings to the what and the one non-obvious why.
recall_sync.py spawned honcho-recall-sync without owner=, so shutdown's
join_plugin_threads((self, manager), ...) never saw it. The worker now
registers under the provider like the other provider threads.
`_flush_session` discarded the observation flags when it rebuilt an evicted SDK session, and the
cached path returned none, so recall fell back to the config snapshot. Both paths now return and
store the flags. A deferred `save()` on a session the cap evicted puts it back in the cache, or
flushes it inline when a newer object owns the key. `save()` and `stop_async_writer()` share the
writer lock, and the writer drains its queue after the join, so a put that raced shutdown is
written. The trim after a flush runs under the cache lock. The shutdown join takes the remaining
budget instead of a fixed ten seconds.
The injection audit file is created owner-only, and `logging: "false"` reads as off. The desktop
passes `<provider>:<user id>` so a basic-auth alice and an OIDC alice are two peers. When a
gateway platform supplies no user id, the peer notice and tool error no longer recommend
peerName, which would merge every user of that gateway onto one peer. README documents
`injection.sessionStart`, `logging`, and what a dashboard login does to peer resolution.
a desktop or cli session with no peerName in honcho.json and no gateway
user id landed on a peer derived from the session key: user-default-<dir>
for per-directory sessions, user-<channel>-<chat> for keyed ones. every
directory got its own phantom peer, so the operator's turns and memory
never reached their real peer and the injected representation went stale
(#93326).
_resolve_user_peer_id now raises HonchoPeerUnresolvedError instead of
deriving a name. a peer is either the declared peerName or an identity
the transport supplied. the provider records the failure, tells the model
once that memory is off and which key to set, returns the same detail
from tool calls, and stops retrying init because a missing config key
does not heal mid-session. the memory-file migration gate loses its
"no owner and no runtime identity" branch: that cohort no longer has a
session to migrate into. whitespace-only peerName is treated as unset
rather than sanitized to "--".
provider shutdown joined the dialectic, sync and memwrite threads for 5s
each and then the async writer, but the session-init thread,
honcho-base-first and honcho-context-prefetch were never joined. any of
them still blocked in httpx when the interpreter finalized aborted the
process with SIGABRT 134 (#37632, #60616, #33485). the 5s join was also
shorter than the 30s http timeout a blocked call can hold (#33485).
spawn_context_thread now registers each thread under its owner (the
provider or the manager) in a weak registry, and shutdown joins every live
thread of both owners inside one deadline: at least 5s, or the configured
http timeout when longer. the manager refuses new prefetch threads once
shutdown began and flushes a late save() inline instead of respawning the
writer. threads that outlive the budget are named in a warning.
the sdk client exposes close() on its http pool. one client is shared by
every manager with the same identity in a gateway process, so a per-agent
shutdown cannot close it; close_honcho_clients() closes all pools and is
registered with atexit when the first client is built, the pattern the
hindsight and mem0 plugins use. follows #69070, #33701, #7627.
the idle sweep from #71463 left three growth paths open. _peers_cache had no
bound at all, _session_observation (from #98941) grew one entry per session
id and kept orphans when an init failed after add_peers, and a burst of
distinct sessions inside one ttl window was not bounded. the sweep also
evicted sessions whose messages had not reached honcho yet, which in
"session" write mode drops the only copy, and _flush_session re-inserted an
evicted session into the cache with no observation flags, so recall for it
routed from the config snapshot instead of its server config.
_cache and _sessions_cache now cap at 128 entries and _peers_cache at 512,
evicting least recently used first (dict order, refreshed on every hit).
a session with unsynced messages is never evicted by the sweep or the cap.
_configure_session_peers returns the synced flags and get_or_create stores
them under _cache_lock next to the cache entry, so the observation dict can
hold no id the cache does not; eviction drops both. recall reads go through
_cached_session, which stamps updated_at, so a read-only session survives
the idle sweep. follows #71461, #71463, #98936.
_flush_session read the unsynced messages, posted them, and only then set
_synced. in a short-lived run the async writer draining save()'s queue and
the exit-time flush_all() both saw the same batch unsynced, so every turn
of a one-shot run landed in honcho twice (#92458).
each HonchoSession now carries an RLock that _flush_session holds around
the whole select, send, mark sequence. the second flusher enters after the
first marked the batch and sends nothing; different sessions still flush
in parallel. the lock lives on the session object instead of a manager
dict keyed by session id (#86094, #92787): it needs no eviction, and two
flushers of one message list can never hold different locks. tests
adapted from #86094 and #92787.
HonchoSession.messages grew forever -- add_message() appended every
turn and _flush_session() only marked entries synced, never trimmed
them. HonchoSessionManager's four caches (_cache, _peers_cache,
_sessions_cache, _context_cache) had no eviction path besides an
explicit /new reset. A long-lived channel that is never manually
reset accumulates both for the gateway's entire uptime.
Honcho is the durable source of truth (get_or_create() already
re-fetches history from Honcho on a cache miss), so both bounds are
safe: evicting an idle local entry only costs one extra Honcho
round-trip next time that key is used.
- Trim already-synced messages beyond a retention window right after
a successful flush; unsynced messages are never touched.
- Add a rate-limited idle-TTL sweep triggered opportunistically from
get_or_create(), so no new background task/watcher wiring is
needed.
Fixes#71461
The four observation booleans on HonchoSessionManager were manager-wide
mutable state initialized from per-session server configs: every session
setup overwrote them, so the last session to initialize retuned recall
routing for all other sessions the manager serves. Store the server-synced
flags under each session's own id instead; the manager-level fields stay
as the config snapshot and sessions that never synced fall back to them.
injection.sessionStart: unset renders every component in table order, an
empty list renders nothing, a pinned list renders only those names in table
order (not config order), a host block pin beats root, a non-list value is
treated as unset, and initialize() reads the pin.
logging: off by default, the logging key or HONCHO_LOGGING turns it on, a
host block can turn it back off, HONCHO_INJECTION_LOG overrides the path,
each record carries reason/turn/session_key/bytes/payload, an unwritable
path never raises, and a tools-mode prefetch logs its reason. the record
holds the user's representation verbatim, which is why the default stays off.
the two session-context tests that asserted exact context() kwargs now expect
tokens= as well (adopted from #92964). config_schema declares the logging
switch so hermes memory setup shows it.
Salvage of #70951 (Willkons / Alice-Willk-bot). Two of three
session.context() sites never forwarded the configured cap, so Honcho
always returned honcho_chat_summary_long.
Adds the missing get_session_context tokens= regression the sweeper
asked for on that PR.
force_refresh_token gated the adopt-from-disk paths behind the failure cooldown.
After one of our exchanges failed transiently, a 401 within the next 30s returned
None even when a sibling process had already rotated and written a valid grant,
so the operation raised HonchoAuthError with a good token sitting on disk.
Adopting is a disk read, not an exchange; the cooldown exists to stop replaying a
single-use refresh token, so the two adopt checks now run before the gates.
_write_config parsed honcho.json twice under the lock and only wrapped the first
read into ConfigWriteRefused; _refuse_unparseable now returns the parsed dict and
the branches use it. The getattr/isinstance duck-typing collapses to one guard.
cli._read_config reuses oauth's tolerant reader (BOM-tolerant, like the strict
reader the write side uses) instead of its own utf-8 copy.
The CLI's _write_config took only the best-effort file lock, so an in-process
refresh thread could still interleave with a command's read-modify-write. The
dashboard's Honcho save (_write_provider_honcho) still seeded its whole-file
rewrite from a tolerant reader, so a honcho.json that exists but does not parse
was replaced by the active host's block alone from the UI - the same bug class
this PR closes on the CLI and refresh paths. `hermes profile create --clone`
swallowed the new ConfigWriteRefused as "plugin not installed".
One parametrized test covers the web writer for the corrupt and parseable cases.
_write_config applied a command's edits relative to the snapshot the read took, but never moved that snapshot after a successful write. A second write on the same object therefore compared A -> B -> A against A, saw no change, and left disk at B. After a write the snapshot and path now follow the caller's dict, so the next write applies only the edits made since.
_read_config() falls back to ~/.honcho/config.json or the default profile when honcho.json does not exist, so cfg.path never matched the write path and _write_config() wrote the whole dict. That overwrote a refresh rotation a serve child had written onto the honcho.json the setup login created moments earlier. When the local file exists at write time, the seed snapshot is now overlaid with the local file and only the command's edits are applied onto it.
save_config read the file and wrote it back without the locks the token refresh holds around its own read, exchange and write. A refresh that landed between the two steps was overwritten and its consumed refresh token was gone. The read and the write now run under both locks.
install_grant writes the login grant to disk before the wizard's later prompts. _apply_grant_to_host wrote it only into the live cfg, so _write_config saw the grant as an edit and copied it over whatever a serve child rotated onto disk meanwhile. The snapshot now takes the grant too, so the final save leaves the newer on-disk grant alone.
_ReadConfig builds its snapshot in __init__; _read_config no longer
assigns the attributes after the fact. cmd_enable prints its hint
inline, since _no_credential_hint had one caller. The split
force_refresh_token signature and call fit on one line. Docstrings and
comments keep the what and the one non-obvious why. No behavior
changes; _apply_edits keeps its original body.
clone and enable accepted HONCHO_API_KEY from the environment as proof the
block could authenticate and wrote enabled: true. the variable can be absent
from the next process, leaving an enabled block with nothing behind it, which
is the cohort the previous commit set out to remove.
_resolve_api_key takes env=False at both write sites, so a block is enabled
only when honcho.json itself holds a key, an oauth grant, or a self-hosted
baseUrl. status and setup keep the environment fallback for display.
every hermes honcho command read honcho.json, changed a field, and wrote the
whole dict back with no lock. a token refresh in another process that landed
between the read and the write was overwritten with the old access and
refresh tokens, and the next refresh replayed a rotated single-use token.
_write_config now holds the same cross-process file lock the refresh path
holds, re-reads disk under it, and applies only the keys the command changed
since its _read_config(). untouched keys keep their on-disk value, so a
rotation survives; a credential the command set on purpose still wins. a
write with no prior read keeps today's whole-file behavior.
hermes memory setup writes through HonchoMemoryProvider.save_config, which
merged the new values over {} when the existing file could not be parsed
and then replaced the file. same wipe as the oauth and cli write paths,
through a different door. it now reads through _read_config_strict and
raises, so the caller sees the error and the file stays as it was.
a named profile cloned from a default profile that signed in with oauth
got a host block with enabled: true and nothing to authenticate with.
hosts.hermes holds the grant, its apiKey is not inherited (#66125), and
copying the oauth block would make two blocks replay one single-use
refresh token. 'hermes honcho enable' on a fresh profile wrote the same
shape. status then showed the profile as enabled while every honcho
call ran without memory and the plugin quietly stayed inactive.
clone_honcho_for_profile and cmd_enable now resolve a credential for the
target block (its own apiKey, the root apiKey, HONCHO_API_KEY, or a base
url) before writing enabled: true. _resolve_api_key takes the block to
check so both share one definition of "can authenticate".
cohorts:
clone from an oauth default block, no root key: block written without
enabled; the client's auto-enable rule turns it on once a credential
appears (setup apikey writes the root key, or a per-profile login)
clone from a default block with a host-level static key only: same
clone with a root apiKey, an env key, or a base url: enabled as before
enable on an empty or fresh block with no credential: refused, one
message names the profile's setup command and the hosts.<name> key,
nothing is written
legacy blocks already on disk as enabled with no credential: nothing
rewrites them; the plugin already treats them as unusable and stays
inactive; enable now prints the same message instead of "already
enabled"
env-only key: counted as a credential at write time, as the client
does at run time; if the variable later disappears the client still
refuses to initialize the block
after the token endpoint revoked a grant (invalid_grant), running
'hermes honcho setup', choosing apikey and pasting a valid key changed
nothing. the wizard wrote the key to the root apiKey only. hosts.<name>
still held the dead access token under apiKey and the grant under oauth,
and the host block wins the lookup, so status kept reporting the revoked
grant and every call kept failing (#97990).
the apikey branch now drops the host's oauth block and writes the chosen
key onto the host block as well as the root. a dead access token is no
longer shown as the current key, so a blank answer with no other key
aborts instead of keeping the grant. a static host key without a grant
is kept as before.
a login finishing while a refresh was rotating the same honcho.json ran
its read-modify-write unlocked. the two writers could interleave: the
login read the file, the refresh persisted a rotated token, the login
wrote its own copy back and the rotated token was gone.
install_grant now holds _refresh_lock and _config_refresh_lock(path)
around the strict read, the config merge and the persist, the same way
force_refresh_token does. the token response is parsed before the locks
so a malformed grant never holds them.
a truncated or hand-edited honcho.json reads as {} on the tolerant read
path. the next write then replaced the file with only the current host
block: an oauth refresh, a login, or any 'hermes honcho' command that
saves a setting wiped every other host and the root keys.
_read_config_strict now raises on a parse error the same way it raises
on a read error, and logs one sentence naming the file. _rotate_and_persist
treats that as a refresh failure and enters the cooldown without spending
the refresh token. install_grant raises into the setup flow. in cli.py
every write goes through _write_config, which runs the same check first
and raises ConfigWriteRefused; the honcho router, the setup wizard and the
profile sync print the sentence instead of a traceback. the wizard checks
before asking its questions. read paths keep the {} fallback.
follows #95860, which added the strict reader for unreadable files and
kept a .corrupt copy on a parse error. leaving the original file in place
keeps the same bytes without a second copy of the tokens on disk.
_persist_credential promises "leaving all else intact", but it seeded
its write from _read_config, which returns {} on ANY read failure, and
_atomic_write_config replaces the whole file via os.replace — which
needs only a writable parent, so a present-but-unreadable honcho.json
did not stop the overwrite. One OSError (EACCES after a root-owned
write, EIO, a stalled mount) during an automatic token refresh or a
fresh login therefore replaced the store with a single-host file,
destroying every other host's credentials and honcho.json's root
config. No user action is required to trigger the refresh path.
Same defect class as #75206 (P1, fixed for the core auth store in
Add _read_config_strict for the write paths: a missing file still
bootstraps as {}; an unreadable file raises with the store untouched;
genuine corruption still degrades but preserves a .corrupt copy first,
since a truncated store usually holds the other hosts' tokens verbatim.
_rotate_and_persist now takes its strict read BEFORE the exchange —
rotation is single-use, so an exchange whose result cannot be persisted
loses the grant — and threads the dict through to _persist_credential,
which also closes the re-read race between the locked read and the
persist. install_grant seeds its root-merge from the strict reader for
the same reason. Read paths keep their fail-open contract untouched;
both readers now use utf-8-sig so a BOM'd store is not misclassified
as corruption (the wipe vector needing no filesystem fault at all).
Adds TestPersistReadFailure: six tests, four of which fail against the
previous source; the rotate-ordering test additionally pins that no
exchange is attempted against an unreadable store.
reading the live client's api_key inside _force_reauth races the in-place
rotation: a sibling waiter's apply_token_to_client (or the proactive
refresh entered via the honcho property) can swap the bearer before the
read, so force_refresh_token receives the already-rotated token, disk
matches it, and the adopt branch never fires — reproducing the very
force-refresh burst this PR removes.
_authed_call now snapshots the bearer before invoking the operation and
passes that exact token through to force_refresh_token.
desktop spawns multiple serve processes that share one rotating honcho
refresh token. force_refresh_token treated every 401 as "rotate now" even
when a sibling had already persisted a new grant, which can replay a
single-use refresh token and revoke the whole grant.
re-read honcho.json under the existing file lock and adopt if the token
that 401'd is no longer on disk. on invalid_grant, re-read once more
before marking the grant dead.
The retained comment still claimed a scope-less multiplex caller could load an
OSS config; identity reads above it raise UnscopedSecretError first. Both the
comment and the regression test now say the same thing: callers are scoped, an
OSS profile whose scope lacks MEM0_API_KEY initializes, and a scope-less caller
is a spawn-site bug that raises.
read_json_or_empty returns {} for malformed JSON; returning that directly from
_load_config made a corrupt profile config.json yield an empty (silently
unconfigured) mapping where the pre-dedup loop kept walking to the legacy
file and then the env branch. Only a non-empty parsed object is now returned.
Four bundled plugins wrapped get_hermes_home() in a try/except that fell back
to ~/.hermes on ImportError (a2a/protocol._hermes_home, photon/auth
._auth_json_path, google_chat adapter inline, openviking done in the previous
commit). A bundled plugin cannot lose hermes_constants -- each already imports
gateway.* / agent.* from the same tree -- so the fallback was dead code that
was also wrong on Windows (%LOCALAPPDATA%/hermes) and under a profile override.
telegram's gmail-triage verb path used Path.home()/".hermes" outright, ignoring
profiles. mem0/_oss_providers baked os.path.expanduser("~/.hermes/mem0_qdrant")
into VECTOR_PROVIDERS at import time, so the Qdrant default landed in the
user's ~/.hermes for every profile; the default is now a lazy callable resolved
by vector_default_config(provider_id) at setup time. a2a/security.py only
changes its import (it borrowed protocol._hermes_home).
Behavior change: on Windows and under profiles these paths now follow the
active HERMES_HOME (they were previously anchored to ~/.hermes in the impossible
fallback / at import time); the default-profile POSIX layout is unchanged.
Test: tests/plugins/test_plugin_paths_follow_profile.py asserts each resolver
(a2a conversations, photon auth.json, mem0 qdrant default, openviking log)
lands inside a HERMES_HOME ContextVar override; sabotage red for the mem0
import-time constant and the photon ~/.hermes fallback.
Three shims caught UnscopedSecretError and degraded to "" (mem0._scoped_env)
or to os.environ (langfuse._secret, azure_identity_adapter._scoped_env). Under
multiplex os.environ holds the DEFAULT profile's .env, so the langfuse/azure
fallback could ship another profile's keys, and the mem0 fallback silently
routed a mis-spawned turn's memories into the default profile's account. The
exception exists to surface exactly that spawn-site bug (agent/AGENTS.md:
never add environ fallthrough, never swallow it). All three now call
agent.secret_scope.get_secret directly: with a scope installed a miss returns
the default; single-profile deployments (multiplex off) still read the process
env inside get_secret; a scope-less multiplex caller raises.
Behavior change: a mis-spawned child under gateway.multiplex_profiles now
fails loud with UnscopedSecretError instead of running silently unauthenticated
/ on the default profile's identity. The #99121 contract (OSS mode needs no
MEM0_API_KEY in scope) is unchanged and its test now installs an empty profile
scope, which is the situation the issue described; a genuinely scope-less
caller is asserted to raise in a new test.
Tests: tests/plugins/test_scoped_secret_readers_fail_closed.py (scope wins over
environ; scope-less multiplex raises) for langfuse + azure, sabotage red when
the langfuse fallthrough is restored; tests/plugins/memory/test_mem0_v3.py::
test_load_config_fails_closed_without_scope_even_for_identity_settings,
sabotage red with a swallowing wrapper reinstated.
Six of eight memory providers spawned plain threading.Thread for prefetch/sync/
writer work. A plain thread starts with an EMPTY contextvars.Context, so under
multiplex profiles the worker resolved the DEFAULT profile's HERMES_HOME (and
fails closed on scoped secrets). honcho and hindsight had each noticed and
written their own copy_context() wrapper; core had a third in memory_manager.
One canonical pair now lives on the ABC module every provider already imports:
agent/memory_provider.py::ctx_bound / spawn_context_thread. memory_manager,
honcho, hindsight, mem0, retaindb, byterover, supermemory and openviking all use
it; the honcho and hindsight wrappers and memory_manager._ctx_bound are deleted.
Five "json.loads(path.read_text()) or {}" readers (mem0._read_mem0_json,
honcho client/oauth/cli _read_config, hindsight save_config/_load_config) fold
into utils.read_json_or_empty, the read half of every read-merge-atomic_json_write
sidecar store.
holographic.save_config was the only config.yaml writer in the tree that
bypassed hermes_cli.config.save_config: raw open("w") + yaml.dump with no config
lock, no managed-mode refusal, no atomic replace, and a swallowed exception. It
now calls save_config(..., merge_existing=True). Behavior change: a managed
install refuses the write (previously silently rewrote config.yaml); other
sections are deep-merged instead of round-tripped through a raw dump.
openviking._hermes_home_path guarded an impossible ImportError of
hermes_constants (the module already imports agent.*) with a ~/.hermes fallback
that is wrong on Windows and under profile overrides; it is replaced by
get_hermes_home() directly.
Tests: tests/plugins/memory/test_provider_threads_inherit_profile.py drives each
provider's real spawn path with a fake backend and asserts the thread sees the
spawner's HERMES_HOME override (sabotage: retaindb back on threading.Thread ->
red). tests/plugins/memory/test_holographic_save_config.py pins merge-with-
existing-sections and managed-mode refusal (sabotage: raw yaml.dump -> red).
Twelve modules each carried their own sqlite3.connect + PRAGMA + `with conn:`
stack. The #69567 fd-leak fix (a `with conn:` commits but never closes, so each
call leaked a connection and its WAL/SHM fds until GC) was pasted as code plus
docstring into six of them and hosted_room_policy_checkpoint never received
it; plugins/plugin_storage.plugin_db was the only production caller issuing a
raw `PRAGMA journal_mode=WAL`, bypassing the network-FS fallback, the
WAL-reset-bug gate and the never-live-downgrade invariant that
hermes_state_wal.apply_wal_with_fallback carries.
hermes_cli/sqlite_util.py (already home to add_column_if_missing/write_txn,
imported by cron, gateway and hermes_cli alike) gains `open_db(path, *,
db_label, busy_timeout_ms, wal, foreign_keys, synchronous_full, row_factory,
check_same_thread, wal_lock_retries, initialize)` and `transaction(conn,
immediate=)`; cron/ledger.py is deleted and hosted_rooms_common's
open_sqlite/connect/transaction become 1-3 line forwarders. Migrated:
agent/verification_evidence, cron/{executions,incidents,notepad,
delivery_queue}, gateway/{delivery_ledger,hosted_room_policy_checkpoint,
hosted_rooms_common (-> hosted_rooms, hosted_room_driver)}, hermes_cli/
projects_db, tools/async_delegation, plugins/plugin_storage.
Behavior changes (each module keeps its effective PRAGMA set otherwise):
- hosted_room_policy_checkpoint: connection now closed after every use and
on init failure (was leaked per call), busy_timeout PRAGMA set explicitly.
- projects_db: gains busy_timeout=5000 (was the sqlite3 default 5 s connect
timeout with no PRAGMA); explicit and observable.
- delivery_ledger / async_delegation: busy_timeout PRAGMA now mirrors the
10 s connect timeout they already had.
- plugin_storage.plugin_db: WAL through apply_wal_with_fallback (DELETE on
network filesystems / WAL-reset-vulnerable builds instead of raw WAL);
busy_timeout=5000.
- cron/incidents._redact_error: redact_sensitive_text(force=True) — the
error text is persisted to disk.
- delivery_ledger's private duplicate-column guard and the unguarded
`ALTER TABLE ADD COLUMN` sites (shared_metrics, api_server_run_idempotency,
holographic store, kanban model_override) go through add_column_if_missing.
- hermes_state.py::_scrub_surrogates: dead byte-copy of
hermes_state_messages._scrub_surrogates (0 callers) deleted.
The a2a invariant test skipped any synthesized token the canonical
redactor itself let through, so a boundary/shape regression would have
passed silently. Every class scrubs today, so the escape hatch goes and
the soft ">= 50" count becomes the exact registry size.
The gateway body test still asserted a "[REDACTED]" fallback marker that
no longer exists; redact_for_egress masks via _mask_token ("***"), so
assert that alone. honcho oauth.py's `re` import became unused when its
private pattern list moved to the registry.
Six independent line-parsers with three different quoting/comment semantics
read the same .env files: tools/skills_tool.load_env (strip("\"'"), no inline
comments), hermes_cli/managed_scope._parse_env (same, no export, no BOM),
web_server_cron._profile_env_value (plain utf-8, no BOM), profile_cmd
._env_file_has_key, env_loader._env_keys_defined_in_dotenv (utf-8, so a BOM'd
first key stayed "\ufeffKEY" and the dashboard profile scrub missed line 1),
mem0/_setup._prompt_api_key (startswith scan, no quote strip). The boundary
parsers (scrub key set, skill secret capture) therefore disagreed with the
parser that installs the profile scope.
Now every one is a 1-3 line forwarder onto load_env_file, and
hermes_cli.config.load_env is memo over it (public signature unchanged).
_parse_env_value moves next to its only caller in secret_scope.
load_env_file gains the same latin-1 fallback env_loader uses to install
into os.environ, so a mis-encoded file yields the same key set on both sides.
Managed .env keeps its fail-LOUD contract (decode error logs and ignores the
file) instead of load_env_file's fail-soft {}.
Behavior change: managed .env, skills_tool and mem0 setup now honour
`export`, quoted-value escapes and inline comments the way the profile scope
does; web_server_cron and the dashboard scrub tolerate a BOM.
Invariant test: a BOM'd/export/quoted/commented .env yields the same key set
via load_hermes_dotenv (installer), load_env_file (scope) and
_env_keys_defined_in_dotenv (scrub); fails with the old scrub parser.
plugins/platforms/a2a/security.py::redact_outbound shipped text to a REMOTE peer
through 8 private regexes (sk-, sk-ant-, ghp_ only, xox[bap] only, AKIA, JWT,
Bearer, email) and never called redact_sensitive_text, so every prefix added to
agent/redact.py (hf_, glpat-, xapp-, npm_, Telegram bot tokens, private keys,
DB URLs, env assignments, auth headers, plugin-registered patterns) was absent
on the A2A path. gateway/run.py::_GATEWAY_SECRET_PATTERNS and
agent/monitoring/redaction.py::_TOKEN_RE/_BEARER_RE were two more parallel
"fallback" lists to maintain.
Now agent/redact.py::redact_for_egress is the one egress scrub:
redact_sensitive_text(force=True) + a bearer sweep for prefix-less opaque
tokens, fail-closed ("[redaction-unavailable]"). Gateway user-facing text,
monitoring export and A2A outbound call it; A2A keeps only its e-mail pass.
Behavior changes: a2a egress now masks the full canonical set; the gateway
chat path returns the fail-closed sentinel instead of a raw string when the
redactor raises; honcho plugin registers hch-at-/hch-rt- with
register_redaction_patterns (masked on every surface; mask shape is the
shared head/tail form instead of "hch-at-[redacted]"); proxy_cli token
display uses mask_secret (4 visible prefix chars instead of 12).
Invariant test: redact_outbound masks a synthesized token for every
registered prefix pattern (fails when reverted to the private list).
_truncate_for_sync documents "the last sentence boundary within max_len", but it
looped over separator KINDS and returned on the first kind that qualified. An early
"。" therefore outranked a "." 240 characters later, and in pure ASCII "." outranked
a later "!" or "?" purely because it comes first in the tuple.
With the 450-char default, "a"*200 + "。" + "b"*240 + "." + "c"*100 kept 201 of the
442 characters available: 241 characters the embedder would have accepted were
discarded, so any fact in the second half of the turn never reached extraction. The
add() call succeeds, so unlike #106235 nothing is logged — the turn is simply
remembered from its first sentence. Raising sync_max_chars widens the gap rather
than closing it.
Take the max over every separator instead, from a named tuple so the set is not
buried in the loop. ".\n" is dropped: its index can never exceed the bare "." it
starts with, so under a max it is unreachable. The first-third guard and the hard-cut
fallback for unsegmented input are unchanged.
Fixes#108868
Ramp Router efforts cache + warm/disk flags, xAI and OpenRouter image catalogs,
Hindsight append-capability verdict, memory-provider skill registry, OpenViking
atexit provider, Honcho loopback flow status, Langfuse client (os.environ-only
credentials) and disk-cleanup's protected cron paths held one profile's
credential- or home-derived value process-wide; YuanbaoAdapter._active_instance
was last-connected-wins across profiles.
Keyed by home key / credential fingerprint under an override, credentials read
through the secret scope, warm threads run under copy_context(); unscoped module
slots stay for the single-profile path and the existing monkeypatch tests.
Under gateway.multiplex_profiles, os.environ holds the DEFAULT profile's .env; a
secondary profile's values exist only in the per-turn secret scope. Every reader
below still read os.environ/os.getenv at call time, so a secondary profile's turn
silently used the default profile's value.
Credentials (F6): FIRECRAWL_API_KEY (read_file hosted OCR), OPENVIKING_API_KEY,
mem0-OSS OPENAI_API_KEY, MODAL_TOKEN_ID/SECRET and BROWSER_USE_API_KEY presence
gates, and the xAI video plugin's os.getenv("XAI_API_KEY") fallback AFTER the
scoped resolver had already missed — the exact fallback-after-miss shape
gateway/AGENTS.md forbids. Deleted, not re-scoped: the resolver is the scope.
Identity / tenant (F7): MEM0_USER_ID/AGENT_ID/HOST/MODE, SUPERMEMORY_CONTAINER_TAG,
RETAINDB_PROJECT, OPENVIKING_ACCOUNT/USER/AGENT (and the whole layered() env
read), HINDSIGHT_BANK_ID/MODE/retain shaping, HERMES_HONCHO_HOST. A raw read
put a secondary profile's memories into the default profile's account/bank/
project/tenant and recalled them back into the default's turns. Each now uses
get_secret with the provider's own per-profile default on a miss.
Endpoints (F8): OPENAI_BASE_URL (aux custom runtime + direct-alias expansion),
XAI_BASE_URL/HERMES_XAI_BASE_URL (aux OAuth), NOUS_INFERENCE_BASE_URL (#65941,
both the aux builder and hermes_cli.auth_nous._nous_inference_env_override),
GATEWAY_PROXY_URL (same UnscopedSecretError-only fallback shape as
GATEWAY_PROXY_KEY three lines below), FIRECRAWL_API_URL, BROWSERBASE_BASE_URL,
SUPERMEMORY/RETAINDB/HONCHO/HINDSIGHT URLs. The keys beside them were already
scoped, so a secondary's key was sent to the default profile's proxy or host.
Targets / display (F11): WEIXIN_HOME_CHANNEL (message posted into the default's
chat), HERMES_LANGUAGE, and agent/i18n's process-wide lru_cache of
display.language — now keyed by HERMES_HOME.
Outbound webhooks: hooks.outbound[].secret_env resolved from os.environ while
the gateway registers each profile's targets inside that profile's scope, so a
secondary's deliveries were signed with the default's secret or left unsigned.
Agent-cache eviction: _spawn_release_thread started a bare threading.Thread, so
commit_memory_session -> provider on_session_end ran with an EMPTY context. The
thread now runs copy_context() and, for the unscoped housekeeping sweep, enters
the owning profile's _profile_runtime_scope resolved from the session key
(agent:<profile>:...). The pressure batch does the same per key.
session_search (#82903): agent/inline_tool_executors.py::_session_search
forwarded every schema argument except `profile`, so a gateway agent could
never select a named profile's store. Forwarded; the ownership-scoping design
in #87779/#87847 is a separate design call and is not attempted here.
Live repro (/tmp/mux_audit/fix-tool-memory-reads/repro.py): 28 FAIL on
origin/main -> 0 FAIL with this change; 10 new invariant tests red on base.
Fixes#82903Fixes#65941Fixes#99121
Addresses #87779
Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Co-authored-by: Michael Versluis (Berry) <michael@wve.nl>
Conflict resolution on top of main also routes MEM0_MODE / MEM0_HOST / MEM0_AGENT_ID /
MEM0_USER_ID through get_secret: identity and host are .env values like the key, and a
raw environ read hands a secondary profile the default profile's mem0 account.
(cherry picked from commit 7a5863ebafdb193d8ec2f89a7aca309774d6d69a)
The vault item is named HINDSIGHT_API_LLM_API_KEY (matching the daemon env var), while the setup wizard uses HINDSIGHT_LLM_API_KEY. Vault-fed scopes carrying only the API-prefixed name resolved empty, so the scopeless path — and the whole fail-closed machinery — never engaged on real deployments. Accept both names.
Proven end-to-end: hydrated scope now resolves the 73-char key and the rewrite gate passes.
(cherry picked from commit 2ce8c2980188ae8e4a3ed398e14fa8ed89a40644)
The embedded daemon-start worker usually runs with no secret scope, so _embedded_llm_api_key resolved empty and both the profile-env compare and the upstream manager merge overwrote the file's good key with emptiness.
- _embedded_llm_api_key: explicit config, then secret scope, then on-disk profile env fallback; UnscopedSecretError treated as empty without touching os.environ (multiplex safety).
- Add _may_rewrite_profile_env gate: refuse rewrite on keyless-build plus keyed-disk, and skip the daemon stop in that case.
- Document key resolution order in plugins/memory/hindsight/README.md.
- Add 5 tests pinning disk fallback, the gate's refuse branch, rotation, keyless providers, and file survival.
(cherry picked from commit 48a4bcd536fa799b666a8d5eaae73c1816ca9e33)
