Commit Graph

6403 Commits

Author SHA1 Message Date
kshitijk4poor
a606a4f94b refactor(context-refs): read HOME_CREDENTIAL_DIRS directly for the @file block
Fold2 PROFX finding 1 (reuse/quality): `_SENSITIVE_HOME_DIRS` had become a
bare alias of agent.file_safety.HOME_CREDENTIAL_DIRS with one reader. Drop
the alias and use the shared constant at that reader so there is one name
for the list. No other references exist in the repo (tests/ and plugins/
included).
2026-10-04 13:53:34 +05:30
kshitijk4poor
625fab3fde fix(context-refs): derive the @file credential-dir block from HOME_CREDENTIAL_DIRS
Gate r1 (2c note): agent/context_references._SENSITIVE_HOME_DIRS was a hand
copy of the home credential dirs and missed .config/gcloud, so
`@file:~/.config/gcloud/application_default_credentials.json` expanded the
gcloud refresh token into context; the canonical read deny-list does not
cover it either. The gateway feeds untrusted remote text through this
expansion, so this is the same leak class the profile-export fix closes.

_SENSITIVE_HOME_DIRS now is agent.file_safety.HOME_CREDENTIAL_DIRS, the
single list the write denylist and profile export already derive from.
file_safety imports stdlib only, so there is no import cycle.

The existing canonical-denylist test gains a .config/gcloud ADC row.

Co-authored-by: Keiran Haax <196126709+keiranhaax@users.noreply.github.com>
2026-10-04 13:53:34 +05:30
kshitijk4poor
404b1fec55 fix(profiles): keep .docker/.azure/.config/gh/.config/gcloud out of profile exports
#132201 stopped profile export shipping .ssh/.aws/.gnupg/.kube/.envrc, but
the other credential dirs the file tools already write-deny still shipped:
a skill dir copied from a home carries .docker/config.json (registry auth),
.azure/accessTokens.json, .config/gh/hosts.yml (GitHub token) and
.config/gcloud/ (ADC / credentials.db), and the default-profile export plus
named exports archived them verbatim (binary DBs skip the redact pass).

The home-relative credential dirs now live once in
agent.file_safety.HOME_CREDENTIAL_DIRS; build_write_denied_prefixes and the
profile export (_OS_CREDENTIAL_STORES) both derive from it, so the write
denylist and the export filter cannot drift. Nested stores (.config/gh) are
matched on trailing path components, so only .config/gh and .config/gcloud
are dropped and the rest of .config still ships. .config/gcloud is included
for parity with file_safety. Distribution installs treat the same paths as
installer-owned via PROFILE_CREDENTIAL_PATHS.

Co-authored-by: Keiran Haax <196126709+keiranhaax@users.noreply.github.com>
2026-10-04 13:53:34 +05:30
teknium1
8e3b5b5446 fix: agent-start left-core migration tolerates agents without a startup-warning sink
With the try/except gone, stub agents (SimpleNamespace in tests/agent/test_system_prompt.py)
raised AttributeError on the say= argument before recover_at_startup ran.
2026-10-04 00:08:20 -07:00
teknium1
c63bc3dcbd fix: every profile a multiplex gateway serves gets its left-core plugin before its adapters start
Gateway start ran the left-core migration only for the launch home, so a secondary profile that
used Home Assistant as its event platform started without the plugin and without its adapter (and a
named launcher's default-root primary was never migrated). Each secondary now runs the migration in
its own runtime scope before its plugins are discovered, and gateway start covers the launch home and
the default root (the multiplex primary). The gateway-start helpers live in gateway/run_startup.py;
run.py and agent_init.py lose the try/except wrappers, since recover_at_startup never raises.

An agent-start install of a row that ships a platform now tells the user to restart the gateway
when one serves the home: the running gateway loaded its platforms before the plugin existed.
2026-10-04 00:08:20 -07:00
teknium1
fb9fc8a3dd refactor: move Home Assistant out of core into the homeassistant catalog plugin
The Home Assistant gateway platform (plugins/platforms/homeassistant) and the
homeassistant toolset (tools/homeassistant_tool.py) now ship as the official
catalog plugin `homeassistant` (NousResearch/hermes-homeassistant), with the
same platform name, toolset/tool names, config keys and env vars.

Migration (hermes_cli/left_core_migration.py, a generic "left core" table
reusing the memory-provider migration's helpers): every profile home that
used Home Assistant (HASS_TOKEN in its .env, platforms.homeassistant enabled
or holding a token, or the homeassistant toolset selected) gets the plugin
installed from the catalog at its reviewed pin by `hermes update`, and lazily
once per process at agent start / gateway start (honours
security.allow_lazy_installs). Every outcome reaches terminal, Desktop and
chat; a gateway-start outcome is handed to that home's first agent.

Core HA special cases become generic PlatformEntry seams the plugin uses:
trusted_inbound (allowlist bypass for event-bus platforms), display_tier
(built-in display defaults) and shared_env_prefixes (profile-clone ownership
rule for env shared with the plugin's tools). Cron delivery uses the existing
cron_deliver_env_var seam; HASS_TOKEN stays stripped from subprocesses via the
installed plugin's manifest secrets.
2026-10-04 00:08:20 -07:00
kshitijk4poor
5762cce4c9 docs(compression): drop stale split-turn clause from task-snapshot comment
Gate r1 (2c Low): the comment in _latest_user_task_snapshot claimed the
split-turn gate measures the same authored text, which no longer holds.
Keep only the reason the reply quote is dropped first. No net line growth.
2026-10-04 12:02:15 +05:30
Brian Le
41c6729421 fix(compression): split oversized turns whose request is long
The oversized-active-turn split (#80449) refuses to fire when the
authored turn-opening request exceeds 1,400 characters. A long /goal
continuation prompt therefore pins its entire autonomous run in the
protected tail: every automatic pass finds an empty compress window,
records no_progress, arms the 300 s structural backoff, and the session
grows into the provider context limit. /compress hits the same empty
window.

The token soft ceiling already bounds the request row, and the split
request is restated verbatim after the handoff by
_reappend_inflight_user_task, so the character cap only blocks
progress. Drop it from the split gate. The historical task snapshot
keeps its own cap and reply-quote handling.

(cherry picked from commit 0081611ff1f088ae0cca86d916ee802e74fe465a)
2026-10-04 12:02:15 +05:30
teknium1
64ad33e32d refactor: remove Skill Sync (hermes sync) and the org skill mirror from core
Skill Sync only worked against a Nous Portal sync plane that was never made
publicly accessible, so no install has ever used it. We don't want it in core
right now, so this deletes it outright instead of moving it behind a plugin
hook surface nobody needs yet:

- `hermes sync` command, `tools/skills_sync_client*.py` and their tests
- the personal pull ticks (gateway housekeeping, classic CLI startup,
  `hermes serve` maintenance) and the debounced post-write push
- the per-skill `sync` usage flag (`set_sync` / `is_sync_enabled`)
- the `skills/_org/<org_id>/` mirror: index gating on `.active_org`, the
  `[org-shared]` / name-collision labels, the org provenance header and
  `org_provenance` field on skill_view, the org delete guard and the
  auto-propose `org_sharing` note on skill_manage

The code is preserved in the private NousResearch/hermes-skill-sync repo if
the feature comes back.
2026-10-03 23:05:48 -07:00
kshitijk4poor
df25bbf469 refactor(skills): one disabled rule in the prompt index; hoist prefix import
r2 gate (quality, Low cleanups):
- is_disabled_entry: drop the dead `bool(disabled) and`, because isdisjoint
  on an empty set already gives False.
- agent/skill_commands.py: import AMBIGUOUS_SKILL_PREFIX once at module
  level instead of inside two adjacent functions. There is no cycle:
  agent/skill_utils.py imports neither skill_commands nor prompt_builder.
- prompt_builder hides(): drop `frontmatter_name in disabled`. The
  post-resolve is_disabled_entry already matches the declared name, and
  hidden copies still take part in resolution, so the index output is
  unchanged. The bare `skill_name` (dir name) term stays: it is base
  behaviour that list/view never had, and it is not changed here.
2026-10-04 02:54:56 +05:30
kshitijk4poor
b82b4ac65c fix(skills): a disabled load path hides only the duplicate it names
r2 gate (2c, Low): `skills.disabled: [a/one]` — what `hermes skills` / the
web toggle save for the same-tier duplicate shown as `a/one` — matched every
copy's relative path, so an unrelated, uniquely named skill in another tier
at the same relative path (external `e1/a/one` named `other`) vanished from
list, index, view, preload and cron while the config UI showed it enabled.

is_disabled_entry now matches the declared name or the entry's load_name
(the relative path only for an ambiguous same-tier row). skill_view applies
the path rule only when that path is the resolved copy's catalog load_name.
This is the reviewer's intent but not the literal `name == rel` check: that
check would let the disabled a/one copy load through its directory alias
(`one`), which r2 verified is refused. The extended test pins both cases.
2026-10-04 02:54:56 +05:30
kshitijk4poor
719f020046 refactor(skills): clear_skills_cache() for reload; fix stale candidate docstring
Gate quality findings (Low): reload_skills reached into
tools.skills_tool._SKILLS_CACHE directly -- add clear_skills_cache() next
to the cache and call it. _collect_skill_candidates still said "the
caller refuses >1", but after the precedence change the caller resolves
candidates via pick_skill_candidate and only refuses same-tier ties.
2026-10-04 02:54:56 +05:30
kshitijk4poor
4e4027ae41 refactor(skills): prompt index takes tiered roots from get_skill_search_roots
Gate reuse+quality finding (Low, convergent): build_skills_system_prompt
took get_all_skills_dirs()[1:] plus get_project_skills_dirs() and the
inner builder re-derived each root's tier (TIER_CREATE_DIR if it equals
get_skill_create_dir() else TIER_EXTERNAL) -- a second copy of the "one
ordering" the stack introduced. Pass the non-local (tier, dir) pairs from
get_skill_search_roots straight through instead. Both probes print
identical output before and after.
2026-10-04 02:54:56 +05:30
kshitijk4poor
905ef4efdf refactor(skills): one same-root rank for skill_view and the catalog
Gate reuse+quality finding (Low, convergent): resolve_skill_catalog
hand-copied skill_view's same-root rank as (False, len(parts) + 1), so
listing/slash and skill_view agreed only while two tuples stayed in sync.
Move the rank to agent.skill_utils.skill_candidate_rank(skill_md, root)
and call it from both; _rank_same_root_candidate is gone. Both probes
(gate fixture + 2c fixture) print identical output before and after.
2026-10-04 02:54:56 +05:30
kshitijk4poor
de0e8b3bb4 perf(skills): memoize the shadow identity decision, not only the warning
Gate efficiency finding (Low): resolve_skill_catalog only remembered a
shadowed path once it had warned, so a byte-identical (non-symlink) copy
of a higher-tier skill -- e.g. an external_dirs mirror of local skills --
was re-read and SHA-256-hashed against every owner on each catalog
rebuild / prompt-index miss. Key the memo by (shadowed path, *sorted
higher-tier paths) and record it whether or not a warning fires.
Probe: provably_same_skill calls over 5 resolves of one identical
shadowed copy 5 -> 1. Warn-once behaviour is unchanged.
2026-10-04 02:54:56 +05:30
kshitijk4poor
1d85eaa240 fix(skills): unloadable-copy note names the tier, not one directory
Gate 2c finding (Low): the prompt index note for a copy skill_view can't
load said it collides "with a different skill in the same skills
directory", but the common trigger is two external_dirs that both hold
productivity/xdup -- two directories in one precedence tier. Say "the
same skills directory tier (e.g. another external_dirs entry)", matching
skill_view's refusal text.
2026-10-04 02:54:56 +05:30
kshitijk4poor
fb6b900bc0 fix(skills): partial preload warning keeps the Ambiguous wording
Gate 2c/quality finding (Low): when some -s skills loaded, cli.py,
hermes_cli/oneshot.py and tui_gateway/server.py still logged "Unknown
skill(s) requested, skipping: Ambiguous skill name dup-demo: ..." --
calling an ambiguous name unknown, the exact wording 3b6f5eb813 removed
from the all-missing error. All three now log format_missing_skills(),
and the "Ambiguous skill name " prefix lives in one constant
(agent.skill_utils.AMBIGUOUS_SKILL_PREFIX) shared by the skill_view
error, the preload/cron label and format_missing_skills' classifier.
2026-10-04 02:54:56 +05:30
kshitijk4poor
a08d099059 fix(skills): skills.disabled also matches a duplicate's exact load path
Gate 2c finding (Medium): a same-tier duplicate is listed under its exact
relative path (`a/one`), so `hermes skills` and the web toggle save that
path into skills.disabled -- but the list filter, skill_view and the
prompt index only matched the declared name (`dup-demo`). Disabling the
row silently did nothing while the UI showed it disabled.

One helper, agent.skill_utils.is_disabled_entry(entry, disabled), matches
the declared name OR the relative path; the catalog (skills_list, config
UI, slash commands) and the prompt index use it, and skill_view also
refuses when the resolved copy's exact path is disabled.
2026-10-04 02:54:56 +05:30
kshitijk4poor
7025dd1cee fix(skills): preload and cron say "Ambiguous" instead of "Unknown"/"not found"
`hermes -s dup-demo` (CLI, oneshot, TUI/kanban worker) died with "Unknown
skill(s): dup-demo" and cron logged "skill not found, skipping" for a name that
exists twice. skill_view's refusal now carries load_names; preload labels the
failed identifier "Ambiguous skill name X: use one of <paths>" (the label is
computed only on the failure path) and format_missing_skills() keeps that
wording in the raised error; cron skips with the same message and lists it in
the job's skipped-skills notice.
2026-10-04 02:54:56 +05:30
kshitijk4poor
340be3750d fix(skills): slash commands register only names that resolve uniquely
scan_skill_commands kept the first copy of a duplicated name (project > local >
external, no create_dir), so /dup-demo silently ran a/one while skill_view
refused dup-demo. Build the map from the shared skill catalog instead: only
`unique` entries get a /command; same-tier duplicates stay reachable by exact
path (`/skill a/one`), shadowed copies never register. /reload-skills also
clears the catalog cache so a reload never serves a TTL-stale view.
2026-10-04 02:54:56 +05:30
yu-xin-c
a24d7382d3 fix(skills): list and prompt index advertise only names skill_view loads
skills_list / `hermes skills list` kept the first copy of a duplicated name, the
prompt index listed every copy under the same bare name, and skill_view refused
that name — so the model was told to load identifiers that could never load.

Add agent/skill_utils.resolve_skill_catalog(): every scanned skill is matched
against the aliases skill_view accepts (declared name, directory name, path
relative to its root) and the shared pick_skill_candidate() rule, yielding a
status (unique / ambiguous / shadowed) and the exact load_name. The list
(tools/skills_tool._skill_catalog -> _find_all_skills, `hermes skills list`) and
the prompt index (project, local, create_dir and external roots resolved
together) both render load_name: a same-tier duplicate appears as `a/one`,
`b/two`; a shadowed lower-tier copy is hidden with a one-time warning; a copy
whose name AND path are both shared is omitted with a rename note. `hermes
skills list` also labels project / create_dir / external rows instead of
"local". Prompt snapshot version bumped to 4 (entries gain `rel`); ordering
stays deterministic.

Rebuilt from ef3d8ba385e (#64462) onto current main: the resolver keeps the
original alias-ownership design, adds tier precedence, and runs over the
refactored prompt_builder collection path.
2026-10-04 02:54:56 +05:30
Tal Orbach
f525ecdb9c fix(skills): skill_view applies cross-directory precedence instead of refusing
skill_view refused any name that matched in more than one skills directory, so a
local skill shadowing an external one (even at the same relative path, e.g.
productivity/xdup) could not be loaded by ANY identifier, while skills_list, the
prompt index and slash commands all showed the local copy.

Resolve by one tier order everywhere: project > local > skills.create_dir >
external_dirs. get_skill_search_roots() in agent/skill_utils.py is now the single
directory ordering (get_all_skills_dirs and tools.skills_tool._skill_search_dirs
both derive from it; the latter previously omitted skills.create_dir). The tier
is classified from the live profile root (_skills_dir()), not the import-time
SKILLS_DIR. Two DIFFERENT skills sharing a name inside one tier still refuse
(59da8ec4e); the refusal now names the exact paths to load instead.

Rebuilt from aba7772261f (#55634) onto current main: the original preferred a
unique local candidate over external_dirs only; this generalises it to the full
tier order and moves the same-root identical-copy ranking into the shared
pick_skill_candidate() helper.
2026-10-04 02:54:56 +05:30
Brooklyn Nicholson
f10a763d9b fix(agent): stamp the activity clock at turn end and dump thread stacks on watchdog fire
The #131740 residual: a Desktop turn wedged between the 'Turn ended' log
line and run_conversation returning. The liveness watchdog fired ~600s
later naming 'API call #8 completed' as the last activity - a stamp from
the last provider response - so the stall surface read like a hung API
call, nothing pinned the blocking frame, and the abort hit a turn whose
loop had already finished.

- finalize_turn stamps the activity clock at 'Turn ended'
  (_log_turn_exit): a stall in the post-loop tail is measured from when
  the loop actually ended, and the watchdog's 'last activity' names the
  finalizer instead of a stale API call. Guarded: a broken
  activity-clock seam never raises into the finalizer.
- TurnLivenessWatchdog._surface_stall dumps every thread's stack (turn
  thread marked, 40-thread cap) at the moment it fires, so the wedged
  frame lands in errors.log on the next occurrence without py-spy.

Refs #131740
2026-10-03 16:18:04 -05:00
kshitijk4poor
91586d448c refactor(tui_gateway): keep the session.create tier parse out of the oversized handler module
methods_session.py is already past 2000 lines, so the exact-tier parse that
#130990 grew in place moves with the rest of the composer overrides into
methods_session_model_guard.py (the session.create param gate), and the
strict "unknown word is an error, normal words pin ''" rule becomes
agent.fast_mode.parse_exact_service_tier next to the tier word table it
reads. The Daybreak/cyber exclusion folds into _is_openai_fast_model's
single return so hermes_cli/models.py does not grow.

Drops the PR's unused contextlib import in prompt_turn.py (no use was
added with it).

Co-authored-by: FroRaut <79215537+Froraut@users.noreply.github.com>
2026-10-04 02:12:57 +05:30
teknium1
7e19f4138b fix(telemetry): record auxiliary model calls with their real outcome
Every failed auxiliary model_route row read error_class=unknown because the
aux logical scope only carried outcome/model/provider and the projection
hard-coded unknown for any non-success. Every exception escaping call_llm,
including Hermes' own aborts (Ctrl+C / /stop mid-compression, interrupted
aux streams, task cancellation), was recorded as failed.

- classify the error that ended an aux call (and the last one a successful
  call recovered from) with agent.error_classifier; HTTP-200 bodies that
  carry a provider error object are classified from that object
- Hermes aborts (non-Exception BaseExceptions, InterruptedError) close the
  call as cancelled/none, matching the contract
- MoA aggregator streams report the same way
- aux calls that finish under the turn's own live scopes (title generation
  runs beside the turn) were never counted: the scope could not be popped
  and the turn-close drain closed it as an orphan without a route. The
  drain now closes it with the stored real result.
2026-10-03 13:38:29 -07:00
kshitijk4poor
26f0ed2e42 refactor(debug): reuse agent.redact lists in support redaction
The support scrub re-listed agent.redact's sensitive query params and
secret header names; two copies of one pattern list drift. Derive the
support-only compact names from _SENSITIVE_QUERY_PARAMS (plus the five
support-only extras) and expose the header names as a tuple that both
the agent regex and the support module build from. agent.redact is a
hard dependency, so import it at module level and drop the try/except
around is_secret_field_name.
2026-10-03 21:15:17 +05:30
kshitijk4poor
2667c960d5 fix(compressor): don't move a merged tool-call carrier during replay dedup
Gate r3 finding 2c: when _summary_placement merged the handoff into
tail[0] and that row is an assistant(tool_calls), the dedup's
move-carrier-to-window-start step popped it away from its tool results
(broken pairing) and still left an assistant row first.

Snapshot the window before the dedup filter; if the filter removed a row
and the carrier is a tool-call row (_template_visible_role is None),
restore the snapshot and skip the dedup this cycle. The output then
matches the pre-stack layout for that shape; the standalone-carrier
move-and-flip path is unchanged.
2026-10-03 18:23:16 +05:30
kshitijk4poor
d20a55cb5c fix(compressor): keep the window opening on a user turn after replay dedup
Gate finding 2ab (BN05 final): when the deduped in-flight original was the
first non-system row (cron / first-compression shape), removing it left the
window opening system -> assistant(tool_calls) -> tool -> carrier. The
carrier-flip guard judges template-visible roles, which skip tool_calls
rows, so it saw nothing wrong; native Gemini maps that leading row to a
model functionCall turn, which its API rejects as the first turn.

After the dedup, if the first non-system row is not a user row, move the
carrier there (the existing flip then makes it role=user). The kept test now
asserts the raw role of compressed[1], since the template-visible check
cannot see a leading tool-call row.
2026-10-03 18:23:16 +05:30
kshitijk4poor
876e9740de refactor(compressor): key the in-flight replay dedup on message_uid_or_none
Gate finding (reuse + quality, Low, convergent): the replay-uid block
hand-rolled agent.message_metadata.message_uid_or_none (non-empty str or
None) and repeated the "message_uid" literal that MESSAGE_UID names. Use
the helper and the constant, hoisting the function's existing lazy
message_metadata import (record_absorbed_message) so it covers both.
Behaviour-equivalent: same uid acceptance, same row filter.
2026-10-03 18:23:16 +05:30
kshitijk4poor
2ceca30d57 refactor(compressor): test the carrier's predecessor role with != "user"
Gate finding (quality, Low): `in {None, "system", "assistant"}` lists every
value _last_template_visible_role yields in a compacted window except
"user", so the set spelled out a complement. `!= "user"` states the actual
condition (no visible user turn precedes the carrier). No behaviour change.
2026-10-03 18:23:16 +05:30
kshitijk4poor
7b3fc99ce8 fix(compressor): keep alternation when the in-flight replay drops the head user row
When the in-flight request survived in the protected head, the replay now
removes that original (one active row per message_uid). But the summary
role was chosen against a head that still ended on that user row, so the
compacted window then opened system -> assistant(tool_calls) -> tool ->
assistant summary: no user turn before the carrier, which strict
alternation templates (#58753 class) reject.

When the removal leaves the assistant carrier with no preceding visible
user, switch to the _force_user_leading layout: carrier role=user with the
request merged after _SUMMARY_END_MARKER (the existing merge path records
the uid as absorbed, so it stays single).

Co-authored-by: Yuan Li <dskwelmcy@163.com>
2026-10-03 18:23:16 +05:30
Yuan Li
a14e929ec0 fix(agent): replace duplicate in-flight task replay
(cherry picked from commit 581719d1daed7a6e6ce804ff756b6b383de379bd)
2026-10-03 18:23:16 +05:30
kshitijk4poor
b1cd88a689 refactor(compression): hoist the commit's helper imports above the in_place split
Gate quality (BS02 r1, Low): the rotation branch re-imported
rejoin_compressed_head_and_tail and stamp_db_persisted_markers locally, which
the in-place branch already imports. Import both once above `if in_place:`
(net -2 lines for the file). The duplicated test setup the same finding
named belonged to the fallback test removed earlier in this stack, so no
helper is needed. No behaviour change.
2026-10-03 18:22:19 +05:30
kshitijk4poor
b75c29d9b4 fix(compression): publish the /compress here N tail with a rotated child
Rotation published only head + summary; compress_now rejoined the kept
verbatim tail in memory and the gateway's second rewrite was what made it
durable. With that rewrite now skipped for a published child (it deleted
the atomically cloned foreign tail), a rotated `/compress here N` lost the
kept exchanges from the child. CLI/TUI/ACP never rewrote, so they already
persisted head-only.

Publish rejoin_compressed_head_and_tail(compressed, verbatim_tail) in the
rotation handoff exactly like the in-place branch stores it, and stamp the
tail rows so compress_now skips its in-memory rejoin. Fixes every surface.
2026-10-03 18:22:19 +05:30
kshitijk4poor
05f3f25d67 refactor(compression): name the repair row stamps by their constants
gateway/run.py copied the merged/unnamed/retired durable-row stamps by
string literal; a rename of the constants in conversation_compression_archive
would silently stop the gateway replay from carrying them. Loop over the
constants instead.

The in-place tail widening in _commit_compaction counted
`_absorbed_row_ids` by hand under a comment about merged users only. Since
the stack, that list also holds the orphan tool/assistant rows the repair
drops, so reword the comment, and use ABSORBED_ROW_IDS and _positive_id
(which also rejects bools) rather than re-rolling them.

The dropped-tool-call repair test asserts the exact repair count (1)
instead of hedging with >= 1.
2026-10-03 18:17:35 +05:30
unsupportedpastels
3f04ecb789 fix(compression): preserve exact coverage through composed repairs
(cherry picked from commit ff3e655f1e9cc8182dbb06df3922fb3b341225b2)
2026-10-03 18:17:35 +05:30
unsupportedpastels
da88c23c02 fix(compression): a row the repair drops is named, not a reason to fall back to the watermark
On a reload without row ids, a dict that stood for rows the alternation repair
dropped or folded made coverage_for_commit return (None, None). The watermark
path then archived every active row below it, including rows another surface
appended that the compressor never saw: they left the model's history without
being in the summary (the unseen-gap contract of 1674499d00).

The repair now records role, content and tool-call ids of each row it retires
without an id, and of an assistant survivor before a fold rewrites its text.
The commit matches each to exactly one active row at or below the watermark and
keeps exact coverage; a retired row widens the carried-tail rewind like a merged
row. An ambiguous or missing match, or a count the recorded rows do not cover,
still takes the watermark path.

The approach of stashing id-less dropped rows as unresolved held coverage comes
from #129162.

Co-authored-by: JerryLiu369 <953639086@qq.com>
(cherry picked from commit 312474b424ede18f01433860309d23526c08e6b3)
2026-10-03 18:17:35 +05:30
John Paul Soliva
6eb90cac31 fix(compression): a row the repair drops ahead of the first survivor is accounted for
The repair retires a dropped row onto the survivor before it. With nothing kept
before it (a stray tool result or an unanswered tool call as the first active row)
the row stayed unnamed, and an in-place compaction cloned it behind the compacted
set. The first dict the pass keeps now stands for the rows dropped ahead of it, by
row id or by count.

The leading-drop case comes from #129162.

Co-authored-by: JerryLiu369 <953639086@qq.com>
(cherry picked from commit 2facd175120acfd99b3e566bd1647e0f4b7767fb)
(cherry picked from commit 90faa5025ba4fd88a8cce967856c592f23776f27)
2026-10-03 18:17:35 +05:30
John Paul Soliva
bbacab4201 fix(compression): the repair's row counts are persistence-only message fields
`_merged_durable_rows` and `_unnamed_durable_rows` ride on live history dicts so an
in-place compaction can count the durable rows behind a repaired dict. The request
copy built in turn_context kept both keys. The four in-tree transports drop them
while converting, but membership in PERSISTENCE_ONLY_MESSAGE_FIELDS is the contract
a transport relies on, and the token estimator priced the two keys.

Reported by JoaoMarcos44 on #125567.

(cherry picked from commit 62e24924f0db520c09681bf9d4ff757e599497ad)
(cherry picked from commit 9b66d804f2bb5f3e174810b63e583415a4c57864)
2026-10-03 18:17:35 +05:30
John Paul Soliva
63c34ec442 fix(compression): the repair counts every durable row it retires without an id
Two more shapes reached the in-place commit with rows nothing named:

- Two killed tool calls in a row. The repair merges the two assistant rows,
  which pops the persist marker, and then drops the merged dict. The drop
  was gated on that marker, so neither row was retired and carried
  messages were recalled twice on every reload shape.
- Two assistant rows in a row on a reload without row ids. The merged dict
  equals neither row and has no marker, so the commit read it as a turn
  that was never persisted: the gateway hid a summarized message and the
  CLI restore recalled the oldest carried message twice.

A dropped dict's row ids are now retired whatever its marker says, as
assistant turn as well as a dropped one, and a dict that carries a count
is known to have been loaded. The key is renamed _unnamed_durable_rows to
say what it counts.

(cherry picked from commit 9bf1653471ecceed23794f09f7659986c43b7404)
(cherry picked from commit 7f4c8f540ea035d68a8cb1696b5da39deeba5f37)
2026-10-03 18:17:35 +05:30
John Paul Soliva
7e3bc5b60d fix(compression): in-place compaction accounts for the rows the repair drops
A turn killed after its tool call was saved leaves a call with no result in
state.db, and the alternation repair drops that row from every reload (a
result it cannot pair goes the same way). The in-place commit then had a
durable row nobody named:

- with row ids (--resume, TUI, Desktop) it was taken for a row another
  surface appended and re-sequenced behind the running turn, again at
  every later compaction;
- without row ids (gateway, ACP) it was either re-sequenced the same way
  or took a slot of the positional tail rewind, so the oldest carried
  message stayed compacted=1 beside its live copy and was recalled twice.

The repair now retires a dropped durable row onto the survivor before it:
its row id joins _absorbed_row_ids, and a reload without ids counts it in
_dropped_durable_rows, which the gateway replay entry carries. A held dict
with such a count keeps the commit on the watermark path, and the tail
rewind widens by it.

On that path _uncounted_merged_rows skipped any dict that listed absorbed
ids. A merged dict can list some of its rows and not others (two unanswered
prompts from a reload without ids, then a persisted one), so it now widens
by the stamp less the ids listed.

(cherry picked from commit d27dec2be6a8e371ff5370e3e0fa5d3f2548622c)
(cherry picked from commit 4f74851c1aa7412d9ac5129ec8e52ca9fcf50798)
2026-10-03 18:17:35 +05:30
John Paul Soliva
3154ccfe76 fix(compression): a merged user dict names rows by provenance, not by text
The commit resolved a marker-less user dict to a run of active rows whose
joined text equals it. A prompt that was never persisted can carry the same
text as a user;user pair another surface appended after the snapshot, so
those unseen rows were archived as summarized instead of being cloned after
the compacted set.

The alternation repair now stamps the dict with how many durable rows it
folded, the gateway replay entry carries that stamp, and the commit resolves
a run only for a stamped dict of that exact width. Folding an unpersisted
turn into the dict drops the stamp.

(cherry picked from commit 13bbb4d1c4c13874bd825d212503f9d9539060f0)
(cherry picked from commit df3fca28cba8a2758c5c711cc0ee582a8e374d51)
2026-10-03 18:17:35 +05:30
John Paul Soliva
44cf71f231 fix(compression): in-place compaction accounts for every row behind a merged user turn
A prompt that never got its reply leaves a durable user;user pair. The
alternation repair hands the compressor one dict for both rows, and the
in-place commit counted it as one original.

On a reload without row ids (gateway) the merged dict matched no row, read
as an unpersisted turn, and its two rows were re-sequenced after the
compacted set: the old prompts landed behind the running turn, and the
tail rewind reached one row into the summarized region and hid it.

With row ids the absorbed row was covered but not counted, so the oldest
carried original stayed compacted=1 beside its live copy.

The commit now resolves a merged user dict to the run of active rows it
was built from, and the tail rewind covers every row a tail dict absorbed.

(cherry picked from commit ba02ecf54487ca5235b99f70e6370db66aba94f4)
(cherry picked from commit 09715419ac7de957b757b5446208d05e26e75f3b)
2026-10-03 18:17:35 +05:30
kshitijk4poor
f6c78be31e refactor(compression): reuse _result_failure_suffix for the write_file refusal stub
#132141 landed the shared outcome helper; render the write_file refusal through it
so the stub reads like the other marked stubs ("[write_file] a.md FAILED: ...") and
fold the test into the TestSummarizeToolResultOutcome tables (one failed row, one
success row) instead of a separate parametrized function.
2026-10-03 17:46:47 +05:30
Yuan Li
2d5ed7dd88 fix(compression): keep a refused write_file's outcome in its stub
_sum_write_file always rendered "[write_file] wrote to PATH (N lines)" from
the call's args, so a write the stale-write guard refused
({"error": "Refusing to overwrite ...", "stale_write_blocked": true}) - or
any other write_file error - compressed into the same stub as a successful
write, and the post-compaction agent reported a change that never landed.
Read the result's error first and say FAILED with a bounded preview.

Salvaged from #131247 (write_file half only; the patch-tool half is covered
by the templated-stub outcome fix in #132141).
2026-10-03 17:46:47 +05:30
kshitijk4poor
815f03abd4 refactor(state): hoist pidns imports and trim duplicated call-site prose
Gate quality findings (Low): hermes_state_pidns is stdlib-only (os, re,
sys, typing), so the function-local imports guarded no cycle — verified
with fresh-interpreter imports of hermes_state, hermes_state_common,
agent.conversation_compression and agent.turn_facade_lease. Import it at
module level. Compare the namespace stamp as the str it already is,
make recorded_namespace private (no external user), and replace the
STRICT/LEGACY explanations repeated at each call site with one-line
pointers to the hermes_state_pidns module docstring that owns them.
conversation_compression.py stays at its base line count.
2026-10-03 17:44:58 +05:30
aleesposito85
6c6ce6149e fix(state): qualify lock/lease holder pid probes by PID namespace
A pid=<n> holder (compression lock, session turn lease, flock holder
record) is namespace-relative: process siblings sharing one state.db from
different PID namespaces (gateway + webui containers on one volume,
systemd PrivatePIDs=) see disjoint PID sets, so a sibling's live pid=
reads as absent to the namespace-blind probe and its unexpired row is
reclaimed under it. For a turn lease that ends the sibling's in-flight
turn ('another Hermes process took over this session'); for a compression
lock it splits the lineage; for a flock record it breaks a live lock.

Stamp every structured holder with the writer's PID-namespace identity
(/proc/self/ns/pid inode, via hermes_state_pidns) and qualify each probe:

* TTL-bounded rows (compression locks, turn leases): strict — a foreign
  or unstamped holder is never probed, it defers to its own expiry (at
  most the remaining TTL). A false defer self-heals at TTL; a false
  reclaim loses a reply. Strictness also stops the steal during a
  mixed-version rollout as soon as the reader restarts (the reader is
  the long-lived side of a container pair that restarts at different
  times), before every writer has restarted to stamp.
* No-expiry flock holder records: legal rollout — a foreign-namespace
  record is never probed, while an unstamped one keeps main's behavior
  (refusing to probe it would permanently disable orphaned-lock cleanup,
  the #100108 path). Protection begins as writers restamp.

Probe doubt already defers for the same reason (a wrongly-kept lease
self-heals; a wrongly-stolen one does not), so this extends an existing
contract rather than adding a new one.

(cherry picked from commit 4b1a29f63b8e6dec0a2724b5c2c4fac2bcc31c0e)
2026-10-03 17:44:58 +05:30
kshitijk4poor
2cad7ef208 fix(compression): only an error-bearing result counts as a refused call
Gate finding BN03.r1.2c (Medium): status == "blocked" alone was treated
as a refusal, but successful kanban results use it as a task state --
kanban_block (and kanban_create with initial_status="blocked") return
{"ok": true, "status": "blocked", ...}. Those summarized as
"[kanban_block] BLOCKED, not run", so after pruning or fallback
compaction the model would read its own successful block/create as
refused and might repeat it (e.g. a duplicate card).

Every real refusal envelope carries a non-empty error string
(terminal _error_json / _blocked_json, tool_error); the kanban success
payloads have none. Require it. Adds the kanban_block success payload
as a negative row in the existing parametrize.
2026-10-03 17:38:45 +05:30
EloquentBrush0x
1b9594ef14 fix(compression): a denied tool call is not summarized as if it ran
Phase-1 pruning and the fallback summary describe each old tool result with
a per-tool one-liner. For a call an approval or write guard refused, those
read as done: a denied `rm -rf build` became "[terminal] ran `rm -rf build`
-> exit -1, 1 lines output", a denied protected write became
"[write_file] wrote to AGENTS.md (2 lines)", and the "The user has NOT
consented ... Do NOT retry" instruction was dropped. Denials are 300+ chars,
over the 200-char prune floor, so this hits every long session.

Summarize a refused result (BLOCKED error, status blocked/pending_approval)
as "BLOCKED, not run" and keep the no-consent note, for every tool. Like the
clarify summary (#128346), it stays below _PRUNE_MIN_CHARS so later passes
keep it. Other results are summarized exactly as before.

(cherry picked from commit fd506bbccd3df8783dce0c0a1895f1aaf1585cb4)
2026-10-03 17:38:45 +05:30
kshitijk4poor
da0bcfca83 fix(compression): price the assistant-anchor escape like the tail walk
The #131412 escape gate summed _estimate_msg_budget_tokens(m) with its
default charge_stale_thinking=True, so every older assistant row was
charged for reasoning the wire never carries, and the native-Anthropic
budget was ignored. On reasoning routes that strip stale thinking the
gate fired on regions under the soft ceiling on the wire, dropped the
last visible reply from the tail and reopened #29824 (probe: no
reasoning cut=2 reply kept; 1200-char reasoning_content on two older
tool-call rows cut=5 reply dropped; main keeps cut=2).

Factor the walk's per-row pricer out of _walk_tail_budget into
_tail_row_pricer and use it for both the walk and the gate, so the
escape agrees with the #84371 accounting. The existing anchor test now
parametrizes the reasoning-heavy shape; it is red without this change
(cut 5 != 2) and green with it. (BN04 2c High; quality/reuse Medium.)
2026-10-03 17:38:02 +05:30