Change-detectors, tautologies, source-reading tests, redundant duplicates,
mock-echo tests and dead/unrunnable tests. Per-test rationale in the lane
ledger (category + reason for every removal).
Non-secret settings live in config.yaml: the extraction cap override becomes
terminal.sync_back_max_bytes in DEFAULT_CONFIG (read via load_config() like the
rest of the terminal section) and the HERMES_SYNC_BACK_MAX_BYTES env var goes
away; docs and the cap tests follow. psutil is a pinned core dependency, so
_temp_entry_owner_alive imports it unconditionally instead of guarding an
ImportError that cannot happen.
Follow-up to the salvaged #114440 commits:
- Stale sweep: a dead owner's entry goes at once, but a live (or recycled)
PID no longer exempts an entry from the 30 min age cutoff — a recycled PID
must not pin a multi-GB leak forever. Liveness via psutil.pid_exists (the
footgun scanner rejects os.kill(pid, 0): it terminates on Windows), so the
os.name guard and the two extra helpers collapse into one.
- HERMES_SYNC_BACK_MAX_BYTES overrides the 2 GiB extraction cap for trees that
legitimately exceed it (the reporter's 4.26 GB tree was downloaded and
discarded on every attempt); the skip warning names the override.
- Vercel sandbox bulk download excludes *.sock like SSH/Modal/Daytona.
- Tests trimmed to two invariants per file; docs mention the cap override,
the socket skip and the per-PID temp naming.
Review follow-ups on #114440:
- Drop the HERMES_SYNC_BACK_MAX_BYTES env override: a malformed value
crashed the whole backend at import time, and the rubric keeps env
vars for secrets only. The 2 GiB cap stays hardcoded (raising it can
be a separate change).
- Embed the owning PID in sync-back temp names and reclaim dead-owner
entries immediately, mirroring daytona's PID-suffixed remote temp.
Age no longer decides liveness (a staging dir's mtime does not move
while content streams into subdirectories, and the download bound is
SSH/Modal-only), and concurrent gateway processes under different
HERMES_HOMEs never sweep each other's transfers. Legacy (pre-PID)
names and Windows hosts keep the 30-minute cutoff.
- Anchor the rc=2 socket tolerance to lines ending in ': socket
ignored' and require a non-empty diagnostic, so whitespace-only
stderr or a filename merely containing the marker still fails.
- Add --exclude='*.sock' to the Modal and Daytona bulk-download tars
(same failure class as the SSH backend).
Every SSH sync-back on a host with a live gateway.sock failed: GNU tar
cannot archive a Unix socket, prints "socket ignored", and exits 2 —
so the download raised, retried three times, and (when the process was
hard-killed mid-download, e.g. by the gateway's tool-subprocess kill)
left its multi-GB temp tar behind. With the stale-temp sweep window at
6 h, a crash loop accumulated 45 GB in /tmp and filled the rootfs
(#114437).
- Exclude *.sock from the remote tar up front; tolerate a rc=2 whose
stderr is only "socket ignored" lines (defensive for tars without
--exclude support or unmatched socket names) — anything else still
fails the transfer.
- Shrink the stale sweep cutoff from 6 h to 30 min; the download is
bounded by a 120 s subprocess timeout, so a live transfer is minutes
old at most and the cutoff only has to sit above that.
- Make the 2 GiB extraction cap overridable via HERMES_SYNC_BACK_MAX_BYTES
for synced trees that legitimately exceed it.
Fixes#114437
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
The salvaged commit only swept `hermes-sync-back-*.tar`. The extraction
staging dir (`tempfile.TemporaryDirectory(prefix="hermes-sync-back-")`) is
leaked by the same hard kill, so the sweep now reclaims both shapes and
returns the count. The window drops from 24 h to 6 h (the reporter's
value): archives appear every few minutes on a busy gateway and a live
transfer is never hours old. Tests trimmed to two invariants: the sweep
touches only stale prefixed entries, and a real sync_back reclaims a
leaked archive while writing its own tar under the identifiable prefix.
Merge upstream 5e645791ac.
Retain the PM feature-flag owner and add upstream connection options.
Use the deny-only window-open policy while trusted external links keep
the existing IPC path. Keep both session-import and external-link copy.
Preserve captured timeout output when adding terminal yield handoff.
Quickstart tests patch the explicit upstream model-assignment owner.
Migrate incoming legacy OS markers to the branch's platforms gate.
Desktop renderer and Electron typechecks passed. Targeted Electron tests
passed (42 tests), Python conflict checks passed (26 tests, 3 skips),
and the plugin-compat import checker passed. CI owns the broad merge gate.
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.
Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
(DEFAULT_CONFIG smart-approval leaked in) — pinned approval
mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
silently probed live provider catalogs (~2s/test) — stubbed
cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
MagicMock agents — interrupt.side_effect now clears _running_agents
(22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
(24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
with rationale comment — the watchdog-dump assertion needs the loop
blocked well past deadline+grace under parallel load (flaked once in
the 40-worker verification run at a 0.2s margin).
Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
Remove unused imports (F401) and duplicate/shadowed import
redefinitions (F811) across the codebase using ruff's safe
autofixes. No behavioral changes -- imports only.
- ~1400 safe autofixes applied across 644 files (net -1072 lines)
- __init__.py re-exports preserved (excluded from F401 removal so
public re-export surfaces stay intact)
- Re-exports that are imported or monkeypatched by tests but look
unused in their defining module are kept with explicit # noqa:
F401 (gateway/run.py load_dotenv; run_agent re-exports from
agent.message_sanitization, agent.context_compressor,
agent.retry_utils, agent.prompt_builder, agent.process_bootstrap,
agent.codex_responses_adapter)
- Unsafe F841 (unused-variable) fixes deliberately skipped -- those
can change behavior when the RHS has side effects
- ruff lints remain disabled in pyproject.toml (only PLW1514 is
selected); this is a one-time cleanup, not a config change
Verification:
- python -m compileall: clean
- pytest --collect-only: all 27161 tests collect (zero import errors)
- core entry points import clean (run_agent, model_tools, cli,
toolsets, hermes_state, batch_runner, gateway)
- static scan: every name any test imports directly from an edited
module still resolves
The original PR placed 'pwd = pytest.importorskip("pwd")' on line 4
but 'import pytest' on line 9 — NameError on module load. Same for
test_file_sync_back.py. Plus, the in-function 'pwd = pytest.importorskip'
calls in test_auto_detected_root_is_rejected confused Python's scope
analysis (later 'import pytest' made pytest local everywhere in the
function) and caused UnboundLocalError. Drop the now-redundant
in-function importorskip calls and rely on the module-level guard.
Follow-ups on top of kshitijk4poor's cherry-picked salvage of PR #8018:
tools/environments/daytona.py
- PID-suffix /tmp/.hermes_sync.<pid>.tar so concurrent sync_back calls
against the same sandbox don't collide on the remote temp path
- Move sync_back() inside the cleanup lock and after the _sandbox-None
guard, with its own try/except. Previously a no-op cleanup (sandbox
already cleared) still fired sync_back → 3-attempt retry storm against
a nil sandbox (~6s of sleep). Now short-circuits cleanly.
tools/environments/file_sync.py
- Add _SYNC_BACK_MAX_BYTES (2 GiB) defensive cap: refuse to extract a
tar larger than the limit. Protects against runaway sandboxes
producing arbitrary-size archives.
- Add 'nothing previously pushed' guard at the top of sync_back(). If
_pushed_hashes and _synced_files are both empty, the FileSyncManager
was never initialized from the host side — there is nothing coherent
to sync back. Skips the retry/backoff machinery on uninitialized
managers and eliminates test-suite slowdown from pre-existing cleanup
tests that don't mock the sync layer.
tests/tools/test_file_sync_back.py
- Update _make_manager helper to seed a _pushed_hashes entry by default
so sync_back() exercises its real path. A seed_pushed_state=False
opt-out is available for noop-path tests.
- Add TestSyncBackSizeCap with positive and negative coverage of the
new cap.
tests/tools/test_sync_back_backends.py
- Update Daytona bulk download test to assert the PID-suffixed path
pattern instead of the fixed /tmp/.hermes_sync.tar.
Salvage of PR #8018 by @alt-glitch onto current main.
On sandbox teardown, FileSyncManager now downloads the remote .hermes/
directory, diffs against SHA-256 hashes of what was originally pushed,
and applies only changed files back to the host.
Core (tools/environments/file_sync.py):
- sync_back(): orchestrates download -> unpack -> diff -> apply with:
- Retry with exponential backoff (3 attempts, 2s/4s/8s)
- SIGINT trap + defer (prevents partial writes on Ctrl-C)
- fcntl.flock serialization (concurrent gateway sandboxes)
- Last-write-wins conflict resolution with warning
- New remote files pulled back via _infer_host_path prefix matching
Backends:
- SSH: _ssh_bulk_download — tar cf - piped over SSH
- Modal: _modal_bulk_download — exec tar cf - -> proc.stdout.read
- Daytona: _daytona_bulk_download — exec tar cf -> SDK download_file
- All three call sync_back() at the top of cleanup()
Fixes applied during salvage (vs original PR #8018):
| # | Issue | Fix |
|---|-------|-----|
| C1 | import fcntl unconditional — crashes Windows | try/except with fallback; _sync_back_locked skips locking when fcntl=None |
| W1 | assert for runtime guard (stripped by -O) | Replaced with proper if/raise RuntimeError |
| W2 | O(n*m) from _get_files_fn() called per file | Cache mapping once at start of _sync_back_impl, pass to resolve/infer |
| W3 | Dead BulkDownloadFn imports in 3 backends | Removed unused imports |
| W4 | Modal hardcodes root/.hermes, no explanation | Added docstring comment explaining Modal always runs as root |
| S1 | SHA-256 computed for new files where pushed_hash=None | Skip hashing when pushed_hash is None (comparison always False) |
| S2 | Daytona /tmp/.hermes_sync.tar never cleaned up | Added rm -f after download (best-effort) |
Tests: 49 passing (17 new: _infer_host_path edge cases, SIGINT
main/worker thread, Windows fcntl=None fallback, Daytona tar cleanup).
Based on #8018 by @alt-glitch.
