fix(pm): retry a plugin fetch failure once, then disable it

Sitting a plugin out after a fetch failure left the recorded stamp stale on
purpose, so every launch resynced, and the spawned-process guard in
prepare_launch raised "dependency sync left this install out of date".
One retry covers a blip; after that the plugin is disabled with the reason,
and hermes plugins enable restores it. Only requires_hermes still sits out,
which boot skips the same way.

pm/plugin_eviction.py

@@ -8,11 +8,12 @@ enables it, the reason reaches the operator and the receipt, and the update cont
 with the rest. Only a core that cannot build on its own still fails.
 
 Disabling needs evidence about the plugin itself: its requires-python against the pinned
-interpreter, its manifest contract, a resolver proof, or its build failing. Anything that
-might be about us or the moment instead (requires_hermes against a version identity that
-can lag, a fetch or tooling failure) only sits the plugin out of this build: config is
-untouched and it rejoins by itself. A secondary profile whose config cannot be read sits
-out the same way until its config is fixed.
+interpreter, its manifest contract, a resolver proof, or its build failing. A fetch or
+tooling failure could be the moment, so the plugin gets one retry before it is disabled.
+requires_hermes is judged against a version identity that can lag (a checkout without its
+release tags), so a misfit there only sits out: config is untouched, boot skips it the same
+way, and it rejoins when the verdict flips. A secondary profile whose config cannot be read
+sits out until its config is fixed.
 """
 from __future__ import annotations
 
@@ -57,10 +58,7 @@ def static_verdicts(entries: list[Entry], python_version: str) -> tuple[dict[Pat
             continue
         try:
             declaration = read_python_declaration(plugin_dir)
-        except OSError as exc:
-            waiting[key] = f"its dependency declaration could not be read: {exc}"
-            continue
-        except (ValueError, TypeError) as exc:
+        except (OSError, ValueError, TypeError) as exc:
             reasons[key] = f"its dependency declaration is invalid: {exc}"
             continue
         # Mirrors enabled_member_dirs, so the recorded stamp is the one boot expects.
@@ -127,6 +125,21 @@ class PluginEviction:
             durable_write_bytes(path, proposed)
 
 
+def _trial(package, enabled, explicit: bool, plugin_dirs: list[Path]) -> str | None:
+    """Why the last of *plugin_dirs* cannot join the build, or None when it builds."""
+    cause = ""
+    # A fetch or tooling failure can be the moment rather than the plugin: one more try.
+    for _attempt in range(2):
+        try:
+            package.apply(enabled, explicit=explicit, plugin_dirs=plugin_dirs, skip_invalid_secondary=True)
+            return None
+        except (ResolutionConflict, BuildFailure) as exc:
+            return f"the dependency environment no longer builds with it: {exc.cause[-400:]}"
+        except InstallError as exc:
+            cause = exc.cause
+    return f"its dependencies could not be prepared, twice: {cause[-400:]}"
+
+
 def sync_evicting(package, facts, fact: dict, *, extras, shipped, frozen, explicit: bool) -> None:
     """Build the discovered selection, disabling whatever plugin keeps it from building.
 
@@ -176,13 +189,9 @@ def sync_evicting(package, facts, fact: dict, *, extras, shipped, frozen, explic
             raise failure from None
         fitting: list[Path] = []
         for member in kept:
-            try:
-                package.apply(enabled, explicit=explicit, plugin_dirs=[*fitting, member], skip_invalid_secondary=True)
-            except (ResolutionConflict, BuildFailure) as exc:
-                reasons[member.resolve()] = f"the dependency environment no longer builds with it: {exc.cause[-400:]}"
-            except InstallError as exc:
-                # A fetch or tooling failure says nothing about the plugin: retry next sync.
-                waiting[member.resolve()] = f"its dependencies could not be prepared: {exc.cause[-400:]}"
+            reason = _trial(package, enabled, explicit, [*fitting, member])
+            if reason:
+                reasons[member.resolve()] = reason
             else:
                 fitting.append(member)
         commit()
@@ -192,7 +201,7 @@ def sync_evicting(package, facts, fact: dict, *, extras, shipped, frozen, explic
             notices.append(f"Disabled plugin '{name}' in {plugins_dir.parent}: {reasons[key]}")
         elif key in waiting:
             notices.append(f"Left plugin '{name}' in {plugins_dir.parent} out of this update: {waiting[key]}; "
-                           "it stays enabled and rejoins once that clears")
+                           "it stays enabled and rejoins once Hermes reports a version it accepts")
     for message in notices:
         print(f"⚠ {message}", file=sys.stderr, flush=True)
         receipt.record_warning(message)

tests/pm/test_plugin_survival_contract.py

@@ -469,10 +469,21 @@ def test_plugin_our_version_rejects_sits_out_without_being_disabled(admission_en
 
 
 @pytest.mark.skipif(not _uv_available(), reason="uv not on PATH")
-def test_update_sync_disables_only_on_evidence_about_the_plugin(admission_env, monkeypatch):
-    """A plugin whose own build fails is disabled; one whose dependency cannot be fetched
-    says nothing about the plugin, so it stays enabled and the next sync retries it."""
+def test_update_sync_retries_a_fetch_failure_once_before_disabling(admission_env, monkeypatch):
+    """Build evidence disables a plugin at once. A fetch failure could be the moment, so it
+    gets one more try first. Either way the recorded state is the one boot expects."""
     from pm.install import sync_venv, venv_is_current
+    from pm.packages import Venv
+
+    trials: list[str] = []
+    real_apply = Venv.apply
+
+    def counting_apply(self, extras, *, plugin_dirs=None, **kwargs):
+        if plugin_dirs:
+            trials.append(Path(plugin_dirs[-1]).name)
+        return real_apply(self, extras, plugin_dirs=plugin_dirs, **kwargs)
+
+    monkeypatch.setattr(Venv, "apply", counting_apply)
 
     tmp_path, home = admission_env
     monkeypatch.setenv("UV_HTTP_RETRIES", "0")
@@ -500,10 +511,11 @@ def test_update_sync_disables_only_on_evidence_about_the_plugin(admission_env, m
     sync_venv(explicit=True, evict_incompatible_plugins=True)
 
     cfg = yaml.safe_load((home / "config.yaml").read_text(encoding="utf-8"))
-    assert cfg["plugins"]["disabled"] == ["wont-build"]
-    assert "offline-dep" in cfg["plugins"]["enabled"]
-    assert "Left plugin 'offline-dep'" in json.dumps(_latest_receipt(home).get("warnings"))
-    assert venv_is_current(project_root=tmp_path / "core") is False  # the next sync retries it
+    assert cfg["plugins"]["disabled"] == ["wont-build", "offline-dep"]
+    # After the full selection's own attempt: build evidence once, the fetch failure twice.
+    assert trials[1:] == ["wont-build", "offline-dep", "offline-dep"]
+    assert "could not be prepared, twice" in json.dumps(_latest_receipt(home).get("warnings"))
+    assert venv_is_current(project_root=tmp_path / "core") is True
 
 
 def test_active_context_home_exported_to_wrapper_subprocess(monkeypatch, tmp_path):

website/docs/getting-started/updating.md

@@ -136,7 +136,7 @@ For an admitted source checkout, `hermes update` runs these phases:
 1. **Pre-update snapshot** — Hermes saves selected state files for every profile in that profile's `state-snapshots/` directory. These include pairing data, cron jobs, `config.yaml`, `.env`, and `auth.json`. Automatic quick snapshots skip individual files larger than 1 GiB. `updates.pre_update_backup` selects `quick`, `full`, or `off`. Full archives use the [backup exclusions](../reference/faq.md#hermes-backup-vs-hermes-profile-export). Recovery uses [Snapshots and rollback](../user-guide/checkpoints-and-rollback.md). Quick snapshots recover state files, not application code. The snapshot is best-effort: if it fails, the update prints a `⚠ Pre-update snapshot FAILED` warning and continues, and the receipt records `pre_update_backup` as a failed step (a deliberate `off`/`--no-backup` lands in the receipt's skips with its reason instead).
 2. **Code update** — applies the configured source branch or stable release tag and updates submodules.
 3. **Post-pull syntax validation + auto-rollback** — after the pull, Hermes compiles the nine critical files every `hermes` invocation imports at startup. If any fails to parse (e.g. an orphan merge-conflict marker, an accidentally truncated file), Hermes runs `git reset --hard <pre-pull-sha>` to roll the install back so your shell stays bootable. Re-run `hermes update` once the upstream fix lands.
-4. **Dependency preparation** — PM provisions required tools and prepares a complete Python environment from the new lock, existing extras, and enabled plugin requirements. It validates that environment before publishing its selection. A plugin never fails the update. A plugin that no longer fits the new core is added to `plugins.disabled` in every profile that enables it (a memory provider has `memory.provider` cleared). That covers a `requires-python` that excludes Hermes's Python, a `manifest_version` newer than this Hermes supports, and dependencies that the resolver proves can't resolve alongside core and earlier plugins in config order, or that fail their own build. The update prints `⚠ Disabled plugin '<name>' in <home>: <reason>`, records it in the receipt's warnings, and continues. Re-enable it with `hermes plugins enable <name>` once the plugin ships a compatible release. Failures that might be about Hermes or the moment rather than the plugin never disable it. That means a `requires_hermes` range the running version misses (a source checkout without release tags can read as an older release), a download or network failure, or an unreadable secondary profile config. The plugin sits out of this build instead (`⚠ Left plugin '<name>' … out of this update`), stays enabled, and rejoins on the next sync once the cause clears. Only a core that cannot build on its own fails this step.
+4. **Dependency preparation** — PM provisions required tools and prepares a complete Python environment from the new lock, existing extras, and enabled plugin requirements. It validates that environment before publishing its selection. A plugin never fails the update. A plugin that no longer fits the new core is added to `plugins.disabled` in every profile that enables it (a memory provider has `memory.provider` cleared). That covers a `requires-python` that excludes Hermes's Python, a `manifest_version` newer than this Hermes supports, and dependencies that the resolver proves can't resolve alongside core and earlier plugins in config order, or that fail their own build. A download or network failure gets one retry and is disabled if it fails again. The update prints `⚠ Disabled plugin '<name>' in <home>: <reason>`, records it in the receipt's warnings, and continues. Re-enable it with `hermes plugins enable <name>` once the plugin ships a compatible release, or once the network is back. A `requires_hermes` range the running version misses never disables a plugin, because a source checkout without release tags can read as an older release. That plugin sits out instead (`⚠ Left plugin '<name>' … out of this update`), stays enabled, and rejoins once Hermes reports a version it accepts. A secondary profile whose config cannot be read sits out the same way until the config is fixed. Only a core that cannot build on its own fails this step.
 5. **Config migration** — detects new config options added since your version and prompts you to set them
 6. **Desktop rebuild (stage-and-swap)** — if the Hermes Desktop app was built from this checkout, it is rebuilt so the GUI matches the new code. The rebuild packs into a temporary staging directory next to `apps/desktop/release/`, verifies the staged app, and only then renames it over the previous build (on Windows a real-time scanner briefly holding `release/win-unpacked` is ridden out with a few short retries). A rebuild that fails at any point — corrupt Electron download, missing dependency, disk full — leaves the previous app untouched and launchable; the update fails at that step, and `hermes desktop --build-only --force-build` or the next `hermes update` retries the rebuild. On macOS the rebuilt bundle is then copied (with `ditto`, signature intact) over a stale `/Applications/Hermes.app` or `~/Applications/Hermes.app`, so the copy Finder and the Dock launch matches the backend; an installed copy that is currently running is left alone and the update tells you to quit it and run `hermes update` again.
 7. **Gateway auto-restart**: running gateways are refreshed after the update completes. Service-managed gateways (systemd on Linux, launchd on macOS) restart through the service manager. Manual gateways are relaunched when Hermes can map their PID to a profile. Manually launched `hermes serve` / `hermes dashboard` backends are different: the updater leaves them running and asks their owner to restart them. See [Manual backend restart reminders](#manual-backend-restart-reminders). Backends owned by a running Desktop app remain the app's responsibility.