fix(config): one bounded backups/config/ dir replaces four config.yaml.bak schemes

Four writers each dropped their own uniquely-named copy of config.yaml next to
the real file and none of them ever deleted anything: hermes setup
(config.yaml.bak.YYYYMMDD_HHMMSS, one per run even with no change), the
corrupt-YAML snapshot (config.yaml.corrupt.<ts>.bak), hermes migrate xai
(config.yaml.bak-pre-migrate-xai-<ts>) and the Docker boot migration
(config.yaml.bak-<ts>, .env.bak-<ts>). A home dir accumulated a dozen variants
with no way to tell which mattered.

hermes_cli/config_backups.py::backup_config is now the single writer:
backups/config/config.yaml.<reason>.<YYYYMMDD-HHMMSS>, skipped when the newest
copy for that reason is byte-identical, rotated to the newest five per reason.
backups/ is already excluded from full backups so nothing nests. Legacy
siblings written by the old schemes are moved into the dir on first use;
hand-named copies (config.yaml.bak-my-note) are left alone.

Live: three `hermes setup --non-interactive` runs against an unchanged config
went from three .bak files in HERMES_HOME to one pre-setup copy under
backups/config/; repeated loads of broken YAML produce one corrupt copy
instead of one per process (deduped by content).
This commit is contained in:
Teknium
2026-09-09 02:02:06 -07:00
parent 06dc51d62d
commit bf53ff00a7
12 changed files with 158 additions and 91 deletions

View File

@@ -49,35 +49,6 @@ class InvalidUserConfigError(RuntimeError):
"""Raised when a run that cannot repair config finds invalid user YAML."""
def _backup_corrupt_config(config_path: Path) -> Optional[Path]:
"""Copy an unparseable ``config.yaml`` to a timestamped ``.corrupt.*.bak``; None on skip/failure.
Symlinks are not followed (never clobber whatever a malicious symlink points at). A sibling
backup of the same size means this corruption was already snapshotted — skip to avoid churn.
Returns the backup path on success, else ``None``. See #21541.
"""
try:
if config_path.is_symlink():
return None
st = config_path.stat()
if st.st_size == 0:
return None
ts = time.strftime("%Y%m%d-%H%M%S")
backup_path = config_path.with_name(f"{config_path.name}.corrupt.{ts}.bak")
for existing in config_path.parent.glob(f"{config_path.name}.corrupt.*.bak"):
try:
if existing.stat().st_size == st.st_size:
return None
except OSError:
continue
if backup_path.exists():
return None
shutil.copy2(config_path, backup_path)
return backup_path
except Exception:
return None
_PARSE_FAILURE_FALLBACK_MSG = {
"last-known-good": (
"Keeping the previously loaded config for this process — "
@@ -108,7 +79,8 @@ def _warn_config_parse_failure(
if key in _CONFIG_PARSE_WARNED:
return
_CONFIG_PARSE_WARNED.add(key)
backup_path = _backup_corrupt_config(config_path)
from hermes_cli.config_backups import backup_config
backup_path = backup_config(config_path, "corrupt")
msg = f"Failed to parse {config_path}: {exc}. " + _PARSE_FAILURE_FALLBACK_MSG.get(
fallback, _PARSE_FAILURE_DEFAULTS_MSG)
if backup_path is not None:
@@ -515,7 +487,8 @@ def require_parseable_user_config(*, ignore_user_config: bool = False) -> None:
return
parse_error = TypeError(f"top-level YAML value must be a mapping, got {type(data).__name__}")
backup_path = _backup_corrupt_config(config_path)
from hermes_cli.config_backups import backup_config
backup_path = backup_config(config_path, "corrupt")
message = (
f"Refusing non-interactive startup because {config_path} is invalid: "
f"{parse_error}. Repair the file or pass --ignore-user-config to "
@@ -1944,7 +1917,7 @@ def _refuse_overwrite(config_path: Path, reason: str, exc: Exception, fix: str)
_FIX_PERMS = "Fix the file permissions or move it aside first."
_FIX_YAML = "Fix the file or restore from a .corrupt.*.bak backup first."
_FIX_YAML = "Fix the file or restore a copy from backups/config/ first."
def require_readable_config_before_write(config_path: Optional[Path] = None) -> Dict[str, Any]:
@@ -1977,7 +1950,7 @@ def require_readable_config_before_write(config_path: Optional[Path] = None) ->
_warn_config_parse_failure(config_path, exc, fallback="refuse-write")
raise RuntimeError(
f"Refusing to overwrite {config_path}: top-level YAML must be a mapping, got "
f"{type(loaded).__name__}. Fix the file or restore from a .corrupt.*.bak backup first."
f"{type(loaded).__name__}. Fix the file or restore a copy from backups/config/ first."
) from exc
return loaded

View File

@@ -0,0 +1,80 @@
"""Point-in-time copies of ``config.yaml``: one directory, one naming scheme, bounded count.
Every writer that wants a "before" copy of the user's config (setup wizard, corrupt-file
snapshot, model migrations) goes through :func:`backup_config`. Copies live in
``<HERMES_HOME>/backups/config/`` — ``backups/`` is already excluded from full backups, so they
never nest — as ``config.yaml.<reason>.<YYYYMMDD-HHMMSS>``. A copy identical to the newest one
for the same reason is skipped, and only the newest ``keep`` per reason survive, so repeated
``hermes setup`` runs or a gateway restarting against broken YAML cannot litter the home dir.
"""
from __future__ import annotations
import filecmp
import logging
import shutil
import time
from pathlib import Path
from typing import Optional
logger = logging.getLogger(__name__)
BACKUPS_SUBDIR = Path("backups") / "config"
DEFAULT_KEEP = 5
# Names earlier code wrote next to config.yaml (setup wizard, corrupt snapshot, xai migration).
# Moved into the backups dir on first use so they stop accumulating in the home root; hand-named
# copies (``config.yaml.bak-my-note``) are the user's and are never touched.
_LEGACY_SIBLING_GLOBS = ("config.yaml.bak.[0-9]*", "config.yaml.corrupt.*.bak", "config.yaml.bak-pre-migrate-*")
def backups_dir(config_path: Path) -> Path:
return config_path.parent / BACKUPS_SUBDIR
def list_config_backups(config_path: Path, reason: Optional[str] = None) -> list[Path]:
"""Existing backups, newest first; filtered to one *reason* when given."""
root = backups_dir(config_path)
if not root.is_dir():
return []
prefix = f"{config_path.name}.{reason}." if reason else f"{config_path.name}."
return sorted((p for p in root.iterdir() if p.is_file() and p.name.startswith(prefix)),
key=lambda p: p.name, reverse=True)
def backup_config(config_path: Path, reason: str, *, keep: int = DEFAULT_KEEP) -> Optional[Path]:
"""Copy *config_path* to the backups dir; return the new path, or None when skipped/failed.
Skips when the file is missing/empty, or when the newest backup for *reason* already holds
identical bytes. Never raises: a failed backup must not block the write it precedes.
"""
try:
if not config_path.is_file() or config_path.stat().st_size == 0:
return None
root = backups_dir(config_path)
root.mkdir(parents=True, exist_ok=True)
_sweep_legacy_siblings(config_path, root)
existing = list_config_backups(config_path, reason)
if existing and filecmp.cmp(config_path, existing[0], shallow=False):
return None
dest = root / f"{config_path.name}.{reason}.{time.strftime('%Y%m%d-%H%M%S')}"
if dest.is_symlink() or dest.exists(): # never write through a planted link
return None
shutil.copy2(config_path, dest)
for stale in [dest, *existing][keep:]:
stale.unlink(missing_ok=True)
return dest
except OSError as exc:
logger.warning("Could not back up %s (%s): %s", config_path, reason, exc)
return None
def _sweep_legacy_siblings(config_path: Path, root: Path) -> None:
for pattern in _LEGACY_SIBLING_GLOBS:
for old in config_path.parent.glob(pattern):
if not old.is_file() or old.is_symlink():
continue
try:
old.replace(root / old.name)
except OSError as exc:
logger.debug("Could not move legacy backup %s: %s", old, exc)

View File

@@ -580,20 +580,6 @@ def run_setup_wizard(args):
return None
def _backup_config_file(config_path: Path) -> Path | None:
"""Back up config.yaml before setup modifies it; None when absent or copy fails."""
if not config_path.exists():
return None
import shutil
from datetime import datetime
backup_path = config_path.with_suffix(f".yaml.bak.{datetime.now().strftime('%Y%m%d_%H%M%S')}")
try:
shutil.copy2(config_path, backup_path)
return backup_path
except Exception:
return None
def _run_setup_section(config: dict, section: str) -> None:
"""``hermes setup <section>``: run one SETUP_SECTIONS entry under the banner."""
entry = next(((label, func) for key, label, func in SETUP_SECTIONS if key == section), None)
@@ -672,7 +658,8 @@ def _run_setup_wizard_impl(args):
hermes_home = get_hermes_home()
# Back up existing config before setup modifies it (#3522)
config_path = get_config_path()
_backup_path = _backup_config_file(config_path)
from hermes_cli.config_backups import backup_config
_backup_path = backup_config(config_path, "pre-setup")
# Non-interactive environments (headless SSH, Docker, CI/CD)
if getattr(args, 'non_interactive', False) or not is_interactive_stdin():

View File

@@ -1,9 +1,7 @@
"""Detect xAI models retired on May 15, 2026 and migrate config.yaml references."""
from __future__ import annotations
import datetime as _dt
import io
import shutil
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional
@@ -130,7 +128,7 @@ def apply_migration(
config_path: Path, issues: List[RetirementIssue], backup: bool = True) -> ApplyResult:
"""Rewrite ``config_path`` in place (ruamel round-trip: comments, order, type literals kept).
Unless ``backup=False`` a copy goes to ``<config_path>.bak-pre-migrate-xai-YYYYMMDD-HHMMSS``.
Unless ``backup=False`` a copy goes to ``backups/config/`` (reason ``pre-migrate-xai``).
"""
from ruamel.yaml import YAML # local import — avoid hard dep at module load
config_path = Path(config_path)
@@ -162,9 +160,8 @@ def apply_migration(
backup_path: Optional[Path] = None
if backup:
ts = _dt.datetime.now().strftime("%Y%m%d-%H%M%S")
backup_path = config_path.with_name(f"{config_path.name}.bak-pre-migrate-xai-{ts}")
shutil.copy2(config_path, backup_path)
from hermes_cli.config_backups import backup_config
backup_path = backup_config(config_path, "pre-migrate-xai")
from hermes_cli.config import require_readable_config_before_write
from utils import atomic_write_text

View File

@@ -4,7 +4,6 @@ from __future__ import annotations
import shutil
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable
@@ -14,6 +13,7 @@ from hermes_cli.config import (
get_env_path,
migrate_config,
)
from hermes_cli.config_backups import backup_config, list_config_backups
from hermes_cli.config_migrations import (
SUPPORT_FLOOR_VERSION,
support_floor_message,
@@ -21,26 +21,14 @@ from hermes_cli.config_migrations import (
from utils import env_var_enabled
def _backup_path(path: Path, stamp: str) -> Path:
base = path.with_name(f"{path.name}.bak-{stamp}")
if not base.exists():
return base
for index in range(1, 1000):
candidate = path.with_name(f"{path.name}.bak-{stamp}.{index}")
if not candidate.exists():
return candidate
raise RuntimeError(f"could not choose a backup path for {path}")
def _backup_existing(paths: Iterable[Path]) -> dict[Path, Path]:
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
"""Snapshot each file into backups/config/; an identical existing snapshot is reused."""
backups: dict[Path, Path] = {}
for path in paths:
if not path.is_file():
continue
dest = _backup_path(path, stamp)
shutil.copy2(path, dest)
backups[path] = dest
dest = backup_config(path, "pre-docker-migrate") or next(
iter(list_config_backups(path, "pre-docker-migrate")), None)
if dest is not None:
backups[path] = dest
return backups

View File

@@ -178,7 +178,7 @@ class TestLoadConfigParseFailure:
load_config()
err = capsys.readouterr().err
baks = list(tmp_path.glob("config.yaml.corrupt.*.bak"))
baks = list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*"))
assert len(baks) == 1, f"expected one backup, got {baks}"
# Backup preserves the original broken content verbatim
assert baks[0].read_text() == broken
@@ -331,7 +331,7 @@ class TestSaveAndLoadRoundtrip:
set_config_value("model.default", "gpt-4o")
assert config_path.read_text(encoding="utf-8") == original
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")), (
assert list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*")), (
"parse-failure path should snapshot a corrupt backup before refusing"
)
@@ -348,7 +348,7 @@ class TestSaveAndLoadRoundtrip:
assert config_path.read_text(encoding="utf-8") == original
assert (tmp_path / ".env").read_text(encoding="utf-8") == "TERMINAL_TIMEOUT=30\n"
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")), (
assert list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*")), (
"unset parse-failure path should snapshot a corrupt backup before refusing"
)
@@ -363,7 +363,7 @@ class TestSaveAndLoadRoundtrip:
set_config_value("model.default", "gpt-4o")
assert config_path.read_text(encoding="utf-8") == original
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")), (
assert list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*")), (
"non-mapping root should snapshot a corrupt backup before refusing"
)
@@ -378,7 +378,7 @@ class TestSaveAndLoadRoundtrip:
unset_config_value("model.default")
assert config_path.read_text(encoding="utf-8") == original
assert list(tmp_path.glob("config.yaml.corrupt.*.bak"))
assert list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*"))
def test_config_set_allows_valid_empty_mapping(self, tmp_path):
"""A genuine empty {} config must still be writable (not a false refuse)."""
@@ -403,7 +403,7 @@ class TestSaveAndLoadRoundtrip:
atomic_config_write(config_path, {"model": {"provider": "openai"}})
assert config_path.read_text(encoding="utf-8") == original
assert list(tmp_path.glob("config.yaml.corrupt.*.bak"))
assert list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*"))
class TestSaveEnvValueSecure:

View File

@@ -0,0 +1,40 @@
"""config.yaml backups: one dir, deduped, bounded — never a pile of siblings in HERMES_HOME."""
from pathlib import Path
from hermes_cli.config_backups import backup_config, list_config_backups
def test_repeat_backups_dedupe_and_rotate(tmp_path: Path, monkeypatch):
cfg = tmp_path / "config.yaml"
cfg.write_text("model: a\n")
stamps = iter(f"2026010100000{i}" for i in range(10))
monkeypatch.setattr("hermes_cli.config_backups.time.strftime", lambda _fmt: next(stamps))
first = backup_config(cfg, "pre-setup", keep=2)
assert first is not None and first.parent == tmp_path / "backups" / "config"
# Same bytes again → no new file (the hermes-setup-three-times case).
assert backup_config(cfg, "pre-setup", keep=2) is None
for i in range(3):
cfg.write_text(f"model: {i}\n")
backup_config(cfg, "pre-setup", keep=2)
kept = list_config_backups(cfg, "pre-setup")
assert len(kept) == 2 and kept[0].read_text() == "model: 2\n"
# Nothing left beside config.yaml in the home root.
assert [p.name for p in tmp_path.iterdir() if p.is_file()] == ["config.yaml"]
def test_legacy_siblings_move_but_user_named_copies_stay(tmp_path: Path):
cfg = tmp_path / "config.yaml"
cfg.write_text("model: a\n")
for name in ("config.yaml.bak.1778718391", "config.yaml.corrupt.20260729-093706.bak",
"config.yaml.bak-pre-migrate-xai-20260515-120000"):
(tmp_path / name).write_text("old")
(tmp_path / "config.yaml.bak-my-note").write_text("mine")
backup_config(cfg, "corrupt")
root = tmp_path / "backups" / "config"
assert (root / "config.yaml.bak.1778718391").exists()
assert (root / "config.yaml.corrupt.20260729-093706.bak").exists()
assert (tmp_path / "config.yaml.bak-my-note").read_text() == "mine"
assert not list(tmp_path.glob("config.yaml.bak.*")) and not list(tmp_path.glob("config.yaml.corrupt.*"))

View File

@@ -162,7 +162,7 @@ class TestBackup:
result = apply_migration(trap_config, issues, backup=False)
assert result.backup_path is None
# No bak file in the directory
assert not list(trap_config.parent.glob("*.bak-pre-migrate-xai-*"))
assert not list(trap_config.parent.rglob("config.yaml.pre-migrate-xai.*"))

View File

@@ -61,7 +61,7 @@ def test_noninteractive_guard_rejects_malformed_yaml(args, tmp_path, caplog, cap
for record in caplog.records
)
assert config_path.read_text(encoding="utf-8") == broken
backups = list(tmp_path.glob("config.yaml.corrupt.*.bak"))
backups = list((tmp_path / "backups" / "config").glob("config.yaml.corrupt.*"))
assert len(backups) == 1
assert backups[0].read_text(encoding="utf-8") == broken
@@ -131,7 +131,7 @@ def test_explicit_config_bypasses_allow_noninteractive_recovery(args, tmp_path):
main_mod._guard_noninteractive_user_config(args)
assert args._noninteractive_config_validated is True
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")) == []
assert list(tmp_path.rglob("config.yaml.corrupt.*")) == []
def test_interactive_chat_keeps_existing_repair_behavior(tmp_path):
@@ -143,7 +143,7 @@ def test_interactive_chat_keeps_existing_repair_behavior(tmp_path):
main_mod._guard_noninteractive_user_config(args)
assert not hasattr(args, "_noninteractive_config_validated")
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")) == []
assert list(tmp_path.rglob("config.yaml.corrupt.*")) == []
@pytest.mark.parametrize(
@@ -163,7 +163,7 @@ def test_queryless_chat_keeps_interactive_repair_behavior(args, tmp_path):
main_mod._guard_noninteractive_user_config(args)
assert not hasattr(args, "_noninteractive_config_validated")
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")) == []
assert list(tmp_path.rglob("config.yaml.corrupt.*")) == []
def test_env_only_config_bypass_allows_noninteractive_recovery(monkeypatch, tmp_path):
@@ -176,7 +176,7 @@ def test_env_only_config_bypass_allows_noninteractive_recovery(monkeypatch, tmp_
main_mod._guard_noninteractive_user_config(args)
assert args._noninteractive_config_validated is True
assert list(tmp_path.glob("config.yaml.corrupt.*.bak")) == []
assert list(tmp_path.rglob("config.yaml.corrupt.*")) == []
def test_reused_args_can_retry_after_config_repair(tmp_path):

View File

@@ -68,8 +68,8 @@ def test_docker_config_migrate_backs_up_and_migrates_legacy_config(tmp_path: Pat
# max_async_children into max_concurrent_children.
assert "ttl_hours" not in raw["model_catalog"]
assert raw["delegation"] == {"max_concurrent_children": 8}
assert list(tmp_path.glob("config.yaml.bak-*"))
assert list(tmp_path.glob(".env.bak-*"))
assert list((tmp_path / "backups" / "config").glob("config.yaml.pre-docker-migrate.*"))
assert list((tmp_path / "backups" / "config").glob(".env.pre-docker-migrate.*"))
def test_docker_config_migrate_skips_below_floor_config_untouched(tmp_path: Path) -> None:
@@ -170,8 +170,8 @@ def test_docker_config_migrate_restores_backups_after_failed_migration(
assert config_path.read_text(encoding="utf-8") == original_config
assert env_path.read_text(encoding="utf-8") == original_env
assert list(tmp_path.glob("config.yaml.bak-*"))
assert list(tmp_path.glob(".env.bak-*"))
assert list((tmp_path / "backups" / "config").glob("config.yaml.pre-docker-migrate.*"))
assert list((tmp_path / "backups" / "config").glob(".env.pre-docker-migrate.*"))
def test_docker_config_migrate_restores_backups_when_version_does_not_advance(
@@ -244,7 +244,7 @@ def test_docker_config_migrate_second_boot_preserves_env_byte_for_byte(tmp_path:
assert env_path.read_bytes() == env_bytes_before
config_after_first = config_path.read_bytes()
first_boot_backups = sorted(tmp_path.glob("config.yaml.bak-*"))
first_boot_backups = sorted((tmp_path / "backups" / "config").glob("config.yaml.pre-docker-migrate.*"))
# ── Second boot (host reboot): version is current, must be a no-op. ──
second = _run_migration(tmp_path)
@@ -255,4 +255,4 @@ def test_docker_config_migrate_second_boot_preserves_env_byte_for_byte(tmp_path:
assert env_path.read_bytes() == env_bytes_before
# config.yaml is untouched by the second boot, and no new backup is made.
assert config_path.read_bytes() == config_after_first
assert sorted(tmp_path.glob("config.yaml.bak-*")) == first_boot_backups
assert sorted((tmp_path / "backups" / "config").glob("config.yaml.pre-docker-migrate.*")) == first_boot_backups

View File

@@ -355,7 +355,7 @@ No configuration is needed — caching activates automatically when an xAI endpo
xAI also ships a dedicated TTS endpoint (`/v1/tts`). Select **xAI TTS** in `hermes tools` → Voice & TTS, or see the [Voice & TTS](../user-guide/features/tts.md#text-to-speech) page for config.
**Retired xAI model migration (May 15, 2026):** xAI is retiring `grok-4*`, `grok-3`, `grok-code-fast-1`, and `grok-imagine-image-pro` on 2026-05-15. `hermes doctor` and `hermes chat` startup both detect any config still pointing at a retired ref and print the recommended replacement. Use `hermes migrate xai` for a one-shot config rewrite — dry-run by default, add `--apply` to write changes (a timestamped `config.yaml.bak-pre-migrate-xai-*` backup is created automatically).
**Retired xAI model migration (May 15, 2026):** xAI is retiring `grok-4*`, `grok-3`, `grok-code-fast-1`, and `grok-imagine-image-pro` on 2026-05-15. `hermes doctor` and `hermes chat` startup both detect any config still pointing at a retired ref and print the recommended replacement. Use `hermes migrate xai` for a one-shot config rewrite — dry-run by default, add `--apply` to write changes (a timestamped copy of the previous config lands in `backups/config/` first).
```bash
hermes migrate xai # preview replacements

View File

@@ -182,6 +182,8 @@ updates:
`pre_update_backup` is the single pre-update safety knob: `quick` (default) snapshots critical state files (pairing data, cron jobs, config, auth; files over 1 GiB are skipped) into `state-snapshots/`; `full` additionally zips all of `HERMES_HOME` into `backups/` and can add minutes on large homes; `off` disables both. Legacy booleans are honored (`true` → `full`, `false` → `off`).
Point-in-time copies of `config.yaml` itself (taken before `hermes setup` rewrites it, before `hermes migrate` edits it, and when the file fails to parse) go to `backups/config/config.yaml.<reason>.<timestamp>`. Identical repeats are skipped and only the newest five per reason are kept, so they never pile up beside `config.yaml`.
For git installs, Hermes auto-stashes dirty tracked files and untracked files before checking out the update branch or pulling. Interactive terminal updates prompt before restoring that stash. Non-interactive updates (desktop/chat app, gateway, or `--yes`) use `updates.non_interactive_local_changes`: `stash` restores local source edits after a successful pull, while `discard` drops the update-created stash after a successful pull. Use `discard` only on managed installs where local source edits are never meant to persist.
Before that stash step, Hermes also restores tracked `package-lock.json` diffs left by npm install/build churn. Commit or manually stash intentional lockfile edits before updating.