Every pre-apply exit of `hermes update` records one closed token on the
receipt at the exit itself (update_receipt.record_stop_reason ->
stop_class), and shared_metrics_update.update_failure_class maps it to
the run's failure_class. aborted_before_apply stays only as the fallback
for an exit with no recorded reason.
Exits that fire before the receipt opens (update-lock refusal, Git
operation in progress, managed install) write no receipt, as before, but
now record a metrics-only row with the same closed class.
The parked receipt copy keeps stop_class, exit_code, the stop reason's
leading label and the restart/user-action flags, so a parked run
classifies the same as the in-process one. A run that committed and is
owed only an interrupt or the user's parked changes reads outcome
partial (failure_class interrupted / local_changes_parked), never failed.
What: _sync_with_upstream_if_needed compiles the upstream commit's startup inventory before its
fast-forward (the origin target's preflight, applied to the second move) and raises
UpstreamTargetBroken without moving; both callers roll the update back the way the post-pull
guard did (the late one checks the branch first, so an unexpected branch's ref is never reset).
A later move of the run arms its obligations for its pre commit (debt_sha_for_move): every
landing of a fast-forward contains it and the fleet discharge holds the gateways to the actual
HEAD, so a failed merge needs no retarget write. The merge verifies HEAD is the counted commit
before reporting success.
Why: the fork ff dropped its marker on git's exit 0 and only the caller's later guard judged the
code, so a kill in between left a broken commit with no recovery record (G1); a failed merge's
retarget to the pulled commit could itself fail and leave the debt naming the unmerged commit
with HEAD on the pulled one (O4, the residual of F23).
Changed test: test_update_syntax_failure_restores_pre_update_head pinned the post-pull guard
catching the upstream target ("Pulled code has a syntax error") and the broken commit landing on
the unexpected branch; the target is now refused before its move, so it asserts the refusal
text and that the broken commit never landed. Outcomes (exit 1, HEAD back at base, stash) hold.
Tests (RED at fd5c37f17bf): a refused retarget leaves the debt on the pulled commit; a broken
upstream target is refused with HEAD unmoved and no marker.
What: disarm_commit_obligations, refused off the run's start commit, now re-arms the
obligations for the HEAD the checkout is actually on instead of leaving them on the failed
move's target; owe_restore_of arms them for the move's pre when the tree is torn under its
marker (the commit the next launch's restore lands on). Every failed-move site settles through
these: _pull_updates (any settle, not only one back at the baseline), the CP0 switch (exception,
torn and settled paths, and an attempt-1 arm left behind when attempt 2 is unresolvable), the
syntax rollback's failure path, and the upstream fork sync, whose merge catch now also takes
OSError/SubprocessError (CustodyRefused is an OSError) instead of letting it escape armed.
arm_commit_point no longer says "the checkout was not changed" when an earlier step of the run
(CP0) already moved it. The fork sync refuses a HEAD that is not an object id instead of
arming pre=, and a failed re-arm is logged and printed, not suppressed.
Why: a second move in one run re-armed the host obligation for its own target; when it failed
it settled back to a commit that is not the run start, disarm refused (head != start_head) and
nothing re-armed for the actual HEAD, so the record named a commit the checkout never holds
(undischargeable on a gateway-less host, wrong for the catch-up restart).
Review C1, C11. Tests (RED at 5dfc384669e): CP1 refused after CP0 owes A, not B, and the refusal
names the moved HEAD; a CustodyRefused upstream merge after the pull owes c1.
What: when the upstream ff is refused or settles back to pre, re-arm the obligation for pre
before the (refused-after-a-pull) disarm.
Why: the arm had retargeted the host record to the upstream commit C while HEAD stayed on B;
the fleet discharge refuses an expected_sha HEAD does not contain, so the debt was stuck.
Red (pre-fix): expected_sha == C with HEAD == B.
Green: expected_sha == B, obligations still armed.
What: arm_commit_point arms the obligations then the interrupted-pull marker as one hard
precondition (failure hands the obligations back and the caller stops before git): the origin
pull exits, CP0 returns a failed result (the caller restores the autostash), the upstream ff
skips. The pull resolves its target with a checked rev-parse --verify and refuses a non-OID.
The pull's refusals run inside the try whose finally names the autostash. The syntax rollback
(cannot refuse: the tree is broken) rolls back unmarked with a warning, and
settle_failed_tree_move no longer counts a missing marker as proof the tree is whole, so only a
tree verified at pre reports "Rollback complete".
Why: _best_effort/suppress(OSError) let git move with no marker (a kill left a torn tree no
launch could identify), an unresolved target armed 'target=' and still merged, and an arm
failure after the autostash exited with no stash guidance.
Red (91270e84a26 update_cmd.py/update_cmd_git.py/update_cmd_commit.py): IsADirectoryError escapes after the unmarked move (pull, CP0, upstream, rollback); unresolved target not refused; stdout '' (no stash ref).
Green: SystemExit/failed result/False with HEAD and tree unchanged; stash ref printed; rollback prints the manual recipe.
The tree-move marker's target= re-resolved the short upstream/main and
_push_synced_fork compared HEAD with upstream/main and origin/main: a local
branch named upstream/main shadowed both. The marker now records the commit
the sync counted and merged; the push check reads refs/remotes/*/main.
(cherry picked from commit 6525df9c7eb0feaa6d2406779c1559581b79325e)
(cherry picked from commit e1d17137115665d22443edec56443ebd52293b6a)
Review round 2 on the LP-COMMIT commit point:
- A failed upstream fork fast-forward after a committed origin pull no
longer hands back the obligations the origin pull armed.
`disarm_commit_obligations` now refuses unless HEAD is the commit the
run STARTED from (`record_run_start`), so every caller is covered; the
restore of the snapshot is an atomic replace.
- The ZIP swap journal counts a leftover `<entry>.hermes-update-old` as
"existed" (`_stage_replacement` puts it back before the swap), and the
`swapping` recovery treats any backup as proof the entry existed, so a
kill mid-swap can no longer delete both copies. A pre-run journal the
start-of-run settle could not clear refuses the swap instead of being
overwritten.
- CP0 runs `git checkout --no-guess <branch>` only when the local branch
exists; otherwise only the marked `checkout -B <branch> origin/<branch>`
runs. git's checkout guess used to create the branch and rewrite the
tree while the code believed the checkout could not move.
- The syntax rollback moves HEAD back to the pre-update commit first
(`reset --soft`, or `update-ref --no-deref` for a parked/detached
checkout), then `reset --hard`. The marker names pre = the good commit
and `rollback=<mode>`, so a kill anywhere lands the next launch on the
pre-update commit; recovery redoes the HEAD step if the kill came first.
- The syntax guards (preflight `git show`, ZIP compile gate, post-pull
backstop) skip a target whose `requires-python` excludes the running
interpreter: its syntax may be a newer Python's. Unknown specs keep
checking.
- The no-op exit in `_verify_head_after_pull` disarms (self-verified).
(cherry picked from commit c787a8c3f9823be8639f6f38efbbdff2b78a97cd)
(cherry picked from commit d785ad1c7e919bd7d64d17f2e9d664b84a7e8982)
Arm the source-completion tail and the fleet obligation in the parent before
any tree move; cover the CP0 branch switch, the upstream fork ff and the syntax
rollback with the interrupted-pull marker; settle (never drop) the marker on a
non-kill git failure; refuse foreign-owned venvs and uncompilable targets before
HEAD moves; push the fork only after a validated update; ZIP fallback only from
an untouched checkout; journal the ZIP swap so start-of-run recovery finishes or
rolls it back and no staging/backup sibling wedges the next run.
(cherry picked from commit 584acad44f3d6c552b7c2767d12c612de0436498)
(cherry picked from commit c6567ac2d28c421f384d95e1a6fc28912cb30f86)
Round 6 bound the branch switch, the upstream syncs, the pull and the rollback.
Other steps that run while the Windows pause holds the gateways still rewrote
tracked files unbound:
- `git stash push` and its cleanup `reset --hard`;
- `git stash apply` and the reset after a conflict;
- the rejected-restore `reset --hard` + `clean`;
- the lockfile-churn `checkout -- <locks>` and the line-ending `checkout`.
If one of them was interrupted mid-write, the gate judged the torn file only
against whatever refs a fetch had left. A restore after the pull had no
baseline at the moved HEAD at all. Either way the paused set was admitted.
Each step now runs inside `_checkout_move`. `paths=` binds the files it
rewrites in place, and `revert=True` binds every tracked change it finds,
because it puts them back to HEAD. A restore binds its stash commit. The gate
counts those paths as writable. A restore that applied its tracked changes in
full reports `whole`, so the tree it changed by design retires its baseline
instead of holding the set.
Removed the "already up to date" switch-back `checkout <current_branch>`. It
only ran when no checkout had left current_branch: in-place and release
updates never switch, and a parked switch is excluded. It was a no-op that
nothing gated. Helpers take a `checkout_move` binder (`_moves_for(token)`);
test doubles of `_prepare_git_command` accept its new keyword.
Only the principal pull recorded its target (mark_move) before git wrote the
tree. The first branch switch in _prepare_checkout_for_update (and its
`checkout -B` fallback), a fork's early and late upstream sync, and the
post-pull syntax rollback wrote the tree unrecorded: a torn switch or sync at
an unmoved HEAD was judged against whatever refs a later fetch left, and a
failed second move (upstream sync after a successful pull) had no baseline at
its starting HEAD at all, so the gate admitted the torn tree.
All of them now run inside _checkout_move(token, *targets). Before git runs it
persists the resolved target(s): on the pause's own baseline when the step
starts from the pause HEAD (it knows the edits already there), otherwise on a
new baseline taken at the step's starting HEAD with its dirty digests. A
record that cannot be written raises, so the step does not run. Once git
returns, a per-step baseline is retired if HEAD moved or the tree is exactly as
the step found it, so a clean refusal never holds the set. The upstream sync
takes the context as `checkout_move`; the pull's move is extracted to
_move_checkout_to so _pull_updates does not nest deeper.
The merge used refs/remotes/upstream/main but the commit counts used the short
upstream/main, which a local branch of that name shadows (refs/heads before
refs/remotes): the sync printed '1 behind' and merged 2 commits. The sync now
resolves refs/remotes/upstream/main^{commit} once and uses that SHA for both
counts (against refs/remotes/origin/main) and the fast-forward.
R2: hermes_cli/update_custody.run_git is THE updater git runner;
update_cmd._git_run, update_cmd_git._git_run, update_cmd_stash,
update_cmd_check._git, gitlock, update_cmd_zip and the fleet ancestry probe
all go through it.
- every updater git call carries -c gc.autoDetach=false
-c maintenance.auto=false: no detached gc/maintenance child ever forks
under (POSIX) or outlives (Windows) the checkout lock;
- POSIX: only local mutators (reset, checkout, merge, stash, rebase,
update-ref, clean, apply, commit, ...) inherit the lock fd, with
core.fsmonitor=false; fetch/ls-remote/credential and readers run without
it, so a credential-cache daemon never holds the checkout (reviewer: up to
2 h of installer waits). On Linux those fd-less children die with the
updater (PR_SET_PDEATHSIG) instead of rewriting refs under the next owner;
- Windows: inside an update every child is created suspended, assigned to
the owner's kill-on-close job and only then resumed; the Node build
(pm.progress.run_contained) joins it through a stdlib launcher.
The fork sync fast-forwards locally (merge --ff-only
refs/remotes/upstream/main after its fetch) instead of a networked pull
under the mutator fd.
Git builds without GIT_NO_LAZY_FETCH (upstream 2.43.0-2.43.3 and other
pre-May-2024 releases; the security backports added it to 2.39.4+) ignore the
override, so a parked branch with its own commits still sent git cherry into
an unbounded lazy fetch on a partial clone (#124767, git 2.43 on WSL). Live
with an unpatched git 2.43.0 build: 20 fetch children and 4 -> 24 packs for a
20-commit upstream, one fetch per commit. Cherry's patch-ids need blobs a
partial clone keeps remote, so it almost never refines the count there:
detect the partial clone from local config (remote.*.promisor, or the older
extensions.partialclone) and settle for the commit count. Same git: 0 fetches.
Co-authored-by: Yuan <xuyuann@icloud.com>
Review on #132378 (@andrexibiza) showed the regression oracle was the net pack
count: with foreground auto-GC (gc.auto=1, autoPackLimit=1, autoDetach=false)
a guard with the override removed still made two lazy fetches and left the
count at 2 -> 2. The test now captures Trace2 around the assessment and
requires zero fetch child starts, pins global/system git config, and runs
plain and foreground-GC variants; removing the override fails both.
The cherry child's environment now comes from noninteractive_git_env() like
the other no-lazy-fetch callers, instead of a raw os.environ copy that the
spawn-env guard rejects.
`_assess_parked_branch_switch` still ran `git cherry origin/<target>` whenever the
commit-graph count found parked-only commits. On a tree:0 partial clone that
patch-id walk lazy-fetches a tree batch per upstream commit, and nothing bounded
it: the #131444 trace identifies exactly this child (`git cherry origin/main`,
6 h 45 m, 332 packs / 180 GiB) as the writer behind the silent disk runaway.
Run the refinement with GIT_NO_LAZY_FETCH=1 (the NO_LAZY_FETCH_ENV every other
read-only updater probe already uses). A missing object now fails `git cherry`
fast and the conservative rev-list count stands in, so the guard can never turn
into a fetch loop; full clones keep the patch-equivalence refinement.
Test: a reachable promisor remote, a parked branch one commit ahead — the guard
must not add a single pack. Red on origin/main and on #124777 alone (4 packs
after vs 2 before), green here.
Refs #131444. Supersedes the bound/offline variants in #125529, #126693, #128335.
git cherry's patch-id walk needs upstream-side trees, which a tree:0
partial clone lazy-fetches in many promisor batches; one failed batch
made the guard call a clean checkout unverifiable and skip the update
(#124767). Count parked-only commits with rev-list (commit graph only)
first, keep cherry as a refinement, and degrade a cherry failure to the
conservative commit count instead of unverifiable.
A console-less parent (pythonw backend, detached daemons) allocates a visible
console per bare spawn, so Windows startup flashes ~7-8 console windows
(#117781) and the GPU statusbar flashes two more every 5s (#101895). Threads
creationflags=windows_hide_flags() through every hermes_cli spawn site the
desktop reaches: gitlock (tasklist probe + git plumbing), the scheduled-task
PowerShell probe, the updater's _git_run/_no_prompt_git_kwargs/_GIT_TEXT_KW
runners, the safe-directory config probe, and consolidates the statusbar's two
nvidia-smi reads per poll into one cached hidden query. The statusbar also
stops polling while the document is hidden and resumes on visibilitychange
(#120262).
Supersedes #117805, #117833, #102047, #120285 (thanks @JoaoMarcos44, @Finn763,
@monerostar, @szzhoujiarui).
The pre-fetch heal (#123254) and the post-update tag fetch now share
fetch_full_commit_graph, so both follow one clone-mode rule: a partial clone
repeats its own filter, a full clone fetches unfiltered (#122353), and only a
depth-limited clone whose history is really missing unshallows as tree:0. A
full clone grafted by a user's `git fetch --depth 1` already has its history
locally, so it unshallows unfiltered and stays full.
Also: the network-timeout message no longer claims "no response from the
remote" (a large transfer on a slow link hits the same cap), without adding a
classifier rule that would assert the opposite.
A depth-1 installer checkout far behind main could never update: the plain
main fetch pulled nearly the full object graph (merge-heavy history drags in
side-branch ancestry past the shallow boundary), hit the 300s network cap
mid-transfer every time, and the unshallow step that makes later fetches
incremental only ran after a successful update. Unshallow the checkout first
(commit-graph-only fetch with the same 900s budget the post-update tag fetch
uses, pulling the update branch along), non-fatally, and stop reporting a
wall-clock kill as 'no response from the remote'.
Fixes#123254
The forced `+refs/heads/<b>:refs/remotes/<r>/<b>` refspec was spelled out at
all three fetch sites (update --check, the apply fetch, the fork upstream
sync). Its leading `+` is load-bearing on depth-1 clones, so one helper,
`update_cmd_check.tracking_refspec`, now owns the string and the reason.
Tag-pinned --single-branch clones configure remote.origin.fetch as a tag-only
refspec, so fetching a branch by name writes FETCH_HEAD and never creates
refs/remotes/origin/<branch>. Every downstream rev-parse of the compare branch
then failed with 'Branch not found' even though the remote plainly has it
(#125112).
Cover the whole class: the --check fetch, the apply-path fetch, and the fork
upstream-sync fetch all pass an explicit +refs/heads/<branch>: mapping.
Verified A/B on a real narrow clone: branch-name fetch leaves origin/main
unresolvable, refspec fetch resolves it.
(cherry picked from commit 6ad2ff3f3f6190ad1fdaed4356d00406019e8dcc)
Both update shapes move HEAD off a detached commit: the branch update checks
out main, the release update checks out the release commit. A commit made on
the detached HEAD is on no branch, so after the move only the expiring reflog
reached it, and the branch path printed nothing about it (gated on #124643).
One helper, _park_detached_head, now runs at both sites before HEAD moves.
When HEAD is detached at a commit no ref contains (refs/stash excluded: the
autostash is dropped after the update, which is why the branch path parks
before stashing), it writes refs/hermes-update-backups/detached-<branch>-<ts>-<sha>
(the divergence rescue refs' scheme, now pruned on the same terms) and prints
the ref and how to list the commits. If the write fails the update stops
rather than orphan the work. A detached HEAD already on a branch or tag (a
pinned release) gets no ref. It replaces the release path's silent, never
pruned refs/hermes/pre-release/<sha>.
git wraps OpenSSH's rejection as "Could not read from remote repository",
never "Authentication failed", so an SSH origin with no GitHub key fell
through to the generic "Failed to fetch updates from origin." The new rule
names the cause and gives the one-line HTTPS switch.
Print the count of commits not on origin/<branch> next to the rescue ref and
a `git log origin/<branch>..<ref>` command, keep pruning the other ref kind
when one for-each-ref call fails, and drive the regression through the real
_pull_updates path against a temp git repo.
Co-authored-by: Bart Collet <8973350+bartcollet@users.noreply.github.com>
Co-authored-by: nca7777 <213462973+nca7777@users.noreply.github.com>
On the update's target branch a diverged history is resolved with
`reset --hard origin/<branch>`. Divergence there has two causes the checkout
cannot tell apart: an upstream force-push or rebase, where nothing local is
lost, and local commits on that branch, where the reset discards every one of
them. Only the orphan case (no common ancestor) wrote a rescue ref, so the
common case left the reflog as the sole way back: a 90-day expiry the user has
to know to reach for, in a checkout Hermes updates unattended. The custom-branch
path above already treats local commits as worth preserving; this is the same
work on the branch the update targets.
The reset itself is unchanged. `pre_pull_sha` is now anchored for both kinds,
under `refs/hermes-update-backups/diverged-<branch>-…` or `…/orphan-…`, and the
message names the recovery command for the diverged kind.
The #87694 footprint concern does not carry over. There `pre_pull_sha` is an
autostash orphan commit holding a full working-tree snapshot, which is what
could reach multi-GB. Here it is ordinary branch history, whose objects the
reflog already pins for its own expiry window. Both kinds expire under the same
keep-newest and max-age rules, which `_prune_orphan_rescue_refs` now applies per
kind rather than to the orphan prefix alone — otherwise the new refs would
accumulate forever, which is the actual shape of #87694.
Tests: a real two-repo divergence proves the discarded commit stays reachable
through the ref, and the existing mock test for this path now pins the new rule
instead of the old skip.
Signed-off-by: moep90 <volleyballlive@googlemail.com>
(cherry picked from commit 8b9568f9fd2e3811ec7afd8320dc432d135374e2)
Reconcile plugin declarations and validation through PM's atomic generation publication; preserve external runtimes, target markers, and conflict refusal. Keep one source-update completion owner and port upstream lifecycle changes to the PM desktop/runtime paths.
Path.glob raises NotImplementedError for a non-relative pattern, which a string `workspaces` (iterated char by char, so "/") or an absolute entry produces. The (OSError, ValueError, TypeError) catch missed it, so the error escaped to the caller's suppress(Exception) and no lock was reverted at all -- back to autostash every run. Non-list values are now ignored and each pattern is tried on its own so a bad one just owns nothing.
install.sh: read the workspace globs with `while read` instead of an unquoted $(...) so they are never pathname-expanded against the caller's CWD before `case` sees the pattern.
Keep the two discriminating cases from #112396: a dirty workspace manifest preserves the
root lockfile, and a manifest outside the workspace graph does not. The non-workspace case
in the original PR created an untracked file that `git diff --name-only` never lists, so it
passed on unfixed main; the fixture now commits vendor/foo/package.json first. The git
helper pins an author identity so the test runs on CI runners without git config.
Restore the WHY in `_discard_lockfile_churn`'s docstring (the autostash-every-run motive)
and document that `_npm_lockfile_owners` mirrors `update_cmd_deps._npm_manifest_paths`.
Co-authored-by: KoNit-K <124019182+KoNit-K@users.noreply.github.com>
A workspace manifest can change the dependency graph represented by the root lockfile. Discover the root package workspaces and preserve the root lock when a covered manifest is dirty, while retaining same-directory nested ownership and discarding unrelated churn. Fixes#112378.
Run historical updater completion in a fresh interpreter so cached imports
cannot revive retired dependency installers. Share Git and ZIP completion,
carry receipt and recovery state, and preserve child exit status.
Route plugin admission, binary acquisition, desktop launch and build paths
through PM. Replace redundant helpers and tests with real worker, package,
publication and launch checks. Keep the shipped compatibility surface fixed.
Targeted Python and desktop checks pass. Native update journeys and fresh
production image qualification remain pending. This is a checkpoint before
those acceptance runs.
Pin uv and uvx to the PM interpreter instead of ambient Python discovery.
A matching dependency stamp cannot prove that installed files still exist.
Repair now rebuilds the recorded workspace and lock in a fresh generation,
checks startup imports, and publishes the selection only after success.
Run startup recovery before dependency activation. Keep manual PM repair
reachable when the selected environment is damaged. Preserve plugin
selection, retry ownership, and the previous generation on failure.
Remove the separate pip, ensurepip, per-extra, and install-time quarantine
ladders. Keep orphan launcher restoration.
Verification: 717 targeted tests passed on native Windows ARM64, with
56 skipped. Ruff, diff checks, and the source-scoped compat check passed.
A disposable real Hermes install recovered deleted YAML and dotenv files,
then printed CLI help with exit 0. Its lock and stamp stayed unchanged.
The full suite and a release build were not run for this change.
Prepare dependency generations before selecting them. Keep shipped tool
bytes separate from writable additions, and store facts beside their entries.
Validate proposed plugin sets before config publication. Restore the previous
config if the facts write fails.
Consolidate duplicate updater, backup, setup, and voice helpers. Repair
launcher selection, dependency consumers, download ownership, update feeds,
and native Windows process and file handling.
Verification: 206 changed/prior-failing Python files reported 4630 passed,
one failed, and 330 skipped. Fix the remaining Hindsight fixture boundary.
The final targeted rerun reported 234 passed and two skipped. The store
review regression batch reported 83 passed and one skipped. Desktop
TypeScript checks, 56 selected Electron tests, 24 release tests, and the
removed-import/compatibility guards passed.
This is an integration checkpoint, not full audit acceptance. The complete
Python suite has not run on this fixed tree. Crash-atomic plugin publication,
generation cleanup, receipt correlation, and packaged lifecycle acceptance
remain open in docs/pm-audit-status.md.
For each issue anchor present in BASE 63279301bc non-test .py and absent on HEAD, the BASE comment/docstring block was re-attached at the HEAD location of the code it explained (matched by the distinctive code line / enclosing def). Sentences already covered by an existing HEAD comment were deduped; the issue number always survives. Insert-only: no code lines changed.
