744f05e220
worker_bootstrap() logged a failed activate_dependencies() and let the worker carry on through the inherited PYTHONPATH pin. That path is not leased: once the gateway releases its generation the collector deletes it under the live worker and the next dependency import dies mid-job (#122936 review, reproduced with a committed generation that has no site-packages). The failure now propagates, so the worker exits before its ownership ack and _launch_external_cron_worker reports the dispatch failure with the worker's stderr. PM's own no-op cases (venv, wheel, Nix, nothing committed under a venv) still return normally. Tests drive the real PM boot in a marked child: it imports a sentinel that exists only in the committed generation, and when an update commits a newer one mid-job the real collector keeps the worker's generation until the worker exits. A damaged selection exits before cron.jobs is imported. 123456hyu made the same prelude move and fail-closed change on #122290 (dff16e12); this carries them without the argv gate and source-order test. Co-authored-by: 123456hyu <123456hyu@users.noreply.github.com>
86 lines
3.9 KiB
Python
86 lines
3.9 KiB
Python
"""Cron: import path of the restart-safe external worker.
|
|
|
|
The worker is spawned as ``sys.executable -m cron.scheduler``. Its entry module is
|
|
``cron.scheduler``, not ``hermes_cli.main``, so nothing bootstraps the gateway's checkout
|
|
onto its ``sys.path``; historically it imported ``cron`` only through the implicit ``-m``
|
|
cwd entry. That entry is gone under ``PYTHONSAFEPATH`` and useless when the venv's
|
|
editable install maps a moved/deleted checkout -- the worker then dies with
|
|
"No module named 'cron'" before its ownership ack (#112729, hypothesised cause).
|
|
|
|
The shared subprocess sanitizer strips Hermes-owned PYTHONPATH entries because user
|
|
children must not see our tree. This child IS Hermes, so the pin is applied *after* the
|
|
env is built, on the sanitized env. On a self-managed (shell-installer / PM) install the
|
|
sanitizer's drop of the runtime site-packages cannot stand this time: the worker inherits
|
|
this process's interpreter, which is PM's store Python and owns no third-party
|
|
dependencies, so the committed generation's site-packages is restored here too or the
|
|
child dies at its first dependency import (``No module named 'ruamel'``, #122222) before
|
|
it can publish its ownership acknowledgement.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import os
|
|
import sysconfig
|
|
from pathlib import Path
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def _installed_purelib() -> Path | None:
|
|
try:
|
|
return Path(sysconfig.get_paths()["purelib"]).resolve()
|
|
except (KeyError, OSError):
|
|
return None
|
|
|
|
|
|
def _committed_dependency_site_packages(project_root: Path) -> Path | None:
|
|
"""The dependency ``site-packages`` PM committed for this install, or ``None``.
|
|
|
|
Asked of PM's own committed selection -- the record ``activate_dependencies`` resolves
|
|
at process boot -- rather than re-derived from this process's ``sys.path``: the record
|
|
belongs to this install (``install_state_dir``, shared by every profile it serves). A
|
|
runner that owns its dependencies (wheel / pipx / developer venv / Nix: no committed
|
|
generation) has nothing to restore, so ``None`` means "pin the tree only" and nothing
|
|
is invented.
|
|
"""
|
|
try:
|
|
from pm.environments import committed_venv, site_packages
|
|
|
|
environment = committed_venv(project_root)
|
|
except Exception as exc:
|
|
# An unreadable record: pin the tree only. The cron worker's own boot re-reads it and
|
|
# fails the dispatch with PM's error.
|
|
logger.warning(
|
|
"cron worker: could not read the committed dependency environment: %s", exc
|
|
)
|
|
return None
|
|
if environment is None:
|
|
return None
|
|
selected = site_packages(environment)
|
|
return selected if selected.is_dir() else None
|
|
|
|
|
|
def pin_hermes_tree_on_pythonpath(worker_env: dict, repo_root: Path) -> dict:
|
|
"""Prepend ``repo_root`` -- and, when this runner has one, the committed dependency
|
|
generation's ``site-packages`` -- to the worker env's own PYTHONPATH (never
|
|
``os.environ``'s).
|
|
|
|
Skipped when ``repo_root`` is the interpreter's ``purelib``: under a wheel / pipx /
|
|
uv-tool install ``cron/`` lives in site-packages itself, which is already importable,
|
|
and pinning it would move site-packages ahead of the stdlib on ``sys.path``.
|
|
|
|
Order (checkout, generation, sanitizer-kept) mirrors ``activate_dependencies``' own
|
|
``sys.path``. For the cron worker, its boot (``cron/worker_bootstrap.py``) then re-selects
|
|
and leases the committed generation before any third-party import, and exits the worker
|
|
if it cannot.
|
|
"""
|
|
root = str(repo_root)
|
|
if _installed_purelib() == Path(root).resolve():
|
|
return worker_env
|
|
existing = [e for e in worker_env.get("PYTHONPATH", "").split(os.pathsep) if e]
|
|
dependency = _committed_dependency_site_packages(Path(root))
|
|
pinned = [root, *([str(dependency)] if dependency is not None else [])]
|
|
worker_env["PYTHONPATH"] = os.pathsep.join(dict.fromkeys([*pinned, *existing]))
|
|
return worker_env
|