Commit Graph

42 Commits

Author SHA1 Message Date
ethernet
9214174e07 fix(ci): lint legs after the main merge
- check_no_tmp_literals: resolve scratch via tempfile/os.tmpdir; the termux
  container mount point is one marked variable per script
- ruff TID251: desktop E2E fixtures may reach PM internals like tests do;
  the pm.runtime_stage ban message no longer names a module that never existed
- auth_codex: build the capped httpx stream subclass on first use so importing
  hermes_cli.auth_codex no longer forces httpx (the lazy proxy in auth_constants
  was defeated by a module-scope base class; broke lanes without httpx)
- desktop-smoke: launchApp is a parameter; the bundle-env test substitutes a
  refusing launcher instead of letting Playwright spawn a dying binary
  (3 unhandled rejections failed the tests-js lane)
2026-09-19 23:25:18 -04:00
ethernet
cb7c688171 test: mirror the source tree for 27 misfiled root tests; one PM home fixture
Root-level tests/ is for root-level modules; 19 files testing scripts/, 3 testing
pm/ and 5 testing hermes_cli/ move to the mirrored directory (workflow file lists
and cross-imports updated; path arithmetic bumped one level).

tests/pm gains a conftest with the isolated_machine_home fixture that seven
modules had copy-pasted verbatim.
2026-09-18 20:13:26 -04:00
ethernet
55d317eca7 fix(release): accept CalVer majors in the canary tag shape
The canary tag regex capped the major at 3 digits while the stable
shape had no cap. The repo's current stable line is CalVer (v2026.9.14),
so canary_tag_for_date cuts v2026.9.15-canary.<ts> — which
handoff.validate_identity then rejected as 'Invalid release handoff
identity', killing every tag-mode stage leg (Windows and Darwin) while
the same-minor stable staged fine.

Lift the cap in the canonical _CANARY_TAG_RE and the Termux mirror
regex; add the stable-vs-canary shape-parity invariant, proven red on
the base regexes.
2026-09-16 12:07:32 -04:00
ethernet
4fbec9c442 feat(pm): repair retired termux pool pins from pm update --termux
The termux-main pool deletes a package's previous archive when it rebuilds, so
the runtime-lib pin table and the bionic lock rows rot without warning. The last
rotation broke a build on eight rows at once, and the stager's concurrent
downloads only surfaced whichever 404 won the race.

pm now owns the pin table it repairs: scripts/termux/runtime_libs.json moves to
pm/termux_runtime_libs.json, so pins live in pm/ and scripts consume them — the
direction scripts/ci/archive_inputs.py already reads pm/lock.json in.

`hermes pm update --termux` repins exactly the rows whose archive the pool has
replaced, hashing each replacement against the index SHA256 before writing
url/version/hash together. `--check` reports without writing and exits 1, so a
retired pin can fail a cheap preflight instead of a payload build.

It is a repair, not an update: an alive pin is never moved, because a repin can
land a rebuilt library under a moved soname and the table is the payload's
recursive DT_NEEDED closure. A pin whose package the pool has dropped outright
is reported and left alone. `--termux` runs alone — names/--target/--uv/--npm
are ignored, since repairing foreign-target pins is not a version resolution.

Verified: `pm update --termux --check` against the live pool reports 89 rows
served; a table deliberately pinned to the retired libiconv 1.18-1 repins to
1.19 with the pool's hash through the real network path; 19 new tests; the
tests/pm, tests/ci and tests/scripts suites have the same failure set as the
base commit (91 pre-existing Windows environment failures, none new).
2026-09-16 11:46:30 -04:00
ethernet
5c002ac298 fix runtime libs 2026-09-16 11:10:34 -04:00
ethernet
ce4ff47aa3 fix(termux): give pillow-heif libheif in the build container and the payload
pillow-heif's sdist build failed with RequiredDependencyException: the
container toolchain installs pillow's libs but not libheif, so
pkg-config libheif had nothing to answer with. Termux's libheif 1.23.4
ships the headers and libheif.pc directly (no -dev split).

Runtime: the built _pillow_heif extension DT_NEEDEDs libheif.so, which
pulls libx265/libde265/libaom/libx264/libc++_shared. All but libheif
and libde265 were already staged in runtime_libs.json (verified by
reading the .so's DT_NEEDED, not the deb Depends line, which also
lists gdk-pixbuf/glib/rav1e the linker never loads); pin those two.

Import gate: import pillow_heif never fails on a dead link -- its
__init__ swallows the _pillow_heif ImportError into a DeferredError
that fires only on first use. The gate now imports _pillow_heif
directly so a dlopen failure fails the build instead of the phone.
2026-09-15 11:28:36 -04:00
ethernet
176d1c771e fix(build): decode release and Termux inputs consistently 2026-09-13 18:21:59 -04:00
ethernet
9a42d60f25 fix(build): handle large release uploads and trim packaging waste 2026-09-13 11:04:03 -04:00
ethernet
d81c0a3fcc fix(termux): validate updater refusal by artifact identity
Commit builds have no update channel, even when installed through dpkg. Read the installed stamp before checking the refusal message and require exit code 2 for both artifact kinds.

Verified real CLI refusal paths, negative controls, and related tests: 15 passed. Ruff and type checks passed. Full deb validation was not rerun.
2026-09-12 11:07:42 -04:00
ethernet
febb908bd1 fix(build): preserve toolchain state and Termux assembly ownership
Reuse a matching ARM64 Visual Studio environment instead of growing its paths on each initialization. Preserve Rust and Cargo homes before isolating bundle state. Hand the private Termux assembly directory to its container user and restore host ownership afterward.

Verified targeted tests, real ARM64 SDK/OpenSSL execution, Rust compilation, and container facts publication. Full release packages were not rebuilt. The Windows x64 source-build timeout and the unchanged Termux linkage fixture failure remain unresolved.
2026-09-12 02:05:48 -04:00
ethernet
a154b89b9f Route build and CI Python preparation through PM operations 2026-09-11 18:14:43 -04:00
ethernet
1bf588234c refactor(build): share product recipes across distributions
Build TUI, web, desktop UI and runnable agent products from explicit
prepared inputs. Keep dependency preparation separate from distribution
packaging, with PM and native builds sharing uv environment construction.

Docker copies compiled frontend products instead of build dependencies.
Nix retains uv2nix environments and consumes shared assembly through store
references. Native desktop and Termux use the same launcher and frontend
contracts. Preserve the independent PM runtime and source imports from
arbitrary working directories.

Keep failed frontend builds from replacing the previous product, reject
source/output overlap, and bound dependency-process output draining.
Include hermes_wisdom in the Nix wheel: real CLI smoke tests exposed its
missing package declaration on the base revision too.

Verified focused Python and JavaScript suites, Docker build/runtime checks,
Nix desktop and CLI/ACP checks, standalone TUI and packaged Electron PTY,
and real full-Chromium interaction. Native signed installers, Android device
installation and the full repository suite remain CI verification.
2026-09-11 13:16:55 -04:00
ethernet
284dbaf537 fix(pm): isolate bootstrap dependencies and unify YAML on ruamel
Activation reaches plugin discovery before the application dependencies
exist. Give PM its own locked Python project and runtime so it can install
or repair the application without importing that dependency tree.

Keep PM outside the application workspace. A shared uv workspace resolves
the application graph and cannot provide this isolation. Route mutations
through an isolated worker and preserve transaction callbacks, cancellation,
custom package registrations, and correlated receipts.

Use the same runtime builder for source installs and packaged payloads.
Keep offline wheelhouse support in that builder. Nix builds the independent
PM lock as a separate derivation. Refuse lazy-disabled bootstrap before
installing tools or dependencies.

Move first-party YAML readers and writers to ruamel. Keep the application
lock's transitive PyYAML requirements for third-party packages.

Verification:
- Focused canonical Python suite: 177 passed, 1 host-gated skip.
- Electron backend probes: 12 passed. Electron typecheck passed.
- Both uv locks, scoped lint, Bash syntax, and whitespace checks passed.
- Cold activation, corrupt-app repair, offline staging, and relocation ran.
- Built and exercised the Nix PM runtime and standalone YAML merge script.

Six broader caller test files retain the same 24 failing test IDs as an
archive of HEAD. The existing real-home guard blocks those tests before
they can exercise the affected paths. No full-suite pass is claimed.
Native Windows signing and full Bionic package execution remain unverified.
2026-09-11 12:23:51 -04:00
ethernet
c8a9682505 fix(pm): preserve pinned binary inputs in R2
Termux removes old package files, so a pinned URL and hash do not keep
build inputs available. Preserve the exact bytes without changing pins.

Archive every PM HTTP artifact and the Termux runtime inputs by SHA256.
CI reads R2 first. Only a missing object permits an upstream download,
hash verification, immutable upload, and verified readback. Seed the
actual toolchain and payload stores before their consumers run.

Use the public archive as a pinned fallback in PM, bootstrap installers,
and Nix fetchers. Keep network retries bounded and report attempted URLs.
Keep publication credentials in protected CI jobs, not installed clients.

Verification:
- 283 targeted tests passed; five POSIX tests skipped on Windows.
- All 87 preserved Termux packages passed local archive miss/hit checks.
- Native ARM64 ripgrep installed through the mirror and ran successfully.
- Wheel import, workflow lint, Python lint, shell syntax, and pins passed.

Live R2 publication, POSIX tests, and Nix builds remain for native CI.
The real-byte archive checks used loopback HTTP, not the live bucket.
2026-09-10 18:17:08 -04:00
ethernet
063cf4428a feat(release): build and stage admitted commits without channels
Keep commit admission on the trusted workflow checkout and reject mixed
release inputs before loading repository code. Stage every built product
under its commit with receipt-bound summary links, never channel writes.

Build both Windows universal bundles through the existing SDK scripts.
Keep Store calendar versions separate from sideload app versions so zero-
major app versions remain packageable. Reject invalid arguments before
modifying bundles. Bind desktop and Termux versions to the source commit,
and record Termux cache provenance without labeling commits as tags.

Verification: 77 Python tests and 36 JS tests passed. Real makeappx packed
and unpacked disposable per-arch and universal packages. Seven official
workflow-expression checks, actionlint, syntax, lint and prose passed.
No signing, installed-app update, Android build, or remote dispatch ran.
2026-09-10 05:42:18 -04:00
ethernet
3f43d634d2 fix(termux): replace the retired libffi runtime archive
Termux removed libffi 3.5.2 from its rolling package pool. The pinned URL
returns 404 and stops runtime-library staging. Pin the available 3.8.0
archive with its downloaded SHA-256, which matches the package index.

Name the package, version and URL when a download fails. Flush staging
progress so piped build logs preserve the failure order. Update the
bionic Python tests to follow the current supplier and binary layout.

Verified all 83 library/license archives and 265 staged shared libraries.
The complete staging step and its verified cache hit both returned 0.
The pinned Python ctypes extension finds its required libffi symbols.
Focused tests: 19 passed, 2 skipped. Bionic execution remains a CI gate.
2026-09-09 00:01:51 -04:00
ethernet
6590ecdc2d fix(termux): pin upstream psutil Android support
Pin Android psutil to upstream commit
380bd2b59c67b0e1b04bbf3a90b11744f4f96644. It contains the unreleased
platform recognition and disk_partitions fixes. Other platforms keep 7.2.2.
Remove the local psutil source patch.

Retain the Git source through requirement normalization and wheel builds.
Use its locked version for offline installation. Provision Git in the
builder and host parsing dependencies through an isolated uv environment.
Wire the source-pin regression tests into Termux verification.

Targeted tests, lock checks, Ruff and shell/workflow checks passed. A real
wheel from the pinned source built and ran on Windows ARM64. Cold-cache
host normalization also passed without packaging installed on the host.
Full bionic compilation remains unverified.
2026-09-07 18:17:06 -04:00
ethernet
7bb62782cc merge: integrate ethie/py314 into ethie/pm-clean
Bring in the Python 3.14 runtime pins and wake-engine changes while
preserving the staged stable-release gate and review fixes.

The merge has no conflicts. Targeted tests on the existing Python 3.11
dev environment passed: 130 passed, 8 skipped. The lock check passed
with Python 3.14.7. Workflow lint and shell syntax checks also passed.
Full Python 3.14 runtime and native release acceptance remain for CI.
2026-09-07 15:12:55 -04:00
ethernet
b0ab0162b0 feat(release): gate stable promotion through the full release pipeline
Run the entire CI workflow before Docker build and tests. Require Nix,
native payload smoke tests, install/update E2E and signed-package upgrade
acceptance before publishing. Keep Desktop Playwright E2E deferred.

Archive tested Docker images and signed bundle candidates with provenance
and hashes. Publishers consume those exact artifacts without rebuilding.
Advance stable channels only after all required publications succeed.
Keep canaries on their separate path and reject direct stable-builder
publication that bypasses the gate.

Move shared release transport, manifests and gates to Python. Keep native
Electron adapters in JS and share feed/MIME facts as JSON. Replace the
R2/feed JS implementation and move its protocol tests to Python.

Verified targeted Python and JS tests, real loopback transport and CLI
execution, temporary Git admission, workflow graph lint, and typechecks.
No live stable release was run. Native signing, package upgrades and real
registry/Store promotion still need their release-run receipts. Separate
services cannot promote atomically. A promotion failure keeps the run red.
2026-09-07 14:40:10 -04:00
ethernet
cd0f97f833 feat(python): pin bundled runtime to 3.14 everywhere (pm, termux lane, CI, installers)
pm python node: 3.14.7+20260901 (freshest python-build-standalone 3.14
build) for the 6 desktop targets; the bionic row moves from the third-party
TUR python3.11 deb to the official termux-main python_3.14.6-1 deb (which
lags PBS by one patch — pinned manually, documented). All 7 digests fetched
from the live sources (PBS release API + termux-main Packages index).
pm/packages.py: main_bin_rel python3.14, deb_package python, bionic fetch
constant, latest_versions guards bionic (no PBS build exists).

termux lane: PYTHON_ABI cp311->cp314, python3.11->python3.14 paths,
libpython3.11.so->3.14, TARGET_ENV 3.11.15->3.14.6 AND sys_platform
linux->android (CPython 3.13+ reports 'android', docs-verified) — linux-
gated markers no longer admit the termux target. runtime_libs.json needs no
change: every python 3.14.6-1 Depends is already staged.

CI: python-version/--python 3.11->3.14 across all 11 workflows incl. the
uv lockfile-check lane. Installers derive the minor from the lock already;
fallbacks bumped. Sandbox images nikolaik/python-nodejs:python3.11-nodejs20
-> python3.14-nodejs22 (tag exists). runtime_repair fall-forward cap now
tracks the <3.15 requires-python window. Docs/README python version claims
updated.
2026-09-07 14:19:18 -04:00
ethernet
1a09c42414 refactor(bundle): share Python payload assembly across desktop and Termux 2026-09-06 22:21:06 -04:00
ethernet
0b30c2484d merge: integrate termux cli bundles into pm-clean
Merge ethie/cli-bundles at 0765ad689b.
Keep PM runtime publication, install identity, TLS policy, and module
boundaries from pm-clean.

Resolve the Node version-discovery method in its owning class. Preserve
staged tools if a repin download or publication fails. Carry extra-only
memory-provider setup through PM and retain restart-required reporting.
Keep target-specific TUI path assertions and discard obsolete self-lock
fixtures and the orphaned Windows service handler.

Verified locally with the canonical Python runner, root JS checks, TUI
checks/build, shell parsing, and workflow YAML parsing. Existing platform
pins and executable modes are unchanged. No full-suite CI, new bionic
bundle, or phone acceptance is claimed for this merge.
2026-09-06 20:13:45 -04:00
ethernet
0765ad689b fix(termux): publish immutable APT indexes and verify the CDN path
Cloudflare cached Packages.gz while serving a new signed Release. Advertise standard Acquire-By-Hash, publish SHA256/SHA512 index objects first, mark mutable APT metadata no-store, and run a real public APT install after upload.
2026-09-06 17:23:16 -04:00
ethernet
3b9c76b88a test(termux): reap the full TUI process tree before cleanup
Real CLI, ffmpeg, pm, APT refusal and TUI startup passed. A detached gateway child still wrote pycache after SIGTERM; use the existing tree terminator and wait for all descendants before deleting the test home.
2026-09-06 17:05:34 -04:00
ethernet
b0c7d0eb47 test(termux): verify upgrades from successful package artifacts
Use a successful admitted release run as the upgrade source. Install its actual deb, upgrade through the signed APT repository, and check user data retention before publishing.
2026-09-06 16:39:54 -04:00
ethernet
fe40130d62 test(termux): keep durable runtime checks and verify the prebuilt TUI
Remove the one-run linkage diagnostic and its expiring artifact dependency. The production wheel import gate and real ELF regression remain. Check the TUI in isolated CI and assert pm exports by behavior, not reloaded function identity.
2026-09-06 16:19:59 -04:00
ethernet
dd4bcda813 fix(termux): rebuild cached runtime notices with their libraries
A changed runtime table must discard all prior package-owned files, including copyright symlinks. Also isolate updater tests from previously collected native modules and exercise the post-merge deferral ordering instead of inspecting retired source symbols.
2026-09-06 16:10:09 -04:00
ethernet
f73ae58f5f test(termux): install from the signed APT repository before publishing 2026-09-06 15:56:35 -04:00
ethernet
203c0a96aa fix(termux): carry the Vulkan loader and finish runtime diagnostics
The clean non-root install passed native imports but ffmpeg needed libvulkan.so. Bundle Termux's real generic loader and exercise media conversion in the bare runtime. Restore doctor imports, pm-venv recognition, and current diagnostics seams.
2026-09-06 15:41:56 -04:00
ethernet
85d8c651ac fix(termux): preserve copyright links and their shared license files 2026-09-06 14:39:14 -04:00
ethernet
1ce0998ac9 refactor(termux): reuse package requirements and harden launchers
Use one requirements writer for the wheelhouse and installed venv. Do not retry failed hashes or paused downloads. Skip pure-wheel decompression, preserve runtime library notices, and keep the current working directory off the launcher import path. Document the prerelease canary package without migration or downgrade guidance.
2026-09-06 14:34:52 -04:00
ethernet
7f6d63bcbb fix(termux): classify bionic from its interpreter and run the Linux tests
The POSIX runner had a blank line after exec env, so it printed the environment and never ran pytest. Keep the command attached and prove failure propagation. The actual payload records ANDROID_API_LEVEL in sysconfig; use that instead of looking for text in a guessed libc file. Rebuild unproven runtime-library extracts from verified archives.
2026-09-06 14:12:12 -04:00
ethernet
7606b014f5 fix(termux): complete the sealed CLI payload and verify installed startup
Stage npm, ffmpeg and its bionic runtime libraries, and static ARM ripgrep. Bind caches to actual build inputs. Generate entrypoints from the project manifest and verify real CLI/TUI startup and media conversion offline. Keep versions unchanged. Windows service work remains out of scope.
2026-09-06 14:00:42 -04:00
ethernet
5b64b060f4 fix(termux): link native Python wheels to the shipped interpreter
Real bionic CI reproduces anydoc's missing _Py_NoneStruct symbol. The symbol exists in the shipped library, but the wheel has no libpython dependency. Link extensions that use Python symbols explicitly and rebuild RECORD before the unchanged offline import gate. Keep abi3 intact.
2026-09-06 12:54:12 -04:00
ethernet
842e3b0eed test(termux): isolate native Python linkage on an ARM runner 2026-09-06 12:06:22 -04:00
ethernet
f00c47de3a fix(termux): import gate maps firecrawl-anydoc to its anydoc module
main's uv.lock added firecrawl-anydoc (a native dep); the gate's
dist-to-module mapping lacked it and the dash-to-underscore fallback
guessed firecrawl_anydoc, which is not the package's module (anydoc).
2026-09-06 10:51:04 -04:00
ethernet
15a427fbe9 fix(termux): repair the import gate's mangled module-split script
The gate merge dropped the commas from the -c payload: split('') turned
the comma-separated module list into one unimportable name. Restore the
exact original payload (split(','), quoted print label) and verify it
executes.
2026-09-06 10:02:00 -04:00
ethernet
126b8fad78 fix(termux): restore the dropped command substitution in the digest read
The digest-reader consolidation dropped the $( ) around the python
one-liner in build_builder_image.sh (the heredoc form did not carry it
visibly), so DIGEST held the literal command text and the builder image
tag became an invalid reference.
2026-09-06 09:17:43 -04:00
ethernet8023
e57596db6f feat(termux): gpg-signed apt repo staging (dists/pool, xz control)
Pure-stdlib stager for the static apt layout: Packages(+gz), an
apt-valid Release (Date field, checksums in the same deb822 stanza --
learned from a real device rejecting the first shape), InRelease +
Release.gpg signed with the repo key (passphrase support via env), the
signing pubkey exported alongside, per-suite immutability, and nightly
versions that sort below stable. Control members are xz (our dpkg-deb
builds -Zxz; the stager also tolerates gz and zstd-via-binary).
2026-09-05 20:00:00 -04:00
ethernet8023
9e94770057 feat(termux): fat self-contained .deb with prebuilt TUI + runtime libs
Assemble the sealed hermes-agent aarch64 .deb: deps-only venv built
AT its on-device path (per-subdir mounts, staged uv, no app wheel --
Hermes is not pip-installable by design), trampolines that resolve their
symlink chain and export the payload linker path (runtime libs derived
from the suppliers' own Depends metadata), the prebuilt TUI bundle, and
the code-scoped .install_method stamp (restoring the 'apt' lane this
distribution needs). Validation runs in the BARE pinned base -- real
extraction, real postinst, C-extension imports, bundled node, TUI syntax
check, and the steward-refusal contract. A derived builder image
(toolchain pre-baked at uid 1000, content-addressed off the lock digest)
keeps cache-miss runs fast.
2026-09-05 20:00:00 -04:00
ethernet8023
0ae85925f3 feat(termux): pinned termux toolchain via pm (linux-arm64-bionic)
A seventh pm target (linux-arm64-bionic) stages the TUR python3.11
.deb, the termux nodejs/uv .debs, and their runtime-lib deps into the
payload -- same pm-consumer shape as the desktop legs: pm owns the pin,
the hardened download (redirect-safe, retry-wrapped), the ar+tar
DebPackage extraction, and file-evidence verify for binaries the
staging host cannot execute.
2026-09-05 20:00:00 -04:00
ethernet8023
e84e625d87 feat(termux): android wheelhouse builder + PEP 738 retagger
Build the full dependency wheel set for bionic from sdist inside the
digest-pinned termux-docker container, using the STAGED payload python
(so the ABI matches the shipped interpreter by construction). Native
sdists (rust/clang) compile serially against the container toolchain;
wheels land in a payload wheelhouse that later gates an offline,
marker-intact venv install. A cache-hit fast path (index.json verified
sha-by-sha) skips the whole container phase when the restored wheelhouse
is already the proven graph.
2026-09-05 20:00:00 -04:00