feat(ci): public-surface diff vs base (dropped public names, methods, test defs), advisory on PRs

The Sep 2026 whole-codebase refactor (PR #102117) opened with 1,703 public
top-level names dropped across 341 modules, 1,000 public/dunder methods in
166, and 126 `def test_` deleted in 52 files. Reviewers found ~30 of the
names by hand; the rest surfaced as post-merge rework: 10 commits restoring
symbols and facade re-exports, 6 restoring tests, and a qwen OAuth break
that passed import smoke because the caller used `module.attr`. Every one
was catchable in seconds; nothing ran the check because it did not exist.

scripts/ci/check_public_surface.py: AST diff of modules present on both
sides of merge-base..HEAD. Public top-level names (defs, classes,
assignments, imported/re-exported names), public and dunder methods of
top-level classes, and `def test_` counts per tests/ file. Deleted modules
and deleted test files are visible decisions and are not flagged; private
names are not flagged. Advisory (exit 0, prints the report) by default;
--strict exits 1 so a refactor brief or a CI lane can gate on it. Wired
into lint.yml as an advisory PR step next to the compat-pointer check.

Replayed on the refactor PR at open (63279301bcb..022785a541) it reports
exactly the figures above in 18 s; on this branch vs main it reports 0.

Test: a throwaway git repo with drops, private drops, a move-with-re-export,
a lost test def and a changed non-source module; asserts the exact report
and the advisory/strict exit codes.
This commit is contained in:
Teknium
2026-09-05 01:48:00 -07:00
parent 2ed33fb38e
commit 8cbb2ce764
3 changed files with 240 additions and 0 deletions

View File

@@ -184,3 +184,12 @@ jobs:
# reverting one commit, so in-tree code must never depend on them.
- name: Forbid in-tree use of plugin-compat pointers
run: python scripts/check_compat_pointers.py
# Advisory: dropped public names / methods / test defs vs the PR base, printed into the log.
# A refactor that silently removes a public symbol breaks plugins that import it; the Sep 2026
# decomposition opened with 1,703 such drops that reviewers had to find by hand.
- name: Public-surface diff vs base (advisory)
if: github.event_name == 'pull_request'
run: |
git fetch --no-tags --depth=1 origin "${{ github.base_ref }}"
python scripts/ci/check_public_surface.py --base "origin/${{ github.base_ref }}" --head HEAD