Independent review: a nonexistent base ref under --strict reported zero
drops and exited 0 (a mis-fetched CI job would look clean); the script now
verifies both refs and the merge-base and exits 2 otherwise. And a method
moved from a class into a mixin/base defined in the same module that the
class still derives from was reported as removed although the attribute
still resolves; public_methods now collects the methods REACHABLE on each
class through its in-module bases. Replay of #102117 at open: 1,703 names /
341 modules and 126 test defs / 52 files unchanged; methods 1,000 -> 951
(the 49 were in-module mixin extractions, i.e. the false positives).
Test: unresolvable ref -> exit 2 in both modes; a method extracted into an
in-module base is not reported.
The Sep 2026 whole-codebase refactor (PR #102117) opened with 1,703 public
top-level names dropped across 341 modules, 1,000 public/dunder methods in
166, and 126 `def test_` deleted in 52 files. Reviewers found ~30 of the
names by hand; the rest surfaced as post-merge rework: 10 commits restoring
symbols and facade re-exports, 6 restoring tests, and a qwen OAuth break
that passed import smoke because the caller used `module.attr`. Every one
was catchable in seconds; nothing ran the check because it did not exist.
scripts/ci/check_public_surface.py: AST diff of modules present on both
sides of merge-base..HEAD. Public top-level names (defs, classes,
assignments, imported/re-exported names), public and dunder methods of
top-level classes, and `def test_` counts per tests/ file. Deleted modules
and deleted test files are visible decisions and are not flagged; private
names are not flagged. Advisory (exit 0, prints the report) by default;
--strict exits 1 so a refactor brief or a CI lane can gate on it. Wired
into lint.yml as an advisory PR step next to the compat-pointer check.
Replayed on the refactor PR at open (63279301bcb..022785a541) it reports
exactly the figures above in 18 s; on this branch vs main it reports 0.
Test: a throwaway git repo with drops, private drops, a move-with-re-export,
a lost test def and a changed non-source module; asserts the exact report
and the advisory/strict exit codes.
