Commit Graph

2 Commits

Author SHA1 Message Date
ethernet
8f6d98e4c3 fix activation of devenv, use /usr/bin/env bash everywhere 2026-09-11 11:17:18 -04:00
Teknium
ced8e30217 feat(scripts): reproducible core-toolset A/B eval harness (toolperf_abeval)
Ships the hard A/B evaluation used for the August 2026 core-toolset
performance batch (#77056) as a reusable harness: 9 error-inducing trap
tasks derived from measured production waste classes, two-arm
PYTHONPATH-only comparison, ATOF-trace-based scoring, resume-safe
batteries.

Hardened from the original one-off: paths de-hardcoded (ABEVAL_ROOT /
ABEVAL_HOME), encoding= on all file IO, startup crashes retry on resume
instead of polluting cells, post-hoc grading fix for err_inline_script
baked in. Live-smoked end to end (baseline arm, qwen3-coder-30b,
err_multi_dir: exit 0, correct on-disk verification, resume record
written).
2026-08-05 13:43:30 -07:00