2 Commits

Author	SHA1	Message	Date
Teknium	0ee5ae61e2	docs(evals): real Codex CLI head-to-head arm + results ... scripts/codex_arm.py drives OpenAI Codex CLI end-to-end on the same transcripts: chunk-file reads until its REAL auto-compaction fires (verified via compacted events in the rollout jsonl; peak 455-483K vs its 258K window), then quizzes post-compaction with the identical question banks and judge. Results (results/codex-arm-2026-08-15/): codex 36.7% avg vs lean closed-book 40.0% vs lean+recovery 68.3%. Codex has no runtime re-access over its rollout history — the session_search differentiator, measured.	2026-08-15 18:41:24 -07:00
Teknium	e2990428e7	docs(evals): ship transcript-building scripts + full eval detail in-repo ... - scripts/reconstruct_lineage.py: rebuild full uncompacted lineage transcripts from a state.db COPY (descendant-tree walk, content-hash dedupe, synthetic-artifact strip, system_prompts hash resolution) - scripts/replay_lineage.py + scripts/build_html_report.py: replay a 500K prefix through any checkout's compressor and render before/after side-by-side with compaction artifacts color-coded - README: transcript-building workflow, scoping-tripwire section - results/SCORECARD-2026-08-15.md: full per-transcript scorecards, all 4 exam question banks, survival analysis, methodology + caveats	2026-08-15 17:40:43 -07:00