Stronger-teacher (D2 continuation) afterstate corpus: ranking gate and frozen override rule
The iteration-3 valid negative used a weak phase-greedy D1 continuation for its H40 labels.
On this page
- Created
- Updated
No explanation has been written for this record yet.
Technical recordThe registered protocol
- Hypothesis
- The iteration-3 valid negative used a weak phase-greedy D1 continuation for its H40 labels. Regenerating the successor-closed corpus with a stronger public teacher (fair D2, five chance samples) at K=64 over the same training roots produces labels whose trained model ranks fresh held-out siblings closer to fair D4 than the D1-teacher model did (top-1 gap to D4 shrinks by at least half), and the frozen override rule then passes on fresh origins.
- Arms
Arm Name Entry point Manifest Candidate distributional-afterstate-ranker-d2 approaches/afterstate-learning/distributional-afterstate/common.hpp– Comparator fair-d4 approaches/fair-expectimax/reference/fair-only-depth4.cpp– - Classification
- algorithmic
- Information boundary
- public-policy
- Benchmark tier
- CHECK
- Lifecycle
- completed
- Primary metric
- Fresh held-out top-1 accuracy vs fair D4 against the scenario-mean H40 D1-continuation target (K=256), compared with the D1-teacher model's iteration-3 result (0.4245 vs D4 0.4986).
- Secondary metrics
- pairwise accuracy and normalized regret vs D4
- eligible-root override regret vs D4 per half-fold under the frozen rule
- override rate
- decisive-root label stability (floor 0.5)
- quantile coverage
- corpus generation wall time and work per root
- Statistical unit
- whole-origin
- Uncertainty method
- Two deterministic held-out half-folds by origin-game hash; comparisons must hold in both halves.
- Data role
- public-development
- Seed leases
SL-20260820T083000Z-5da70000
- Dataset references
DS-20260820-afterstate-h40-k256-9d31c47a
- Whole-origin split
- yes
- Reuse disclosure
- Training: the same harvested training roots (seeds 0x5da70000-0x5da700ff) relabeled with the D2 teacher at K=64, H=40. Ranking gate: fresh seeds 0x5da70500-0x5da7053f (64 origins, D1-continuation K=256 target for cross-model comparability). Override gate: fresh seeds 0x5da70600-0x5da7063f (64 origins, same frozen rule and target as EX-20260820-d4-toptwo-override-gate-0bdb39a1).
- Pass criteria
- Corpus successor-closed at K=64 with the D2 teacher; completeness 1.0.
- Ranking gate: model top-1 >= (0.4245 + 0.5 * (0.4986 - 0.4245)) = 0.4616 on each half-fold, i.e. the D1-teacher gap to D4 shrinks by at least half.
- Override gate: eligible-root override regret <= D4 regret - 0.01 in EACH half-fold, with override rate >= 5%.
- Decisive-root label stability >= 0.5 for both evaluation corpora (else inconclusive).
- Quantile coverage within [0.66, 0.87].
- On pass
- Preregister a SCREEN gameplay experiment with the frozen override rule and the D2-teacher checkpoint.
- On fail
- Record a valid negative. If a stronger teacher does not shrink the gap, the label-quality hypothesis is weakened and the afterstate line is parked.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 28800 CPU threads 24 Max host bytes 51539607552 Max GPU bytes 32212254720 GPU devices gfx1151 - Stop conditions
- Stop on any rules, information-boundary, legality, determinism, or parity failure.
- Stop immediately on out-of-memory or on host available memory falling below 16 GiB.
- Stop on any read of a seed outside the named lease.
- Stop corpus generation after 6 hours wall.
- Stop training after 3 hours GPU wall or 30 epochs.
- Expected artifacts
runs/<run-id>/corpus-d2/*.ndjsonruns/<run-id>/model/*.ptruns/<run-id>/gate/*.jsonresearch/results/<result-id>.json
- Amendments
Timestamp Before controlled data Reason 2026-08-21T13:45:00Z no lifecycle advanced preregistered -> completed after the run and result records were written; protocol content otherwise unchanged.
Technical recordResults recorded against this protocol
The stronger-teacher hypothesis fails as tested. A successor-closed corpus relabeled with a fair-D2 (five-sample) continuation teacher at K=64 (2.88M rows over 6,535 roots; partial at the generator's 4h default wall stop) trained a model that, on the fresh 0x5da70500 ranking gate against the fixed D1-continuation H40 target, reached top-1 0.3365 - far below fair D4's 0.5020, below the D1-teacher model's 0.4245 from iteration 3, and only at exact-D1's own 0.3339. The frozen criterion (top-1 >= 0.4616 on each half, i.e. closing half the iteration-3 gap to D4) failed by a wide margin in both halves (0.342, 0.331). The frozen override gate on fresh 0x5da70600 roots also failed (eligible-root regret: half1 -0.0089, half2 +0.0110, pooled +0.0012). IMPORTANT CONFOUND, disclosed: the D2-teacher model was evaluated against D1-continuation outcomes (frozen for comparability with iteration 3), so part of its regression may reflect the teacher/target mismatch rather than teacher quality alone. Read narrowly, the result says a stronger-teacher corpus did not produce a better ranker of the fixed public-continuation outcome, and the afterstate line's ranking deficit is robust to the teacher choice within the tested configurations.
- ✓Corpus successor-closed at K=64 with the D2 teacher, completeness 1.0 — observed: per-root completeness 1.0 over 6,535 fully labeled roots; the corpus is partial (6,535 of 8,192 planned) at the generator's 4h default wall stop
- ✕Ranking gate: model top-1 >= 0.4616 on each half-fold — observed: half1 0.3418, half2 0.3314
- ✕Override gate: eligible-root override regret <= D4 regret - 0.01 in EACH half-fold, rate >= 5% — observed: half1 -0.0089, half2 +0.0110, pooled +0.0012; override rate 35.9%
- ✓Decisive-root label stability >= 0.5 for both evaluation corpora — observed: 0.8058 (override corpus), 0.8010 (ranking corpus)
- ✓Quantile coverage within [0.66, 0.87] — observed: 0.7244
Technical recordRecorded metrics
- Teacher/target mismatch confound: the model was trained on D2-continuation outcomes but evaluated against D1-continuation outcomes (frozen for comparability with iteration 3). A matched D2-continuation target was not generated (cost); the regression may partly reflect the mismatch.
- The D2 corpus is partial (6,535 of 8,192 roots) at the generator's 4h default wall stop.
- The compact 3.4M-param architecture and the D1-harvested root distribution are unchanged from iteration 3.
- Single machine profile; FP32 on the shared-memory iGPU.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260821-afterstate-d2-teacher-58772f17.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260821-afterstate-d2-teacher-58772f17.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 89aa9fc6f1cde64e5626ce377c388f06d2dee17ec5d821466d2d03c61b600b70.
Registered by OpenCode / moonshotai/Kimi-K3.