Full training of the K=256 afterstate model, then the frozen top-two override rule on fresh origins
The iteration-3 model was stopped at epoch 11 of 20 with its ranking loss still decreasing, so it was undertrained.
On this page
- Created
- Updated
The near-tie override gate missed by a quarter of a percentage point, and its own record named the bottleneck: the network's ranking quality. The obvious next move is the one every practitioner reaches for. The checkpoint had been trained for 11 epochs on a bounded pilot budget. Perhaps it was simply undertrained.
This experiment removes that explanation. The same architecture, the same data, double the budget: 22 epochs over the two-million-row subsample, 44 million row-updates, a cosine schedule carried to completion. About five and a half hours on the machine's integrated GPU.
Training worked exactly as intended. The ranking loss fell from 0.630 to 0.584 — the model learned what it was being taught, more thoroughly than before.
On held-out positions it got worse. Scored on the same roots as the earlier checkpoint, top-1 accuracy fell from 0.424 to 0.361 and normalized regret rose from 0.241 to 0.281. Training loss down, held-out ranking down. That is the textbook signature of overfitting, and it means the extra epochs were spent memorising the teacher's labels rather than learning the function behind them.
The frozen gate then failed on fresh data, and failed in an informative way. The override fired on 41.1% of eligible near-ties and cut regret by 0.0228 in the first half-fold — but raised it by 0.0297 in the second, where the intervention was actively harmful. Pooled, it is +0.0018: indistinguishable from doing nothing. Determinism, calibration and label stability all passed; the failure is entirely in generalization.
What this closes. Direct action-override by this model family is shut, per the frozen failure action. Not because the idea is wrong — the previous experiment showed the mechanism firing correctly and helping in both halves — but because the thing being consulted cannot be made reliable by training it harder. The limitation that matters is stated in the record: the student is a compact 3.4M-parameter network, and the claim established here is about compact students, not about learned evaluators in general.
The companion experiment run the same night, the stage-1 ordering test, takes that limitation and makes it precise.
Technical recordThe registered protocol
- Hypothesis
- The iteration-3 model was stopped at epoch 11 of 20 with its ranking loss still decreasing, so it was undertrained. Fully training the identical architecture on the identical K=256 successor-closed corpus (up to 40 epochs, 4M-row deterministic subsample, 6h GPU budget) improves its calibrated ranking enough that the FROZEN top-two near-tie override rule (epsilon=500, paired-bootstrap 95% lower bound > 0, 1000 resamples, domain 0x4f565252) passes its frozen margin on fresh held-out origins.
- Arms
Arm Name Entry point Manifest Candidate distributional-afterstate-ranker approaches/afterstate-learning/distributional-afterstate/train.py– Comparator fair-d4 approaches/fair-expectimax/reference/fair-only-depth4.cpp– - Classification
- algorithmic
- Information boundary
- public-policy
- Benchmark tier
- CHECK
- Lifecycle
- completed
- Primary metric
- Eligible-root mean normalized regret of the override policy vs unchanged fair D4 on fresh held-out roots (scenario-mean H40 D1-continuation target, K=256), per origin-hash half-fold.
- Secondary metrics
- override rate among eligible roots
- whole-set regret
- decisive-root breakdown
- decisive-root label stability (floor 0.5)
- quantile coverage in [0.66,0.87]
- byte-identical repeated gate reports
- training epochs completed and final ranking loss
- Statistical unit
- whole-origin
- Uncertainty method
- Two deterministic held-out half-folds by origin-game hash (mix32 of origin seed); the primary comparison must hold in both halves.
- Data role
- public-development
- Seed leases
SL-20260820T083000Z-5da70000
- Dataset references
DS-20260820-afterstate-h40-k256-9d31c47a
- Whole-origin split
- yes
- Reuse disclosure
- Training corpus: the existing K=256 corpus-A (seeds 0x5da70000-0x5da700ff, training-role). Evaluation: fresh seeds 0x5da70400-0x5da7043f (64 origins), read once by this experiment's gate. The override RULE is frozen from EX-20260820-d4-toptwo-override-gate-0bdb39a1; only the model changes.
- Pass criteria
- Training completes 22 epochs on the 2M-row deterministic subsample within the 6h GPU budget (44M row-updates, 2x iteration 3's 22M), with the cosine LR schedule completing.
- Eligible-root override regret <= D4 regret - 0.01 in EACH half-fold.
- Override rate >= 5% of eligible roots.
- Decisive-root label stability >= 0.5 (else inconclusive).
- Quantile coverage within [0.66, 0.87].
- Gate report byte-identical across two runs.
- On pass
- Preregister a SCREEN gameplay experiment (32 paired development games) with the frozen rule and this checkpoint.
- On fail
- Record a valid negative. If full training still fails the frozen margin, the direct-override use of this model family is closed and the line's remaining value is the corpus infrastructure.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 28800 CPU threads 24 Max host bytes 51539607552 Max GPU bytes 32212254720 GPU devices gfx1151 - Stop conditions
- Stop on any rules, information-boundary, legality, determinism, or parity failure.
- Stop immediately on out-of-memory or on host available memory falling below 16 GiB.
- Stop on any read of a seed outside the named lease.
- Stop training after 6 hours GPU wall or 40 epochs, whichever comes first.
- Expected artifacts
runs/<run-id>/model/*.ptruns/<run-id>/gate/override-gate-report.json (x2 byte-identical)research/results/<result-id>.json
- Amendments
Timestamp Before controlled data Reason 2026-08-21T03:25:00Z yes Training-completeness criterion redefined in row-update terms before this experiment's controlled corpus (0x5da70400-0x5da7043f) was generated: the iGPU sustains ~28 min/epoch on 4M rows, so a fixed 30-epoch requirement was not reachable in the 6h GPU budget. The undertraining hypothesis is tested against iteration 3's 11-epoch/2M-row checkpoint by total row-updates instead. 2026-08-21T03:50:00Z yes Row-update target rescaled to measured iGPU throughput (~1.7k rows/s FP32, compute-bound; 512-group batches measured slower than 256) before the controlled corpus was generated: 2M-row subsample (matching iteration 3's training set size) x 22 epochs = 44M row-updates = 2x iteration 3, with the cosine schedule completing. 2026-08-21T09:45:00Z no lifecycle advanced preregistered -> completed after the run and result records were written; protocol content otherwise unchanged.
Technical recordResults recorded against this protocol
Full training does not rescue the afterstate model; it overfits. The model was trained to 22 epochs on the 2M-row K=256 subsample (44M row-updates, 2x iteration 3, cosine schedule completed, ranking loss 0.584 vs iteration 3's 0.630). On the SAME held-out roots as iteration 3 (corpus-C, a labeled diagnostic reuse), the fully-trained model ranks WORSE than iteration 3's 11-epoch checkpoint (top-1 0.361 vs 0.424, pairwise 0.658 vs 0.685, regret 0.281 vs 0.241) - training loss improved while held-out ranking degraded, a textbook overfitting signature against the D1-continuation H40 labels. The frozen override gate on fresh corpus-E (2,867 roots, 1,106 near-tie eligible, 41% override rate) then FAILED: eligible-root regret half1 0.2289 vs D4 0.2517 (+0.0228) but half2 0.2778 vs 0.2481 (-0.0297, the override is actively harmful there), pooled +0.0018. Stability (0.824), calibration (0.800), determinism (byte-identical) all passed. Conclusion: the model's limitation is not training completeness but generalization to held-out roots under a weak D1 teacher; the direct-override use of this model family is closed per the frozen failure action.
- ✓Training completes 22 epochs on the 2M-row subsample within budget (44M row-updates, 2x iteration 3) — observed: 22 epochs, cosine schedule completed, ~5.5h GPU
- ✕Eligible-root override regret <= D4 regret - 0.01 in EACH half-fold — observed: half1 +0.0228 (passes); half2 -0.0297 (override harmful); pooled +0.0018
- ✓Override rate >= 5% of eligible roots — observed: 41.1%
- ✓Decisive-root label stability >= 0.5 — observed: 0.8244
- ✓Quantile coverage within [0.66, 0.87] — observed: 0.7997
- ✓Gate report byte-identical across two runs — observed: byte-identical
Technical recordRecorded metrics
- The diagnostic comparison to iteration 3 reuses corpus-C (iteration 3's held-out), a labeled diagnostic; the frozen override gate used fresh corpus-E.
- The model family evaluated is the compact 3.4M-param ResNet over D1-continuation H40 labels; the result does not bound a stronger-teacher or different-architecture variant.
- The iGPU is compute-bound for this model (~2-3k rows/s FP32); larger-scale training was not attempted within the overnight budget.
- Two killed training attempts (tooling: memory blowup, output buffering) preceded the recorded run; they produced no artifacts and are disclosed in the run record.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260821-afterstate-full-train-override-ec7f28f4.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260821-afterstate-full-train-override-ec7f28f4.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 720733b8b618314cd535f7e2bb54bae653151cb50ceb7aa31df77e29b5eb7339.
Registered by OpenCode / moonshotai/Kimi-K3.