Scale-out stage 1: successor-closed fair-D4 search-value labels; held-out D4-ranking agreement
A successor-closed corpus of exact fair-D4 per-action search values (every legal sibling labeled by the pinned reference itself) lets the action-free afterstate model learn fair D4's within-root ordering well enough to reach preregistered agreement thresholds on fresh held-out roots - the stage-1 prerequisite of the registered search-guided self-play loop (TH-20260821-search-guided-self-play-at-scale-299ed02f).
On this page
- Created
- Updated
Every failed learned policy in this repository has had the same explanation attached to it: sibling extrapolation. A model watches the move that was actually played, learns what happened next, and is then asked at deployment to choose between several alternatives it never saw evaluated. The data was never action-complete, so the model could not rank actions. Fix the data and the models should work.
This experiment fixes the data completely, and the models still do not work.
The setup removes every excuse in turn. The training labels are not noisy rollouts or human play — they are the pinned reference search's own exact values, computed under its own chance quadrature. Not just for the played move but for every legal sibling, at every root: 291,890 labelled afterstates across 8,639 training roots, with label completeness of 1.0. And the task is not "be better than the search". It is only "reproduce the search" — agree with fair D4 about which column is best.
It reached top-1 agreement of 0.375, against a frozen threshold of 0.60. Pairwise agreement 0.643 against 0.78; normalized regret 0.291 against 0.13. Every criterion failed, in both half-folds independently.
The number that gives this result its force is the comparison the record chose: exact fair D1 — a one-ply search, the cheapest legal lookahead there is — agrees with D4 at 0.486. Depth 2 reaches 0.568. The trained network is a worse approximation of the four-ply search than the four-ply search's own shallowest version, despite having been handed that search's exact answers as labels.
What this relocates. The obstacle is not sibling coverage. It is the representational capacity of a compact board evaluator for a 4-ply search-value function. That is a different problem with different remedies, and — usefully — it is falsifiable: the claim predicts that a materially larger student does better, and nothing larger has been tried. The integrated GPU's throughput is what bounded the model size here, and the record says so in its limitations.
For the scale-out programme this experiment is a gate, and the gate is shut. A search-guided self-play loop needs a student that can at least hold the teacher's ordering before it is worth spending a cluster on generating more of the teacher's games. At this model scale, stage 1's prerequisite is not met.
Technical recordThe registered protocol
- Hypothesis
- A successor-closed corpus of exact fair-D4 per-action search values (every legal sibling labeled by the pinned reference itself) lets the action-free afterstate model learn fair D4's within-root ordering well enough to reach preregistered agreement thresholds on fresh held-out roots - the stage-1 prerequisite of the registered search-guided self-play loop (TH-20260821-search-guided-self-play-at-scale-299ed02f). Prior D4 clones failed on played-action or sparse-sibling data; this configuration is successor-closed.
- Arms
Arm Name Entry point Manifest Candidate afterstate-d4q-student approaches/afterstate-learning/distributional-afterstate/train.py– Comparator fair-d4 approaches/fair-expectimax/reference/fair-only-depth4.cpp– - Classification
- algorithmic
- Information boundary
- public-policy
- Benchmark tier
- CHECK
- Lifecycle
- completed
- Theories tested
- Primary metric
- Held-out top-1 agreement between the model's within-root ordering and exact fair D4's ordering on fresh roots.
- Secondary metrics
- pairwise agreement
- normalized regret against D4's value ordering
- per half-fold stability
- model value calibration (Pearson) on held-out roots
- Statistical unit
- whole-origin
- Uncertainty method
- Two deterministic held-out half-folds by origin-game hash; thresholds must hold in both halves.
- Data role
- public-development
- Seed leases
SL-20260820T083000Z-5da70000
- Whole-origin split
- yes
- Reuse disclosure
- Training: exact fair-D4 per-action values computed by the pinned reference on training roots harvested from seeds 0x5da70000-0x5da700ff (training-role). Agreement gate: fresh seeds 0x5da70700-0x5da7073f (64 origins); only D4 comparator labels are computed there - the target is D4's own ordering, so no continuation corpus is needed for this gate.
- Pass criteria
- Successor-closed D4-value labels on >= 8,000 training roots with completeness 1.0.
- Held-out top-1 agreement >= 0.60 on each half-fold (for scale: exact D1 agrees with D4 at 0.486 and exact D2 at 0.568 on the historical 1,508/465-root panel; 0.60 is beyond D2).
- Held-out pairwise agreement >= 0.78 on each half-fold (D2: 0.767).
- Normalized regret <= 0.13 on each half-fold (D2: 0.141).
- All CHECK-tier self-tests pass before any label is inspected.
- On pass
- Register stage 2: use the frozen student as the leaf of a bounded search and compare against fair D2/D4 on paired development games.
- On fail
- Record a valid negative; the scale-out loop's stage 1 is blocked and the registered theory's first falsification condition is engaged.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 28800 CPU threads 24 Max host bytes 51539607552 Max GPU bytes 32212254720 GPU devices gfx1151 - Stop conditions
- Stop on any rules, information-boundary, legality, determinism, or parity failure.
- Stop immediately on out-of-memory or on host available memory falling below 16 GiB.
- Stop on any read of a seed outside the named lease.
- Stop D4 labeling after 4 hours wall.
- Stop training after 3 hours GPU wall or 30 epochs.
- Expected artifacts
runs/<run-id>/d4q-labels/*.tsvruns/<run-id>/model/*.ptruns/<run-id>/gate/*.jsonresearch/results/<result-id>.json
- Amendments
Timestamp Before controlled data Reason 2026-08-21T10:45:00Z no lifecycle advanced preregistered -> completed after the run and result records were written; protocol content otherwise unchanged.
Technical recordResults recorded against this protocol
Scale-out stage 1 fails its prerequisite: a compact action-free afterstate model cannot learn fair D4's within-root ordering even from successor-closed, exactly-labeled search values. Training labels were the pinned reference's own depth-3 values of every legal sibling's resolved afterstate under its own five-stratum quadrature (291,890 labeled afterstates over 8,639 training roots, completeness 1.0). On 3,030 fresh held-out roots the model's one-ply chance-averaged ordering agreed with exact fair D4 at top-1 0.375 (frozen threshold >= 0.60), pairwise 0.643 (>= 0.78), normalized regret 0.291 (<= 0.13), failing every criterion in both origin-hash half-folds. For scale, exact fair D1 agrees with D4 at 0.486 top-1 on the historical panel - the learned student is WORSE than the cheapest exact search. Combined with the repository's prior played-action distillation failures, this strengthens the conclusion to: the obstacle to learning D4's ranking is not sibling coverage but the representational capacity of a compact board evaluator for the 4-ply search-value function. The registered self-play loop's stage-1 prerequisite is not met at this model scale.
- ✓Successor-closed D4-value labels on >= 8,000 training roots, completeness 1.0 — observed: 8,639 roots, 291,890 afterstate labels, every legal sibling x 5 strata
- ✕Held-out top-1 agreement >= 0.60 on each half-fold — observed: half1 0.3628, half2 0.3864
- ✕Held-out pairwise agreement >= 0.78 on each half-fold — observed: half1 0.6375, half2 0.6473
- ✕Normalized regret <= 0.13 on each half-fold — observed: half1 0.3055, half2 0.2778
- ✓All CHECK-tier self-tests pass before any label is inspected — observed: SELFTEST OK (10 checks including D2-continuation determinism)
Technical recordRecorded metrics
- The student is the compact 3.4M-param ResNet; a materially larger model was not tested (the iGPU's FP32 throughput bounds what is trainable overnight).
- Labels are the depth-3 value of afterstates (the value one ply below the D4 root), so the student approximates D4's search through its own horizon, not an oracle's.
- The gate measures agreement with D4's ordering, which is itself a strong-but-not-optimal reference; a student below D4's agreement could in principle still add value inside a different search, which this experiment does not test.
- Single machine profile; FP32 on the shared-memory iGPU.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260821-afterstate-d4q-stage1-40136e9e.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260821-afterstate-d4q-stage1-40136e9e.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 9641df72c64fb44ef3a40acccae44dc35b5868e57b573e1fd6c63883e73d530b.
Registered by OpenCode / moonshotai/Kimi-K3.