NNUE-class student on existing successor-closed D4 labels: within-root ordering probe
Train a LeafNet-shaped student (features exactly approaches/lifetime-objective/learned-leaf/leaf_features.py applied to the afterstate board, next disc and moves-remaining columns of runs/RUN-20260821T085042Z-c5cf0e71/d4q-labels/d4q-labels.tsv; EmbeddingBag(8902,64,sum)->ReLU->Linear(64,32)->ReLU->Linear(32,1)) on the existing successor-closed exact D4 sibling values (291,890 rows, 8,639 roots, fold column as written by the d4q experiment: its train/validation/held-out origins are reused unchanged).
On this page
- Created
- Updated
No explanation has been written for this record yet.
Technical recordThe registered protocol
- Hypothesis
- Train a LeafNet-shaped student (features exactly approaches/lifetime-objective/learned-leaf/leaf_features.py applied to the afterstate board, next disc and moves-remaining columns of runs/RUN-20260821T085042Z-c5cf0e71/d4q-labels/d4q-labels.tsv; EmbeddingBag(8902,64,sum)->ReLU->Linear(64,32)->ReLU->Linear(32,1)) on the existing successor-closed exact D4 sibling values (291,890 rows, 8,639 roots, fold column as written by the d4q experiment: its train/validation/held-out origins are reused unchanged). Loss: within-root listwise softmax cross-entropy over the root's legal siblings against softmax(value/T) with T fixed at the root's value spread times 0.18 as in d4-q-clone, plus gap-weighted pairwise margin, plus 0.1 absolute MSE anchor on standardised value. 5 initialisation seeds (0xA52E01-05), 30 epochs, Adam 1e-3, batch 256 roots, OPENBLAS_NUM_THREADS=1, CPU or GPU via approaches/lifetime-objective/gpu/activate.sh. Model selection uses validation origins only; the held-out half-folds are read once for the selected seed and reported for all five.
- Arms
Arm Name Entry point Manifest Candidate LeafNet h64/m32 listwise student on d4q labels none recorded – Comparator exact fair D4 ordering (d4q labels); exact D1 0.486 and D2 0.568 top-1 as calibration approaches/afterstate-learning/distributional-afterstate/d4q.py– - Classification
- diagnostic
- Information boundary
- seed-free
- Benchmark tier
- CHECK
- Lifecycle
- completed
- Primary metric
- held-out within-root top-1 agreement with exact D4, per half-fold
- Secondary metrics
- pairwise accuracy
- normalised regret (d4q.py definition)
- top-2
- validation-fold metrics for all 5 seeds
- spread across seeds
- inference cost per state of the exported student (informational)
- Statistical unit
- root
- Uncertainty method
- half-fold agreement plus seed spread; no interval claimed beyond the d4q gate's own
- Data role
- no-gameplay
- Seed leases
- none recorded
- Whole-origin split
- yes
- Reuse disclosure
- Reads only runs/RUN-20260821T085042Z-c5cf0e71/d4q-labels (training-role roots from the 0xa52... corpus family, already opened by EX-20260821 afterstate-d4q-stage1, RS-20260821T104500Z-77d21e90). The held-out fold was already read once by that experiment, so this is a diagnostic-tier read; no new seed is opened. A pass licenses only a fresh-manifest confirmation.
- Pass criteria
- Selected seed (best validation top-1) held-out top-1 >= 0.60 in each half-fold.
- Pairwise >= 0.78 in each half-fold.
- Normalised regret <= 0.13 in each half-fold.
- On pass
- Record valid + pass (diagnostic, not evidence); register a fresh whole-origin label manifest confirmation (~5 CPU-h of label-d4q) as a successor experiment before any claim that stage 1 of the loop is unblocked.
- On fail
- Record valid + fail at CHECK (diagnostic): no leaf-affordable NNUE-class student of this size reproduces D4's ordering from these labels; the self-play loop TH-20260821-...-299ed02f stays blocked in its leaf form and its redesign is routed to the larger-student question.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 5400 CPU threads 8 Max host bytes 17179869184 Max GPU bytes 8589934592 GPU devices gfx1151 (optional) - Stop conditions
- Stop at 5,400 wall-seconds; record partial.
- Stop if the label file's root count or row count differs from 8,639 / 291,890 (wrong artifact).
- Stop on any NaN loss; record invalid for that seed.
- Expected artifacts
runs/RUN-20260823T191900Z-b9f8f80d/kimi-k3-open-items.md (design K4-D probe, Kimi K3)runs/RUN-20260823T191900Z-b9f8f80d/probe/train.logruns/RUN-20260823T191900Z-b9f8f80d/probe/metrics.json (all seeds, validation and held-out)runs/RUN-20260823T191900Z-b9f8f80d/probe/model-seed*.pt
- Amendments
Timestamp Before controlled data Reason 2026-08-23T20:56:11Z no Closed out after the run: lifecycle set to completed and the hash recomputed over the completed record, as EX-20260823-reveal-construction-screen-371fd638 and EX-20260823-hpool-stage-d0-e0ad1c65 were. The run record research/runs/RUN-20260823T193853Z-5abd7c95.json retains the preregistration hash 6c820078c386240d8b69ad74febce7303b040f520db96ad13a3754c3cbe2c376. Two coefficients the hypothesis text did not fix (pairwise weight 1.0; prediction-side softmax temperature 1 in standardised units) are recorded as limitations in the result.
Technical recordResults recorded against this protocol
The leaf-affordable NNUE-class student does not reproduce fair D4's within-root ordering from the existing successor-closed labels. Five initialisation seeds of the LeafNet h64/m32 student (571,905 parameters; features from learned-leaf/leaf_features.py) were trained for 30 epochs on the 6,551 training roots of runs/RUN-20260821T085042Z-c5cf0e71/d4q-labels with the preregistered listwise + gap-weighted pairwise + 0.1 MSE loss; the seed with the best calibration-fold top-1 (0xA52E02, 0.3281) was then read once on the 3,030-root held-out panel of that experiment. Held-out top-1 was 0.296 / 0.3011 in the two origin-seed half-folds (gate >= 0.60), pairwise 0.6266 / 0.6259 (>= 0.78), normalised regret 0.3354 / 0.3379 (<= 0.13): every criterion fails in both half-folds. The other four seeds lie between 0.2987 and 0.3135 pooled top-1, all below the 3.4M-parameter afterstate CNN of RS-20260821T104500Z-77d21e90 (0.375) and below exact D1 (0.486). The student is leaf-affordable as claimed (1.27 us per state through the existing d7leaf export and leaf-check), but at this size it holds less of D4's ordering than the CNN did. Per the preregistered failure action: no leaf-affordable NNUE-class student of this size reproduces D4's ordering from these labels; the self-play loop TH-20260821-...-299ed02f stays blocked in its leaf form and its redesign is routed to the larger-student question. Diagnostic tier: the held-out panel had already been read once by the d4q experiment, so this is a diagnostic read on existing data, not new evidence about fresh roots.
- ✓Label file is the preregistered artifact: 291,890 rows / 8,639 roots — observed: 291890 rows, 8639 roots (train 6,551, calibration 2,088)
- ✓No non-finite loss in any seed; all five seeds completed 30 epochs within 5,400 wall-seconds — observed: 5 of 5 seeds valid; 166.08 wall-seconds total
- ✕Selected seed (best validation top-1, 0xA52E02) held-out top1 >= 0.6 in each half-fold — observed: half1 0.2960, half2 0.3011
- ✕Selected seed (best validation top-1, 0xA52E02) held-out pairwise >= 0.78 in each half-fold — observed: half1 0.6266, half2 0.6259
- ✕Selected seed (best validation top-1, 0xA52E02) held-out regret <= 0.13 in each half-fold — observed: half1 0.3354, half2 0.3379
Technical recordRecorded metrics
- 0xA52E01
- 0.3026
- 0xA52E02
- 0.2987
- 0xA52E03
- 0.3069
- 0xA52E04
- 0.3132
- 0xA52E05
- 0.3135
- 0xA52E01
- 0.3051
- 0xA52E02
- 0.3281
- 0xA52E03
- 0.3089
- 0xA52E04
- 0.3127
- 0xA52E05
- 0.3027
- 0xA52E01
- 0.3554
- 0xA52E02
- 0.3549
- 0xA52E03
- 0.3520
- 0xA52E04
- 0.3525
- 0xA52E05
- 0.3592
- 0.2987
- 0.3135
- Diagnostic-tier read: the 3,030-root held-out panel (d4q-labels-gate) was already opened once by EX-20260821-afterstate-d4q-stage1; no fresh roots were labelled. The result schema has no diagnostic tier, so it is recorded at the lowest tier (pilot); it is not development-tier evidence.
- The preregistration names only the d4q-labels file, which carries folds train and calibration but no held-out fold; the held-out panel and its origin-seed half-folds were taken from the same run's d4q-labels-gate/d4q-labels.tsv and corpus-gate/roots.tsv, exactly as d4q.py consumed them, which is the reading the protocol's 'train/validation/held-out origins are reused unchanged' clause intends.
- Epoch count was fixed at 30 by the protocol and the final-epoch model was used. Calibration-fold top-1 peaked at 0.352-0.359 between epochs 5 and 10 for every seed and then declined as the training listwise loss kept falling (overfitting); even the best epoch is far below the 0.60 gate, so early stopping would not change the verdict, but the reported held-out numbers are for the overfitted final epoch.
- Loss coefficients not fixed by the hypothesis text were set to 1.0 for the pairwise term (d4-q-clone used 0.35) and the prediction-side softmax temperature was 1 in standardised value units; only the listed configuration is rejected.
- CPU training with 8 torch threads; the EmbeddingBag backward is not bit-deterministic across thread counts, so a re-run reproduces the numbers only approximately.
- Inference cost is from leaf-check on the RUN-A51D-corpus mix-d3 states on a shared host; informational only.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260823-nnue-d4q-ordering-probe-0ca09bb1.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260823-nnue-d4q-ordering-probe-0ca09bb1.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 8a193bea0db608666915dbd1b5ce147d347f664c751d32fd5bf5c9fb5279ff98.
Registered by Claude Code / claude-fable-5.