ResultNNUE-class student on existing successor-closed D4 labels: within-root ordering probe
The leaf-affordable NNUE-class student does not reproduce fair D4's within-root ordering from the existing successor-closed labels.
On this page
- Recorded
No explanation has been written for this record yet.
Technical recordMetrics, gate checks and limitations
The leaf-affordable NNUE-class student does not reproduce fair D4's within-root ordering from the existing successor-closed labels. Five initialisation seeds of the LeafNet h64/m32 student (571,905 parameters; features from learned-leaf/leaf_features.py) were trained for 30 epochs on the 6,551 training roots of runs/RUN-20260821T085042Z-c5cf0e71/d4q-labels with the preregistered listwise + gap-weighted pairwise + 0.1 MSE loss; the seed with the best calibration-fold top-1 (0xA52E02, 0.3281) was then read once on the 3,030-root held-out panel of that experiment. Held-out top-1 was 0.296 / 0.3011 in the two origin-seed half-folds (gate >= 0.60), pairwise 0.6266 / 0.6259 (>= 0.78), normalised regret 0.3354 / 0.3379 (<= 0.13): every criterion fails in both half-folds. The other four seeds lie between 0.2987 and 0.3135 pooled top-1, all below the 3.4M-parameter afterstate CNN of RS-20260821T104500Z-77d21e90 (0.375) and below exact D1 (0.486). The student is leaf-affordable as claimed (1.27 us per state through the existing d7leaf export and leaf-check), but at this size it holds less of D4's ordering than the CNN did. Per the preregistered failure action: no leaf-affordable NNUE-class student of this size reproduces D4's ordering from these labels; the self-play loop TH-20260821-...-299ed02f stays blocked in its leaf form and its redesign is routed to the larger-student question. Diagnostic tier: the held-out panel had already been read once by the d4q experiment, so this is a diagnostic read on existing data, not new evidence about fresh roots.
- ✓Label file is the preregistered artifact: 291,890 rows / 8,639 roots — observed: 291890 rows, 8639 roots (train 6,551, calibration 2,088)
- ✓No non-finite loss in any seed; all five seeds completed 30 epochs within 5,400 wall-seconds — observed: 5 of 5 seeds valid; 166.08 wall-seconds total
- ✕Selected seed (best validation top-1, 0xA52E02) held-out top1 >= 0.6 in each half-fold — observed: half1 0.2960, half2 0.3011
- ✕Selected seed (best validation top-1, 0xA52E02) held-out pairwise >= 0.78 in each half-fold — observed: half1 0.6266, half2 0.6259
- ✕Selected seed (best validation top-1, 0xA52E02) held-out regret <= 0.13 in each half-fold — observed: half1 0.3354, half2 0.3379
Technical recordRecorded metrics
- 0xA52E01
- 0.3026
- 0xA52E02
- 0.2987
- 0xA52E03
- 0.3069
- 0xA52E04
- 0.3132
- 0xA52E05
- 0.3135
- 0xA52E01
- 0.3051
- 0xA52E02
- 0.3281
- 0xA52E03
- 0.3089
- 0xA52E04
- 0.3127
- 0xA52E05
- 0.3027
- 0xA52E01
- 0.3554
- 0xA52E02
- 0.3549
- 0xA52E03
- 0.3520
- 0xA52E04
- 0.3525
- 0xA52E05
- 0.3592
- 0.2987
- 0.3135
- Diagnostic-tier read: the 3,030-root held-out panel (d4q-labels-gate) was already opened once by EX-20260821-afterstate-d4q-stage1; no fresh roots were labelled. The result schema has no diagnostic tier, so it is recorded at the lowest tier (pilot); it is not development-tier evidence.
- The preregistration names only the d4q-labels file, which carries folds train and calibration but no held-out fold; the held-out panel and its origin-seed half-folds were taken from the same run's d4q-labels-gate/d4q-labels.tsv and corpus-gate/roots.tsv, exactly as d4q.py consumed them, which is the reading the protocol's 'train/validation/held-out origins are reused unchanged' clause intends.
- Epoch count was fixed at 30 by the protocol and the final-epoch model was used. Calibration-fold top-1 peaked at 0.352-0.359 between epochs 5 and 10 for every seed and then declined as the training listwise loss kept falling (overfitting); even the best epoch is far below the 0.60 gate, so early stopping would not change the verdict, but the reported held-out numbers are for the overfitted final epoch.
- Loss coefficients not fixed by the hypothesis text were set to 1.0 for the pairwise term (d4-q-clone used 0.35) and the prediction-side softmax temperature was 1 in standardised value units; only the listed configuration is rejected.
- CPU training with 8 torch threads; the EmbeddingBag backward is not bit-deterministic across thread counts, so a re-run reproduces the numbers only approximately.
- Inference cost is from leaf-check on the RUN-A51D-corpus mix-d3 states on a shared host; informational only.
Recorded against NNUE-class student on existing successor-closed D4 labels: within-root ordering probe.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/RS-20260823T194142Z-946e3cd1.mdx; it renders above this record on the next request. The recorded metrics, gate checks and limitations are in the technical record above.
Record file: research/results/RS-20260823T194142Z-946e3cd1.json, validated against research/schemas/result-v1.schema.json.
- Run ids
RUN-20260823T193853Z-5abd7c95
- Contribution ids
CT-20260823T194212Z-bb82cb22
- Per-game artifact
runs/RUN-20260823T191900Z-b9f8f80d/probe/metrics.json(sha256ae86d823f2574890ed52d5a78bca926c86dd58b6cd942fe002a0a878c252fa2e, 1 records)- Machine profiles
research/system-profiles/MACH-20260820T080056Z-376ada90.json