Offline capacity and training sweep of the NNUE-shaped survival leaf on the existing corpus

Training approaches/lifetime-objective/learned-leaf/train_leaf.py unchanged on runs/RUN-A51D-corpus/all.states (explored == 0 records, whole-origin split seed 0x59117000) over the 13-configuration grid in sweep.sh — hidden 32/64/128/256/512, mid 16/32/64, 10 or 20 epochs, lr 1e-3 or 3e-3, plus three training seeds at the baseline — yields a best held-out lifetime Pearson within 0.01 of the h64/m32/e10/lr3e-3 baseline (finding-08 recorded 0.8564 on the same split).

preregisteredtier: CHECKdiagnosticpreviously-evaluated-developmentprivileged-teacherEX-20260822-nnue-leaf-capacity-sweep-79632fd9

On this page

Linked approachIs the small survival network as accurate as it can be?

Theories

TH-20260822-nnue-leaf-capacity-saturation-9c45e42e

Results

RS-20260822T024228Z-94090db1

Dates

Created: 2026-08-22T02:05:48Z
Updated: 2026-08-22T02:06:23Z

Record idEX-20260822-nnue-leaf-capacity-sweep-79632fd9

No explanation has been written for this record yet.

Technical recordThe registered protocolEX-20260822-nnue-leaf-capacity-sweep-79632fd9

Hypothesis: Training approaches/lifetime-objective/learned-leaf/train_leaf.py unchanged on runs/RUN-A51D-corpus/all.states (explored == 0 records, whole-origin split seed 0x59117000) over the 13-configuration grid in sweep.sh — hidden 32/64/128/256/512, mid 16/32/64, 10 or 20 epochs, lr 1e-3 or 3e-3, plus three training seeds at the baseline — yields a best held-out lifetime Pearson within 0.01 of the h64/m32/e10/lr3e-3 baseline (finding-08 recorded 0.8564 on the same split).
Arms: Arm Name Entry point Manifest
Candidate leafnet-capacity-grid approaches/lifetime-objective/leaf-capacity-sweep/sweep.sh –
Comparator fair-d4 approaches/fair-expectimax/reference/fair-only-depth4.cpp –
Classification: diagnostic
Information boundary: privileged-teacher
Benchmark tier: CHECK
Lifecycle: preregistered
Theories tested: The NNUE-shaped survival leaf is accuracy-saturated at hidden width 64 on the existing corpus
Primary metric: held-out lifetime Pearson correlation on the whole-origin test split (486,819 examples), best configuration minus baseline
Secondary metrics: held-out lifetime mean absolute error in moves
mean hazard accuracy over 12 rises
mirrored-board Pearson
parameter count
training wall seconds per run
seed-to-seed spread of the baseline over three training seeds
Statistical unit: whole-origin
Uncertainty method: seed-to-seed spread of the baseline configuration over three training seeds (0xA52A, 0xA52B, 0xA52C); no bootstrap
Data role: previously-evaluated-development
Seed leases: none recorded
Whole-origin split: yes
Reuse disclosure: Reads only the existing training corpus runs/RUN-A51D-corpus/all.states (SEEDLEASE-A52 training ranges 0xa5200000-0xa52152ff as recorded in runs/RUN-A51D-corpus/gen.sh; the same file finding-08 trained on). No seed is opened and no game is played. Held-out metrics use the corpus's own whole-origin test split, which finding-08 also reported on; they are diagnostic, never strength evidence.
Pass criteria: All 13 runs complete with finite metrics on the same split (summary.json lists 13 rows).
Gate verdict from summarize.py: supported-as-tested if delta <= max(0.005, 2 x seed spread); not-supported-as-tested if delta >= 0.02 and delta > 3 x spread; otherwise inconclusive.
Every reported number is copied from the trainer's own <tag>.json output; nothing is recomputed.
On pass: Record valid + pass (saturated); the next learned-leaf experiment must change the target or the feature space, not the width.
On fail: Record valid + fail (capacity or training length does matter offline); register a separate gameplay experiment before any strength claim; do not adopt any model.
Gate fixed before controlled data: yes
Resources: Wall seconds 3600
CPU threads 2
Max host bytes 17179869184
Max GPU bytes 17179869184
GPU devices Radeon 8060S (gfx1151), ROCm 7.13, torch 2.13.0+rocm7.1 in .venv-rocm
Stop conditions: Stop on any rules, information-boundary, legality, determinism, or parity failure.
Stop at 3,600 wall-seconds; a partial grid is recorded as partial.
A failed training run is recorded with its log and the grid continues; the gate needs all 13 rows.
Expected artifacts: runs/<sweep-run-id>/models/<tag>.json and .pt for 13 tags
runs/<sweep-run-id>/summary.json
runs/<sweep-run-id>/inputs.sha256 and environment.txt
Amendments: none recorded

Arm	Name	Entry point	Manifest
Candidate	leafnet-capacity-grid	`approaches/lifetime-objective/leaf-capacity-sweep/sweep.sh`	–
Comparator	fair-d4	`approaches/fair-expectimax/reference/fair-only-depth4.cpp`	–

Wall seconds	3600
CPU threads	2
Max host bytes	17179869184
Max GPU bytes	17179869184
GPU devices	Radeon 8060S (gfx1151), ROCm 7.13, torch 2.13.0+rocm7.1 in .venv-rocm

Technical recordResults recorded against this protocol1 record

valid runoutcome: passsupported-as-testedtier: mechanics-onlyRS-20260822T024228Z-94090db1

Offline capacity/training sweep of the NNUE-shaped survival leaf on runs/RUN-A51D-corpus/all.states (13 runs, same whole-origin test split, 486,819 held-out examples). Baseline h64/m32/e10/lr3e-3 held-out lifetime Pearson 0.8546 (finding-08 recorded 0.8564 on an earlier torch build); seed-to-seed spread 0.0023 over three seeds. Best configuration h64-m64-e10-lr3e3-s0 at 0.8568, delta +0.0022 <= max(0.005, 2 x spread): gate verdict supported-as-tested. Width hurts: Pearson by width (params, Pearson, final train loss) h32-m32-e10-lr3e3-s0 (286,447 params: 0.855, loss 0.5238); h64-m32-e10-lr3e3-s0 (572,367 params: 0.8546, loss 0.511); h128-m32-e10-lr3e3-s0 (1,144,207 params: 0.8543, loss 0.5006); h256-m32-e10-lr3e3-s0 (2,287,887 params: 0.8515, loss 0.4857); h256-m64-e20-lr3e3-s0 (2,296,591 params: 0.8446, loss 0.4596); h512-m64-e20-lr3e3-s0 (4,592,143 params: 0.8378, loss 0.4197). Training loss falls monotonically with width while held-out accuracy falls, i.e. larger students overfit this corpus; 20 epochs change nothing at h64 and hurt at h128/h256; lr 1e-3 and mid 16/64 are within spread. The accuracy of this feature space on this corpus is saturated near 0.855-0.857; it is not limited by hidden width, middle width, epochs or learning rate.

What it had to pass

✓All 13 runs complete with finite metrics on the same split — observed: 13 rows, 0 FAILED lines in sweep.log
✓Gate verdict from summarize.py — observed: delta +0.0022 vs threshold max(0.005, 2 x 0.0023) -> supported-as-tested
✓Every number copied from the trainer's own <tag>.json output — observed: summarize.py reads heldOut/heldOutMirrored/history fields verbatim

Technical recordRecorded metricsRS-20260822T024228Z-94090db1

basePearson

0.8546

seedSpread

0.0023

bestTag

h64-m64-e10-lr3e3-s0

bestPearson

0.8568

deltaBestMinusBase

0.0022

gateVerdict

supported-as-tested

runs

tag
h128-m32-e10-lr3e3-s0
hidden
128
mid
32
epochs
10
lr
0.0030
seed
42,282
parameters
1,144,207
lifetimePearson
0.8543
mirroredPearson
0.8541
lifetimeMAE
14.4423
hazardAccMean
0.8387
finalTrainLoss
0.5006
trainWallSeconds
10.7382
tag
h128-m32-e20-lr3e3-s0
hidden
128
mid
32
epochs
20
lr
0.0030
seed
42,282
parameters
1,144,207
lifetimePearson
0.8520
mirroredPearson
0.8516
lifetimeMAE
14.5026
hazardAccMean
0.8374
finalTrainLoss
0.4887
trainWallSeconds
10.7461
tag
h256-m32-e10-lr3e3-s0
hidden
256
mid
32
epochs
10
lr
0.0030
seed
42,282
parameters
2,287,887
lifetimePearson
0.8515
mirroredPearson
0.8512
lifetimeMAE
14.5446
hazardAccMean
0.8365
finalTrainLoss
0.4857
trainWallSeconds
15.5636
tag
h256-m64-e20-lr3e3-s0
hidden
256
mid
64
epochs
20
lr
0.0030
seed
42,282
parameters
2,296,591
lifetimePearson
0.8446
mirroredPearson
0.8444
lifetimeMAE
14.7638
hazardAccMean
0.8322
finalTrainLoss
0.4596
trainWallSeconds
15.2548
tag
h32-m32-e10-lr3e3-s0
hidden
32
mid
32
epochs
10
lr
0.0030
seed
42,282
parameters
286,447
lifetimePearson
0.8550
mirroredPearson
0.8548
lifetimeMAE
14.4879
hazardAccMean
0.8384
finalTrainLoss
0.5238
trainWallSeconds
7.5850
tag
h512-m64-e20-lr3e3-s0
hidden
512
mid
64
epochs
20
lr
0.0030
seed
42,282
parameters
4,592,143
lifetimePearson
0.8378
mirroredPearson
0.8379
lifetimeMAE
15.1885
hazardAccMean
0.8256
finalTrainLoss
0.4197
trainWallSeconds
13.9648
tag
h64-m16-e10-lr3e3-s0
hidden
64
mid
16
epochs
10
lr
0.0030
seed
42,282
parameters
571,087
lifetimePearson
0.8542
mirroredPearson
0.8543
lifetimeMAE
14.4476
hazardAccMean
0.8388
finalTrainLoss
0.5151
trainWallSeconds
8.3707
tag
h64-m32-e10-lr1e3-s0
hidden
64
mid
32
epochs
10
lr
0.0010
seed
42,282
parameters
572,367
lifetimePearson
0.8552
mirroredPearson
0.8552
lifetimeMAE
14.4757
hazardAccMean
0.8386
finalTrainLoss
0.5219
trainWallSeconds
3.2629
tag
h64-m32-e10-lr3e3-s0
hidden
64
mid
32
epochs
10
lr
0.0030
seed
42,282
parameters
572,367
lifetimePearson
0.8546
mirroredPearson
0.8546
lifetimeMAE
14.4145
hazardAccMean
0.8390
finalTrainLoss
0.5110
trainWallSeconds
8.5447
tag
h64-m32-e10-lr3e3-s1
hidden
64
mid
32
epochs
10
lr
0.0030
seed
42,283
parameters
572,367
lifetimePearson
0.8567
mirroredPearson
0.8565
lifetimeMAE
14.4525
hazardAccMean
0.8391
finalTrainLoss
0.5114
trainWallSeconds
7.2195
tag
h64-m32-e10-lr3e3-s2
hidden
64
mid
32
epochs
10
lr
0.0030
seed
42,284
parameters
572,367
lifetimePearson
0.8569
mirroredPearson
0.8569
lifetimeMAE
14.4411
hazardAccMean
0.8391
finalTrainLoss
0.5131
trainWallSeconds
8.8389
tag
h64-m32-e20-lr3e3-s0
hidden
64
mid
32
epochs
20
lr
0.0030
seed
42,282
parameters
572,367
lifetimePearson
0.8564
mirroredPearson
0.8563
lifetimeMAE
14.4448
hazardAccMean
0.8389
finalTrainLoss
0.5051
trainWallSeconds
8.4661
tag
h64-m64-e10-lr3e3-s0
hidden
64
mid
64
epochs
10
lr
0.0030
seed
42,282
parameters
574,927
lifetimePearson
0.8568
mirroredPearson
0.8567
lifetimeMAE
14.4299
hazardAccMean
0.8391
finalTrainLoss
0.5097
trainWallSeconds
5.2192

environment

python 3.13.5 torch 2.13.0+rocm7.1 hip 7.1.52802 device Radeon 8060S Graphics

Limitations

Offline diagnostic on teacher labels from already-played games; says nothing about playing strength (finding-08: a 0.008 Pearson gap did not separate two models in play).
Single corpus (5.26M records, whole-origin split); a corpus several times larger could move the saturation point and is the natural follow-up, since the wider models overfit rather than underfit.
Per-board inference cost was not measured; irrelevant to the verdict because no wider model won.
GPU ran concurrently with a 30-thread CPU evolution; wall times are not timing-grade; metrics are unaffected.
The baseline reproduction differs from finding-08's 0.8564 by 0.0018 on a newer torch build; within the measured seed spread.

Open the result record

Agent contextHow to extend this record

To add a reader-facing explanation, write web/content/research/EX-20260822-nnue-leaf-capacity-sweep-79632fd9.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.

Record file: research/experiments/EX-20260822-nnue-leaf-capacity-sweep-79632fd9.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 8e38473a52bc956da3917a312a1747d4d2795e9bf473df0220275409097ec030.

Registered by Claude Code / claude-fable-5.