Is the small survival network as accurate as it can be?

Retrain the tiny board-scoring network that already fits inside the search at several widths, training lengths and learning rates, and ask whether any of them predicts remaining lifetime better than the one that played — without playing a single game.

What happened

Nothing beat the network that already played. Across thirteen retrainings the best configuration was 0.0022 more accurate than the baseline, and the three training seeds of the baseline itself spread by 0.0023, so the difference is training noise, and the gate fixed in advance reads supported-as-tested: the network's accuracy on this corpus is saturated. The revealing part is the shape of the failure. Making the network wider lowered its training loss step by step and lowered its held-out accuracy step by step; the eight-times wider version was the worst predictor of games it had not seen. That is a network memorising a corpus that is too small for it, not one that is too small for the job. The next learned-leaf experiment should change the data or the target, not the width.

Technical recordThe resultopen ↓close ↑

Result RS-20260822T024228Z-94090db1 for EX-20260822-nnue-leaf-capacity-sweep-79632fd9, run RUN-20260822T020655Z-feaf4331, valid, pass (supported-as-tested), evidence tier mechanics-only (offline, no game played). Held-out lifetime Pearson on the whole-origin test split (486,819 examples): baseline h64/m32/10 epochs 0.8546 (seeds 42283 and 42284: 0.8567, 0.8569; spread 0.0023); h32 0.8550; h128 0.8543; h256 0.8515; h256/m64/20 epochs 0.8446; h512/m64/20 epochs 0.8378 (final training loss 0.5110 → 0.4197 over that ladder); h64 at 20 epochs 0.8564; mid 16 / mid 64 0.8542 / 0.8568; lr 1e-3 0.8552. Mirrored-board Pearson matches each row within 0.0004. The recorded finding-08 baseline was 0.8564 on an earlier torch build. Full table in runs/RUN-20260822T020655Z-feaf4331/summary.json.

The intuition

The learned leaf is the one learned evaluator in this repository that helped inside the search, and its page ends with a claim: "train a bigger survival model is not the next experiment", because a network five times its size was only 0.008 more accurate. That claim rests on a single comparison between two quite different architectures. If the small network's accuracy is set by its inputs — 135 active features per board, mostly single cells and adjacent pairs — then making it wider or training it longer will not help, and the next learned-leaf experiment should change what the network sees or what it is asked to predict. If instead the network is simply too small or under-trained, wider versions will be more accurate, and the claim is wrong.

Each training run costs about half a minute on the integrated GPU and reads only the corpus that already exists, so the question is cheap to settle either way.

How it works, step by step

1. Same data, same split. Every run trains on the existing 5.26-million-position corpus with the whole-origin split the learned-leaf work used, so held-out numbers are comparable with the recorded baseline.
2. A small grid. Hidden width 32 to 512, middle layer 16 to 64, ten or twenty epochs, two learning rates — thirteen runs, including three training seeds of the baseline so that a difference can be compared with the noise of training itself.
3. One number per run. Held-out correlation between predicted and actual remaining lifetime, read straight from the trainer's own output file; mean error in moves, hazard accuracy and the mirrored-board check are reported beside it.
4. A gate fixed in advance. The claim "saturated" is supported if the best run beats the baseline by no more than twice the seed-to-seed spread (or 0.005, whichever is larger); it is rejected if the best run is at least 0.02 better and more than three spreads away; anything between is inconclusive.

Technical recordThe recordopen ↓close ↑

• Theory TH-20260822-nnue-leaf-capacity-saturation-9c45e42e; experiment EX-20260822-nnue-leaf-capacity-sweep-79632fd9 (frozen, CHECK tier, diagnostic, no seed opened).
• Trainer: approaches/lifetime-objective/learned-leaf/train_leaf.py, unchanged; corpus runs/RUN-A51D-corpus/all.states; the recorded baseline is held-out lifetime Pearson 0.8564 (runs/RUN-A52-LEAF/model/leafnet-h64.json).
• sweep.sh runs the grid; summarize.py tabulates and applies the gate.
• Limitation stated in advance: inference cost per board is not measured here, so a wider network that wins offline is not yet known to fit the roughly one-microsecond leaf budget the search imposes.

Sources

• sweep.sh: the thirteen-run grid, reading the existing corpus only
• summarize.py: the table and the preregistered gate