Drop7 Research
← Theories

The NNUE-shaped survival leaf is accuracy-saturated at hidden width 64 on the existing corpus

supported-as-testedassessedevidence: mechanics-onlydiagnosticTH-20260822-nnue-leaf-capacity-saturation-9c45e42e
No explanation has been written for this theory yet. Add web/content/research/TH-20260822-nnue-leaf-capacity-saturation-9c45e42e.mdx and it will appear here. The registered record is shown below.

The registered record

Claim

On the existing 5.26M-record training corpus (runs/RUN-A51D-corpus/all.states, whole-origin split of dataset.py), the NNUE-shaped survival leaf's held-out lifetime correlation does not improve by more than 0.01 when hidden width is raised from 64 to 512, the middle layer from 32 to 64, epochs from 10 to 20, or the learning rate is changed: its accuracy is saturated by the feature space and the data, not by capacity or training length.

Mechanism

finding-08 reports held-out lifetime correlation 0.8564 for the 572k-parameter h64 student against 0.8646 for a 3.0M-parameter convolutional teacher, and concludes that 'the accuracy is not there to be had'. That conclusion rests on one pair of architectures. If the 135-active-feature first layer is the bottleneck, wider hidden layers and longer training will not move the held-out correlation; if the student is under-parameterised or under-trained, they will. Either answer is cheap (about 35 s of GPU per run) and neither requires a game.

What would prove it wrong

  • The best sweep configuration's held-out lifetime Pearson exceeds the h64/m32/10-epoch/lr 3e-3 baseline by at least 0.02 and by more than three times the seed-to-seed spread of the baseline: the claim is not supported as tested (capacity or training length matters).
  • A delta between max(0.005, 2 x seed spread) and that threshold: inconclusive at this corpus size.
  • Out of scope: whether a more accurate leaf plays better. finding-08 already shows a 0.008 correlation gap did not separate the CNN from the student in play; a gameplay test is a separate preregistered experiment.
registered 2026-08-22T02:04:57Z by Claude Code / claude-fable-5

Experiments that test it

Results recorded against it

valid run · outcome: passsupported-as-testedmechanics-onlyRS-20260822T024228Z-94090db1

Offline capacity/training sweep of the NNUE-shaped survival leaf on runs/RUN-A51D-corpus/all.states (13 runs, same whole-origin test split, 486,819 held-out examples). Baseline h64/m32/e10/lr3e-3 held-out lifetime Pearson 0.8546 (finding-08 recorded 0.8564 on an earlier torch build); seed-to-seed spread 0.0023 over three seeds. Best configuration h64-m64-e10-lr3e3-s0 at 0.8568, delta +0.0022 <= max(0.005, 2 x spread): gate verdict supported-as-tested. Width hurts: Pearson by width (params, Pearson, final train loss) h32-m32-e10-lr3e3-s0 (286,447 params: 0.855, loss 0.5238); h64-m32-e10-lr3e3-s0 (572,367 params: 0.8546, loss 0.511); h128-m32-e10-lr3e3-s0 (1,144,207 params: 0.8543, loss 0.5006); h256-m32-e10-lr3e3-s0 (2,287,887 params: 0.8515, loss 0.4857); h256-m64-e20-lr3e3-s0 (2,296,591 params: 0.8446, loss 0.4596); h512-m64-e20-lr3e3-s0 (4,592,143 params: 0.8378, loss 0.4197). Training loss falls monotonically with width while held-out accuracy falls, i.e. larger students overfit this corpus; 20 epochs change nothing at h64 and hurt at h128/h256; lr 1e-3 and mid 16/64 are within spread. The accuracy of this feature space on this corpus is saturated near 0.855-0.857; it is not limited by hidden width, middle width, epochs or learning rate.

What it had to pass
  • ✓All 13 runs complete with finite metrics on the same split — observed: 13 rows, 0 FAILED lines in sweep.log
  • ✓Gate verdict from summarize.py — observed: delta +0.0022 vs threshold max(0.005, 2 x 0.0023) -> supported-as-tested
  • ✓Every number copied from the trainer's own <tag>.json output — observed: summarize.py reads heldOut/heldOutMirrored/history fields verbatim
Recorded metrics
basePearson
0.8546
seedSpread
0.0023
bestTag
h64-m64-e10-lr3e3-s0
bestPearson
0.8568
deltaBestMinusBase
0.0022
gateVerdict
supported-as-tested
runs
  1. tag
    h128-m32-e10-lr3e3-s0
    hidden
    128
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    1,144,207
    lifetimePearson
    0.8543
    mirroredPearson
    0.8541
    lifetimeMAE
    14.4423
    hazardAccMean
    0.8387
    finalTrainLoss
    0.5006
    trainWallSeconds
    10.7382
  2. tag
    h128-m32-e20-lr3e3-s0
    hidden
    128
    mid
    32
    epochs
    20
    lr
    0.0030
    seed
    42,282
    parameters
    1,144,207
    lifetimePearson
    0.8520
    mirroredPearson
    0.8516
    lifetimeMAE
    14.5026
    hazardAccMean
    0.8374
    finalTrainLoss
    0.4887
    trainWallSeconds
    10.7461
  3. tag
    h256-m32-e10-lr3e3-s0
    hidden
    256
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    2,287,887
    lifetimePearson
    0.8515
    mirroredPearson
    0.8512
    lifetimeMAE
    14.5446
    hazardAccMean
    0.8365
    finalTrainLoss
    0.4857
    trainWallSeconds
    15.5636
  4. tag
    h256-m64-e20-lr3e3-s0
    hidden
    256
    mid
    64
    epochs
    20
    lr
    0.0030
    seed
    42,282
    parameters
    2,296,591
    lifetimePearson
    0.8446
    mirroredPearson
    0.8444
    lifetimeMAE
    14.7638
    hazardAccMean
    0.8322
    finalTrainLoss
    0.4596
    trainWallSeconds
    15.2548
  5. tag
    h32-m32-e10-lr3e3-s0
    hidden
    32
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    286,447
    lifetimePearson
    0.8550
    mirroredPearson
    0.8548
    lifetimeMAE
    14.4879
    hazardAccMean
    0.8384
    finalTrainLoss
    0.5238
    trainWallSeconds
    7.5850
  6. tag
    h512-m64-e20-lr3e3-s0
    hidden
    512
    mid
    64
    epochs
    20
    lr
    0.0030
    seed
    42,282
    parameters
    4,592,143
    lifetimePearson
    0.8378
    mirroredPearson
    0.8379
    lifetimeMAE
    15.1885
    hazardAccMean
    0.8256
    finalTrainLoss
    0.4197
    trainWallSeconds
    13.9648
  7. tag
    h64-m16-e10-lr3e3-s0
    hidden
    64
    mid
    16
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    571,087
    lifetimePearson
    0.8542
    mirroredPearson
    0.8543
    lifetimeMAE
    14.4476
    hazardAccMean
    0.8388
    finalTrainLoss
    0.5151
    trainWallSeconds
    8.3707
  8. tag
    h64-m32-e10-lr1e3-s0
    hidden
    64
    mid
    32
    epochs
    10
    lr
    0.0010
    seed
    42,282
    parameters
    572,367
    lifetimePearson
    0.8552
    mirroredPearson
    0.8552
    lifetimeMAE
    14.4757
    hazardAccMean
    0.8386
    finalTrainLoss
    0.5219
    trainWallSeconds
    3.2629
  9. tag
    h64-m32-e10-lr3e3-s0
    hidden
    64
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    572,367
    lifetimePearson
    0.8546
    mirroredPearson
    0.8546
    lifetimeMAE
    14.4145
    hazardAccMean
    0.8390
    finalTrainLoss
    0.5110
    trainWallSeconds
    8.5447
  10. tag
    h64-m32-e10-lr3e3-s1
    hidden
    64
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,283
    parameters
    572,367
    lifetimePearson
    0.8567
    mirroredPearson
    0.8565
    lifetimeMAE
    14.4525
    hazardAccMean
    0.8391
    finalTrainLoss
    0.5114
    trainWallSeconds
    7.2195
  11. tag
    h64-m32-e10-lr3e3-s2
    hidden
    64
    mid
    32
    epochs
    10
    lr
    0.0030
    seed
    42,284
    parameters
    572,367
    lifetimePearson
    0.8569
    mirroredPearson
    0.8569
    lifetimeMAE
    14.4411
    hazardAccMean
    0.8391
    finalTrainLoss
    0.5131
    trainWallSeconds
    8.8389
  12. tag
    h64-m32-e20-lr3e3-s0
    hidden
    64
    mid
    32
    epochs
    20
    lr
    0.0030
    seed
    42,282
    parameters
    572,367
    lifetimePearson
    0.8564
    mirroredPearson
    0.8563
    lifetimeMAE
    14.4448
    hazardAccMean
    0.8389
    finalTrainLoss
    0.5051
    trainWallSeconds
    8.4661
  13. tag
    h64-m64-e10-lr3e3-s0
    hidden
    64
    mid
    64
    epochs
    10
    lr
    0.0030
    seed
    42,282
    parameters
    574,927
    lifetimePearson
    0.8568
    mirroredPearson
    0.8567
    lifetimeMAE
    14.4299
    hazardAccMean
    0.8391
    finalTrainLoss
    0.5097
    trainWallSeconds
    5.2192
environment
python 3.13.5 torch 2.13.0+rocm7.1 hip 7.1.52802 device Radeon 8060S Graphics
Limitations
  • Offline diagnostic on teacher labels from already-played games; says nothing about playing strength (finding-08: a 0.008 Pearson gap did not separate two models in play).
  • Single corpus (5.26M records, whole-origin split); a corpus several times larger could move the saturation point and is the natural follow-up, since the wider models overfit rather than underfit.
  • Per-board inference cost was not measured; irrelevant to the verdict because no wider model won.
  • GPU ran concurrently with a 30-thread CPU evolution; wall times are not timing-grade; metrics are unaffected.
  • The baseline reproduction differs from finding-08's 0.8564 by 0.0018 on a newer torch build; within the measured seed spread.