Drop7 Research

Type to search every page, record and vocabulary term.

GitHubPlay

An evolved NNUE leaf, distilled from a depth-5 teacher

A compact neural leaf evaluator for the depth-3 fair search, warm-started by distilling a depth-5 teacher's sibling-complete root values and then refined by an evolutionary search whose fitness is whole-game score itself.

rejectedrecorded
fitnessfair leafwarm startshort12 candidates, generation by generation
Technique

Evolutionary optimisation

Try many slightly different versions, keep the ones that did best, make the next generation from them, and repeat; no gradient needed.

Read the primer: Evolutionary optimisation

On this page
Records8
Theories1
Experiments4
Results3
Family

Lifetime objective

Sources

The problem

Every search on this site has the same shape: imagine each legal column, imagine the next disc, imagine the reply, and so on for a few moves, then stop and ask a leaf evaluator how good the board at the end of each imagined line is. The points won along the way are known exactly; the leaf guesses the rest. The frozen fair leaf is a sum of eighteen hand-written terms.

How much the leaf matters was measured before any training signal was read. On a frozen snapshot of thirty teacher games, every root was searched twice at each depth, once with the fair leaf and once with a leaf that returns zero, so the only difference between the two sets of column values is the leaf term.

search depthrootsleaf's share of the column spreadin-tree score spread (points)leaf-term spread (points)decisions that change without the leafagreement with the depth-5 teacher, fair leaf / zero leaf
3 plies4000.973934,03355.5%72.0% / 45.0%
4 plies800.956434,09343.8%73.8% / 47.5%
5 plies80.971,1413,81350.0%100.0% / 50.0%
Leaf-swing diagnostic on a frozen 30-game snapshot of the corpus (run artifacts leaf-swing-d3.txt, leaf-swing-d4.txt, leaf-swing-d5.txt). Spreads are medians over roots of the difference between the best and worst legal column. The leaf's share is the median ratio of leaf-term spread to total value spread on roots with no losing column.

At every depth the leaf term carries about nineteen twentieths of the spread between columns, and removing it changes the chosen column at roughly half of the roots. The search multiplies the leaf's influence, because the same evaluator is applied at every frontier position of every line. A better leaf is worth pursuing for the same reason a worse one is dangerous.

Two earlier lines of work had each failed in an instructive way. Networks trained to imitate a search learned the positions the teacher visited and then could not rank the six sibling moves it never played (the sibling trap). And the one time a leaf was tuned by evolution on whole-game score directly, the eighteen hand-written weights of the fair leaf under CMA-ES, the optimiser followed selection noise into a worse evaluator (the first leaf evolution). Its post-mortem found that with 32 games per candidate and a paired standard deviation near 200,000 points, fresh seeds every generation left selection noise alone to steer.

Proposed solution

Replace the formula, for the three-move search, with a small neural network of the kind chess engines call an NNUE, an "efficiently updatable neural network" (primer). It is sparse enough to evaluate in a microsecond, which a leaf called hundreds of thousands of times per decision can afford, and expressive enough to encode board patterns that eighteen linear terms cannot. The one on this page has about 572,000 weights, almost all of them in the first table.

visible boardnext disc · moves to rise8,902 features135 are onSum 135 rowsof a table with one 64-widerow per feature (the big part:570k of the 572k weights)ReLU → 32small dense layerReLU → 1one number× 17,000points (one rise)Evolution changes only the 572,000 weights. The features, the shape and the depth-3 search around it stay fixed.The network never sees the score, the level, the move number or the seed.
The NNUE reads exactly what a player can see, turns it into 135 active features out of 8,902 possible ones, sums the matching rows of a weight table, passes the sum through two small layers, and returns one number in units of a row rise (17,000 points). Evolution changes the weights only.

The design keeps what each earlier failure taught. Imitation is used only as a warm start, from a corpus that records a depth-5 teacher's value for every legal column at every position, so the sibling-coverage hole is closed at the source. Evolution then refines the warm start on the deployed objective, the final score of complete games, with the noise handled: every candidate in a generation plays the same games (common random numbers), and selection compares ranks. The network reads only the visible board, the visible next disc, the moves until the next rise and whether the game is over, and the depth-5 teacher reads the same public state.

one fitness block: the same 32 seeds for everyone (8 shown)seed 1seed 2seed 3seed 4seed 5seed 6seed 7seed 8candidate Acandidate Bfair leaf (control)same discs, same rises, same reveals in every row: the only thing that differs is the evaluator, so A minus B is the evaluator, not luck
Common random numbers. Because every candidate and both controls play exactly the same seeds in a generation, the difference between two candidates' mean scores is a difference in evaluators and owes nothing to the discs they were dealt. Selection then compares ranks, so one lucky game moves a candidate up a few places instead of dragging the whole generation.

How it works

Depth-5 teacherplays gamesfrozen fair leaf, 7 stratatraining-lease seedsLabelled rootsevery position it faced,with all 7 column valuesSupervisedwarm startNNUE fits the teacher'svalues; whole-origin splitEvolution60 generations of pairedgames inside depth 3Elitere-selectiontop 8 replay 128 freshgames; winner frozenHeld-out screen64 never-read games,played exactly oncetraining seeds 0xa52e0300 onward: teacher games, every fitness block, the re-selectionscreen seeds0xa52e1300, oncestage A · stage B · stage C · stage D — the candidate never sees a screen seed before it is frozen
Four stages. The teacher, the corpus, the warm start, the evolution and the elite re-selection all read seeds from the training lease, which may be read again. The screen reads a separate block of seeds that nothing has ever opened, exactly once, after the candidate's weights are frozen and hashed.
  1. Teacher corpus (stage A). The frozen fair evaluator, driven by the fast Rust bitboard engine, plays training games at depth 5 with seven chance strata. Five moves is one full row-rise cycle, so every rise consequence of a root move is inside the teacher's horizon. At every position the corpus records the board, the next disc, the moves until the rise, and the teacher's value for all seven columns; the search computes those anyway, so the corpus is sibling-complete for free. Each teacher game is capped at 500 moves, and the stage stops starting new games at a fixed wall-clock budget; whatever whole games have completed are the corpus.
  2. Warm start (stage B). The network is fitted to the teacher's values with the Adam optimiser on a Huber loss measured in rise units, with a whole-origin split so that validation never sees a position from a game it trained on. Training runs for 16 epochs and the epoch with the lowest validation loss is frozen. A deployment-faithful probe then puts that network inside the real depth-3 search on held-out positions and asks how often the search picks the teacher's column.
  3. Evolution (stage C). Generation 0 is the warm start plus 31 copies with Gaussian noise added to every weight. Each generation draws a fresh block of 32 seeds that no candidate has seen, and all 32 candidates play all 32 seeds as the leaf of the depth-3, seven-stratum fair search with the ordinary 2,000-move cap; the frozen fair leaf and the unevolved warm start play the same seeds as two controls that are never selected. A candidate's fitness is the plain mean of its 32 final scores. The four fittest are copied unchanged; the other 28 slots are filled by tournaments of three whose winner is copied and has every weight perturbed by Gaussian noise at 5% of its tensor's standard deviation. The next population is written to disk before the generation's artifact is committed, so a crash replays the generation from identical inputs.
1 · Population32 candidate weight sets(generation 0: the warm start plus 31 noisy copies)2 · One fresh seed block32 seeds nobody has played,the same 32 for every candidate3 · Everyone playseach candidate is the leaf of thedepth-3 search on all 32 games;fitness = mean final score4 · Keep the best fourcopied unchanged into thenext generation (elites)5 · Fill the other 28 slotsdraw 3 at random, keep the fittest;copy it; add Gaussian noise to everyweight: σ = 5% of its tensor's spreadnext generation(4 elites + 28 mutants)Two controls play the same 32 seeds every generationthe frozen fair leaf (is the population beating what it replaced?) and the unevolved warm start(did evolution move anything at all?). Neither is ever selected or mutated.checkpoint: population writtenbefore the generation is marked doneafter generation 59: top 8 replay128 fresh games; the winner is frozen
One generation. Thirty-two candidates play the same fresh block of thirty-two seeds as the leaf of the real depth-3 search; the four fittest are copied unchanged; the other twenty-eight slots are filled by tournaments of three whose winner is copied and mutated. The frozen fair leaf and the unevolved warm start play every block as controls and are never selected.
  1. Re-selection. After the last generation, the eight fittest candidates each replay a fresh 128-game block, and the one with the highest mean is frozen as the candidate. Its SHA-256 is recorded before the screen block is opened, so the vector that is screened is provably the one that was selected. The best of thirty-two noisy estimates is biased upward, and 128 fresh games shrink that bias before anything is claimed.
  2. Screen (stage D). The frozen candidate plays 64 games on a seed block nothing has ever opened, against three arms on the identical seeds: the same depth-3 search with the frozen fair leaf (the comparator), the same search with the unevolved warm start (the ablation, isolating what evolution added), and the fair leaf at depth 4 (the program's standing reference, for context). The screen is played exactly once, whatever it shows. The network's output at every step is one number per board, in units of a row rise, consumed by the search.

What happened

Run RUN-20260902T035644Z-c1fd8987 under EX-20260902-nnue-evolution-d3-v2-49c18bc2 · run record lifecycle: completedsnapshot 2026-09-03T02:57:58Z
  1. A · teacher corpusdone177 complete games, 21,618 labelled roots
  2. B · supervised warm startdonebest epoch 8 of 16, validation Huber 0.6873
  3. C · evolutiondone60 of up to 60 generations completed
  4. C · elite re-selectiondonecandidate frozen, SHA-256 edd0d2efd181…
  5. D · held-out screendonepreregistered gate not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts.

On the 64 never-read held-out games, the evolved leaf scored 106,964 points less per game than the frozen fair leaf inside the same depth-3 search (bootstrap 95% lower bound −146,580), so the preregistered gate fails and the theory's claim is not supported as tested (RS-20260903T025751Z-6577b33e, run valid, outcome fail, ledger-recorded). The same screen shows what evolution did do: the evolved leaf beat its own unevolved warm start by 35,375 points on the same seeds, a gain whose lower bound sits well above zero, from a warm start that played at about half the fair leaf's level.

The stages tell the rest. The depth-5 teacher ran four to seven times slower per position than the pilot had projected, so the corpus closed at about a third of the games the protocol allowed for. The warm start fitted the teacher's values closely and, placed inside the real depth-3 search, picked the teacher's column less than half the time, about what a leaf that returns zero manages. Evolution then separated the population from the warm start steadily across all sixty generations without ever approaching the fair control, and the curve had not flattened when the protocol stopped it.

Every teacher game the depth-5 search has completed

recorded median 348,847
complete games
177
labelled roots
21,618
mean score
433,131
median score
348,847
best game
1,889,608
mean moves
122.1
games stopped at the cap
3
teacher time per root
77.5 s
mean time per game
2.6 h
Stage A. Each dot is one complete teacher game, in the order the games finished; hover a dot for its seed, score, length and how long the depth-5 search took to play it. Games are heavy-tailed: most end in the low hundreds of thousands and a few run far longer. The teacher turned out four to seven times slower per position than the pilot had projected, so the stage reached its wall budget well short of the 512 games the protocol allowed for; the protocol makes the completed games the corpus and does not depend on the count.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/analysis.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/corpus/parts
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/corpus.err

Supervised warm start: loss by epoch on a whole-origin split

epoch 8 frozen as the warm start
  • training loss
  • validation loss (whole-origin held-out games)
roots / games
21,618 / 177
train / validation roots
17,641 / 3,977
epoch frozen
8 of 16
validation Huber
0.6873
ordering probe roots
256
top-1 agreement with teacher
44.1%
mean teacher-value regret
2,395 points
Stage B. Training and validation loss by epoch, in rise units; the dashed line marks the epoch frozen as the warm start. The ordering probe below the chart is the number that matters for deployment: how often the real depth-3 search, with this network as its leaf, picks the column the depth-5 teacher picked on held-out positions.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/analysis.json

Fitness by generation against the two controls that played the same games

population meanbest candidatefair leaf control (same seeds)unevolved init control (same seeds)
  • population mean
  • best candidate
  • top-4 mean
  • population range (lowest to highest candidate)
  • fair leaf control (same seeds)
  • unevolved init control (same seeds)
generations completed
60
latest best candidate
219,577
latest population mean
185,958
latest fair-leaf control
328,339
latest init control
146,372
mean above fair, last 10
0 of 10
mean margin over fair
-121,896
training-signal falsifier
no signal
illegal / incomplete decisions
0 / 0
Stage C. Best candidate, top-four mean and population mean by generation, with the two controls that played the same seeds each generation. Because every line in a generation was measured on the same 32 games, the vertical distance between a line and the fair control is a paired difference. The shaded band is the whole population's range. The theory's training-signal falsifier asks whether the population mean sits above the fair control in most of the final ten generations.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/analysis.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/evolve/progress.jsonl

Every candidate in every generation

  • one candidate
  • fair leaf on the same 32 games
  • unevolved init on the same 32 games
The same generations with every candidate drawn. Selection pressure shows up as the cloud tightening and drifting; a population that stays centred on the pink control line has not moved away from the warm start.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/evolve/progress.jsonl

Held-out screen: evolved candidate minus fair leaf, both at depth 3

recorded paired mean -106,964 (band: 95 % bootstrap)first half | second half
  • candidate scored higher on that game
  • reference scored higher on that game
  • recorded paired mean -106,964 (band: 95 % bootstrap)
paired mean difference
-106,964 points
bootstrap 95% lower bound
-146,580
Student-t 95% lower bound
-145,890
wins / ties / losses
14 / 0 / 50
first half / second half
-109,139 / -104,790
paired sd
186,538
detection floor
38,357
lower-quartile difference
-36,134
Stage D, the primary contrast. One bar per held-out game: the evolved candidate's score minus the frozen fair leaf's score on the same seed, in seed order. The dashed line is the recorded paired mean and the vertical divider separates the two halves the gate checks separately.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/heldout.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json

The four arms on the same 64 held-out games, and the preregistered gate

armmeanmedianlower quartilebest gamemean movescensored
evolved NNUE leaf, depth 3190,961160,108122,253477,75858.50
unevolved (supervised) NNUE leaf, depth 3155,586140,282107,423401,03848.80
frozen fair leaf, depth 3297,926255,000158,387844,31888.00
frozen fair leaf, depth 4 (reference)394,984302,556198,5251,663,637114.30
  • screen artifact: illegalDecisions 0 and incompleteDecisions 0 in every arm
  • bootstrap 95% lower bound of paired score delta > 0
  • Student-t 95% lower bound > 0
  • paired mean delta > 0 in both halves
  • candidate Q25 >= fair-d3s7 Q25
  • the preregistered gate was not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260902T035644Z-c1fd8987.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/heldout.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json

Because the paired margin over the fair leaf was still rising at generation 60, a second, longer experiment was registered under the same theory, EX-20260903-nnue-evolution-continuation-d3-f8ce9181, and launched on 2026-09-03 as run RUN-20260903T032832Z-a76a6cf7. It starts from the population the first run's final selection produced and never played, adds the first run's frozen candidate as a third paired control so the curve shows directly how far the population has moved, anneals the mutation size from the first run's setting toward a floor, and stops when a straight line fitted to the paired margin over the fair leaf across the last hundred generations no longer has a positive lower bound on its slope, checked every fifty generations from the hundredth, or at 1,000 generations or 72 hours. It plays fresh training and screen leases and ends with the same one-shot 64-game screen.

The plateau rule stopped evolution after generation 149. On a fresh 64-game screen the continued leaf still lost to the fair leaf, and beat the first run's frozen candidate (RS-20260903T163321Z-733076b5, run valid, outcome fail, ledger-recorded). The figures below are the continuation's own artifacts.

Run RUN-20260903T032832Z-a76a6cf7 under EX-20260903-nnue-evolution-continuation-d3-f8ce9181 · run record lifecycle: completedsnapshot 2026-09-03T16:33:27Z
  1. A · teacher corpusdoneinherited from the first run (177 games, 21,618 roots); no new teacher game
  2. B · supervised warm startdoneinherited from the first run; the population resumes from its generation-60 checkpoint
  3. C · evolutiondone150 of up to 1000 generations completed; stopped by the plateau rule
  4. C · elite re-selectiondonecandidate frozen, SHA-256 759084fab975…
  5. D · held-out screendonepreregistered gate not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts.

Fitness by generation against the two controls that played the same games

population meanbest candidatetop-4 meanunevolved init control (same seeds)
  • population mean
  • best candidate
  • top-4 mean
  • population range (lowest to highest candidate)
  • fair leaf control (same seeds)
  • unevolved init control (same seeds)
  • first run's frozen candidate (same seeds)
generations completed
150
latest best candidate
288,774
latest population mean
232,897
latest fair-leaf control
296,966
latest init control
146,339
mean above fair, last 10
0 of 10
mean margin over fair
-98,026
training-signal falsifier
no signal
latest first-run-candidate control
237,283
latest mean minus first run's candidate
-4,385
mutation sigma (latest)
0.0345
plateau check after gen 149
slope +37.31/gen, lower bound -158.88 → stop
stopped by the plateau rule
yes
illegal / incomplete decisions
0 / 0
Continuation, stage C. Generation 0 is the first run's generation-60 population. The violet dashed line is the first run's frozen candidate playing the same seeds each generation: the vertical distance between the population lines and it is a paired measure of what the continuation has added. Hover a generation for its sigma; the latest plateau check appears in the summary below the chart.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/analysis.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/evolve/progress.jsonl

Every candidate in every generation

  • one candidate
  • fair leaf on the same 32 games
  • unevolved init on the same 32 games
Every candidate of the continuation in every generation. As sigma anneals, the cloud should tighten; whether it also keeps drifting upward relative to the amber fair-leaf ticks is the question the plateau rule answers.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/evolve/progress.jsonl

Held-out screen: evolved candidate minus fair leaf, both at depth 3

recorded paired mean -68,441 (band: 95 % bootstrap)first half | second half
  • candidate scored higher on that game
  • reference scored higher on that game
  • recorded paired mean -68,441 (band: 95 % bootstrap)
paired mean difference
-68,441 points
bootstrap 95% lower bound
-112,090
Student-t 95% lower bound
-112,274
wins / ties / losses
25 / 0 / 39
first half / second half
-112,383 / -24,500
paired sd
210,055
detection floor
43,193
lower-quartile difference
-17,974
Continuation, stage D, the primary contrast on the new held-out block: continued candidate minus the fair leaf, one bar per game.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

Held-out screen: continued candidate minus the first run's candidate, both at depth 3

recorded paired mean +36,278 (band: 95 % bootstrap)first half | second half
  • candidate scored higher on that game
  • reference scored higher on that game
  • recorded paired mean +36,278 (band: 95 % bootstrap)
paired mean difference
+36,278 points
bootstrap 95% lower bound
+9,085
Student-t 95% lower bound
+8,543
wins / ties / losses
41 / 0 / 23
first half / second half
-7,140 / +79,697
paired sd
132,910
detection floor
27,330
lower-quartile difference
+33,068
The continuation's own question, out of sample: continued candidate minus the first run's frozen candidate on the same 64 never-read games.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

The four arms on the same 64 held-out games, and the preregistered gate

armmeanmedianlower quartilebest gamemean movescensored
evolved NNUE leaf, depth 3251,667224,341171,440571,36775.40
first run's frozen candidate, depth 3215,389185,102138,372505,06165.30
unevolved (supervised) NNUE leaf, depth 3143,674123,543107,000341,05445.30
frozen fair leaf, depth 3320,108276,827189,414940,70394.20
frozen fair leaf, depth 4 (reference)391,907351,132215,823962,024113.30
  • screen artifact: illegalDecisions 0 and incompleteDecisions 0 in every arm
  • bootstrap 95% lower bound of paired score delta > 0
  • Student-t 95% lower bound > 0
  • paired mean delta > 0 in both halves
  • candidate Q25 >= fair-d3s7 Q25
  • the preregistered gate was not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T032832Z-a76a6cf7.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

The second continuation resumed that plateau population with a slower decay in mutation size, keeping the same search and stopping rule. The run stopped at the same generation. On its fresh 64-game screen, the candidate still lost to the fair leaf, and its gain over the immediately prior candidate was not statistically distinguishable from zero (RS-20260904T090620Z-e5731bf0, run valid, outcome fail, ledger-recorded). This alternative schedule did not produce a confirmed further gain.

Run RUN-20260903T190433Z-a87fd7fc under EX-20260903-nnue-evolution-continuation2-d3-80eebad3 · run record lifecycle: completedsnapshot 2026-09-04T09:06:27Z
  1. A · teacher corpusdoneinherited from the first run (177 games, 21,618 roots); no new teacher game
  2. B · supervised warm startdoneinherited from the first run; the population resumes from its generation-60 checkpoint
  3. C · evolutiondone150 of up to 1000 generations completed; stopped by the plateau rule
  4. C · elite re-selectiondonecandidate frozen, SHA-256 5318e70e8e1c…
  5. D · held-out screendonepreregistered gate not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts.

Fitness by generation against the two controls that played the same games

population meanbest candidatetop-4 meanfair leaf control (same seeds)unevolved init control (same seeds)
  • population mean
  • best candidate
  • top-4 mean
  • population range (lowest to highest candidate)
  • fair leaf control (same seeds)
  • unevolved init control (same seeds)
  • first run's frozen candidate (same seeds)
generations completed
150
latest best candidate
305,455
latest population mean
244,380
latest fair-leaf control
367,660
latest init control
152,262
mean above fair, last 10
0 of 10
mean margin over fair
-70,199
training-signal falsifier
no signal
latest first-run-candidate control
229,454
latest mean minus first run's candidate
+14,926
mutation sigma (latest)
0.0453
plateau check after gen 149
slope +164.02/gen, lower bound -28.52 → stop
stopped by the plateau rule
yes
illegal / incomplete decisions
0 / 0
Second continuation, stage C. Generation 0 is the first continuation's generation-150 population. The violet dashed line is that run's frozen candidate playing the same seeds each generation.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/analysis.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/evolve/progress.jsonl

Every candidate in every generation

  • one candidate
  • fair leaf on the same 32 games
  • unevolved init on the same 32 games
Every candidate of the second continuation in every generation.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/evolve/progress.jsonl

Held-out screen: evolved candidate minus fair leaf, both at depth 3

recorded paired mean -85,509 (band: 95 % bootstrap)first half | second half
  • candidate scored higher on that game
  • reference scored higher on that game
  • recorded paired mean -85,509 (band: 95 % bootstrap)
paired mean difference
-85,509 points
bootstrap 95% lower bound
-128,482
Student-t 95% lower bound
-129,423
wins / ties / losses
22 / 0 / 42
first half / second half
-59,644 / -111,374
paired sd
210,440
detection floor
43,272
lower-quartile difference
-19,573
Second continuation, stage D, the primary contrast on the new held-out block: candidate minus the fair leaf.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

Held-out screen: continued candidate minus the first run's candidate, both at depth 3

recorded paired mean +13,573 (band: 95 % bootstrap)first half | second half
  • candidate scored higher on that game
  • reference scored higher on that game
  • recorded paired mean +13,573 (band: 95 % bootstrap)
paired mean difference
+13,573 points
bootstrap 95% lower bound
-30,165
Student-t 95% lower bound
-30,869
wins / ties / losses
31 / 0 / 33
first half / second half
+2,355 / +24,790
paired sd
212,968
detection floor
43,791
lower-quartile difference
+15,137
The second continuation's own question, out of sample: this candidate minus the immediately prior candidate on the same 64 never-read games.
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

The four arms on the same 64 held-out games, and the preregistered gate

armmeanmedianlower quartilebest gamemean movescensored
evolved NNUE leaf, depth 3249,757192,147156,535710,29775.10
first run's frozen candidate, depth 3236,184209,823141,398660,00370.80
unevolved (supervised) NNUE leaf, depth 3142,911124,024106,530269,98445.30
frozen fair leaf, depth 3335,266272,318176,1081,080,13398.20
frozen fair leaf, depth 4 (reference)333,857285,125194,039800,13097.20
  • screen artifact: illegalDecisions 0 and incompleteDecisions 0 in every arm
  • bootstrap 95% lower bound of paired score delta > 0
  • Student-t 95% lower bound > 0
  • paired mean delta > 0 in both halves
  • candidate Q25 >= fair-d3s7 Q25
  • the preregistered gate was not passed
Source

Values copied from the run snapshot web/content/figures/nnue-evolution/RUN-20260903T190433Z-a87fd7fc.json, which web/scripts/extract-nnue-evolution.ts writes from the run's artifacts:

  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/heldout.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-init-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-init.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-fair-d4s7-vs-fair-d3s7.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-candidate-vs-baseline-run1.json
  • runs/RUN-20260903T190433Z-a87fd7fc/nnue-evolution/screen/compare-baseline-run1-vs-fair-d3s7.json

What we learned

Whole-game evolution with common random numbers moves a 572,000-weight leaf. That is the finding the first, CMA-ES leaf evolution could not produce. The population separated from its warm start steadily across all sixty generations, the ten-generation paired margins over the warm start rising from +4,682 to +40,431 for the population mean and reaching +76,679 for the best candidate in the last block, and the held-out ablation confirmed it out of sample at +35,375 on the 64 screen games (RS-20260903T025751Z-6577b33e).

Imitation of state values is the weak link. The warm start matched the teacher's numbers (validation Pearson 0.9504 at the frozen epoch) yet, placed inside the real depth-3 search, picked the teacher's column on 44.14% of 256 held-out roots, and on the screen it played 142,340 paired points behind the fair leaf; evolution closed 35,375 of that, about a quarter of the way (same record). The teacher is the expensive part: a depth-5, seven-stratum search cost 77.5 seconds per root at full parallelism, so the corpus closed at 177 games instead of 512, and whether a larger corpus would have produced a warm start that holds the ordering is untested. The leaf decides: the screen agrees with the leaf-swing diagnostic above, since swapping the leaf moved the mean by more than 100,000 points in both directions.

The continuation kept working for roughly a hundred more generations and then levelled. That plateau sits about 45,000 paired points above where the continuation started and about 90,000 below the fair leaf on the training blocks, and the screen agrees (RS-20260903T163321Z-733076b5). The design could not separate whether the leaf class ran out of improvement or the annealed step size, which had fallen from 0.050 to 0.034, starved the search. The second continuation tested a slower decay and still stopped at the same generation, with no confirmed further gain on its fresh screen (RS-20260904T090620Z-e5731bf0). It tested one alternative schedule. More games per candidate or a larger population remain open ways to ask whether selection can see the small differences left as candidates converge. The other open question is whether a warm start trained on the teacher's ordering, instead of its values, would start close enough to the fair leaf for evolution to finish the job.

Agent contextRecords and provenance
  • Theory TH-20260825-evolved-nnue-leaf-d3-0f47e46c; frozen protocol EX-20260902-nnue-evolution-d3-v2-49c18bc2 (successor to the frozen EX-20260825-nnue-evolution-d3-bca7f330, whose scientific protocol it carries verbatim plus the operational parameters that record left open); run RUN-20260902T035644Z-c1fd8987, started 2026-09-02T04:24:10Z with 32 threads on the Ryzen AI MAX+ 395 workstation (research/system-profiles/MACH-20260902T035644Z-f5e59b6e.json). Result RS-20260903T025751Z-6577b33e (valid run, scientific outcome fail, assessment not-supported-as-tested, evidence tier public-development); compact evidence promoted with a manifest under artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/, including the frozen candidate and the warm start weights.
  • Seed leases: training SL-20260825T063000Z-a52e0300 (opened at launch; teacher corpus from 0xa52e0300, fitness blocks from 0xa52e0500, generation g playing the 32 seeds from 0xa52e0500 + 32·g, re-selection at 0xa52e0c80) and screen SL-20260825T063000Z-a52e1300 (64 seeds; opened only by the screen stage, at 2026-09-03T02:41:29Z, after the candidate's SHA-256 was recorded). Training and screen ranges are disjoint by construction.
  • Search: fair expectimax, depth 3, seven stratified chance samples, terminal utility −1,000,000, policy seed 0xd7075eed, completion-guaranteeing work bound, 64k-entry depth-gated table; the teacher is the same search at depth 5 with the frozen fair leaf. Network: 8,902 sparse features, 135 active per position; EmbeddingBag(8,902 × 64) → ReLU → 32 → ReLU → 1, output scaled by 17,000 points; about 572k parameters.
  • Stage parameters. A: 512 seeds requested, 500-move cap, new games stop at 46,800 s of wall clock. B: Adam, learning rate 3e-4, batch 64, Huber δ = 1 rise, 16 epochs, seed 0x0e701e57, 256 probe roots. C: population 32, 60 generations, 32 paired games per candidate per generation, 4 elites, tournament size 3, per-tensor σ = 5% of tensor standard deviation (floor 1e-4), initial cloud 2σ, evolution seed 0x0e701e58, 21,600 s sub-budget; top 8 re-evaluated on 128 fresh games. D: four arms on 64 seeds, compare.py with 20,000 bootstrap resamples at seed 0xb0071eaf.
  • Artifacts under runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/: gates.log, the preflight logs, corpus/parts/*.jsonl (one file per teacher game, resumable), corpus.err (one line per completed game), pretrain/{init.bin,report.json,probe.json}, evolve/{config.json,progress.jsonl,gen-*.json,population-*.bin,selection.json,candidate-weights.bin,candidate-weights.sha256}, screen/{heldout.json,compare-*.json}, rusage.jsonl (kernel resource usage per stage), analysis.json and analysis.md, and the leaf-swing diagnostic (leaf-swing-d{3,4,5}.txt with the frozen 30-game input under leaf-swing-input/). The page's figures read web/content/figures/nnue-evolution/<run>.json, written by web/scripts/extract-nnue-evolution.ts from the run's artifacts.
  • Implementation: a std-only Rust crate (approaches/lifetime-objective/nnue-evolution/) with a path dependency on the proven engine; the reference crate is not modified. Five binaries: gate (the CHECK-tier mechanics suite), teacher_corpus (stage A), pretrain (stage B), evolve (stage C, checkpointed and resumable, with --select for the elite re-selection), and screen (stage D, emitting the population-artifact format the existing compare.py consumes unchanged). The stages are driven by scripts/pipeline.sh (RUN_ID=… THREADS=32 scripts/pipeline.sh corpus|pretrain|evolve|select|screen|compare|chain), which wraps every stage in scripts/with-rusage.py so wall, CPU and peak memory land in rusage.jsonl, opens the screen lease through scripts/open-screen-lease.py only when the candidate hash file exists, refuses to run the screen twice, and passes the original total evolve allowance on every resume while the binary's durable deadline (wall-budget.json) prevents that allowance from starting over. A STOP file or the wall budget ends the loop at a generation boundary; a crash after the completion marker rebuilds any missing summary or plateau decision from the committed artifact before another seed can be read. scripts/analyze.py --run <run> summarises whatever artifacts exist into analysis.json and analysis.md.
  • Continuation: EX-20260903-nnue-evolution-continuation-d3-f8ce9181, run RUN-20260903T032832Z-a76a6cf7, started 2026-09-03T03:34:49Z. Result RS-20260903T163321Z-733076b5 (valid run, scientific outcome fail, assessment not-supported-as-tested, evidence tier public-development); compact evidence promoted with a manifest under artifacts/results/EX-20260903-nnue-evolution-continuation-d3-f8ce9181/RUN-20260903T032832Z-a76a6cf7/, including the frozen candidate, the plateau log and the launch script. The plateau rule stopped the run after generation 149. Leases: training SL-20260903T033000Z-a52e2000 (fitness blocks from 0xa52e2000, generation g playing [0xa52e2000 + 32·g, +32); the re-selection block follows the last fitness block played) and screen SL-20260903T033000Z-a52ea000 (64 seeds), both checked against every constant in the repository. Generation 0 is runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/evolve/population-060.bin (SHA-256 3da5021531898f08254d35cad5486fbbc12092770216bcdaea988d3724422cb8) played exactly as saved, with no re-noising. Three paired controls, never selected: the frozen fair leaf, the first run's warm start (init.bin, SHA-256 7d6fa5563de6716a4a895fb224bbe7f81c568391b6b737f85acc110872c075e0), and the first run's frozen candidate (candidate-weights.bin, SHA-256 edd0d2efd181de43f35d62c4df784cb2c789db3ae296be93a1c6c59082034a9f). Evolution seed 0x0e701e59. Annealed mutation: relative sigma max(0.01, 0.05 × exp(−g / 400)), so 0.05 at generation 0, 0.030 at 200, 0.018 at 400, and the 0.01 floor from generation 644 onward. Plateau rule: after every 50th completed generation from the 100th, ordinary least squares regresses the paired margin over the fair control across the last 100 generations; the run stops when the slope's one-sided 95% lower bound (slope − 1.645 × standard error) is not greater than zero. From the first run's artifacts the residual scatter about the trend was 38,813 points, at which the window detects slopes of about 221 points per generation (22,100 per 100 generations); the first run's own slope of +676 points per generation (one-sided 95% lower bound +200) would have kept it going. Other stops: 1,000 generations, a 259,200 s (72 h) evolve wall budget, a STOP file. Every check is appended to plateau.jsonl. Since the launch, a code-review pass repaired two interruption-only faults in the continuation driver (a completed generation could be stranded without its progress row or plateau marker, and a restart granted a fresh wall allowance); the launched run remains attributable to its recorded source commit.
  • Second continuation: EX-20260903-nnue-evolution-continuation2-d3-80eebad3, run RUN-20260903T190433Z-a87fd7fc, launched 2026-09-03. Result RS-20260904T090620Z-e5731bf0 (valid run, scientific outcome fail); compact evidence promoted under artifacts/results/EX-20260903-nnue-evolution-continuation2-d3-80eebad3/RUN-20260903T190433Z-a87fd7fc/. Generation 0 is the first continuation's generation-150 population. The mutation-decay time constant is 1,500 generations instead of 400, a 3.75-fold slowdown; sigma is still 0.045 at the equivalent point where the first continuation reached 0.034, and does not reach its floor within the 1,000-generation cap. The search, population, fitness blocks, plateau window and checks remain unchanged. The immediately prior candidate plays every block as the third control; the screen artifact calls this arm baseline-run1, with its identity recorded in evolve/config.json's baselineSha256 field. Training and screen use fresh leases.
  • How to continue from here: every generation's population is on disk, so a later experiment can resume the loop from any population-NNN.bin, start a new run from candidate-weights.bin, or change one operator at a time. Doing so is a new experiment: it needs its own record, its own training lease (the fitness blocks above are spent), and a fresh never-read screen block.
Technical recordFull results tableRS-20260903T025751Z-6577b33e

Screen (stage D), 64 never-read games from 0xa52e1300, 2,000-move cap, corrected 17,000-point Hardcore scoring, zero censored, illegal or incomplete decisions in every arm:

Arm (depth-3 search unless stated)MeanMedianLower quartileMean movesClears/moveReveals/move
evolved candidate190,961160,108122,25358.501.71310.9124
unevolved warm start (ablation)155,586140,282107,42348.751.55100.7554
frozen fair leaf (comparator)297,926255,000158,38788.031.95491.0722
frozen fair leaf, depth 4 (reference arm)394,984302,556198,525114.302.05471.1494

Paired contrasts, 20,000 bootstrap resamples:

ContrastMean deltaBootstrap 95% boundsStudent-t lower 95%Detection floorW-T-LHalves
candidate − fair leaf−106,964−146,580 to −69,983−145,89038,35714-0-50−109,139 / −104,790
warm start − fair leaf−142,340−178,492 to −107,771−178,51135,6429-0-55−156,143 / −128,537
candidate − warm start+35,375lower +16,89918,94936-0-28both positive
fair depth 4 − fair depth 3+97,059lower +29,330+27,01169,02341-0-23+75,458 / +118,660

The candidate's lower quartile (122,253) fell below the fair leaf's (158,387), a gate criterion in its own right.

Stage A: 177 complete teacher games (21,618 sibling-complete labelled roots, 3 stopped at the 500-move cap) before the 46,800 s new-game cutoff; 77.5 s per root, four to seven times slower than the pilot projected, so the corpus is about a third of the 512 games the protocol allowed for. Stage B: validation Huber 0.6873 rise units at epoch 8 of 16 on a whole-origin split of 17,641 / 3,977 roots; validation Pearson 0.9504 at that epoch; the deployment-faithful probe put the depth-3 search with the warm start at top-1 agreement 0.4414 with the teacher on 256 held-out roots, mean teacher-value regret 2,395 points. Stage C: 60 generations; ten-generation paired mean margins over the warm-start control +4,682, +15,853, +22,704, +32,278, +34,037, +40,431 points, best candidate +76,679 in the last block; the population mean was above the fair leaf in 0 of the final 10 generations and the best candidate in 0 (mean margin −121,896), so the theory's training-signal falsifier fails. Elite re-selection on 128 fresh games froze candidate-29 at 202,237 (finalists spanned 187,352 to 202,237).

Technical recordContinuation screenRS-20260903T163321Z-733076b5

Continuation screen (stage D), 64 never-read games from 0xa52ea000, 2,000-move cap, corrected 17,000-point Hardcore scoring, zero censored, illegal or incomplete decisions in every arm:

Arm (depth-3 search unless stated)MeanMedianLower quartileMean movesClears/moveReveals/move
continued candidate251,667224,341171,44075.361.88681.0404
first-run candidate215,389185,102138,37265.331.78980.9589
unevolved warm start143,674123,543107,00045.311.51240.7314
frozen fair leaf (comparator)320,108276,827189,41494.171.99351.1120
frozen fair leaf, depth 4 (reference arm)391,907351,132215,823113.282.05261.1506

Paired contrasts, 20,000 bootstrap resamples:

ContrastMean deltaBootstrap 95% boundsStudent-t lower 95%Detection floorW-T-LHalves
continued − fair leaf−68,441−112,090 to −26,694−112,27443,19325-0-39−112,383 / −24,500
continued − first-run candidate+36,278lower +9,085+8,54327,33041-0-23−7,140 / +79,697
continued − warm start+107,994lower +82,769+82,11925,49754-0-10both positive
first-run candidate − fair leaf−104,719lower −143,479−143,89538,60220-0-44both negative
fair depth 4 − fair depth 3+71,799lower +25,857+23,28647,80439-0-25+31,746 / +111,851

The plateau rule stopped the run after generation 149: slope +239 per generation (lower bound +30) at the generation-99 check, then +37 (lower bound −159) across generations 50 to 149. Fifty-generation averages of the population mean: 204,748, 228,706, 237,946. Elite re-selection on 128 fresh games froze candidate-12 at 264,466 (finalists spanned 228,212 to 264,466). The continued candidate's lower quartile (171,440) fell below the fair leaf's (189,414).

Technical recordSecond continuation screenRS-20260904T090620Z-e5731bf0

Second continuation screen (stage D), 64 fresh held-out games. Values below are recorded in RS-20260904T090620Z-e5731bf0:

Arm (depth-3 search)MeanLower quartile
second-continuation candidate249,757156,534
frozen fair leaf (comparator)335,266176,108
Paired contrast on the same 64 gamesMean deltaBootstrap 95% lower bound
candidate − fair leaf−85,509−128,482
candidate − immediately prior candidate+13,572−30,165

The gain over the prior candidate is not statistically distinguishable from zero. The first continuation's equivalent contrast against its own predecessor, on its separate 64-game screen, was +36,278 with a lower bound of +9,085 (RS-20260903T163321Z-733076b5).

The plateau rule stopped both continuations after generation 149. The second continuation's slope at that check was +164 points per generation, with a lower bound of −29, against the first continuation's +37. Its best candidate beat the fair control on 26 of 150 training blocks, against 9 before. The population mean rose through all three fifty-generation blocks (235,353, 238,797, 244,800). Each generation used 32 paired games per candidate. These training observations did not translate into a confirmed further gain on the screen.

The comparison tests one slower schedule. It cannot separate limited room for improvement in this leaf class at this population size and search depth from a schedule that still shrinks too quickly or selection noise that hides small improvements. A further schedule comparison, more games per candidate or a larger population would each require a new experiment.

Agent contextValidity, gates and limitations
  • The preregistered gate, fixed before any leased seed was read: on the 64 held-out games the evolved candidate must beat the frozen fair leaf at the identical depth-3 configuration with a paired mean difference whose one-sided 95% bootstrap lower bound and Student-t lower bound are both above zero, the difference must be positive in both halves of the block, and the candidate's lower-quartile score may not fall below the fair leaf's. Every generation artifact and the screen artifact must record zero illegal and zero incomplete decisions, and the screened weights must hash to the value recorded before the screen lease opened. Outcome: every pass criterion except artifact integrity and candidate identity fails. The continuation screen (RS-20260903T163321Z-733076b5) fails the same gate on a fresh 64-game block. The second continuation (RS-20260904T090620Z-e5731bf0) also fails the same gate on its fresh 64-game block; the gain over the immediately prior candidate is unconfirmed.
  • What the rejection covers: this exact configuration, this model class, this teacher depth, this optimiser and budget, this deployment depth. It does not reject the idea of a learned leaf. Had the candidate passed while the unevolved warm start passed equally on the same seeds, the evolutionary stage would have contributed nothing and the theory's second leg would have been refuted even with the network standing. The retained corpus, warm start and populations remain available for a successor experiment with its own record and leases.
  • Gates before the first leased seed: 11 CHECK gates on the already-open probe block (feature determinism and bounds, information-boundary blindness to score, level and move count, reflection consistency, fresh-searcher and multi-worker determinism, legality and completed depth under random, zero and saturated weights, leaf finiteness, serialisation round-trip), all passed; a seed-free SMT preflight on the same probe block chose 32 threads over 16. The information-boundary gate proves that states differing only in score, level or move count produce bit-identical evaluations.
  • Limitations: the corpus reached about a third of its planned size, so the distillation leg was tested with less data than designed; 64 screen games with a paired standard deviation of 186,538 points give a detection floor of 38,357 for the primary contrast; the fitness blocks are development data and the screen block is now read.
Agent contextScoring mode

Corrected 17,000-point Hardcore scoring throughout: fitness blocks and the screen use the ordinary 2,000-move cap, teacher games a 500-move cap (3 of 177 stopped there). The network's output and the Huber loss are in units of one row rise, 17,000 points. No historical 7,000-point number appears on this page.

RecordsTheories, experiments and results that reference this directory
theorynot-supported-as-testedassessedA depth-5-distilled, whole-game-evolved NNUE leaf beats the frozen fair leaf inside the same depth-3 search

Claim: A leaf-affordable NNUE evaluator (the 8,902-feature, 135-active sparse class of approaches/lifetime-objective/learned-leaf), whose weights are (a) initialised by distilling the sibling-complete root values of a depth-5 seven-stratum fair-expectimax teacher and (b) then refined by a mutation-only genetic algorithm whose fitness is the mean score of complete paired depth-3 seven-stratum games, deployed as the leaf of that same depth-3 seven-stratum search, achieves a higher mean whole-game score on never-read paired development games than the identical search using the frozen fair leaf.

This theory is currently not-supported-as-tested at the public-development (a cohort for deciding what to try next, not confirmation) level.

experimentsupersededDepth-5-distilled NNUE leaf refined by paired-fitness whole-game evolution inside the depth-3 fair search

It compares d3s7-evolved-nnue-leaf against fair-d3s7 at the SCREEN (a 32-game paired screen) level, using public-development data.

No result has been recorded for it.

experimentcompletedDepth-5-distilled NNUE leaf refined by paired-fitness whole-game evolution inside the depth-3 fair search (v2: operational parameters fixed in the record body)

It compares d3s7-evolved-nnue-leaf against fair-d3s7 at the SCREEN (a 32-game paired screen) level, using public-development data.

valid run outcome: fail The run was valid and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.

experimentcompletedContinued whole-game evolution of the depth-5-distilled NNUE leaf from the first run's generation-60 population, with annealed mutation and a preregistered plateau stop (up to 1,000 generations)

It compares d3s7-evolved-nnue-leaf-continued against fair-d3s7 at the SCREEN (a 32-game paired screen) level, using public-development data.

valid run outcome: fail The run was valid and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.

experimentcompletedSecond continuation of the depth-5-distilled NNUE leaf evolution: resume from the plateau population with a slower mutation-size decay (up to 1,000 further generations)

It compares d3s7-evolved-nnue-leaf-continued2 against fair-d3s7 at the SCREEN (a 32-game paired screen) level, using public-development data.

valid run outcome: fail The run was valid and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.

resultvalid runoutcome: failtier: public-developmentRS-20260903T025751Z-6577b33e

The run was valid; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 9 preregistered checks, 4 passed and 4 failed.

Valid run of the frozen protocol EX-20260902-nnue-evolution-d3-v2-49c18bc2 on the whole workstation (32 threads), every stage completed, every artifact with zero illegal and zero incomplete decisions. Stage A: the depth-5 seven-stratum teacher played 177 complete games (21,618 sibling-complete labelled roots, 3 stopped at the 500-move cap) before the 46,800 s new-game cutoff; the teacher ran 77.5 s per root, four to seven times slower than the pilot projected, so the corpus is about a third of the 512 games the protocol allowed for. Stage B: the supervised warm start reached validation Huber 0.6873 rise units at epoch 8 of 16 on a whole-origin split (17,641/3,977 roots), and the deployment-faithful ordering probe put the depth-3 search with that leaf at top-1 agreement 0.4414 with the teacher on 256 held-out roots (mean teacher-value regret 2,395 points). Stage C: 60 generations of the mutation-only GA (population 32, 32 paired games per candidate per fresh block, fair-d3s7 and warm-start controls on every block). The population separated steadily from its warm start - ten-generation paired mean margins over the init control of +4,682, +15,853, +22,704, +32,278, +34,037, +40,431 points, best candidate +76,679 in the last block - but never approached the fair control: the population mean was above the fair leaf in 0 of the final 10 generations and the best candidate in 0 (mean margin -121,896), so the theory's training-signal falsifier fails. Elite re-selection on 128 fresh games froze candidate-29 at 202,237 (finalists spanned 187,352 to 202,237). Stage D: on the 64 never-read held-out games (0xa52e1300, opened once at 2026-09-03T02:41:29Z after the candidate's SHA-256 was recorded), the evolved candidate averaged 190,961 against the frozen fair leaf's 297,926 at the identical depth-3 configuration: paired delta -106,964 (bootstrap 95% bounds -146,580 to -69,983, Student-t lower bound -145,890, detection floor 38,357), W-T-L 14-0-50, both halves negative (-109,139 / -104,790), lower quartile 122,253 against 158,387. Every preregistered pass criterion except artifact integrity and candidate identity fails: scientific outcome fail for this exact configuration. The ablation arm shows what evolution did contribute: the unevolved warm start averaged 155,586 (-142,340 against the fair leaf), and the evolved candidate beat it on the same seeds by +35,375 (bootstrap lower bound +16,899, floor 18,949, W-T-L 36-0-28, both halves positive). Whole-game evolution with common random numbers therefore moves a 572k-weight leaf on the deployed objective, which the first (CMA-ES) leaf evolution could not show; it moved it about a quarter of the way from a warm start that plays at half the fair leaf's level. The reference arm reproduced the program's standing result: fair depth 4 over fair depth 3 +97,059 (lower bound +29,330). Read: the claim is not supported as tested; the mechanism's evolutionary leg is supported, its distillation leg is the weak link (a warm start that holds the teacher's values but not its ordering), and the budget (177 teacher games, 60 generations) was too small for evolution to cover the distance.

Technical recordLimitations recorded with the resultRS-20260903T025751Z-6577b33e
  • Single 64-game held-out screen: the paired detection floor is about 38,000 points for the primary contrast and 19,000 for the ablation; the primary result is far outside its floor, the ablation clears its own.
  • The teacher corpus reached 177 of the 512 games the protocol allowed for because the depth-5 teacher ran four to seven times slower than the pilot projected; the protocol makes the completed games the corpus, so the result is valid, but it rejects this configuration at this corpus size, not the design at 512 games.
  • The supervised warm start plays at about half the fair leaf's level; the deployment-faithful probe (0.441 top-1) and the leaf-swing diagnostic (zero leaf 0.450 on a different sample) suggest imitation of state values barely improved the search's ordering over no leaf at all. Evolution then had roughly 150,000 paired points to make up in 60 generations and made up about 35,000 to 40,000.
  • The per-game artifact holds four arms x 64 games (256 rows); the primary contrast pairs the candidate and fair-d3s7 rows by seed.
  • The fair-d4s7 arm is diagnostic only and reproduces the standing depth-4-over-depth-3 result on these seeds; it is not part of the gate.
  • Two operational faults during the unattended chain (a false liveness reading, then a script replacement that crashed the corpus stage's shell after its binary had exited 0) are recorded in the run record; the 'corpus: done' marker in pipeline.log was appended by hand with a note. No artifact was affected.
  • The elite re-selection block (0xa52e0c80, 128 games) and every fitness block are training-lease seeds; the last generation's 32-game leaders read 20,000 to 60,000 above their 128-game re-selection means, which is the best-of-32 bias the re-selection exists to remove.

Full record →

resultvalid runoutcome: failtier: public-developmentRS-20260903T163321Z-733076b5

The run was valid; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 9 preregistered checks, 4 passed and 4 failed.

Valid run of the frozen protocol EX-20260903-nnue-evolution-continuation-d3-f8ce9181 (successor to RS-20260903T025751Z-6577b33e), every stage completed, every artifact with zero illegal and zero incomplete decisions. Stage C resumed from the first run's checkpointed generation-60 population (SHA-256 3da5021531898f08254d35cad5486fbbc12092770216bcdaea988d3724422cb8) with the first run's frozen candidate (SHA-256 edd0d2efd181de43f35d62c4df784cb2c789db3ae296be93a1c6c59082034a9f) as a third paired control and an annealed mutation size (sigma_rel from 0.05, time constant 400 generations, floor 0.01; it reached 0.0345 at the last generation). The preregistered plateau rule stopped the run after generation 149: the first check after generation 99 found the paired margin over the fair control rising at +238.6 points per generation (standard error 127.1, one-sided 95% lower bound +29.6) and continued; the second, after generation 149, found +37.3 per generation (standard error 119.3, lower bound -158.9), no detectable improvement over generations 50-149, and stopped. Fifty-generation averages of the population mean: 204,748, 228,706, 237,946; paired margin over the first run's candidate +8,982, +36,076, +45,613; over the fair control -116,503, -99,833, -88,948. The best candidate beat the fair control on 9 of 150 blocks (generations 59, 66, 73, 97, 100, 109, 115, 125, 137); the population mean beat it on none, so the theory's training-signal falsifier fails again (mean above fair in 0 of the last 10, margin -98,026). Elite re-selection on 128 fresh games (0xa52e32c0) froze candidate-12 at 264,466 (finalists 228,212 to 264,466). Stage D, 64 never-read games (0xa52ea000, opened once at 2026-09-03T16:21:00Z after the candidate's SHA-256 was recorded), five arms: the continued candidate averaged 251,667 against the frozen fair leaf's 320,108 at the identical depth-3 configuration, paired -68,441 (bootstrap 95% bounds -112,090 to -26,694, Student-t lower bound -112,274, floor 43,193), W-T-L 25-0-39, halves -112,383 / -24,500, lower quartile 171,440 against 189,414: every screen criterion fails, scientific outcome fail for this configuration. The preregistered secondary contrast answers the continuation's own question: the continued candidate beat the first run's frozen candidate on the same seeds by +36,278 (bootstrap lower bound +9,085, Student-t lower bound +8,543, floor 27,330, W-T-L 41-0-23, halves -7,140 / +79,697), so 150 further generations improved the leaf out of sample, by about a third of the remaining distance. The first run's candidate reproduced its earlier result on fresh seeds: -104,719 against the fair leaf here (lower bound -143,479) against -106,964 on the first screen. Over the warm start the continued candidate is +107,994 (lower bound +82,769, W-T-L 54-0-10). The reference arm gave fair depth 4 over fair depth 3 +71,799 (lower bound +25,857). Read: whole-game evolution keeps improving the leaf until roughly generation 100 of the continuation and then levels off about 70,000 paired points short of the frozen fair leaf on held-out games; the claim is not supported at this budget, and the plateau is the new fact. Whether the levelling is a property of the leaf class or of the annealed step size cannot be separated in this design.

Technical recordLimitations recorded with the resultRS-20260903T163321Z-733076b5
  • Single 64-game held-out screen: paired detection floors of about 43,000 points for the primary contrast and 27,000 for the continuation contrast; the primary result sits far outside its floor, the continuation contrast just outside its own, and its first-half estimate is negative, so 'improved out of sample' is established at the whole-block level only.
  • The plateau rule and the annealed mutation size are confounded: the step size had fallen from 0.050 to 0.034 when the rule fired, so the design cannot say whether the leaf class ran out of improvement or the search did. A constant-sigma continuation from the same population would separate them.
  • The plateau rule tests a fitted slope over 100 generations with a one-sided 95% lower bound; at the observed scatter it can miss real improvements below about 220 points per generation. The observed point estimate over generations 50-149 was +37 per generation.
  • The per-game artifact holds five arms x 64 games (320 rows); each contrast pairs two arms by seed.
  • The starting population, the warm-start control and the baseline control are products of the first run's training data; nothing in this run re-read that lease.
  • The owner's commit e2b0d19 (wall-budget pinning and progress recovery on resume) landed on the branch mid-run; the run never resumed, the driver shells held the original script by an unlinked inode, and every stage ran the binaries built from commit e4fd018 as the run record states.
  • The fair-d4s7 arm is diagnostic only. The first run's candidate reproduced its earlier screen result on these fresh seeds (-104,719 against -106,964), which is the closest thing to a replication the program has for that number.

Full record →

resultvalid runoutcome: failtier: public-developmentRS-20260904T090620Z-e5731bf0

The run was valid; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 8 preregistered checks, 3 passed and 4 failed.

Valid run of the frozen protocol EX-20260903-nnue-evolution-continuation2-d3-80eebad3 (second successor to the theory, third experiment in the series), every stage completed, every artifact with zero illegal and zero incomplete decisions. Stage C resumed from the second experiment's checkpointed generation-150 population (SHA-256 33dcc25b88ab56a0a80329bf11983b539883aa922900b03de3cea71e1da3529d) with that experiment's frozen candidate (SHA-256 759084fab97599818d01a5da6d18b9cc77172249f09a68d5104361bca385e09b) as the third paired control, testing whether a 3.75x slower mutation-size decay (time constant 1,500 generations vs. 400 before; sigma stayed at 0.0500 at generation 0, 0.0453 at the stop, versus 0.0345 at the equivalent generation last time) would let evolution get further past the point where the prior run plateaued. The identical preregistered plateau rule stopped this run at the identical generation as before, 149: the check after generation 99 found the paired margin over the fair control rising at +201.7 points per generation (lower bound +11.7, versus +239 and +30 in the second experiment) and continued; the check after generation 149 found +164.0 per generation (lower bound -28.5, versus +37 and -159 before) and stopped. The improvement rate at the stopping point was more than four times the prior run's (164 against 37 points/generation), and the best candidate beat the fair control on 26 of 150 blocks against 9 in the second experiment, but the lower bound of the fitted slope still crossed zero at the same 100-generation checkpoint, so the rule fired regardless. Fifty-generation population-mean averages: 235,353, 238,797, 244,800 (rising monotonically, unlike the second experiment's flat third block); paired margin over the immediately-prior candidate +4,301, -949, +13,589; over the fair control -99,479, -88,006, -77,654. Training-signal falsifier fails again (population mean above fair in 0 of the last 10 generations, margin -70,199; best above fair in 4 of the last 10). Elite re-selection on 128 fresh games (0xa52eb3c0) froze candidate-28 at 257,314 (finalists 232,499 to 257,314, a tighter spread than the second experiment's). Stage D, 64 never-read games (0xa52f2100, opened once at 2026-09-04T08:56:37Z after the candidate's SHA-256 was recorded), five arms: this candidate averaged 249,757 against the frozen fair leaf's 335,266, paired -85,509 (bootstrap 95% bounds -128,482 to -43,591, Student-t lower bound -129,423, floor 43,272), W-T-L 22-0-42, both halves negative (-59,644 / -111,374), lower quartile 156,534 against 176,108: every screen criterion fails, scientific outcome fail for this configuration, consistent with both prior screens. The preregistered secondary contrast is the key negative finding of this run: the candidate beat the immediately-prior (second experiment's) frozen candidate by only +13,572 on the same seeds, with a bootstrap 95% lower bound of -30,165 and a Student-t lower bound of -30,869 -- both crossing zero (detection floor 43,791), W-T-L 31-0-33. Unlike the second experiment's clearly positive +36,278 (lower bound +9,085) over the first experiment's candidate, this third experiment's gain over the second is NOT statistically distinguishable from zero at this screen size: three months of relatively larger mutations bought a point estimate about a third the size of the previous continuation's out-of-sample gain, and it is not confidently positive. The second experiment's candidate itself scored -99,081 against the fair leaf on this fresh block (lower bound -154,094), broadly consistent with its own screen result of -68,441 (lower bound -112,090), a second informal replication. Over the warm start the candidate is +106,846 (lower bound +76,970). The reference arm gave fair depth 4 minus fair depth 3 of -1,409 (bounds -62,290 to +58,933, crossing zero on this cohort, a diagnostic-only reading and not comparable across screens with different fresh seeds). Read together with the second experiment: slowing the mutation-size decay produced a visibly healthier training curve (a still-rising population mean through all three 50-generation blocks, more generations where the best candidate beat the fair control) but did NOT produce a statistically confirmed improvement over the immediately-prior candidate on held-out games, and the plateau rule still stopped the run at the same generation. The most defensible reading is that the annealing schedule was not the dominant cause of the earlier plateau -- something else (population size, games per candidate, or the leaf class's genuine ceiling under this search depth) is the binding constraint, though a schedule 3.75x slower is not proof that no schedule would help; a much slower schedule, or removing the anneal-driven exploration decay entirely in favour of a fixed sigma with more games per candidate, remains untested.

Technical recordLimitations recorded with the resultRS-20260904T090620Z-e5731bf0
  • Single 64-game held-out screen: paired detection floors of about 43,000 points for the primary contrast and 44,000 for the baseline contrast; the baseline-run1 (secondary) contrast's point estimate of +13,572 sits well inside its own floor, so 'no confirmed improvement over the prior candidate' is the correct reading, not 'no improvement occurred' -- a true effect below about 44,000 points could not be distinguished from zero at this sample size.
  • The plateau rule fired at the identical generation (149) as the second experiment despite a 3.75x slower decay constant; this is evidence against the annealing schedule being the dominant cause of the second experiment's plateau, but it is a single comparison at one alternative time constant, not a sweep, and cannot rule out that some other (e.g. much slower, or non-exponential) schedule would behave differently.
  • The per-game artifact holds five arms x 64 games (320 rows); each contrast pairs two arms by seed.
  • The starting population and both controls are products of the first and second experiments' training data; nothing in this run re-read either prior lease.
  • pipeline.sh's screen and compare stages hardcode the third control's arm name as 'baseline-run1' regardless of which run's candidate is actually supplied via $BASELINE; in this run that arm holds the SECOND experiment's frozen candidate, not the first. The mapping is recorded accurately in evolve/config.json's baselineSha256 field (matches the second experiment's candidate hash) and in this record's metrics.priorRuns; the label itself is cosmetic and should be parameterised in a future pipeline.sh edit made only while no stage is executing.
  • The fair-d4s7 arm is diagnostic only and its contrast against fair-d3s7 crossed zero on this cohort's fresh seeds (-1,409, bounds -62,290 to +58,933); this is expected cohort-to-cohort variation on a small paired sample and is not comparable across the three screens, which drew different seed blocks.

Full record →

Agent contextSource files, operational notes and how to reproduce

Directory: approaches/lifetime-objective/nnue-evolution