On this page
Dates
Recorded
Record idRS-20260903T163321Z-733076b5

No explanation has been written for this record yet.

Technical recordMetrics, gate checks and limitationsRS-20260903T163321Z-733076b5
valid runoutcome: failnot-supported-as-testedtier: public-developmentRS-20260903T163321Z-733076b5

Valid run of the frozen protocol EX-20260903-nnue-evolution-continuation-d3-f8ce9181 (successor to RS-20260903T025751Z-6577b33e), every stage completed, every artifact with zero illegal and zero incomplete decisions. Stage C resumed from the first run's checkpointed generation-60 population (SHA-256 3da5021531898f08254d35cad5486fbbc12092770216bcdaea988d3724422cb8) with the first run's frozen candidate (SHA-256 edd0d2efd181de43f35d62c4df784cb2c789db3ae296be93a1c6c59082034a9f) as a third paired control and an annealed mutation size (sigma_rel from 0.05, time constant 400 generations, floor 0.01; it reached 0.0345 at the last generation). The preregistered plateau rule stopped the run after generation 149: the first check after generation 99 found the paired margin over the fair control rising at +238.6 points per generation (standard error 127.1, one-sided 95% lower bound +29.6) and continued; the second, after generation 149, found +37.3 per generation (standard error 119.3, lower bound -158.9), no detectable improvement over generations 50-149, and stopped. Fifty-generation averages of the population mean: 204,748, 228,706, 237,946; paired margin over the first run's candidate +8,982, +36,076, +45,613; over the fair control -116,503, -99,833, -88,948. The best candidate beat the fair control on 9 of 150 blocks (generations 59, 66, 73, 97, 100, 109, 115, 125, 137); the population mean beat it on none, so the theory's training-signal falsifier fails again (mean above fair in 0 of the last 10, margin -98,026). Elite re-selection on 128 fresh games (0xa52e32c0) froze candidate-12 at 264,466 (finalists 228,212 to 264,466). Stage D, 64 never-read games (0xa52ea000, opened once at 2026-09-03T16:21:00Z after the candidate's SHA-256 was recorded), five arms: the continued candidate averaged 251,667 against the frozen fair leaf's 320,108 at the identical depth-3 configuration, paired -68,441 (bootstrap 95% bounds -112,090 to -26,694, Student-t lower bound -112,274, floor 43,193), W-T-L 25-0-39, halves -112,383 / -24,500, lower quartile 171,440 against 189,414: every screen criterion fails, scientific outcome fail for this configuration. The preregistered secondary contrast answers the continuation's own question: the continued candidate beat the first run's frozen candidate on the same seeds by +36,278 (bootstrap lower bound +9,085, Student-t lower bound +8,543, floor 27,330, W-T-L 41-0-23, halves -7,140 / +79,697), so 150 further generations improved the leaf out of sample, by about a third of the remaining distance. The first run's candidate reproduced its earlier result on fresh seeds: -104,719 against the fair leaf here (lower bound -143,479) against -106,964 on the first screen. Over the warm start the continued candidate is +107,994 (lower bound +82,769, W-T-L 54-0-10). The reference arm gave fair depth 4 over fair depth 3 +71,799 (lower bound +25,857). Read: whole-game evolution keeps improving the leaf until roughly generation 100 of the continuation and then levels off about 70,000 paired points short of the frozen fair leaf on held-out games; the claim is not supported at this budget, and the plateau is the new fact. Whether the levelling is a property of the leaf class or of the annealed step size cannot be separated in this design.

What it had to pass
  • All 11 CHECK gates passed on the rebuilt binaries before the first leased seed was read — observed: ALL GATES PASSED (gates.log), 13/13 unit tests, smoke test of every new flag on the already-open probe block
  • Every generation artifact and the screen artifact: illegalDecisions 0, incompleteDecisions 0, censored games reported — observed: 150 generation artifacts: 0 illegal, 0 incomplete, 0 censored; screen: 0 illegal, 0 incomplete, 0 censored in all five arms
  • Held-out screen, candidate vs fair-d3s7: bootstrap 95% lower bound of the paired score delta > 0 AND Student-t 95% lower bound > 0 — observed: delta -68,441; bootstrap LB -112,090, UB -26,694; t LB -112,274; paired sd 210,055; floor 43,193; W-T-L 25-0-39
  • Held-out screen: paired mean score delta > 0 in both halves — observed: first half -112,383, second half -24,500
  • Held-out screen: candidate lower-quartile score >= fair-d3s7 lower-quartile score — observed: candidate Q25 171,440 vs fair-d3s7 Q25 189,414 (delta -17,974)
  • The screened candidate is the exact frozen winner of the elite re-selection; SHA-256 recorded before the screen lease opened — observed: candidate-12 of generation 149, mean 264,466 over 128 fresh games at 0xa52e32c0; sha256 759084fab97599818d01a5da6d18b9cc77172249f09a68d5104361bca385e09b; lease opened 2026-09-03T16:21:00Z by the screen stage
  • Reported alongside the gate: candidate vs baseline-run1 bootstrap 95% lower bound > 0 (the continuation improved the leaf out of sample) — observed: delta +36,278; bootstrap LB +9,085, UB +63,768; t LB +8,543; floor 27,330; W-T-L 41-0-23; halves -7,140 / +79,697 (the first half alone is negative)
  • Reported alongside the gate: the generation of the stop and the plateau statistics — observed: stopped by the plateau rule after generation 149 (150 completed): check after generation 99 slope +238.6/gen, se 127.1, LB +29.6 (continue); after generation 149 slope +37.3/gen, se 119.3, LB -158.9 (stop); window means -99,833 then -88,948
  • Theory falsifier (training signal): population mean fitness > paired fair-d3s7 control in the majority of the final 10 generations — observed: mean above fair in 0/10, best in 0/10; mean margin -98,026; best candidate above fair on 9 of 150 blocks
Technical recordRecorded metricsRS-20260903T163321Z-733076b5
cohort
seedsStartHex
0xa52ea000
games
64
moveCap
2,000
role
public-development, never read before this screen (lease SL-20260903T033000Z-a52ea000, opened once)
arms
candidate
games
64
mean
251667.3906
median
224340.5000
q25
171439.7500
max
571,367
movesMean
75.3594
clearsPerMove
1.8868
revealsPerMove
1.0404
censored
0
illegal
0
incomplete
0
wallSeconds
576.4845
baseline-run1
games
64
mean
215389.0156
median
185101.5000
q25
138371.7500
max
505,061
movesMean
65.3281
clearsPerMove
1.7898
revealsPerMove
0.9589
censored
0
illegal
0
incomplete
0
wallSeconds
502.0007
init-d3s7
games
64
mean
143673.7031
median
123,543
q25
106999.5000
max
341,054
movesMean
45.3125
clearsPerMove
1.5124
revealsPerMove
0.7314
censored
0
illegal
0
incomplete
0
wallSeconds
331.6452
fair-d3s7
games
64
mean
320108.4844
median
276826.5000
q25
189,414
max
940,703
movesMean
94.1719
clearsPerMove
1.9935
revealsPerMove
1.1120
censored
0
illegal
0
incomplete
0
wallSeconds
272.6142
fair-d4s7
games
64
mean
391907.1719
median
351,132
q25
215822.7500
max
962,024
movesMean
113.2813
clearsPerMove
2.0526
revealsPerMove
1.1506
censored
0
illegal
0
incomplete
0
wallSeconds
11087.7038
paired
candidate-vs-fair-d3s7
meanDelta
-68441.0938
bootstrapLower95
-112089.9094
bootstrapUpper95
-26693.8359
studentTLower95
-112274.3615
pairedSd
210054.9151
detectionFloor
43192.5419
wtl
  1. 25
  2. 0
  3. 39
halves
  1. -112382.5625
  2. -24499.6250
q25Delta
-17974.2500
movesDelta
-18.8125
init-vs-fair-d3s7
meanDelta
-176434.7813
bootstrapLower95
-213633.3297
bootstrapUpper95
-140506.4984
studentTLower95
-214034.1823
pairedSd
180181.3872
detectionFloor
37049.7977
wtl
  1. 8
  2. 0
  3. 56
halves
  1. -178178.6250
  2. -174690.9375
q25Delta
-82414.5000
movesDelta
-48.8594
candidate-vs-init
meanDelta
107993.6875
bootstrapLower95
82769.3031
bootstrapUpper95
133701.0930
studentTLower95
82118.8189
pairedSd
123995.8508
detectionFloor
25496.6468
wtl
  1. 54
  2. 0
  3. 10
halves
  1. 65796.0625
  2. 150191.3125
q25Delta
64440.2500
movesDelta
30.0469
fair-d4s7-vs-fair-d3s7
meanDelta
71798.6875
bootstrapLower95
25857.3078
bootstrapUpper95
119451.2141
studentTLower95
23285.7939
pairedSd
232480.3118
detectionFloor
47803.7641
wtl
  1. 39
  2. 0
  3. 25
halves
  1. 31746.1563
  2. 111851.2188
q25Delta
26408.7500
movesDelta
19.1094
candidate-vs-baseline-run1
meanDelta
36278.3750
bootstrapLower95
9085.1570
bootstrapUpper95
63768.1234
studentTLower95
8543.2666
pairedSd
132910.3700
detectionFloor
27329.6948
wtl
  1. 41
  2. 0
  3. 23
halves
  1. -7139.9375
  2. 79696.6875
q25Delta
33,068
movesDelta
10.0313
baseline-run1-vs-fair-d3s7
meanDelta
-104719.4688
bootstrapLower95
-143478.7719
bootstrapUpper95
-67444.3852
studentTLower95
-143894.5037
pairedSd
187732.0370
detectionFloor
38602.4001
wtl
  1. 20
  2. 0
  3. 44
halves
  1. -105242.6250
  2. -104196.3125
q25Delta
-51042.2500
movesDelta
-28.8438
training
generationsCompleted
150
stoppedOnPlateau
true
plateauChecks
  1. generation
    99
    window
    100
    slopePerGeneration
    238.6249
    standardError
    127.0788
    lowerBound95
    29.5988
    windowMeanFirstHalf
    -116503.0471
    windowMeanSecondHalf
    -99833.0190
    stop
    false
  2. generation
    149
    window
    100
    slopePerGeneration
    37.3143
    standardError
    119.2761
    lowerBound95
    -158.8775
    windowMeanFirstHalf
    -99833.0190
    windowMeanSecondHalf
    -88948.3871
    stop
    true
fiftyGenerationBlockAverages
  1. generations
    0-49
    populationMean
    204747.8623
    best
    242098.4525
    meanMinusFair
    -116503.0471
    bestMinusFair
    -79152.4569
    meanMinusBaseline
    8982.3529
    bestMinusBaseline
    46332.9431
    sigmaAtEnd
    0.0442
  2. generations
    50-99
    populationMean
    228705.8372
    best
    275125.1206
    meanMinusFair
    -99833.0190
    bestMinusFair
    -53413.7356
    meanMinusBaseline
    36076.1941
    bestMinusBaseline
    82495.4775
    sigmaAtEnd
    0.0390
  3. generations
    100-149
    populationMean
    237945.8410
    best
    285994.4269
    meanMinusFair
    -88948.3871
    bestMinusFair
    -40899.8012
    meanMinusBaseline
    45613.4210
    bestMinusBaseline
    93662.0069
    sigmaAtEnd
    0.0345
tenGenerationBlocks
  1. generations
    0-9
    populationMean
    198114.0884
    top4Mean
    221970.0750
    best
    229839.3250
    meanMinusFair
    -123398.6366
    bestMinusFair
    -91673.4000
    meanMinusInit
    49351.0696
    meanMinusBaseline
    -3548.5304
  2. generations
    10-19
    populationMean
    202549.9282
    top4Mean
    231257.9602
    best
    242869.1125
    meanMinusFair
    -138529.6343
    bestMinusFair
    -98210.4500
    meanMinusInit
    52363.4938
    meanMinusBaseline
    16067.6188
  3. generations
    20-29
    populationMean
    203141.2692
    top4Mean
    228807.5289
    best
    237100.1719
    meanMinusFair
    -92208.5808
    bestMinusFair
    -58249.6781
    meanMinusInit
    49689.4161
    meanMinusBaseline
    5352.8380
  4. generations
    30-39
    populationMean
    206259.3706
    top4Mean
    236366.9516
    best
    244948.4188
    meanMinusFair
    -118918.7013
    bestMinusFair
    -80229.6531
    meanMinusInit
    52443.9112
    meanMinusBaseline
    12523.5987
  5. generations
    40-49
    populationMean
    213674.6552
    top4Mean
    243235.7219
    best
    255735.2344
    meanMinusFair
    -109459.6823
    bestMinusFair
    -67399.1031
    meanMinusInit
    58796.8646
    meanMinusBaseline
    14516.2396
  6. generations
    50-59
    populationMean
    225622.1756
    top4Mean
    258711.7734
    best
    269100.2500
    meanMinusFair
    -86547.8275
    bestMinusFair
    -43069.7531
    meanMinusInit
    73783.6818
    meanMinusBaseline
    24289.6631
  7. generations
    60-69
    populationMean
    219346.4406
    top4Mean
    252135.9828
    best
    263647.7344
    meanMinusFair
    -106459.8656
    bestMinusFair
    -62158.5719
    meanMinusInit
    73687.9219
    meanMinusBaseline
    33851.7719
  8. generations
    70-79
    populationMean
    231187.5366
    top4Mean
    265508.5672
    best
    274636.9562
    meanMinusFair
    -85917.9884
    bestMinusFair
    -42468.5687
    meanMinusInit
    80509.0147
    meanMinusBaseline
    31926.8991
  9. generations
    80-89
    populationMean
    233843.3086
    top4Mean
    268375.5172
    best
    281134.5187
    meanMinusFair
    -101265.8602
    bestMinusFair
    -53974.6500
    meanMinusInit
    79471.4273
    meanMinusBaseline
    45444.1617
  10. generations
    90-99
    populationMean
    233529.7248
    top4Mean
    272108.8586
    best
    287106.1437
    meanMinusFair
    -118973.5533
    bestMinusFair
    -65397.1344
    meanMinusInit
    77933.4779
    meanMinusBaseline
    44868.4748
  11. generations
    100-109
    populationMean
    240531.0065
    top4Mean
    275976.7289
    best
    288242.7156
    meanMinusFair
    -78512.1810
    bestMinusFair
    -30800.4719
    meanMinusInit
    88217.0190
    meanMinusBaseline
    45300.6128
  12. generations
    110-119
    populationMean
    237897.3521
    top4Mean
    273658.6984
    best
    286345.5625
    meanMinusFair
    -83419.3511
    bestMinusFair
    -34971.1406
    meanMinusInit
    82886.6739
    meanMinusBaseline
    45329.8771
  13. generations
    120-129
    populationMean
    238849.4075
    top4Mean
    274787.2523
    best
    288351.2813
    meanMinusFair
    -93764.6144
    bestMinusFair
    -44262.7406
    meanMinusInit
    83077.8513
    meanMinusBaseline
    44751.7106
  14. generations
    130-139
    populationMean
    240464.9854
    top4Mean
    275556.8977
    best
    286090.1531
    meanMinusFair
    -91019.8646
    bestMinusFair
    -45394.6969
    meanMinusInit
    91141.3886
    meanMinusBaseline
    54606.0511
  15. generations
    140-149
    populationMean
    231986.4536
    top4Mean
    267803.5086
    best
    280942.4219
    meanMinusFair
    -98025.9245
    bestMinusFair
    -49069.9563
    meanMinusInit
    76973.6505
    meanMinusBaseline
    38078.8536
bestAboveFairGenerations
  1. 59
  2. 66
  3. 73
  4. 97
  5. 100
  6. 109
  7. 115
  8. 125
  9. 137
trainingSignalCheck
rule
population mean fitness > paired fair-d3s7 control in the majority of the final 10 completed generations (theory falsifier 2)
window
  1. 140
  2. 141
  3. 142
  4. 143
  5. 144
  6. 145
  7. 146
  8. 147
  9. 148
  10. 149
generationsMeanAboveFair
0
generationsTop4AboveFair
0
generationsBestAboveFair
0
generationsInitAboveFair
0
meanMarginLast10
-98025.9245
top4MarginLast10
-62208.8695
passed
false
artifactIntegrity
generationArtifacts
150
illegalDecisions
0
incompleteDecisions
0
censoredGames
0
config
experiment
EX-20260903-nnue-evolution-continuation-d3-f8ce9181
population
32
games
32
generations
1,000
elites
4
tournament
3
sigmaRel
0.0500
sigmaDecayTau
400
sigmaFloor
0.0100
seed
0x0e701e59
leaseStart
0xa52e2000
moveCap
2,000
wallSeconds
259,200
resumePopulation
/home/keshav/Developer/drop7-bench/runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/evolve/population-060.bin
resumePopulationSha256
3da5021531898f08254d35cad5486fbbc12092770216bcdaea988d3724422cb8
baseline
/home/keshav/Developer/drop7-bench/artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/evolve/candidate-weights.bin
baselineSha256
edd0d2efd181de43f35d62c4df784cb2c789db3ae296be93a1c6c59082034a9f
plateauWindow
100
plateauCheckEvery
50
plateauMinGenerations
100
selection
finalGeneration
149
finalists
  1. 25
  2. 6
  3. 5
  4. 29
  5. 0
  6. 16
  7. 12
  8. 9
blockStart
0xa52e32c0
selectGames
128
winner
candidate-12
winnerMean
264466.4688
resources
  1. label
    evolve
    command
    1. /home/keshav/Developer/drop7-bench/approaches/lifetime-objective/nnue-evolution/target/release/evolve
    2. --init
    3. /home/keshav/Developer/drop7-bench/artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/pretrain/init.bin
    4. --lease-start
    5. 0xa52e2000
    6. --out
    7. /home/keshav/Developer/drop7-bench/runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/evolve
    8. --population
    9. 32
    10. --games
    11. 32
    12. --generations
    13. 1000
    14. --elites
    15. 4
    16. --tournament
    17. 3
    18. --sigma-rel
    19. 0.05
    20. --seed
    21. 0x0e701e59
    22. --threads
    23. 32
    24. --wall-seconds
    25. 259200
    26. --move-cap
    27. 2000
    28. --experiment-id
    29. EX-20260903-nnue-evolution-continuation-d3-f8ce9181
    30. --resume-population
    31. /home/keshav/Developer/drop7-bench/runs/RUN-20260902T035644Z-c1fd8987/nnue-evolution/evolve/population-060.bin
    32. --baseline
    33. /home/keshav/Developer/drop7-bench/artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/evolve/candidate-weights.bin
    34. --sigma-decay-tau
    35. 400
    36. --sigma-floor
    37. 0.01
    38. --plateau-window
    39. 100
    40. --plateau-check-every
    41. 50
    42. --plateau-min-generations
    43. 100
    startedAt
    2026-09-03T03:34:50Z
    endedAt
    2026-09-03T16:15:39Z
    wallSeconds
    45649.2030
    userSeconds
    1431451.9050
    systemSeconds
    580.6870
    peakRssBytes
    479,240,192
    exitCode
    0
  2. label
    select
    command
    1. /home/keshav/Developer/drop7-bench/approaches/lifetime-objective/nnue-evolution/target/release/evolve
    2. --select
    3. --lease-start
    4. 0xa52e2000
    5. --out
    6. /home/keshav/Developer/drop7-bench/runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/evolve
    7. --generations
    8. 1000
    9. --games
    10. 32
    11. --select-games
    12. 128
    13. --threads
    14. 32
    15. --move-cap
    16. 2000
    startedAt
    2026-09-03T16:15:39Z
    endedAt
    2026-09-03T16:21:00Z
    wallSeconds
    320.7660
    userSeconds
    9947.1370
    systemSeconds
    7.3760
    peakRssBytes
    251,973,632
    exitCode
    0
  3. label
    screen
    command
    1. /home/keshav/Developer/drop7-bench/approaches/lifetime-objective/nnue-evolution/target/release/screen
    2. --candidate
    3. /home/keshav/Developer/drop7-bench/runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/evolve/candidate-weights.bin
    4. --init
    5. /home/keshav/Developer/drop7-bench/artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/pretrain/init.bin
    6. --seeds-start
    7. 0xa52ea000
    8. --games
    9. 64
    10. --threads
    11. 32
    12. --move-cap
    13. 2000
    14. --out
    15. /home/keshav/Developer/drop7-bench/runs/RUN-20260903T032832Z-a76a6cf7/nnue-evolution/screen/heldout.json
    16. --experiment-id
    17. EX-20260903-nnue-evolution-continuation-d3-f8ce9181
    18. --arm
    19. baseline-run1=/home/keshav/Developer/drop7-bench/artifacts/results/EX-20260902-nnue-evolution-d3-v2-49c18bc2/RUN-20260902T035644Z-c1fd8987/evolve/candidate-weights.bin
    startedAt
    2026-09-03T16:21:00Z
    endedAt
    2026-09-03T16:30:44Z
    wallSeconds
    584.0530
    userSeconds
    12716.1680
    systemSeconds
    7.1320
    peakRssBytes
    1,793,974,272
    exitCode
    0
firstRun
resultId
RS-20260903T025751Z-6577b33e
runId
RUN-20260902T035644Z-c1fd8987
screenCandidateVsFair
-106,964
Limitations
  • Single 64-game held-out screen: paired detection floors of about 43,000 points for the primary contrast and 27,000 for the continuation contrast; the primary result sits far outside its floor, the continuation contrast just outside its own, and its first-half estimate is negative, so 'improved out of sample' is established at the whole-block level only.
  • The plateau rule and the annealed mutation size are confounded: the step size had fallen from 0.050 to 0.034 when the rule fired, so the design cannot say whether the leaf class ran out of improvement or the search did. A constant-sigma continuation from the same population would separate them.
  • The plateau rule tests a fitted slope over 100 generations with a one-sided 95% lower bound; at the observed scatter it can miss real improvements below about 220 points per generation. The observed point estimate over generations 50-149 was +37 per generation.
  • The per-game artifact holds five arms x 64 games (320 rows); each contrast pairs two arms by seed.
  • The starting population, the warm-start control and the baseline control are products of the first run's training data; nothing in this run re-read that lease.
  • The owner's commit e2b0d19 (wall-budget pinning and progress recovery on resume) landed on the branch mid-run; the run never resumed, the driver shells held the original script by an unlinked inode, and every stage ran the binaries built from commit e4fd018 as the run record states.
  • The fair-d4s7 arm is diagnostic only. The first run's candidate reproduced its earlier screen result on these fresh seeds (-104,719 against -106,964), which is the closest thing to a replication the program has for that number.

Recorded against Continued whole-game evolution of the depth-5-distilled NNUE leaf from the first run's generation-60 population, with annealed mutation and a preregistered plateau stop (up to 1,000 generations).

Agent contextHow to extend this record

To add a reader-facing explanation, write web/content/research/RS-20260903T163321Z-733076b5.mdx; it renders above this record on the next request. The recorded metrics, gate checks and limitations are in the technical record above.

Record file: research/results/RS-20260903T163321Z-733076b5.json, validated against research/schemas/result-v1.schema.json.

Run ids
  • RUN-20260903T032832Z-a76a6cf7
Contribution ids
  • CT-20260902T042620Z-3b4bf7a8
Per-game artifact
artifacts/results/EX-20260903-nnue-evolution-continuation-d3-f8ce9181/RUN-20260903T032832Z-a76a6cf7/screen/heldout.json (sha256 ffc73a1cf78d74818fb58b3ef9e72bd62eea436176cce02209b634f6ba4434a7, 320 records)
Artifact manifest
artifacts/results/EX-20260903-nnue-evolution-continuation-d3-f8ce9181/RUN-20260903T032832Z-a76a6cf7/manifest.json
Machine profiles
  • research/system-profiles/MACH-20260903T032430Z-4e906a6e.json