Turning the reference search's constants into dials
Makes death penalty, search depth, and chance samples configurable to test which settings improve survival.
When some of what happens next is your choice and some is luck, look ahead by taking the best of your options and the average of luck's.
On this page
- Depth x chance-exactness factorial: the fifth ply at five and at seven strata, with an end-to-end reproduction control (2026-08-21)
- The two arms finding-09 left unfinished: reveal sampling on top of the fourth ply, and the depth-3 ladder at full joint coverage (2026-08-21)
- The two arms finding-09 left unfinished: reveal sampling on top of the fourth ply, and the depth-3 ladder at full joint coverage (2026-08-21)
- Depth x chance-exactness factorial: the fifth ply at five and at seven strata, with an end-to-end reproduction control (2026-08-21)
The strongest policy in this repository has a handful of numbers baked into it that nobody had ever varied. This approach makes three of them adjustable without changing anything else, so they can be swept and measured instead of assumed.
Two sweeps have been run through it. The first: the one the approach is named after — is a valid negative: the death penalty is at its stop and is not a lever. The second is a replicated positive: the number of chance samples was too small to represent the game's own randomness, and fixing it was worth about a third more score.
The intuition
At the bottom of its look-ahead, the search puts a number on every board it can reach. A board where the game is over gets a fixed, very large negative number — the terminal utility, set to −1,000,000. Every other board gets a score from a hand-written leaf evaluator, and in practice those come out between roughly 200 and 11,000.
So the death penalty is about a hundred times larger than the entire spread of everything else. The decision rule is effectively: first avoid dying within four moves; only then prefer the nicer-looking board. Given that surviving is where 94% of the points come from (score decomposition), that constant is the single number pointed most directly at the objective — and it had never been swept. Perhaps the search was too timid, or not timid enough.
The second dial is subtler. At a chance node the search has to average over the next disc, which is equally likely to be any of seven values. The frozen reference averages over five representative samples. Five buckets cannot cover seven equally likely outcomes, so some disc values get no weight at all: the same ones, every time, at every node. That is not noise that cancels out; it is a fixed bias built into every estimate the search makes.
How it works, step by step
- Input. Exactly what the reference reads: the visible board, the visible next disc, and how many drops remain before the next rise. Nothing else.
- Reuse, not reimplementation. The leaf evaluator, the chance stratification, the board canonicalisation, the cache keys, the column ordering, the work accounting and the legal fallback are the unmodified frozen code, included as a library. Only the depth-limited driver is rewritten, so that three constants become command-line options.
- A parity gate before any game is played. At its default settings the parameterised driver must pick the identical column as the frozen reference on every move. Recorded result: 50 moves compared, 0 mismatches.
- Paired sweeps. Each setting plays the same ordered list of game seeds as the reference, so scores can be compared game by game rather than as two separate averages.
- Output. A column, and per-game records through the family's shared harness, including the survival-flow rates that the whole family watches.
What happened: the death penalty
Nothing at all, in the direction that mattered. Making death fifty times more expensive produced byte-identical games: same scores, same lifetimes, 0 wins, 64 ties, 0 losses. The search is already as risk-averse as the constant can make it. Moving the other way, towards recklessness, only cost, and it cost in an instructive shape: at a penalty of −100,000 the policy cleared slightly more discs per move and still died sooner. Throughput is not the objective; throughput sustained without dying is.
That closes a cheap hypothesis. The gap between a 94-move average and the roughly 294 moves a million-point average would need cannot be bought by re-pricing death.
Technical recordThe technical record — terminal utility
Source: finding-04-terminal-utility-saturated,
status exploratory, evidence tier development, recorded as a valid
negative result. Six arms, 64 paired games each, seeds
0xa51d0000–0xa51d003f on the exploratory development lease SEEDLEASE-A51D,
2,000-move cap, common seeds across arms.
| Terminal utility | Mean score | Mean moves | Clears/move | Paired delta vs reference | W–T–L | 95% lower bound |
|---|---|---|---|---|---|---|
| −50,000,000 | 321,992 | 94.06 | 1.973 | 0 | 0–64–0 | 0 |
| −10,000,000 | 321,992 | 94.06 | 1.973 | 0 | 0–64–0 | 0 |
| −3,000,000 | 321,992 | 94.06 | 1.973 | 0 | 0–64–0 | 0 |
| −1,000,000 (frozen) | 321,992 | 94.06 | 1.973 | 0 | — | — |
| −300,000 | 321,652 | 93.98 | 1.973 | −340 | 1–62–1 | −1,039 |
| −100,000 | 320,161 | 92.92 | 1.987 | −1,831 | 4–29–31 | −4,563 |
The identical-game result at magnitudes at or beyond 1,000,000 is exact and needs no statistics. The −300,000 and −100,000 deltas are single-cohort estimates. Only the terminal utility was varied; the leaf weights are frozen, and a jointly re-tuned leaf could in principle move the saturation point, which is what the leaf reweight work went on to test, using this same parameterised search as its base.
What happened: the number of chance samples
Averaging over seven disc values instead of five was worth about a third more score, and it replicated on a second, previously unread set of games. On the confirmation cohort the seven-sample search scored 398,498 against 297,327, and lasted 114.66 moves against 87.16. Median, lower quartile and minimum all improved, so this is not one lucky long game; and the survival-flow rates moved in the direction the mechanism predicts, closing roughly a fifth of the gap to the clears and reveals per move that indefinite survival requires.
The more interesting part is that this is an interaction, not a "more samples is better" result. With the biased five-sample estimator, adding a fourth move of look-ahead bought nothing measurable. With the exact seven-sample estimator, the fourth move was worth about 86,000 points. Depth and estimator quality are complements. That offers a mechanism for one of the repository's most repeated conclusions: that deeper search is not automatically stronger because every historical depth experiment sat on top of the same biased estimator.
It is also not free: the exact estimator costs 3.82 times the work per move at the same depth, and a variant one move shallower with the exact estimator ties the frozen reference at an eighth of the work.
Technical recordThe technical record — chance samples
Source: finding-05-chance-strata,
status exploratory, evidence tier development, replicated across two
independent cohorts for the headline arm. Positive result. Same parity gate
(50 moves, 0 mismatches).
| Comparison | Delta score | 95% lower bound | Delta moves | W–T–L | Verdict |
|---|---|---|---|---|---|
| depth 4: 7 samples − 5 samples | +101,171 | +47,457 | +27.50 | 41–0–23 | significant |
| depth 3: 7 samples − 5 samples | +7,276 | −45,961 | +2.42 | 34–0–30 | not significant |
| 7 samples: depth 4 − depth 3 | +86,172 | +26,468 | +22.39 | 40–0–24 | significant |
| 5 samples: depth 3 − depth 4 | +7,723 | −42,743 | +2.69 | 35–0–29 | not significant |
Cohorts: a first cohort on 0xa51d0000–0xa51d003f, whose seeds had already
influenced other decisions and therefore cannot serve as confirmation
(+71,138, lower bound +5,826); and a confirmation cohort on previously unread
seeds 0xa51d1000–0xa51d103f (+101,171, lower bound +47,457). 64 paired
games each.
Confirmation cohort detail: mean score 297,327 versus 398,498; mean moves 87.16 versus 114.66; clears per move 1.9489 versus 2.0571; reveals per move 1.0697 versus 1.1549; 0 censored games and 0 score-identity violations in both arms; games at or above one million: 0 versus 2.
Cost, at fixed depth 4: work per move 1,296,034 versus 4,956,614, a ratio of 3.824 against a predicted 3.79. Depth 3 with seven samples costs 156,834 per move and ties depth 4 with five samples (+14,999, 95% lower bound −31,029, 32–0–32).
Limitations as recorded. 64 paired games per arm; only the headline arm is replicated, and every other row of the interaction table rests on a single cohort. Seven samples makes the next-disc expectation exact, but gray-disc reveals are still sampled, so the reveal expectation is not exact — whether reveal sampling is now the binding bias was tested separately in the reveal sampling work. No fixed-time comparison is reported. And the headline mean of 398,498 is far below the 1,050,000 the frozen protocol requires before a candidate may even be frozen: no protected or final seed was opened or is justified by this result.
What this taught us, and what is still open
Two constants, two opposite outcomes, and the difference between them is the lesson. The risk constant was saturated: it had no reachable setting that changed a decision, so no amount of tuning could have helped. The sample count was biased: it was systematically wrong in a way that no amount of extra depth could average away. Sweeping a constant is cheap; assuming which way it will go is not.
The negative result also narrowed the search for the remaining headroom. The search already minimises death within its horizon as its first priority, and its horizon is four moves against a rise every five. Whatever is missing has to live in the long-horizon content of the leaf evaluator, not in the search's appetite for risk.
Still open, and cautionary: whether the depth gradient keeps climbing once the estimator is exact. A later, still in-progress test in the fast engine approach reports the fifth move of look-ahead reversing rather than continuing the gain, on interim partial cohorts. Read that record's own caveats before quoting it.
RecordsTheories, experiments and results that reference this directory
Claim: With seven chance strata (one stratum per disc value, so every individual chance marginal is exact), adding a fifth ply of look-ahead to the fair expectimax search raises mean score on the shared 64-game development cohort by a paired margin whose one-sided 95% whole-game bootstrap lower bound is above zero, as the third-to-fourth ply does (+86,172 with a lower bound of +26,468, finding-05). With five strata, where the third-to-fourth ply gradient is absent (-7,723, not significant), no fourth-to-fifth ply gradient is expected either. Depth and chance-estimator exactness are therefore complements, and the fourth ply is not a special stopping point.
This theory is currently mixed at the public-development (a cohort for deciding what to try next, not confirmation) level.
Claim: Factoring the fair search's chance node into independent next-disc and reveal draws pays at every depth, so applying it on top of the fourth ply raises mean score over the same four-ply search with a single reveal sample by a paired margin whose one-sided 95% whole-game bootstrap lower bound is above zero. The depth-3 gain finding-09 measured (+64,116 with a lower bound of +7,475 from one to six reveal samples) is therefore an increment that adds to depth rather than an alternative way of buying the same thing.
This theory is currently not-supported-as-tested at the public-development (a cohort for deciding what to try next, not confirmation) level.
It compares fast-engine-parameterized-fair-search, depth 5 (arms d5s7 and d5s5) against the recorded depth-4 arms on the same seeds (d4s7 = runs/RUN-A51D-s7confirm/fresh-s7.json, d4s5 = runs/RUN-A51D-s7confirm/fresh-s5.json), plus a same-cohort fast-engine reproduction of d4s7 as the control at the STANDARD (a 64-game paired development cohort) level, using previously-evaluated-development data.
partial run outcome: fail The run was partial and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.
partial run outcome: inconclusive The run was partial and the outcome was inconclusive (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.
It compares factored-chance-fair-search, arms d4 N=7 M=2 and d3 N=7 M=12 against the recorded single-reveal-sample arms on the same seeds (d4 N=7 M=1 = runs/RUN-A51D-s7confirm/fresh-s7.json; d4 N=5 M=1 = runs/RUN-A51D-s7confirm/fresh-s5.json) and finding-09's completed depth-3 ladder (M=1, 3, 6) at the STANDARD (a 64-game paired development cohort) level, using previously-evaluated-development data.
partial run outcome: fail The run was partial and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.
valid run outcome: fail The run was valid and the outcome was fail (public-development (a cohort for deciding what to try next, not confirmation)). Read the result.
The run was partial; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 6 preregistered checks, 3 passed and 2 failed.
The fifth ply buys nothing at either chance resolution, and the earlier interim reading that it was actively harmful is withdrawn. Complete leg, 64 of 64 games: depth 5 at five strata scores 288,704 against depth 4 at five strata's 297,327, a paired -8,624 with a one-sided 95% whole-game bootstrap lower bound of -55,134 and W-T-L 33-0-31, for 23.29x the logical work per move. That is a wash, not a reversal. Partial leg, 16 of 64 games and still running: depth 5 at seven strata is -1,581 against the depth-4 seven-stratum control (95% lower bound -173,154, W-T-L 7-0-9) at 34.32x the work, and +16,622 against depth 3 at seven strata (95% lower bound -130,027, W-T-L 8-0-8) at 1,084.78x the work. At a fixed stratum count, depth 3 -> 4 -> 5 does not separate. READ THIS BEFORE QUOTING THE MEANS: the eye-catching gap between the 398,498 of d4s7 and the 288,704 of d5s5 is a chance-samples effect and not a depth effect, because those two arms differ in both factors; the correct paired depth contrasts at fixed chance resolution are d5s5 - d4s5 = -8,624 and d5s7 - d4s7 = -1,581, both indistinguishable from zero, and the correct paired stratum contrast at fixed depth is finding-05's d4s7 - d4s5 = +101,171. The interim slice reported in finding-15 section 2.2 (-268,611 over 8 paired games) was completion-order biased against depth 5 exactly as that section warned; at 16 games the bias is gone and the delta is -1,581. The engine control is clean and is the other retained result here: the fast engine's depth-4 seven-stratum arm reproduces the recorded unoptimised arm over 64 paired games x 11 fields with 0 mismatches, and the depth-5 five-stratum arm reproduces the recorded 32-game unoptimised arm over 32 paired games x 11 fields with 0 mismatches across two binaries and two different cache capacities. Every arm audited 0 incomplete decisions at its requested depth, 0 censored games and 0 score-decomposition identity failures. Flow rates fall with depth at five strata (1.9387 clears and 1.0651 reveals per move against depth 4's 1.9489 and 1.0697, and against the 2.400 and 1.400 indefinite survival needs), so nothing here moves toward the target.
Technical recordLimitations recorded with the result
- PARTIAL ARM: the depth-5 seven-stratum arm holds 16 of 64 games and was still executing (process 127323) when this result was written. Every seven-stratum number here is over those 16 paired games and none of them can decide the primary gate. The frozen snapshot assessed is runs/RUN-20260821T060358Z-895d0a79/d5s7-partial-16games.json; the live artifact will be rewritten to 32 games and beyond and will no longer match this record's hash.
- The 16 finished games are the cohort's first 16 seeds (0xa51d1000-0xa51d100f) because the runner completes whole 16-game chunks, so they are a fixed block rather than a random subset; they are paired seed-for-seed against comparators recorded on exactly those seeds, which is what makes the paired delta fair even though the 16-game mean is not an estimate of the arm's 64-game mean.
- 64 paired games with a score standard deviation of 38-64% of the mean; at this sample size a true depth effect smaller than roughly 55,000 points cannot be resolved, so 'no gradient' means 'no gradient this cohort can see', not 'exactly zero'.
- The cohort 0xa51d1000-0xa51d103f was opened by finding-05 and is permanently development data. This is a diagnostic comparison at STANDARD tier; it is not a freeze gate and can never become confirmation evidence.
- The depth-5 arms declare a 200,000-entry transposition cache and the depth-4 comparators 60,000. Capacity provably cannot change play (verified three ways in finding-15 section 3, including whole-game agreement across two binaries) but work per move is not comparable across capacities and each figure must be read with its capacity.
- The experiment record was written after the runs, by a different agent than the one that executed them. The comparison rule predates the arms and is implemented in analyze.py, but this is retroactive registration and not a preregistration; the amendment on the experiment record says so.
- Wall-clock figures are not timing-grade: the machine carried a one-minute load average of 12-63 from other agents' jobs throughout. Scores, moves and logical work are deterministic and unaffected.
- This rejects the exact configurations tested - depth 5 at five and seven strata, this leaf, this terminal utility, these work bounds. It does not show that no five-ply search can help. finding-15 section 5 names two mechanisms that would predict exactly this outcome (the terminal utility has no death-depth shaping, and the leaf is an uncalibrated potential); neither was tested here.
- Cost model correction, recorded because it changes what is affordable rather than what is true: the 78-wall-hour projection that once cancelled this experiment used worst-case iterative-deepening work. Measured work at depth 5 with seven strata is about 10x below that bound, and the arms ran in hours.
The run was partial; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 5 preregistered checks, 3 passed and 1 failed.
Chance-node decorrelation and search depth do not compound; they substitute. The primary arm is complete at 64 of 64 games: depth 4 with seven disc samples and two reveal samples scores 356,548 against the 398,498 of the same search with one reveal sample, a paired -41,950 with a one-sided 95% whole-game bootstrap lower bound of -100,137 and W-T-L 28-0-36, for 4.07x the logical work per move. The gate asked for a lower bound above zero and got a negative point estimate, so the compounding theory is rejected as tested. STATE THE DOSE WHEN QUOTING THIS: two reveal samples raises joint (disc, reveal) coverage from 14.3% to 28.6%, which is a smaller increment than the six samples (85.7%) that first cleared noise at depth 3; at depth 3 the three-sample dose (42.9%) was also not significant (+24,980, lower bound -23,451). This result therefore rejects a doubling of reveal samples on top of the fourth ply, and does not establish that a wide reveal estimator at depth 4 would fail - that arm was never affordable. What the arm does establish is that the depth-4 search is not starved for the thing the extra samples supply. The striking positive finding is an equivalence at near-equal work: depth 3 with six reveal samples costs 4,244,020 work per move and scores 376,442, while depth 4 with one reveal sample costs 4,956,614 and scores 398,498, and the paired delta between them is -22,056 with a lower bound of -89,867 and W-T-L 30-0-34 - two different ways of spending the same budget landing in the same place, which is the same exchangeability the depth factorial shows from the other side. The new arm is not worthless: against the frozen five-stratum depth-4 reference it is +59,221 with a lower bound of +9,134 and W-T-L 37-0-27, so the gain comes from the seven disc samples, not from the reveal samples. Second arm, partial at 32 of 64 games and still running: depth 3 with twelve reveal samples takes joint coverage to 100% and scores -4,495 against the six-sample arm on the 32 shared seeds (lower bound -91,425, W-T-L 15-0-17) and +31,413 against the one-sample arm (lower bound -70,729), so the ladder that ordered with coverage from M=1 to M=6 stops ordering at M=12. Every arm audited 0 decisions below target depth, 0 work-limit events, 0 censored games and 0 score-decomposition identity failures, and the chunk-pooling used to survive interruption was verified to reproduce a single 64-game run field-for-field with identical summed logical work.
Technical recordLimitations recorded with the result
- PARTIAL ARM: the depth-3 twelve-reveal-sample arm holds 32 of 64 games (chunks 0 and 1 of 4) and was still executing when this result was written. Every M=12 number here is over those 32 paired seeds, none of them decides its gate, and the arm's 32-game mean is not an estimate of its 64-game mean. The frozen snapshot assessed is runs/RUN-20260821T143541Z-4c4370b6/d3-n7-m12-partial-32games.json; re-pooling the live artifact after chunk 2 will change its bytes.
- DOSE, NOT AXIS: the depth-4 arm tested two reveal samples (28.6% joint coverage), which is below the depth-3 dose that first cleared noise (six samples, 85.7%). The negative result is about that dose at that depth. A wide depth-4 reveal estimator was not run and remains unmeasured; at the observed 20.2M work per move for M=2, an M=6 depth-4 arm would be roughly another 3x on top and was not affordable on a shared machine.
- The depth-4 M=2 delta is negative but not significantly negative: the same estimator's one-sided 95% upper bound is +17,541. The honest reading is 'buys nothing measurable for 4.07x the work', not 'harms'.
- The comparator arms for the M ladder (d3 M=1, M=3, M=6) and the two depth-4 single-sample arms come from finding-09 and from the earlier chance-strata study; they were re-read here from their retained artifacts but have no machine-readable run record of their own, so this result's run records cover only the two new arms.
- 64 paired games with a score standard deviation of 55-62% of the mean. Effects smaller than roughly 60,000 points are invisible at this sample size.
- The cohort 0xa51d1000-0xa51d103f was opened by finding-05 and read again by finding-09; it is permanently development data. STANDARD tier, diagnostic only, never a freeze gate and never confirmation evidence.
- Each arm ran as four sequential 16-game chunks so it could survive interruption. Pooling was verified to reproduce a single run exactly, but chunking does change thread scheduling and wall time, and chunk 1 of the depth-4 arm ran at 8 threads rather than 12 after a load back-off.
- Work per move is not comparable across arms with different declared cache capacities. The new arms auto-size their cache from their branching factor (960,695 and 346,921 entries) while the recorded comparators declare 60,000; every work figure must be read with its capacity.
- The experiment record was written after the runs, by a different agent than the one that executed them. The two arms and their launch protocol were fixed in prose in finding-09 and in run-arms.sh before either produced a game, but this is retroactive registration and the amendment on the experiment record says so.
- All artifacts for this family live under runs/, which is gitignored, so the evidence is not committed with the record. The frozen snapshots and the content manifest under the two run directories are the durable reference.
- This rejects compounding at the tested dose only. It says nothing about the reveal-by-reveal correlation, which finding-09 identified as untouched by this knob and which no arm here attacks.
The run was valid; the outcome was fail, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 5 preregistered checks, 3 passed and 2 failed.
SUPERSEDES RS-20260821T181918Z-ea7076a3, which assessed the same experiment while its second arm held 32 of 64 games. Both arms are now complete at 64 of 64 and the run validity rises from partial to valid. The old record remains as committed history and is not edited; this record carries the completed figures and everything below replaces the partial ones. Nothing in arm 1 changed and the direction of arm 2 did not change; only its magnitude did, by about 6x. ARM 1, unchanged and complete: depth 4 with seven disc samples and two reveal samples scores 356,548 against the 398,498 of the same search with one reveal sample - a paired -41,950 with a one-sided 95% bootstrap interval of (-100,137, +17,541) and W-T-L 28-0-36, for 4.07x the logical work per move. The gate required a lower bound above zero, so the compounding theory is rejected as tested. The sign is stable across both cohort halves (-11,082 on seeds 0xa51d1000-0xa51d101f and -72,818 on 0xa51d1020-0xa51d103f), independently confirmed by the agent that completed the runs. The +17,541 ceiling is the load-bearing number: had the axes compounded even at a quarter of the +64,116 the same knob is worth at depth 3, the data would have had to show it. The axes substitute in the strong sense. ARM 2, now complete: depth 3 with twelve reveal samples - the configuration at which (disc, reveal) joint coverage reaches 100% - scores 349,345, not the 356,890 its first 32 games suggested. Against the six-sample arm it is -27,097 with an interval of (-83,807, +31,209) and W-T-L 28-0-36; against the one-sample arm +37,019 (-25,076, +102,426), 30-0-34. THE REVEAL AXIS SATURATES AT M ABOUT 6 AND DOES NOT IMPROVE TO FULL COVERAGE. Read this as saturation, not as a demonstrated regression: the M=6 to M=12 step does not clear zero in either direction, so the data are consistent with the curve being flat from M=6 onward and are not consistent with it continuing to climb at the rate M=1 to M=6 showed. Score, moves, numbered clears and covered reveals all peak at M=6 (85.7% coverage) and fall back at M=12 (100%); only mean occupancy improves monotonically across all four points (23.88, 23.81, 23.49, 23.39). This weakens two SUPPORTING arguments in finding-09 without touching its headline, which rests on its own paired tests and stands: the dose-response is no longer monotone once the fourth point is added, with four of five tracked quantities reversing, and section 2's attribution of strength to joint coverage fails exactly at the 100% endpoint that attribution predicts should be best. finding-09 section 5 should now be read as 'M=6 is a local optimum', not 'more M is better'. Validity controls all pass: 0 decisions below target depth, 0 work-limit events, 0 censored games and 0 score-decomposition identity failures in both arms, and the chunk-and-pool procedure on which every cohort in this family depends was verified end to end by running depth 3 (N=5, M=1) as four 16-game chunks at 1 thread and reproducing the existing 64-game 12-thread artifact field-for-field - 0 mismatches, identical summed logical work of 312,966,881, only per-game wallSeconds differing, which also re-confirms worker-count independence.
Technical recordLimitations recorded with the result
- SUPERSESSION: this record replaces RS-20260821T181918Z-ea7076a3 for the same experiment. That record was written and committed while arm 2 held 32 of 64 games and is left byte-unchanged, because this repository has no precedent for annotating a committed result in place; the supersession relationship is carried here and by the theory record's evidenceRefs list, which is how the afterstate line's superseded results are linked. Quote this record, not the partial one.
- DOSE, NOT AXIS, for arm 1: the depth-4 arm tested two reveal samples (28.6% joint coverage), below the six samples (85.7%) that first cleared noise at depth 3. The negative is about that dose at that depth; a wide depth-4 reveal estimator was never affordable and remains unmeasured. What the +17,541 ceiling does bound is how much compounding could have been hiding at this dose.
- SATURATION, NOT REGRESSION, for arm 2: the M=6 to M=12 step does not clear zero in either direction. The defensible statement is that the reveal axis is exhausted by M about 6 and that filling the last of the joint grid buys at most +31k and plausibly nothing. It is not evidence that more reveal samples hurt.
- No adjacent step in the depth-3 M ladder is individually significant, so the whole M >= 3 region is consistent with one plateau. The M=1 to M=6 endpoint test (+64,116 [+7,475]) is what carries finding-09's positive result, and it is unaffected by this record.
- The three readings finding-09's Continuation section 16 lists for the turnover - redundant last atoms, variance reduction rather than coverage per se, or 64 games being unable to resolve steps this size - are not separated by this experiment.
- 64 paired games with a score standard deviation of 55-73% of the mean; the M=12 arm's is 254,059 on a mean of 349,345. Effects below roughly 60,000 points are invisible at this sample size.
- The cohort 0xa51d1000-0xa51d103f was opened by finding-05 and read again by finding-09; it is permanently development data. STANDARD tier, diagnostic only, never a freeze gate and never confirmation evidence. A fresh-block replication is owed before anything here is promoted.
- Work per move is not comparable across arms with different declared cache capacities: the new arms auto-size from their branching factor (960,695 and 346,921 entries) while the recorded comparators declare 60,000.
- Not timing-grade: shared machine throughout, and chunk 1 of the depth-4 arm ran at 8 threads rather than 12 after a load back-off. Scores, moves and logical work are deterministic and unaffected, as the 1-thread-versus-12-thread pooling control demonstrates directly.
- The experiment record was written after the runs. The arms and their launch protocol were fixed in prose in finding-09 and in run-arms.sh before either produced a game, but this is retroactive registration and the experiment record's amendment says so.
- All artifacts live under runs/, which is gitignored, so the evidence is not committed with the record; the content manifest and the retained analysis output under the run directories are the durable reference.
- Nothing here moves the ceiling. The best arm on this cohort remains 398,498 against the 1,050,000 the frozen qualification protocol requires, and every game ended.
The run was partial; the outcome was inconclusive, at the public-development (a cohort for deciding what to try next, not confirmation) level. Of 6 preregistered checks, 3 passed and 2 failed.
SUPERSEDES RS-20260821T181917Z-9a34ba02, which assessed this experiment when the depth-5 seven-stratum arm held 16 games. The arm was stopped by the repository owner's decision at the 32-game chunk boundary and will not be resumed, so its analysis is now FINAL even though the cohort is partial: 32 of 64 planned games, every one of them a whole game, 0 censored, 0 score-decomposition identity failures, 0 incomplete decisions, minimum completed depth 5. The old record remains committed history and is not edited. THE HEADLINE IS A CORRECTION, NOT AN UPDATE. The previous record read the fifth ply as 'does not separate'. That reading was a NON-MEASUREMENT REPORTED AS A NULL. Doubling the sample from 16 to 32 games moved the depth-5-minus-depth-4 seven-stratum contrast from -1,581 to +23,367 and its median from -39,660 to +18,820 - THE SIGN FLIPPED - which is what a quantity being estimated far below its detection floor looks like. By chunk the paired mean is -1,581 on the first 16 seeds and +48,315 on the second 16. Do NOT replace the old reading with 'depth 5 helps': +23,367 is equally unsupported. The one-sided 95% bootstrap lower bound is -83,046 and the contrast's detection floor at n=32 is 107,988, so the estimate sits at 22% of the smallest effect this cohort could have resolved. The correct statement is that THE FOURTH-TO-FIFTH PLY CONTRAST AT SEVEN STRATA WAS NEVER MEASURED, in either record. THE POWER ANALYSIS IS THE MOST USEFUL THING THIS EXPERIMENT PRODUCED. Detection floor, taken as 1.645 x sd / sqrt(n), the smallest true effect whose one-sided 95% bound would clear zero: d4s7-d4s5 +101,171 against a floor of 55,192 (n=64); d4s7-d3s7 +86,172 against 61,457 (n=64); d5s5-d4s5 -8,624 against 47,052 (n=64); d5s7-d4s7 +23,367 against 107,988 (n=32). EVERY SIGNIFICANT RESULT IN THIS FACTORIAL IS ABOVE ITS FLOOR AND EVERY NULL IS BELOW IT - the factorial separated the contrasts it had the power to separate and nothing else. Resolving the observed +23,367 needs about 684 paired games; finishing to the planned 64 would have left a standard error near 46,400 against a 23,367 estimate, still a non-measurement. That is the justification for the stop: the contrast is not answerable at any affordable cohort size, so the marginal machine-day buys no information. The variance is structural, not fixable by tidier running: the five largest single-seed paired deltas are -1,002,862, +958,985, -678,455, +592,546 and -577,069, so individual games swing by more than twice the cohort mean. WHAT IS ACTUALLY MEASURED HERE, and it is the same lesson from the other side: at depth 5, going from five to seven strata is worth +123,613 with a lower bound of +32,575, W-T-L 19-0-13 - SIGNIFICANT, and comfortably above its 95,207 floor - for 5.85x the work. The chance-exactness axis pays at depth 5 exactly as it pays at depth 4 (+101,171 [+47,447] there). The previous record's warning therefore survives and is strengthened: the eye-catching gap between d4s7's 398,498 and d5s5's 288,704 is a CHANCE-SAMPLES effect, not a depth effect, and both stratum contrasts are now significant while no depth contrast is. The engine controls are unchanged and clean: the fast engine's depth-4 arm reproduces the recorded unoptimised arm over 704 field comparisons with 0 mismatches, and the depth-5 five-stratum arm reproduces its recorded 32-game predecessor over 352 comparisons with 0 mismatches across two binaries and two cache capacities.
Technical recordLimitations recorded with the result
- SUPERSESSION: this record replaces RS-20260821T181917Z-9a34ba02 for the same experiment. That record was written and committed when the depth-5 seven-stratum arm held 16 games, and its central depth claim reversed sign when the sample doubled. It is left byte-unchanged, because this repository has no precedent for annotating a committed result in place; the relationship is carried here and by the theory record's evidenceRefs. Quote this record, not its predecessor.
- PARTIAL BUT FINAL: 32 of 64 planned games. The run validity stays partial because the planned cohort was not completed; the analysis is nevertheless final, because the stop was a decision and the arm will not be resumed. These are different things and the record keeps them apart deliberately.
- THE PRIMARY CONTRAST IS UNMEASURED, not null. Neither -1,581 at n=16 nor +23,367 at n=32 is evidence about the fifth ply at seven strata. Anyone quoting either number as a finding is quoting noise.
- The 32 games are the cohort's first two 16-seed blocks, a fixed prefix rather than a random subset, because the runner completes whole chunks. They are paired seed-for-seed against comparators recorded on exactly those seeds, which is what keeps the paired delta fair; the arm's 32-game mean is not an estimate of a 64-game mean.
- The detection floors are a normal approximation (1.645*sd/sqrt(n)) applied to heavy-tailed paired deltas with sample skewness between -0.30 and +0.92. They are planning quantities, accurate to roughly the 1-5% by which they differ from the percentile bootstrap the tooling reports, not exact power guarantees.
- The 684-game and 13-wall-day figures assume the observed effect size is the true one and that throughput matches this run's 1,647 s per game at 14 threads. Throughput varied 2.3x between the two chunks under other agents' load, so the wall estimate spans roughly 8-18 days. If the true effect is smaller than +23,367 the required cohort grows quadratically.
- The one significant new result, d5s7 - d5s5 at +123,613 [+32,575], rests on 32 games rather than 64 and on an arm that was stopped; it is a stratum contrast at fixed depth 5 and should be replicated on a fresh block before it is leaned on.
- The cohort 0xa51d1000-0xa51d103f was opened by finding-05 and is permanently development data. STANDARD tier, diagnostic only, never a freeze gate and never confirmation evidence.
- The depth-5 arms declare a 200,000-entry transposition cache and the depth-4 comparators 60,000. Capacity provably cannot change play but work per move is not comparable across capacities.
- The experiment record was written after the runs, by a different agent than the one that executed them; its amendment says so. The comparison rule predates the arms.
- This still rejects nothing about five-ply search in general - only these depths, this leaf, this terminal utility, these work bounds. The two mechanisms named in finding-15 section 5 remain untested, and a flat-and-unmeasurable depth axis is consistent with both.
- Nothing here moves the ceiling. The best arm on this cohort is 411,874 on 32 games against the 1,050,000 the frozen qualification protocol requires, and every game ended.
Agent contextSource files, operational notes and how to reproduce
Directory: approaches/lifetime-objective/risk-calibration