Pruned fair search against full-width fair d4s7 on the previously evaluated 256-game pilot cohort
On the previously evaluated pilot cohort 0xa5277000-0xa52770ff, the pruned-d4s7 configuration selected by EX-20260902-oneply-q-fit-and-pruned-search-panel-46c75cdf (lowest held-out normalised regret among the preregistered depth-4 configurations with work ratio at most 0.35) plays complete games whose paired score and lifetime are not measurably below full-width fair d4s7 at a fraction of the logical work; and, if EX-4's depth-5 point estimate was favourable, the selected pruned-d5s7 configuration (lowest regret under exact d5 among the preregistered depth-5 configurations with work at most 2.0 times d4s7) is compared to full-width d4s7 on the first 128 seeds with its detection floor stated.
On this page
- Created
- Updated
No explanation has been written for this record yet.
Technical recordThe registered protocol
- Hypothesis
- On the previously evaluated pilot cohort 0xa5277000-0xa52770ff, the pruned-d4s7 configuration selected by EX-20260902-oneply-q-fit-and-pruned-search-panel-46c75cdf (lowest held-out normalised regret among the preregistered depth-4 configurations with work ratio at most 0.35) plays complete games whose paired score and lifetime are not measurably below full-width fair d4s7 at a fraction of the logical work; and, if EX-4's depth-5 point estimate was favourable, the selected pruned-d5s7 configuration (lowest regret under exact d5 among the preregistered depth-5 configurations with work at most 2.0 times d4s7) is compared to full-width d4s7 on the first 128 seeds with its detection floor stated.
- Arms
Arm Name Entry point Manifest Candidate pruned fair expectimax (drop7-oneply-q evaluate --arm pruned:...) approaches/value-policy-learning/oneply-q-prune/rust/src/bin/evaluate.rs– Comparator full-width fair d4s7 (the same FairSearch policy as RUN-20260902T081659Z-d2aa6375's fair-d4s7 arm), 256 games on the same seeds approaches/value-policy-learning/klein-friedmann-linear-q/rust/src/policy.rs– - Classification
- algorithmic
- Information boundary
- public-policy
- Benchmark tier
- PILOT
- Lifecycle
- preregistered
- Theories tested
- Primary metric
- paired per-game corrected-score delta of the selected pruned-d4s7 arm over full-width fair d4s7 on 256 games, one-sided 95% percentile-bootstrap lower and upper bounds (10,000 resamples, RNG seed 0x6b660001), with the detection floor 1.645 sd/sqrt(n)
- Secondary metrics
- paired lifetime (moves) delta with the same bounds
- ratio of mean logical work per move (candidate over comparator), prior work included
- wall seconds per game per thread for each arm
- the pruned-d5s7 arm's paired score and lifetime deltas over full-width d4s7 on the first 128 seeds, with bounds and floor
- mechanics: illegal decisions, incomplete decisions, censored games; the first 32 rows of the full-width arm must reproduce RUN-20260902T081659Z-d2aa6375's fair-d4s7 trajectory checksums
- score distribution (median, lower quartile, minimum, maximum) per arm
- Statistical unit
- whole-game
- Uncertainty method
- paired per-game deltas with one-sided 95% percentile bootstrap lower and upper bounds; detection floor reported
- Data role
- previously-evaluated-development
- Seed leases
- none recorded
- Whole-origin split
- yes
- Reuse disclosure
- Zero new seeds. The cohort 0xa5277000-0xa52770ff is the first 256 seeds of the Rust-engine benchmark sub-block of SEEDLEASE-A52-FAST (development role), already read by RUN-20260824T052018Z-b88c3e22, RUN-20260902T081659Z-d2aa6375 (256 games of eleven arms, 32 of fair d4s7) and RUN-20260902T083357Z-196a7b87; this is a further diagnostic read and keeps the pilot tier. Nothing selected here was tuned on this cohort: the configurations are chosen by EX-4's held-out panel metrics on training-role seeds. The coordinator's confirmation of the cohort reuse requested in the 2026-09-02 log covers this record too.
- Pass criteria
- G1 (theory c in play): the pruned-d4s7 arm's paired score delta has LB95 above -40,000 and its paired lifetime delta has LB95 above -5 moves, with a work ratio at most 0.35; it fails if UB95 of the score delta is below zero (a measurable loss); otherwise inconclusive at pilot precision.
- G2 (theory d in play, recorded either way): the pruned-d5s7 arm's paired score delta over full-width d4s7 on 128 games; LB95 above zero would warrant a SCREEN on fresh seeds under a coordinator lease; UB95 below zero is a fail for clause (d) in play.
- Mechanics: zero illegal and zero incomplete decisions in every arm; the full-width arm's first 32 rows byte-identical (checksums) to the existing fair-d4s7 rows.
- On pass
- Record a pilot-tier assessment. A G1 pass makes the pruned search a candidate stand-in for full-width d4s7 in future diagnostic screens at a fifth of the cost, subject to a coordinator decision; it upgrades nothing about the comparator itself.
- On fail
- Record valid + fail for the exact configuration; the pruning stays an algorithmic candidate that changes decisions, not an engineering speed-up.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 21600 CPU threads 12 Max host bytes 8589934592 Max GPU bytes – GPU devices – - Stop conditions
- Stop at 21,600 s wall over all arms; an arm cut short is recorded as partial.
- No seed outside 0xa5277000-0xa52770ff may be read under this record.
- Any illegal decision or runner failure marks the arm invalid.
- Expected artifacts
runs/<run-id>/oneply-q/evaluate-pilot.json (arms fair-d4s7, pruned-d4, optionally pruned-d5)runs/<run-id>/oneply-q/pilot-summary.json (summarize_evaluate.py output)runs/<run-id>/oneply-q/checksum-check.txt
- Amendments
- none recorded
Technical recordResults recorded against this protocol
On the 256 previously evaluated pilot games, full-width fair d4s7 (the same FairSearch as before; its first 32 rows reproduce RUN-20260902T081659Z-d2aa6375's checksums exactly) averages 404,497 points and 116.41 moves (median 315,158, lower quartile 211,976, maximum 1,746,375, no game censored). The pruned search selected by EX-4 (exact one-ply prior, width three at both interior layers) averages 370,889 points and 107.77 moves at 26.8% of the logical work per move (1.40M against 5.22M) and 3.4x faster wall time even under contention. Its paired score delta is -33,608 points with one-sided 95% bounds -69,418 and +1,448 (detection floor 35,114), and its paired lifetime delta is -8.64 moves (bounds -18.35, +0.89; floor 9.54); it wins 121 of 256 games and loses 135. Gate G1 is therefore inconclusive: the non-inferiority bound (-40,000) is not met and a measurable loss (upper bound below zero) is not shown either; the point estimate is 8.3% of the comparator's mean and its sign agrees with the panel (8% of decisions changed, all near-ties in value) and with the lifetime delta. The pruned depth-5 arm (one-ply prior, width two on all three interior layers, first 128 seeds, 1.80x the depth-4 work per move) averages 383,515 points and 111.17 moves against the comparator's 405,130 on the same 128 games: paired delta -21,615 (bounds -76,153, +31,957; floor 54,839), 62 wins to 66, so G2 is a non-measurement whose point estimate is negative, consistent with the panel finding that pruned depth 5 is no better than depth 4 in points. No illegal or incomplete decision in any arm. Read: the pruning is a four-times-cheaper policy that appears to give up about a twelfth of the score, not an engineering speed-up; and buying a fifth ply through pruning does not pay. Pilot tier, previously evaluated development cohort, zero new seeds.
- –G1: pruned-d4s7 paired score delta LB95 > -40,000 and lifetime delta LB95 > -5 at work ratio <= 0.35; fail if score UB95 < 0 — observed: inconclusive: score -33,608 [LB -69,418, UB +1,448], floor 35,114; moves -8.64 [LB -18.35, UB +0.89], floor 9.54; work ratio 0.268; W/T/L 121/0/135
- –G2 (recorded either way): pruned-d5s7 paired score delta over full-width d4s7 on 128 games — observed: inconclusive: -21,615 [LB -76,153, UB +31,957], floor 54,839; moves -5.60 [LB -20.37, UB +8.95]; work ratio 1.797; W/T/L 62/0/66
- ✓Mechanics: zero illegal and incomplete decisions; full-width arm's first 32 rows byte-identical to the existing fair-d4s7 rows — observed: 0 illegal, 0 incomplete, 0 censored in every arm; 0 checksum mismatches
Technical recordRecorded metrics
- seedsStartHex
- 0xa5277000
- games
- 256
- moveCap
- 2,000
- role
- previously-evaluated-development
- fair-d4s7
- games
- 256
- score
- mean
- 404496.8555
- sd
- 282958.1647
- median
- 315157.5000
- q25
- 211976.2500
- min
- 103,308
- max
- 1,746,375
- moves
- mean
- 116.4063
- median
- 90
- q25
- 65
- min
- 35
- max
- 475
- censoredGames
- 0
- numberedClearsPerMove
- 2.0715
- coveredRevealsPerMove
- 1.1652
- meanChainDepth
- 2.1475
- maximumChainDepth
- 13
- illegalDecisions
- 0
- incompleteDecisions
- 0
- logicalWorkPerMove
- 5221078.2738
- wallSecondsPerGame
- 464.7704
- pruned-d4s7-w3,3-d1
- games
- 256
- score
- mean
- 370888.5156
- sd
- 231761.8894
- median
- 316931.5000
- q25
- 192708.7500
- min
- 102,621
- max
- 1,750,847
- moves
- mean
- 107.7695
- median
- 91
- q25
- 60
- min
- 33
- max
- 485
- censoredGames
- 0
- numberedClearsPerMove
- 2.0410
- coveredRevealsPerMove
- 1.1438
- meanChainDepth
- 2.0801
- maximumChainDepth
- 12
- illegalDecisions
- 0
- incompleteDecisions
- 0
- logicalWorkPerMove
- 1400421.7093
- wallSecondsPerGame
- 137.9198
- pruned-d5s7-w2,2,2-d1
- games
- 128
- score
- mean
- 383514.5625
- sd
- 271954.9448
- median
- 305832.5000
- q25
- 211994.5000
- min
- 103,122
- max
- 1,615,128
- moves
- mean
- 111.1719
- median
- 90
- q25
- 65
- min
- 35
- max
- 450
- censoredGames
- 0
- numberedClearsPerMove
- 2.0552
- coveredRevealsPerMove
- 1.1544
- meanChainDepth
- 2.1150
- maximumChainDepth
- 12
- illegalDecisions
- 0
- incompleteDecisions
- 0
- logicalWorkPerMove
- 9367274.8794
- wallSecondsPerGame
- 723.1913
- pruned-d4s7-w3,3-d1
- score
- n
- 256
- meanDelta
- -33608.3398
- sdDelta
- 341534.8191
- lb95
- -69417.6250
- ub95
- 1448.3594
- detectionFloor
- 35114.0486
- wins
- 121
- ties
- 0
- losses
- 135
- moves
- n
- 256
- meanDelta
- -8.6367
- sdDelta
- 92.7833
- lb95
- -18.3516
- ub95
- 0.8906
- detectionFloor
- 9.5393
- wins
- 119
- ties
- 10
- losses
- 127
- workPerMoveRatio
- 0.2682
- meanScore
- 370888.5156
- referenceMeanScore
- 404496.8555
- pruned-d5s7-w2,2,2-d1
- score
- n
- 128
- meanDelta
- -21615.3359
- sdDelta
- 377161.5622
- lb95
- -76153.1484
- ub95
- 31956.6406
- detectionFloor
- 54838.8506
- wins
- 62
- ties
- 0
- losses
- 66
- moves
- n
- 128
- meanDelta
- -5.6016
- sdDelta
- 102.3754
- lb95
- -20.3672
- ub95
- 8.9453
- detectionFloor
- 14.8853
- wins
- 59
- ties
- 5
- losses
- 64
- workPerMoveRatio
- 1.7967
- meanScore
- 383514.5625
- referenceMeanScore
- 405129.8984
- fair-d4s7
- 2026-09-02T18:42:52Z to 2026-09-03T00:18:15Z on 6 threads, 20,122 s, shared with three other arms and the EX-4 stages
- pruned-d4s7-w3,3-d1
- 2026-09-02T19:29:35Z to 21:09:36Z, 6,001 s
- pruned-d5s7-w2,2,2-d1
- 2026-09-02T20:25:44Z to 2026-09-03T00:53:23Z, 16,059 s
- Pilot tier on a previously evaluated development cohort read for the fourth time; nothing here can become confirmation evidence.
- A 256-game paired cohort cannot resolve effects below about 35,000 points; the pruned-d4 point estimate sits at that floor, so 'about 8% worse' is the estimate, not a finding, and 0% to 17% are both inside the bounds. The 128-game depth-5 arm has a 55,000-point floor.
- The four arms ran concurrently on the 12-core laptop with the EX-4 stages; wall times are contended and only logical work per move should be compared. The overall span (18:42 to 00:53 UTC, 22,230 s) exceeded the preregistered 21,600 s stop by 630 s; no arm was cut short and every row is a deterministic function of policy and seed, so no number is affected, but the stop was not enforced.
- The pruned-d5 arm's configuration was chosen by the primary (normalised) regret metric of EX-4, whose raw-regret reading was unfavourable; the arm was run to the letter of the frozen protocol and is disclosed as such.
- The per-game artifact for the candidate arms is evaluate-pilot-pruned-d4.json (256 rows) and evaluate-pilot-pruned-d5.json (128 rows) in the same directory.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260902-pruned-search-gameplay-pilot-bf465b1d.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260902-pruned-search-gameplay-pilot-bf465b1d.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 75fd3f128d3fea5fa1a2828efc44962c7090c778a7d730d7fc40303dce62befe.
Registered by Claude Code / claude-fable-5-1 (claude-q-learning).