On this page
Dates
Created
Updated
Record idEX-20260902-pruned-search-gameplay-pilot-bf465b1d

No explanation has been written for this record yet.

Technical recordThe registered protocolEX-20260902-pruned-search-gameplay-pilot-bf465b1d
Hypothesis
On the previously evaluated pilot cohort 0xa5277000-0xa52770ff, the pruned-d4s7 configuration selected by EX-20260902-oneply-q-fit-and-pruned-search-panel-46c75cdf (lowest held-out normalised regret among the preregistered depth-4 configurations with work ratio at most 0.35) plays complete games whose paired score and lifetime are not measurably below full-width fair d4s7 at a fraction of the logical work; and, if EX-4's depth-5 point estimate was favourable, the selected pruned-d5s7 configuration (lowest regret under exact d5 among the preregistered depth-5 configurations with work at most 2.0 times d4s7) is compared to full-width d4s7 on the first 128 seeds with its detection floor stated.
Arms
ArmNameEntry pointManifest
Candidatepruned fair expectimax (drop7-oneply-q evaluate --arm pruned:...)approaches/value-policy-learning/oneply-q-prune/rust/src/bin/evaluate.rs
Comparatorfull-width fair d4s7 (the same FairSearch policy as RUN-20260902T081659Z-d2aa6375's fair-d4s7 arm), 256 games on the same seedsapproaches/value-policy-learning/klein-friedmann-linear-q/rust/src/policy.rs
Classification
algorithmic
Information boundary
public-policy
Benchmark tier
PILOT
Lifecycle
preregistered
Primary metric
paired per-game corrected-score delta of the selected pruned-d4s7 arm over full-width fair d4s7 on 256 games, one-sided 95% percentile-bootstrap lower and upper bounds (10,000 resamples, RNG seed 0x6b660001), with the detection floor 1.645 sd/sqrt(n)
Secondary metrics
  • paired lifetime (moves) delta with the same bounds
  • ratio of mean logical work per move (candidate over comparator), prior work included
  • wall seconds per game per thread for each arm
  • the pruned-d5s7 arm's paired score and lifetime deltas over full-width d4s7 on the first 128 seeds, with bounds and floor
  • mechanics: illegal decisions, incomplete decisions, censored games; the first 32 rows of the full-width arm must reproduce RUN-20260902T081659Z-d2aa6375's fair-d4s7 trajectory checksums
  • score distribution (median, lower quartile, minimum, maximum) per arm
Statistical unit
whole-game
Uncertainty method
paired per-game deltas with one-sided 95% percentile bootstrap lower and upper bounds; detection floor reported
Data role
previously-evaluated-development
Seed leases
none recorded
Whole-origin split
yes
Reuse disclosure
Zero new seeds. The cohort 0xa5277000-0xa52770ff is the first 256 seeds of the Rust-engine benchmark sub-block of SEEDLEASE-A52-FAST (development role), already read by RUN-20260824T052018Z-b88c3e22, RUN-20260902T081659Z-d2aa6375 (256 games of eleven arms, 32 of fair d4s7) and RUN-20260902T083357Z-196a7b87; this is a further diagnostic read and keeps the pilot tier. Nothing selected here was tuned on this cohort: the configurations are chosen by EX-4's held-out panel metrics on training-role seeds. The coordinator's confirmation of the cohort reuse requested in the 2026-09-02 log covers this record too.
Pass criteria
  1. G1 (theory c in play): the pruned-d4s7 arm's paired score delta has LB95 above -40,000 and its paired lifetime delta has LB95 above -5 moves, with a work ratio at most 0.35; it fails if UB95 of the score delta is below zero (a measurable loss); otherwise inconclusive at pilot precision.
  2. G2 (theory d in play, recorded either way): the pruned-d5s7 arm's paired score delta over full-width d4s7 on 128 games; LB95 above zero would warrant a SCREEN on fresh seeds under a coordinator lease; UB95 below zero is a fail for clause (d) in play.
  3. Mechanics: zero illegal and zero incomplete decisions in every arm; the full-width arm's first 32 rows byte-identical (checksums) to the existing fair-d4s7 rows.
On pass
Record a pilot-tier assessment. A G1 pass makes the pruned search a candidate stand-in for full-width d4s7 in future diagnostic screens at a fifth of the cost, subject to a coordinator decision; it upgrades nothing about the comparator itself.
On fail
Record valid + fail for the exact configuration; the pruning stays an algorithmic candidate that changes decisions, not an engineering speed-up.
Gate fixed before controlled data
yes
Resources
Wall seconds21600
CPU threads12
Max host bytes8589934592
Max GPU bytes
GPU devices
Stop conditions
  1. Stop at 21,600 s wall over all arms; an arm cut short is recorded as partial.
  2. No seed outside 0xa5277000-0xa52770ff may be read under this record.
  3. Any illegal decision or runner failure marks the arm invalid.
Expected artifacts
  • runs/<run-id>/oneply-q/evaluate-pilot.json (arms fair-d4s7, pruned-d4, optionally pruned-d5)
  • runs/<run-id>/oneply-q/pilot-summary.json (summarize_evaluate.py output)
  • runs/<run-id>/oneply-q/checksum-check.txt
Amendments
none recorded
Technical recordResults recorded against this protocol1 record
valid runoutcome: inconclusivemixedtier: pilotRS-20260903T013022Z-d32ee053

On the 256 previously evaluated pilot games, full-width fair d4s7 (the same FairSearch as before; its first 32 rows reproduce RUN-20260902T081659Z-d2aa6375's checksums exactly) averages 404,497 points and 116.41 moves (median 315,158, lower quartile 211,976, maximum 1,746,375, no game censored). The pruned search selected by EX-4 (exact one-ply prior, width three at both interior layers) averages 370,889 points and 107.77 moves at 26.8% of the logical work per move (1.40M against 5.22M) and 3.4x faster wall time even under contention. Its paired score delta is -33,608 points with one-sided 95% bounds -69,418 and +1,448 (detection floor 35,114), and its paired lifetime delta is -8.64 moves (bounds -18.35, +0.89; floor 9.54); it wins 121 of 256 games and loses 135. Gate G1 is therefore inconclusive: the non-inferiority bound (-40,000) is not met and a measurable loss (upper bound below zero) is not shown either; the point estimate is 8.3% of the comparator's mean and its sign agrees with the panel (8% of decisions changed, all near-ties in value) and with the lifetime delta. The pruned depth-5 arm (one-ply prior, width two on all three interior layers, first 128 seeds, 1.80x the depth-4 work per move) averages 383,515 points and 111.17 moves against the comparator's 405,130 on the same 128 games: paired delta -21,615 (bounds -76,153, +31,957; floor 54,839), 62 wins to 66, so G2 is a non-measurement whose point estimate is negative, consistent with the panel finding that pruned depth 5 is no better than depth 4 in points. No illegal or incomplete decision in any arm. Read: the pruning is a four-times-cheaper policy that appears to give up about a twelfth of the score, not an engineering speed-up; and buying a fifth ply through pruning does not pay. Pilot tier, previously evaluated development cohort, zero new seeds.

What it had to pass
  • G1: pruned-d4s7 paired score delta LB95 > -40,000 and lifetime delta LB95 > -5 at work ratio <= 0.35; fail if score UB95 < 0 — observed: inconclusive: score -33,608 [LB -69,418, UB +1,448], floor 35,114; moves -8.64 [LB -18.35, UB +0.89], floor 9.54; work ratio 0.268; W/T/L 121/0/135
  • G2 (recorded either way): pruned-d5s7 paired score delta over full-width d4s7 on 128 games — observed: inconclusive: -21,615 [LB -76,153, UB +31,957], floor 54,839; moves -5.60 [LB -20.37, UB +8.95]; work ratio 1.797; W/T/L 62/0/66
  • Mechanics: zero illegal and incomplete decisions; full-width arm's first 32 rows byte-identical to the existing fair-d4s7 rows — observed: 0 illegal, 0 incomplete, 0 censored in every arm; 0 checksum mismatches
Technical recordRecorded metricsRS-20260903T013022Z-d32ee053
cohort
seedsStartHex
0xa5277000
games
256
moveCap
2,000
role
previously-evaluated-development
arms
fair-d4s7
games
256
score
mean
404496.8555
sd
282958.1647
median
315157.5000
q25
211976.2500
min
103,308
max
1,746,375
moves
mean
116.4063
median
90
q25
65
min
35
max
475
censoredGames
0
numberedClearsPerMove
2.0715
coveredRevealsPerMove
1.1652
meanChainDepth
2.1475
maximumChainDepth
13
illegalDecisions
0
incompleteDecisions
0
logicalWorkPerMove
5221078.2738
wallSecondsPerGame
464.7704
pruned-d4s7-w3,3-d1
games
256
score
mean
370888.5156
sd
231761.8894
median
316931.5000
q25
192708.7500
min
102,621
max
1,750,847
moves
mean
107.7695
median
91
q25
60
min
33
max
485
censoredGames
0
numberedClearsPerMove
2.0410
coveredRevealsPerMove
1.1438
meanChainDepth
2.0801
maximumChainDepth
12
illegalDecisions
0
incompleteDecisions
0
logicalWorkPerMove
1400421.7093
wallSecondsPerGame
137.9198
pruned-d5s7-w2,2,2-d1
games
128
score
mean
383514.5625
sd
271954.9448
median
305832.5000
q25
211994.5000
min
103,122
max
1,615,128
moves
mean
111.1719
median
90
q25
65
min
35
max
450
censoredGames
0
numberedClearsPerMove
2.0552
coveredRevealsPerMove
1.1544
meanChainDepth
2.1150
maximumChainDepth
12
illegalDecisions
0
incompleteDecisions
0
logicalWorkPerMove
9367274.8794
wallSecondsPerGame
723.1913
pairedVsFairD4s7
pruned-d4s7-w3,3-d1
score
n
256
meanDelta
-33608.3398
sdDelta
341534.8191
lb95
-69417.6250
ub95
1448.3594
detectionFloor
35114.0486
wins
121
ties
0
losses
135
moves
n
256
meanDelta
-8.6367
sdDelta
92.7833
lb95
-18.3516
ub95
0.8906
detectionFloor
9.5393
wins
119
ties
10
losses
127
workPerMoveRatio
0.2682
meanScore
370888.5156
referenceMeanScore
404496.8555
pruned-d5s7-w2,2,2-d1
score
n
128
meanDelta
-21615.3359
sdDelta
377161.5622
lb95
-76153.1484
ub95
31956.6406
detectionFloor
54838.8506
wins
62
ties
0
losses
66
moves
n
128
meanDelta
-5.6016
sdDelta
102.3754
lb95
-20.3672
ub95
8.9453
detectionFloor
14.8853
wins
59
ties
5
losses
64
workPerMoveRatio
1.7967
meanScore
383514.5625
referenceMeanScore
405129.8984
checksumGate
first 32 rows of the new fair-d4s7 arm vs RUN-20260902T081659Z-d2aa6375: 0 mismatches (checksum, score, moves, logicalWork)
wallClock
fair-d4s7
2026-09-02T18:42:52Z to 2026-09-03T00:18:15Z on 6 threads, 20,122 s, shared with three other arms and the EX-4 stages
pruned-d4s7-w3,3-d1
2026-09-02T19:29:35Z to 21:09:36Z, 6,001 s
pruned-d5s7-w2,2,2-d1
2026-09-02T20:25:44Z to 2026-09-03T00:53:23Z, 16,059 s
Limitations
  • Pilot tier on a previously evaluated development cohort read for the fourth time; nothing here can become confirmation evidence.
  • A 256-game paired cohort cannot resolve effects below about 35,000 points; the pruned-d4 point estimate sits at that floor, so 'about 8% worse' is the estimate, not a finding, and 0% to 17% are both inside the bounds. The 128-game depth-5 arm has a 55,000-point floor.
  • The four arms ran concurrently on the 12-core laptop with the EX-4 stages; wall times are contended and only logical work per move should be compared. The overall span (18:42 to 00:53 UTC, 22,230 s) exceeded the preregistered 21,600 s stop by 630 s; no arm was cut short and every row is a deterministic function of policy and seed, so no number is affected, but the stop was not enforced.
  • The pruned-d5 arm's configuration was chosen by the primary (normalised) regret metric of EX-4, whose raw-regret reading was unfavourable; the arm was run to the letter of the frozen protocol and is disclosed as such.
  • The per-game artifact for the candidate arms is evaluate-pilot-pruned-d4.json (256 rows) and evaluate-pilot-pruned-d5.json (128 rows) in the same directory.

Open the result record

Agent contextHow to extend this record

To add a reader-facing explanation, write web/content/research/EX-20260902-pruned-search-gameplay-pilot-bf465b1d.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.

Record file: research/experiments/EX-20260902-pruned-search-gameplay-pilot-bf465b1d.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: 75fd3f128d3fea5fa1a2828efc44962c7090c778a7d730d7fc40303dce62befe.

Registered by Claude Code / claude-fable-5-1 (claude-q-learning).