Rollout veto

The problem

The reference search is tactically strong and structurally short-sighted. It looks four of the player's moves ahead, so it cannot see the second row rise, and it cannot see that a column it likes will be unplayable two rise cycles from now. A twenty-five-move replay can see that. But a long replay is noisy: play the same position forward under different luck and the outcomes differ widely, so a long look used carelessly will override a good move on the strength of one lucky imagined future.

Proposed solution

Let the long look object, never choose. The depth-4 search picks its column as usual. Only on dangerous boards does the policy replay every legal column forward for twenty-five moves, and it gives every column the same seven imagined futures so that the comparison between columns is paired: no column gets luckier tapes than another. An alternative may displace the search's choice only when it survives at least as often, clears at least as much, beats the default by a fixed confidence bound, and is not far below the search's own valuation. Otherwise the depth-4 move plays.

The policy reads the visible board, the visible next disc and the moves until the next rise; the imagined futures are generated from a hash of that public board, and the continuation that plays them is typed so that it cannot receive the real tape, the game's seed, the score, the level or the move number.

How it works

1. Run the unmodified fair depth-4 search and keep its column as the default. If anything below fails, this is what gets played.
2. Route or skip. The expensive part runs only when the board is dangerous, defined as a maximum column height of at least four. On safe boards the policy is plain depth 4, bit for bit.
3. At a routed position, give every legal column the same seven imagined futures, each generated from a hash of the public board.
4. In each future, force the candidate column, then play twenty-five more moves with a fresh, complete depth-2 search at every step. That continuation sees a board, a disc, a rise phase and whether the game is over.
5. Score each line by the points it earns, minus a large penalty if it dies, plus one leaf evaluation if it is still alive at move 25.
6. Apply four conditions, all required, before an alternative may displace depth 4: it must survive in at least as many of the seven futures; it must not clear fewer numbered discs on average; its paired advantage must be positive at a fixed one-sided confidence bound; and it must not be too far below depth 4's own valuation. Among alternatives that pass, the one with the largest confidence bound wins. The output is one column.

What happened

The single registered pilot game scored 404,047 points where plain depth 4 scored 159,616 on the same seed, with a handful of overrides in the whole game (one game, ledger-recorded). Its projected cost was far above its preregistered runtime ceiling, so the run was paused before the second game rather than allowed to become a strength claim on a sample of one; the record at the time says "only one pair, not statistical evidence". Two follow-ups applied only changes that provably cannot alter a decision and saved immaterial wall time. A four-game quality extension kept a higher mean mostly because the original long game was still in it, and failed its frozen gate. Six cheaper teachers replayed against the original's decisions reproduced almost none of its beneficial overrides.

The mechanism was then ported to corrected 17,000-point scoring, verified byte-identical against the frozen source, and run on 32 paired development games (finding-03, run valid, outcome fail, development tier). The veto lost to fair depth 4 by 46,510.5 points per game, and the two flow rates that would have had to rise for the idea to work both fell.

What we learned

The estimator was measuring the wrong game. A column selected because it looks good under 25 moves of depth-2 play is then executed in a game played by depth 4. That mismatch is the most concrete finding here, and it is why the accepted overrides made realised flow worse even though "must not clear fewer discs" was one of the four conditions: the retest's 68 overrides cost an average of −21,887 points each (32 games, finding-03).

A one-game pilot cannot be informative in either direction for a policy whose per-override effect has a standard deviation of roughly 160,000 points (finding-03). The 404,047-point game is an anecdote, and the record treated it as one from the start.

What is ruled out is exact: horizon 25, seven futures, danger height 4, a depth-2 continuation, all legal columns as challengers, at both value bands, on 32 fresh paired games. The engineering survives the negative result: paired imagined futures shared across columns, a continuation whose type cannot carry privileged data, an exact fallback, and a compression pass proven decision-identical. The retest names the cheapest next test, and it is neither a longer horizon nor a wider band: more imagined futures, since the return test appears under-sampled by roughly two to seven times at seven. Whether more futures would change the verdict is the open question.

Agent contextRecords and provenanceopen ↓close ↑

Records: experiment index "D4 and long-outcome research"; ledger sections "Fair-D4 25-move exact-D2 rollout veto", "Exact 25-move rollout-veto compression", "D2 rollout teacher-compression replay" and "Original D4 + D2/s7/h25 quality extension". Retest: finding-03, run validity valid, scientific outcome fail, evidence tier development, reproduced in this checkout, separate namespace and seed lease, frozen source untouched.

Pilot seed 0x3ded0000; quality-extension games 0x3ded0001–0x3ded0003; the runner paused before 0x3ded0001 in the original run.

Agent contextFull results tableopen ↓close ↑

Original pilot (runtime-paused; ledger-recorded), seed 0x3ded0000, one game: stock depth 4 159,616 points / 105 moves, 1.97143 clears and 1.09524 reveals per move; the veto 404,047 / 250, with 2.276 clears and 1.316 reveals per move. 12 switches out of 179 routed decisions (4.8% of 250 moves); 15 alternatives passed every gate. Runtime 852.255 seconds for the pair against a 2,700-second ceiling, projected to 15,340.599 seconds for the full protocol.

Exact compression (runtime-paused; ledger-recorded). Replay asserted the original 404,047 points, 250 moves, 179 routes and 12 switches before writing. The depth-4 value prefilter skipped 120 of 1,242 action roots (9.66%); exact continuation reuse removed a further 26,303 transitions; depth-2 calls fell from 192,983 to 152,884. Whole-game wall time moved 641.689 to 639.321 seconds, the rollout phase from 333.585 to 295.902, which is not the order of magnitude the gate required. Every value, action and selected move stayed bit-identical to the original.

Quality extension (rejected; ledger-recorded). Four games: candidate mean 174,659.5 points / 113.75 moves against 107,007.75 / 72.5. Clears per move rose 1.84483 to 2.05055, reveals 0.98966 to 1.12967, and all four leave-one-out subsets were positive, but only one of the three new pairs was a joint score-and-move win, and lower-half score retention was 88.38% against a 90% requirement. Individual games: 0x3ded0001 109,264/70 became 153,925/100; 0x3ded0002 was identical at 59,004/45 with zero switches; 0x3ded0003 100,147/70 became 81,662/60.

Teacher compression (rejected; ledger-recorded). Replay-only over the 179 exported routed states, six approximations (shallower continuations, fewer futures, shorter horizons, mixed schemes). Best configuration (depth 2 at rise phases 1 or 5, otherwise depth 1) reproduced 166 of 179 exact actions and 2 of 12 beneficial switches, against an eligibility floor of 8 of 12; the five other configurations recovered none. The high overall agreement comes from the 167 positions where the teacher did nothing.

Corrected-scoring retest, finding-03, 32 paired development games:

Arm	mean score	mean moves	clears/move	reveals/move
fair depth 4	339,290	98.66	2.0200	1.1308
rollout veto	292,780	86.34	1.9515	1.0749

Paired mean delta −46,510.5 points and −12.31 moves; one-sided 95% bootstrap lower bound −91,924.6; wins–ties–losses 9–4–19; sign test p = 0.0436 on the 28 non-tied games; 0 censored games, 0 illegal moves. Of 12,314 alternatives scored, 103 passed all four conditions and 68 overrides were taken: an average of −21,887 points per override taken.

Agent contextValidity, gates and limitationsopen ↓close ↑

The pilot's runtime ceiling was 2,700 seconds for the pair; the projected full protocol was 15,340.599 seconds, more than five times that, and the runner paused before the second game. The compression gate required an order-of-magnitude saving and the exact compression delivered 641.689 → 639.321 seconds. The quality extension's frozen gate required 90% lower-half score retention and joint score-and-move wins on the new pairs. The teacher-compression eligibility floor was 8 of 12 beneficial switches reproduced.

In the retest, three games drew zero overrides and were exactly identical to the comparator, which doubles as a fallback check. The value-band correction that an earlier audit had asked for (7,000 to 17,000 in the fourth condition) was run as a full second arm and changed the number of overrides taken by one, from 67 to 68; 29 of 32 games were identical between the two bands. The retest notes that the return test appears under-sampled by roughly two to seven times at seven futures and names more futures as the cheapest next test.

Agent contextScoring modeopen ↓close ↑

Finding-03 ported the mechanism to corrected 17,000-point scoring and reports its 32-game retest under it. The four ledger entries above (pilot, exact compression, quality extension, teacher compression) predate that port; their point totals are kept in separate tables here and are not compared with the retest's.