---
title: N-tuple value with a two-rise look-ahead
family: ntuple-rl
summary: The family's largest run trained a pattern evaluator on 50 million moves and searched two row rises deep. Search made it worse than trusting the network alone.
status: rejected
evidence: ledger-recorded
reads: public
kind: strategy
technique: n-tuple
featured: true
---

## The problem

Nearly every learning experiment in this family taught a network the value of
the one column the policy played, then asked it to rank the six columns it never
played. That is the [sibling trap](/learn/concepts/ranking-siblings), and it has
sunk most learned evaluators on this site.

A second problem is timing. Drop7's deadline is the row rise every five drops
(the [rise](/learn/glossary)), and a look-ahead that stops in the middle of a
cycle is judging a board that is about to be pushed up a row. A search that
looked past the next rise, or the next two, would judge boards at the moment
that decides whether the game continues. Searching that far through every
chance outcome is expensive, so something in the chance handling has to give.

## Proposed solution

Two ideas, stacked. The first is to learn the value of a position before the
next disc is dealt, a [chance-state](/learn/concepts/chance-vs-choice) value, so
one learned number serves every column that leads to that position and no
column is left unlabelled. The evaluator is an
[n-tuple network](/learn/glossary): a table of learned weights for many small
patterns of cells, summed. It starts pooled, with all five positions in the rise
cycle sharing one set of tables, and after 20 million moves the weights are
copied into five phase-specific heads, so the rare phases inherit data from the
common ones before they specialise.

The second is to look across one rise boundary and then two, keeping every legal
column at the root and admitting the best two deeper. To make that affordable
the search does not average over the whole chance tree. For each bundle of
sampled outcomes it follows the single sampled outcome whose value is closest to
the bundle's average and continues down that one line: a bounded, deterministic
representative-outcome rollout. It is an approximation of a
[fair](/learn/glossary) average, and it turned out to be the part that mattered.

The policy reads the visible board, the visible next disc and the moves until
the next rise, and nothing else.

## How it works

1. Train pooled. On-policy [temporal-difference](/learn/glossary) learning over
   184 active four-cell n-tuple windows, canonicalised by reflection, with an
   undiscounted three-step forward view. The input is the public position; the
   output is one value for that position.
2. Split by rise phase at 20 million moves. Copy the pooled weights into five
   phase-specific heads and keep training to the 50-million-move budget.
3. Stabilise at the end. Adaptive per-weight learning rates run only for the
   last ten million moves.
4. Play. For each legal column, take all seven coordinate-stratified gray-disc
   reveal outcomes; deepen across one rise boundary, then two; follow the
   representative outcome down each line; finish both boundaries inside 100,000
   simulator calls per decision. A decision that cannot finish is a fatal error
   rather than a fallback to a shallower answer. The output is the column with
   the best two-rise value.

## What happened

Training ran to its budget with no checkpoint chosen on results, and the
mandatory gate then rejected the search. On the 64-game cohort recorded in the
[ledger](/docs/research/history) (rejected, ledger-recorded), playing the
network's own first choice averaged 181,733 points; adding the two-rise search
to the same weights dropped the average to 113,644. The gate required at least
300,000. The search was worse in both halves of the cohort, on score and on
moves, and every decision finished inside its work budget with no illegal move
and no game cut short. The search believed in worse moves.

## What we learned

Search over a weak evaluator can be worse than no search. Identical weights on
an identical cohort, and two rise cycles of look-ahead lowered the mean from
181,733 to 113,644 points (64 games, ledger). A deeper search compounds the leaf
evaluator's errors instead of averaging them away, and here it also chased the
approximation that made the tree affordable: following one representative
outcome rather than averaging over chance is the kind of unfair chance handling
this site has repeatedly measured as harmful.

The training scale was not the problem. Fifty million moves of on-policy
learning produced a policy roughly level with much cheaper ones in this family;
the [compute concept page](/learn/concepts/does-more-compute-help) collects the
other cases.

The rejection covers this configuration: this learning target, this
pooled-then-split schedule, and this representative-outcome rollout. The open
question is what a phase-conditioned n-tuple network does as the leaf of a fair
search such as the [reference](/approach/fair-expectimax/reference), which
was never run.

<AgentContext summary="Records and provenance">

Status in the [experiment index](/docs/research/experiment-index): rejected,
ledger-recorded, "after 50 million transitions the representative-outcome
search was worse than direct n-tuple play." Ledger:
[history](/docs/research/history), on the burned 64-game cohort
`0x3d200000...0x3d20003f`.

Training: exactly 50,000,000 transitions over 1,057,844 completed games in
1,504.404 seconds, in resumable chunks of at most 5 million, with no
intermediate evaluation or checkpoint selection. The phase model with adaptive
accumulators uses 65.4 MB of parameter storage. The fit lane was the previously
unused seed family `0x6d000000...0x6dffffff`. The 256-game development cohort
`0x6e000000...0x6e0000ff` and every protected and final cohort remain unopened;
no qualification artifact was written and training stopped before the
100-million-move continuation.

Provenance: two independent semantic reviews before the source was frozen,
strict Clang 21 and GCC 14 optimized executable and library builds, sanitizer
runs, and an immutable preregistration at
`artifacts/protocols/optimistic-phase-ntuple/protocol.json`.

</AgentContext>

<AgentContext summary="Full results table">

Stage-A gate on the 64-game cohort `0x3d200000...0x3d20003f`, from the ledger:

| Policy | Mean score | Mean moves |
| --- | ---: | ---: |
| Direct n-tuple play | 181,733.422 | 56.359 |
| Two-rise-boundary search | 113,643.969 | 37.375 |

Ordered halves: direct 180,667 / 182,800 points and 56.125 / 56.594 moves;
search 118,367 / 108,921 points and 38.719 / 36.031 moves. The final training
chunk averaged 176,247 points and 54.811 moves.

</AgentContext>

<AgentContext summary="Validity, gates and limitations">

The Stage-A gate required at least 300,000 points and 90 moves as absolute
floors; the search reached neither. Every decision completed inside the
100,000-call work budget, no move was illegal, and no game was censored, so the
rejection is not a resource artifact.

A recorded defect in the frozen protocol:
[audit-03](/docs/exploratory/audit-03-claim-arithmetic) records as its most
severe finding (C1) that the protocol's 100-million-move qualification clause
requires a corrected-scoring depth-4 search to "reproduce frozen means 176925.25
score and 116.375 moves". That score is the mean of an eight-game confirmation
under historical 7,000-point scoring, archival; the corrected-scoring replay of
the identical trajectories is 400,675.25. A corrected-score run can never
reproduce it, so that clause of the gate is permanently unsatisfiable. The
audit's remedy is a new versioned protocol, because frozen artifacts are not
edited. This did not affect the Stage-A rejection above, which used the absolute
floors rather than that clause.

</AgentContext>

<AgentContext summary="Scoring mode">

The ledger section does not name the level bonus. Audit-03 classifies all of
this experiment's rows as consistent with the corrected 17,000-point award.

</AgentContext>