---
title: Distributional afterstate ranker
family: afterstate-learning
summary: Measures every legal column under aligned futures and trains one network to rank the resulting positions.
status: rejected
evidence: ledger-recorded
reads: public
kind: strategy
technique: afterstate
---

## The problem

A policy is graded on which column it picks, so a network that learns to pick
has to be shown what each column would have led to. The learned evaluators
before this one were shown only the column that was actually played, then asked
at deployment to rank the six they had never seen: the
[sibling trap](/learn/concepts/ranking-siblings).

A second problem sits underneath. Grading a ranker needs a target ranking, and
in a game this noisy the target is itself a measurement, and nobody had checked
whether a simulated long-horizon outcome holds still enough to say which column
was better.

## Proposed solution

Take one position. The player has up to seven legal columns. Drop into each and
let the cascade finish: seven resolved boards, the
[afterstates](/learn/glossary). Play 40 more moves from each with a fixed simple
policy under a future pinned down in advance, record the score, and repeat with
a different pinned future. Average, and you have an opinion about which column
was better with luck held constant, because all seven lived through the same
futures.

<svg viewBox="0 0 660 208" className="my-4 w-full max-w-2xl rounded-xl border border-zinc-800 bg-zinc-900/40" role="img" aria-label="One harvested position expands to every legal column's afterstate, each afterstate is played forward through the same pinned futures, and the resulting spread of outcomes per column is what one network learns to reproduce and order.">
  <g fontSize="11" fill="#e4e4e7">
    <rect x="12" y="78" width="96" height="52" rx="8" fill="#1e3a8a" />
    <text x="60" y="99" textAnchor="middle" fontWeight="700">position</text>
    <text x="60" y="115" textAnchor="middle" fontSize="10" fill="#bfdbfe">harvested</text>
    <rect x="146" y="24" width="120" height="28" rx="6" fill="#14532d" />
    <text x="206" y="42" textAnchor="middle">afterstate 1</text>
    <rect x="146" y="90" width="120" height="28" rx="6" fill="#14532d" />
    <text x="206" y="108" textAnchor="middle">afterstate 2</text>
    <rect x="146" y="156" width="120" height="28" rx="6" fill="#14532d" />
    <text x="206" y="174" textAnchor="middle">…all legal</text>
    <rect x="300" y="24" width="146" height="160" rx="8" fill="#78350f" />
    <text x="373" y="82" textAnchor="middle" fontWeight="700">the same</text>
    <text x="373" y="99" textAnchor="middle" fontWeight="700">pinned futures</text>
    <text x="373" y="120" textAnchor="middle" fontSize="10" fill="#fde68a">40 more moves each,</text>
    <text x="373" y="135" textAnchor="middle" fontSize="10" fill="#fde68a">every column, every time</text>
    <rect x="480" y="24" width="166" height="70" rx="8" fill="#4c1d95" />
    <text x="563" y="48" textAnchor="middle" fontWeight="700">a spread of outcomes</text>
    <text x="563" y="66" textAnchor="middle" fontSize="10" fill="#ddd6fe">16 quantiles per column,</text>
    <text x="563" y="82" textAnchor="middle" fontSize="10" fill="#ddd6fe">and that spread is the label</text>
    <rect x="480" y="114" width="166" height="70" rx="8" fill="#3f3f46" />
    <text x="563" y="140" textAnchor="middle" fontWeight="700">one network learns</text>
    <text x="563" y="158" textAnchor="middle" fontSize="10" fill="#d4d4d8">to reproduce the spread</text>
    <text x="563" y="174" textAnchor="middle" fontSize="10" fill="#d4d4d8">and the ordering</text>
    <g stroke="#3f3f46" strokeWidth="1.5">
      <line x1="108" y1="92" x2="146" y2="38" />
      <line x1="108" y1="104" x2="146" y2="104" />
      <line x1="108" y1="116" x2="146" y2="170" />
      <line x1="266" y1="38" x2="300" y2="80" />
      <line x1="266" y1="104" x2="300" y2="104" />
      <line x1="266" y1="170" x2="300" y2="128" />
      <line x1="446" y1="80" x2="480" y2="60" />
      <line x1="446" y1="128" x2="480" y2="148" />
    </g>
  </g>
</svg>

Two design choices carry the work. The label is a spread: the network predicts
16 quantiles of the outcome distribution, so it can tell "reliably decent" apart
from "usually poor with a rare jackpot", and its 76.5% interval can be checked
against how often it really contains the outcome. The loss is about order too,
because a within-position ranking term sits alongside the quantile loss.

The information boundary runs between the two halves. The corpus builder is a
[teacher](/learn/glossary), since it chooses the hidden gray numbers and makes
the same future happen seven times, which no player can do; the evaluator that
would play reads the resolved public afterstate and nothing else, with seeds,
hidden values, score, level and move number absent from the record format by
construction.

## How it works

1. Harvest. `generate-corpus.cpp` plays fair depth-1 development games and
   records public positions: canonical board, visible next disc, drops until
   rise, legal columns. Folds split by origin game, so a held-out position's own
   game never trains the model.
2. Expand and pin. Every legal column is played out to its resolved afterstate,
   and each scenario index derives one random stream from the canonical public
   position alone, shared by every column there.
3. Label, completely. From each afterstate a fixed public phase-greedy depth-1
   policy plays up to 40 further moves in that stream; the label is the score
   obtained and the moves survived. The loop iterates over the legal-action list
   itself, so a missing sibling is impossible, and every run reports a
   completeness fraction that must be exactly 1.0. These teacher labels never
   reach the evaluator.
4. Train. `train.py` fits a compact residual convolutional network in float32
   with a 16-quantile loss and a within-position ranking loss; mixed precision
   was rejected in the source because it could flip near-tied orderings.
5. Grade. `label-d4.cpp` computes the exact
   [fair depth-4](/approach/fair-expectimax/reference) ordering of the same
   held-out positions, and model and search are scored against one target, the
   ordering implied by the averaged 40-move outcomes: how often the first choice
   is right, how often a pair is ordered correctly, and normalized regret, the
   share of the best-to-worst gap the chooser gives away.
6. Deploy, never reached. Conditional on passing, the plan was a one-ply
   chance-averaged greedy policy over legal afterstates with exact depth-4 as a
   fallback, over 32 paired games. No cohort was ever opened.

## What happened

Three pilots ran in order and only the third could deliver a verdict. With 256
pinned futures per column the target held still, and the network ranked columns
worse than exact fair depth-4 on 2,523 fresh held-out positions, so the theory
was recorded as not supported as tested (valid run, scientific outcome fail,
pilot tier, `RS-20260820T142500Z-8f4a2d17`).

The first two pilots failed on the labels. With eight pinned futures per column,
splitting them into two halves gave two rankings that mostly disagreed: 0.246
against the 0.5 the frozen protocol demanded. The spread of outcomes within one
column was larger than the typical gap between columns, so there was no
trustworthy target to grade anyone against, and the outcome recorded is
inconclusive. Sixty-four futures lifted the agreement and still missed the floor.

A fourth experiment asked the narrow question: leave depth-4 in charge, look only
at positions where its own top two columns are almost tied, and let the model
swap to the second when its advantage survives a resampling test. The override
fired on 37.0% of eligible positions and improved on depth-4 in both halves of
the held-out data, and it failed its frozen gate, because the rule fixed in
advance required at least 0.01 of improvement in each half separately and the
second half delivered 0.0075 (`RS-20260820T184500Z-63c0a8e2`, valid run,
scientific outcome fail).

## What we learned

The label has to be checked before the model is. Two pilots produced corpora,
trained networks and a sheet of comparison numbers in which the model looked
worse than depth-4, and all of it meant nothing, because the thing it was
compared against was noise. Only the preregistered stability floor kept two false
negatives out of the record. Any experiment that grades a policy against
simulated outcomes should measure the reliability of those outcomes first.

Successor-closed data did what it claims: coverage was exactly complete in every
run, and when the model lost it lost on ranking quality, on positions of the kind
it was trained on, rather than on sibling extrapolation.

A conservative override is a live mechanism. It fired often, its uncertainty was
calibrated, it improved on the reference in both halves of held-out data, and it
lost on a frozen margin, which makes it the first preregistered held-out test in
the record where a learned model's intervention beat the reference search at all.
None of that is a claim about play: no complete game was played, and at pilot
tier this says nothing about score, survival or the million-point mean. The open
question is whether a model without the two recorded handicaps, a weak depth-1
continuation teacher and training stopped at 11 of 20 epochs, would clear the
same unchanged bar.

<AgentContext summary="Records and provenance">

Theory `TH-20260820-distributional-afterstate-ranker-7aba7fb3`. Experiments
`EX-20260820-afterstate-pilot-h40-29b8588a`,
`EX-20260820-afterstate-pilot-h40-k64-d7a9faf5`,
`EX-20260820-afterstate-pilot-h40-k256-65a05477` and
`EX-20260820-d4-toptwo-override-gate-0bdb39a1`. Results
`RS-20260820T094500Z-5c1e9a04`, `RS-20260820T114500Z-2b7c9e31`,
`RS-20260820T142500Z-8f4a2d17` and `RS-20260820T184500Z-63c0a8e2`. Index row:
[experiment index](/docs/research/experiment-index), "Afterstate learning". The
theory was registered with the conditions that would kill it, including the one
that turned out to matter most: if the labels themselves are too noisy to be
stable, the verdict is inconclusive rather than a rejection of the theory. Three
of the four falsification criteria are about ranking quality; the fourth is about
whether the corpus can be built successor-closed at all.

All four runs drew only on seed lease `SL-20260820T083000Z-5da70000`, and every
held-out block was read exactly once. Single machine profile
`MACH-20260820T080056Z-376ada90`, float32 on a shared-memory integrated GPU.

</AgentContext>

<AgentContext summary="Full results table">

All figures are read from the machine-readable result records under
`research/results/`. They are pilot tier: offline ranking comparisons on
public-development data, with no complete games played by either side.

Iteration 1, 8 scenarios (`RS-20260820T094500Z-5c1e9a04`, valid run,
inconclusive). 11,379 roots, 616,048 sibling labels, action completeness 1.0;
2,470 held-out roots. Split-half Spearman 0.2457 against the frozen 0.50 floor.
Within-action scenario standard deviation 20,922 points against a median
between-action gap of 7,096; both are statistics of the teacher labels. Against
that noisy target the model trailed depth-4 (pooled top-1 0.247 vs 0.330,
pairwise 0.571 vs 0.637, regret 0.413 vs 0.337) and depth-4 beat depth-1 (0.330
vs 0.254), which matches the historical ordering of those searches. Quantile
interval coverage 0.631, outside its band.

Iteration 2, 64 scenarios (`RS-20260820T114500Z-2b7c9e31`, valid run,
inconclusive). Split-half Spearman 0.4462 on 2,750 fresh held-out roots.
Coverage 0.8069, inside the band. Model against depth-4: top-1 0.3422 vs 0.4331,
pairwise 0.6367 vs 0.6972, regret 0.3029 vs 0.2315; model against its own
depth-1 teacher, top-1 0.3422 vs 0.2985. Training stopped at epoch 15 of 20 on
the GPU budget.

Iteration 3, 256 scenarios (`RS-20260820T142500Z-8f4a2d17`, valid run, fail;
theory assessed not supported as tested). 24,270,592 rows over 14,009 roots,
completeness 1.0; 2,523 held-out roots of which 411 are "decisive", meaning a
best-to-worst spread above 20,000 points, a threshold fixed on earlier
training-role data. Stability 0.8181 decisive, 0.6379 unconditioned, both above
the floor. Coverage 0.8635. Model against depth-4, pooled: top-1 0.4245 vs
0.4986, pairwise 0.6851 vs 0.7366, regret 0.2408 vs 0.1784; on decisive roots the
gap narrows but persists, top-1 0.5255 vs 0.5839, regret 0.0820 vs 0.0643. Model
against the depth-1 teacher, top-1 0.4245 vs 0.3191. Every half-fold moved the
same direction. Compact 3.4M-parameter residual network, stopped at epoch 11 of
20.

Override gate (`RS-20260820T184500Z-63c0a8e2`, valid run, fail). 2,689 fresh
held-out roots; 1,030 eligible, a 38.3% near-tie rate at the frozen 500-point
depth-4 gap; the model may swap only when its advantage survives a resampling
test at 95% confidence, and the override fired on 37.0% of eligible roots against
a 5% minimum.
Mean normalized regret on eligible roots, override against unchanged depth-4:
half 1 0.2484 vs 0.2835 (+0.0351), half 2 0.2343 vs 0.2418 (+0.0075), pooled
0.2414 vs 0.2629 (+0.0214). On the 84 decisive eligible roots, 0.0880 vs 0.1306
(+0.0426). Whole-set regret 0.1850 vs 0.1932. Stability 0.8144, coverage 0.8606,
two gate runs byte-identical. Six of seven gate criteria passed; the one that
failed was the primary one.

</AgentContext>

<AgentContext summary="Validity, gates and limitations">

Why "valid run", "outcome fail" and "positive signal" all fit the record at once.
Valid run means the experiment did what it promised: the corpus was complete, the
labels were stable, the self-tests passed before any label was inspected, the
gate script produced byte-identical reports on two runs, and no seed outside the
registered lease was read. A valid run whose candidate loses is a finished
scientific contribution; an invalid run tells you nothing either way. Outcome
fail means the threshold written down before the data existed was not met. The
pooled improvement passes and the 0.0075 miss is small, and neither changes the
verdict, because a threshold that may be reinterpreted after the data arrives
stops being a threshold. The experiment's own failure action closes the door: no
further override variants of this same checkpoint without new evidence. Positive
signal means the failure was about the checkpoint rather than about the
mechanism, and the recorded limitations name the likely bottleneck. A retry is
allowed with a better-trained or better-taught model, under the same unchanged
rule.

Recorded limitations that apply to every number above. The target is the 40-move
outcome under a phase-greedy depth-1 continuation: a weak, fixed public teacher,
and the same label family the model was trained on. Depth-4 is scored against
that same target, but an offline ranking comparison cannot measure real gameplay
interaction, which only a SCREEN tier could. Roots are harvested from fair
depth-1 games, so the position distribution is depth-1's rather than the
deployment policy's. Both trained models were stopped early by the GPU budget.

</AgentContext>

<AgentContext summary="Scoring mode">

No point total on this page is a complete-game score. The point figures are
label statistics from the teacher's 40-move continuations, and every comparison
between the model and a search is an offline ranking statistic on stored
positions. Nothing here can be placed beside a policy's mean score.

</AgentContext>