---
title: Sibling network on the 200-move panel
family: d4-long-outcome
summary: Start a network as an exact copy of the reference search, train it on a locked panel where every legal column was measured, and watch it get worse.
status: rejected
evidence: task-record only
reads: public
---

Start a network in a state where it *is* the reference search, exactly, then let
it learn from a locked panel of positions in which every legal column was played
forward two hundred moves, and see whether training moves it anywhere better.

<EvidenceLabel status="rejected" evidence="task-record only" reads="public" />

It did not. In every held-out fold it moved away from the reference and ranked
moves worse.

## Why this experiment is unusually clean

Most learning failures in this repository have an easy excuse available. Either
the training data only recorded the move that was actually played — the
[sibling trap](/learn/concepts/ranking-siblings), or the network started from
random weights and might simply have landed in a bad place.

This design removes both excuses:

- **Every legal column is labelled.** The corpus is a locked panel of 477
  positions drawn from 8 whole games, in which each sibling column was
  evaluated with a 200-move continuation. Nothing is extrapolated.
- **The network starts as the reference policy.** Its main output is a
  *residual* over depth-4 search, initialised to exactly zero. Before a single
  gradient step, the model's ranking is precisely fair depth 4's ranking. Any
  movement is training's own doing.
- **Nothing is selected on the answer.** One architecture and one training
  schedule are fixed before evaluation; the source states that there is no
  validation-driven choice of epoch, seed, feature, width or loss. Folds hold out
  one **whole origin game** at a time, and which game a position came from is
  never shown to the model.
- **There is no way to cheat by playing.** The program has no gameplay lane at
  all: it never creates a game, advances one, reads a seed, generates a panel or
  runs a screen.

## How it works, step by step

1. Load the checksum-locked 477-record panel and verify its hash, its game
   count, and the number of records contributed by each of the 8 origin games.
2. Encode each position as sparse public features: the 49 cells, the visible
   next disc, the rise phase, the candidate column — plus summaries of the
   position after each candidate move.
3. Train five outputs at once: the residual over depth 4, and four auxiliary
   quantities — mean moves survived, numbered clears, reveals, and a
   99th-percentile downside, with the ranking losses (pairwise and listwise)
   carrying most of the weight and the auxiliaries a small share.
4. Repeat for each fold, holding out one whole origin game, and compare against
   plain depth 4 on that game's positions: how often the top column matches, how
   often pairs are ordered correctly, and how much value is left on the table.

## What happened

The retained outcome is one sentence: it regressed depth-4 search on top-move
accuracy, pairwise ordering, and leftover value **in every origin fold**.

That is a stronger negative than it first sounds. A model that begins as an
exact copy of the best known policy, on data where every alternative was
actually measured, with no chance to overfit a selection decision, still walked
away from the reference and got worse. Whatever is wrong with learning to rank
moves here, it is not a bad starting point and it is not missing sibling labels.

<Callout title="No numbers are retained for this run" tone="warn">
This experiment is **task-record only**. The
[experiment index](/docs/research/experiment-index) carries its status and its
one-line outcome; the [ledger](/docs/research/history) has **no entry** for it.
No accuracy, regret, fold table or checkpoint is retained, so none is quoted
here.
</Callout>

<TechnicalDetails>

- Status of record: **rejected; task-record only**
  ([experiment index](/docs/research/experiment-index), "D4 and long-outcome
  research"): "it regressed D4 top-1, pairwise ranking, and regret in every
  origin fold."
- Read from the source rather than from a result: 477 expected records across 8
  games from `0x3d6d0010`, corpus SHA-256 beginning `bfda8ae3`; 5 heads, 64
  hidden units, 48 epochs, batch of 32 roots, learning rate 0.0012, weight decay
  1e-5, gradient-norm clip 3.0, loss weights pair 1.0 / list 0.75 / point 0.25 /
  auxiliary 0.12, one fixed network seed, pool reshuffled every epoch.
- [`audit-05`](/docs/exploratory/audit-05-optimistic-curriculum) §2.3 quotes the
  source lines that make the zero-initialised residual exact — "the untrained
  model is exactly always-D4", and §4 lists this as failure 16 of 17, one of the
  two experiments that actively *refute* the idea that these failures are
  optimisation or initialisation problems.
- **Scale for the reader.** Two other studies on this same 477-root panel *are*
  recorded with numbers, and they frame what "worse than depth 4" means here: a
  recomputation of exact search on the panel found depth 4 materially better than
  depth 1 on top-move accuracy, pairwise ranking, regret and every origin
  (completed; task-record only diagnostic), and a hindsight-based ranker charged
  for its information advantage scored **28.93% top-1 against fair depth 4's
  38.16%**, with pairwise 59.85% against 66.82% (ledger-recorded). Both live in
  [terminal policy iteration](/approaches/terminal-policy-iteration).
- **A caveat on the panel itself.** The research roadmap records that this
  477-root panel is "reusable diagnosis, not fresh model-selection evidence"
  (quoted in [`audit-05`](/docs/exploratory/audit-05-optimistic-curriculum)
  §2.5). It has been read; a new model cannot be gated on it. §2.5 also records
  that the corpus file itself lived under `/tmp` and is gone from this machine.
- Source: `d4-h200-sibling-nnue.cpp`.

</TechnicalDetails>

## What this taught us, and what is still open

- **Sibling coverage alone is not enough.** The repository's most-cited failure
  mode was labelling only the move that was played. Here every move was labelled,
  and the ranking still regressed, which points at the *labels* rather than at
  their coverage: at one position, the true long-horizon values of seven columns
  are close together relative to the noise in any affordable measurement of them.
- **Starting at the answer is not a shortcut.** The zero residual is an elegant
  safety property: an untrained model that is exactly the reference, but it
  does not stop gradient descent from leaving it.
- **Still open:** whether a label with a smaller noise-to-gap ratio exists at
  all. The strongest signals measured anywhere in this repository are
  *survival* heads, not return heads, which is the argument
  [`audit-05`](/docs/exploratory/audit-05-optimistic-curriculum) §5.3 makes for
  predicting per-rise survival instead of score.