Research Status
On this page
docs/research/status.md
September 2026
On 2026-09-05 a learned evaluator beat the frozen fair leaf inside the same search for the first time, at screen tier. Lookup tables over every full row and full column of the board (ten million patterns per line), plus small windows, all conditioned on the rise clock, were trained by temporal-difference play on the Rust engine for four billion moves and placed as the leaf of the depth-3 seven-stratum fair search. On 256 never-read paired public-development games the tables scored 484,577 points and 140 moves against 314,438 and 93 for the identical search with the fair leaf: paired +170,139, bootstrap 95% lower bound +130,499, both halves positive, 167 wins to 89 (RS-20260905T215332Z-95d18a5a, approach page). On the same games the tables at depth 3 also beat the fair leaf at depth 4, the standing reference (377,803), by 106,775 with a lower bound of 62,574; that arm was diagnostic, not the preregistered comparator.
What this does and does not change. The evidence tier is SCREEN on public development seeds, opened once; it is not a qualification, no protected or final seed was opened, and the mean is under half the million-point target. The candidate was chosen at the best of twenty training-role validation points, so its screen margin is the honest number and its validation margin (+270,023) is not.
On 2026-09-06 the result replicated on a second, disjoint block. The same frozen tables (SHA-256 verified) played 512 never-read public-development games as the depth-3 leaf and scored 487,066 against 326,717 for the fair leaf in the same search: paired +160,349, bootstrap 95% lower bound +129,753, both halves positive, 330 wins to 182 (RS-20260906T040113Z-6ba93171). The same experiment trained a six-times-wider evaluator (2x4 and 4x2 window families added, 5.8 x 10^9 entries) on fresh training seeds until a 256-game validation margin stopped rising (a window-of-four plateau rule, which fired at 4.5 x 10^9 moves); that candidate passed the same gate on the same 512 games (481,869, paired +155,153, lower bound +126,819) and came out 5,196 behind the first tables with bounds from -40,535 to +29,158, so the preregistered scale verdict is inconclusive: no gain larger than about 35,000 in either direction. Played directly with no search the wide tables averaged 328,039 against 294,323 for the first tables on the same games, level with the fair leaf's depth-3 search; the extra scale improved the one-ply policy and left the leaf where it was. Both candidates beat the fair leaf at depth 4 on these games by more than 84,000.
Later on 2026-09-06 the first tables played one ply deeper. Unchanged and hash-verified, as the leaf of the reference depth-4 seven-stratum search they scored 516,155 against 465,675 for the same tables as the depth-3 leaf on 512 never-read public-development games from a fourth block: paired +50,479, bootstrap 95% lower bound +14,707, both halves positive, 284 wins to 228, a higher lower quartile (RS-20260906T171746Z-1623f833). On the same games the fair leaf's fourth ply was worth +49,599, and the per-game difference between the two depth steps is +881 with bounds from -40,932 to +42,867 (floor 42,550), so the preregistered interaction verdict is inconclusive: the learned leaf pays for depth about as well as the hand-written one, and no better or worse than a 512-game screen can see. At depth 4 the tables beat the fair leaf in the same search by 133,588 (lower bound 102,051); the depth-3 margin on this block was 132,707. The depth-4 search does about 36 times the depth-3 search's logical work per game. The first tables at depth 4 are now the candidate to carry forward; the next steps the record supports are a STANDARD-tier evaluation of that unchanged candidate on fresh development seeds, and a study of a leaf tuned for the search's use rather than for one-ply play (training on the boards the search evaluates, or removing the optimistic starting value from never-updated entries).
Late on 2026-09-06 a fourth experiment asked whether the leaf should change its mind as the board fills. A seed-free pass over the first run's saved accumulators found the frozen tables least trained on the fullest boards (0.98% of legal seven-high column patterns and 1.06% of five-or-more-disc top-row patterns ever updated, the touched full-line entries still at the optimistic starting value), so two repairs were preregistered against the frozen tables on a fifth never-read 512-game block (EX-20260906-ntuple-fill-conditioned-continuation-a9e5cbd3): five copies of every table keyed to a global fill bucket (occupied cells or tallest column), warm-started by copying the frozen tables into every bucket and trained on for up to two billion moves beside an unconditioned continuation as control; and two no-training edits that replace the 901,259,321 never-updated entries' starting value with zero or with the class mean. The occupied-cells candidate as the depth-3 leaf scored 506,494 against 485,455 for the frozen tables: paired +21,039 with bounds from -16,864 to +59,230 and a floor of 37,956, so the preregistered gate fails as a non-measurement; against the control continuation it is +15,093 and the control against the frozen tables +5,946, both inconclusive, and neither fill arm beat the control's best validation margin (RS-20260906T234914Z-a3fae1a9). The zeroed edit, with nothing trained, scored 514,897 at depth 3: paired +29,442 over the frozen tables with a lower bound of +5,813, changing only 194 of the 512 games, the one reading whose fixed verdict is supported. One ply deeper both repairs gave their gains back: the fill candidate at depth 4 was 22,631 behind the frozen tables (bounds -57,779 to +12,806) and the zeroed edit 10,592 behind (bounds -32,045 to +11,133), both inconclusive and both below their own depth-3 scores, while the frozen tables' fourth ply on these games was worth 36,500. The theory is assessed mixed, the frozen tables at depth 4 remain the candidate to carry forward, and the record now points at a leaf trained on the boards the search evaluates, and at a depth-aware price for never-updated entries, rather than at more buckets.
The frozen tables of both training runs, the three screens' per-game rows
and their analyses are published in the public research archive under their
run ids (https://data.drop7.dev/runs/<run-id>/ntuple-scale/..., digests in
the references; dataset records
DS-20260906-ntuple-scale-frozen-tables-ff977178
and
DS-20260906-ntuple-scale-wide-frozen-tables-c5eb52a6),
so another machine can reproduce the screens or continue training from them.
August 2026
Through August the best dependable policy was fair depth-4 expectimax — a four-move look-ahead that treats the game's luck honestly. Even with its best chance model, its measured means sit under half of that target. This page shows where the gap is, what has been measured, and which directions the evidence has closed.
How score responds to search depth, under each of the two chance models:
Mean score against search depth, fair expectimax, by chance-stratum count
- 5 strata (biased next-disc estimate)
- 7 strata (exact next-disc estimate)
- 7 strata, first 32 seeds only (arm stopped)
Notes and sources
All arms play the same 64-seed development cohort 0xa51d1000-0xa51d103f under corrected 17,000-point Hardcore scoring, except the depth-5 seven-stratum arm, which was stopped by decision after the first 32 seeds (0xa51d1000-0xa51d101f) and is drawn as its own one-point series so the two sample sizes are never joined by a line. Depth 2 was run only at seven strata. No confidence band is drawn on means; the paired contrasts with their bounds are in the companion bar figure. Evidence tier: development / public-development. The historical eight-game D3/D4 cohorts used 7,000-point scoring and are not shown.
- docs/exploratory/finding-05-chance-strata.md
- RS-20260821T181917Z-9a34ba02
- RS-20260821T205102Z-d89df4b5
Spec: web/content/figures/score-vs-depth.json · 3 source records · kind line
The same factorial as paired contrasts, which is the form the conclusions rest on:
Paired contrasts in the depth x chance-resolution factorial (mean delta, one-sided 95% lower bound)
- Chance resolution: 7 strata minus 5 strata
- Depth: 5 plies minus 4 plies
Notes and sources
Whisker is the one-sided 95% whole-game bootstrap lower bound; the upper end of the whisker is the point estimate itself (no upper bound is drawn). A contrast is significant when its lower bound clears zero: both stratum contrasts (7 minus 5 strata) do; neither depth contrast (5 minus 4 plies) does, and each sits below its own detection floor (47,052 at n=64, 107,988 at n=32). Cohort 0xa51d1000-0xa51d103f; the n=32 contrasts cover its first 32 seeds. Evidence tier: public-development.
Spec: web/content/figures/strata-5-vs-7.json · 1 source record · kind bar
The simulator and reference searches are mature enough to support reproducible research, but the strategy problem is unsolved. Corrected-score fair depth-4 expectimax is the strongest dependable reference found so far. Its average score across 64 games is 308,296 points, far below the goal of a one-million-point average.
Exploratory work on 2026-08-20/21 reproduced that reference on fresh seeds and completed the depth-by-chance-resolution factorial. Two results from it strongly suggest what to do next:
- The fourth ply is worth a large, significant gain — but only with an exact chance model. With the approximate five-stratum model the same fourth ply is worth slightly less than nothing. Chance resolution does not merely add points; it changes the sign of the depth gradient.
- The fifth ply is not measurable by this design. Both depth-5 contrasts sit far inside their own detection floors, and the seven-stratum estimate changed sign when the cohort grew from 16 games to 32, so the supported statement is that any depth-5 effect is smaller than a 64-game paired cohort can resolve — not that it is zero. What can be said is economic: even the optimistic estimate buys its points at 35.6x the work per move.
- A 64-game paired cohort in this game cannot see an effect below roughly
50,000 points. Every null result in the depth factorial is below its own
detection floor and is therefore a non-measurement rather than evidence of no
effect
(
RS-20260821T205102Z-d89df4b5). Choose experiments whose predicted effect exceeds the floor, or find a lower-variance estimator than complete games.
A fourth day, 2026-08-23, went to learned models and label economics — the reveal-construction probe in live play, the optimistic-state D0 gate, the leaf-cost NNUE student C0, and the P-SOL G0 label-semantics guardrail — and closed each of those four directions as tested; the research log for that day tells the story, and the closed-directions table below carries the headline numbers.
No candidate has qualified for the protected validation protocol. The frozen record marks both the protected and one-shot final cohorts as unopened. See the research status evidence for details.
How the research progressed
Seven eras, one sentence each; every configuration and outcome below is tabled row by row in the experiment index.
- A trustworthy simulator. TypeScript and native Hardcore rules were aligned and a scoring audit corrected the five-move level award from 7,000 to 17,000; results made with 7,000-point scoring remain historical Sequence-scored evidence.
- Hand-written policies and shallow lookahead provided fast baselines and showed that immediate score alone is a poor guide.
- Fair expectimax became the reference: completed fair D4 consistently improved on D3 in the corrected-score small cohorts, while deeper variants were not automatically better.
- Learning public-state values repeatedly hit sibling extrapolation: a model learned the outcome of the action that was played, then deployment asked it to rank actions it had not observed equally well.
- Oracle and long-outcome teachers found predictive signal, but students failed held-out sibling ranking or were too slow; oracle strength is an upper-bound teaching signal, not a legal policy result.
- Constructive cycles and explicit reservoirs look useful as features or options, but no tested controller displaced D4.
- Offline policy improvement around D4 on a locked panel of 477 public roots underperformed or barely overrode D4; neither ranker justified a gameplay run.
The failure mode that recurs through eras 4-7 is easier drawn than described:
Source
diagram-sibling-extrapolation.svg — source and reading guide
Mechanism diagram D1 of runs/RUN-20260823T191900Z-b9f8f80d/kimi-k3-figure-plan.md
("Diagrams (mechanisms, not charts)"). Hand-written, self-contained SVG; no measured
numbers are drawn.
What it explains
The repeated failure mode named in docs/research/status.md §4 ("Learning
public-state values"): a model is trained on the outcome of the action that was
actually played, then deployment asks it to rank all seven legal columns — six
of which it never observed equally. This is failure class (iii) sibling
coverage / within-root discrimination of
docs/exploratory/audit-05-optimistic-curriculum.md §4, the largest class in the
census: 6 of 17 learned-policy experiments (experiments 5, 9, 11, 13, 14, 17).
Element-by-element
- Root board (both panels): a small invented 7×7 position, drawn in engine
orientation (row 0 at top, columns 1–7 left to right). Blue circles are
numbered discs; solid gray circles are covered discs; the gray disc with a
crack mark is cracked (one hit taken). "next disc: 2 · 3 moves to rise" is the
rest of the public state a legal policy may use (
docs/methodology.md, information boundary). - Seven arrows: one per legal column, fanning from the root to the seven successor afterstates.
- Solid accent arrow + "label: H40 return" tag (column 3): the action that
was played during data collection. Its successor is the only one carrying a
training label — an H40 return under D1 continuation, the afterstate label
panel of the 2026-08-20/21 experiments (
docs/research/status.md§7). - Dashed arrows + "?" tags: the six unplayed columns. No label exists for these afterstates; any value the model assigns them is extrapolation.
- Deployment panel: the same fan, but now the model must produce a ranking of all seven. The six never-observed successors carry dashed red outlines and "rank ?" tags; the observed one is marked "observed".
- Caption strip: the failure in one line. The second line quotes the status.md §4 conclusion that low value error on visited states did not guarantee good root-action ranking.
- Hover/focus popovers (the
fig-pt/fig-popconvention of the chart generator): drill-downs on the root board (public-state definition and board orientation), the labelled sibling, the unlabelled siblings, the deployment fan, and the caption.
Simplifications (stated explicitly)
- Successors are drawn as root + dropped disc. Cascades, reveals and gravity after the drop are not simulated; the diagram is about label coverage, not mechanics. The landing cell of the dropped disc is the accent square in each mini-board.
- The root position is invented, not a recorded board; no recorded board image exists in the repository for this purpose, and the mechanism does not depend on the position.
- "H40 return" names the label family of the afterstate experiments (status.md §7); the diagram does not assert that every class-(iii) failure used that exact label — the class spans played-action-only labels, noisy labels, and labels that cannot separate siblings (audit-05 §4 definition).
- The 2026-08-21 result quoted in the deployment popover (successor-closed
coverage, every legal sibling, exact search-value labels, still worse than
one-ply exact search) is from
docs/research/status.md"Directions closed", included because it relocates the obstacle from coverage to capacity.
Sources
docs/research/status.md§4 (sibling extrapolation paragraph) and §7; "Directions closed" table (the successor-closed coverage row).docs/exploratory/audit-05-optimistic-curriculum.md§4 — class (iii) definition and the counts table (6 of 17; experiments 5, 9, 11, 13, 14, 17).docs/methodology.md— the public-state definition.- Figure spec:
runs/RUN-20260823T191900Z-b9f8f80d/kimi-k3-figure-plan.md, D1.
Conventions
CSS variables with light-theme fallbacks (var(--fig-fg, #222),
var(--fig-muted, #888), var(--fig-accent, #2563eb), --fig-grid,
--fig-pop-bg, --fig-pop-border, --fig-cover, --fig-danger) so the diagram
renders standalone and adopts the console's dark theme inside .research-fig
(web/app/globals.css). Popovers are pure SVG/CSS (fig-pt + fig-pop), the
same convention as web/content/figures/score-vs-depth.svg; they need no
JavaScript.
Source: web/content/figures/diagrams/diagram-sibling-extrapolation.source.md
And every learned evaluator's ranking quality, against the exact search it tried to reproduce:
Learned evaluators versus exact search: top-1, pairwise and regret on held-out roots
- top-1 accuracy
- pairwise accuracy
- normalized regret
Notes and sources
Panels differ across points and are named per category: the afterstate models and the fair D4 / fair D1 comparators are gated on the H40 D1-continuation panel; the D4-value student on a D4-ordering panel; the planner-distill student and its comparator on the fair-planner H5K256 panel; martingale-dual B0 and its comparator on the locked 477-root H200 panel; exact D1 / exact D2 on the historical panel of RS-20260821T104500Z-77d21e90. Compare within a panel, not across. Fair D1 pairwise and regret on the afterstate panel are not recorded, and exact D1 / exact D2 are recorded top-1 only, so those series points are omitted. Full-train pairwise 0.658 is from the record's summary prose; its per-half values are not recorded. Per-point labels are omitted to keep the dot grid readable; every popover carries the value and source. All eight students sit below their panel's fair-D4 comparator column. Evidence tier: pilot (afterstate and D4-value line); see each source for its panel definition.
- RS-20260820T094500Z-5c1e9a04
- RS-20260820T114500Z-2b7c9e31
- RS-20260820T142500Z-8f4a2d17
- RS-20260821T094500Z-1a7e3c55
- RS-20260821T134500Z-4b9d2f68
- RS-20260821T104500Z-77d21e90
- docs/exploratory/finding-11-planner-distillation.md
- docs/research/history.md
Spec: web/content/figures/learned-ranking-metrics.json · 8 source records · kind dot
Most useful conclusions so far
- Fair chance handling matters. Optimistic, worst-case, or tiny reused reveal samples can rank moves incorrectly.
- More depth is not automatically more strength. Depth and chance
resolution substitute rather than compound: with an approximate chance
model the best depth is three, with an exact chance model it is four, and
the budget frontier has a flat top at the fair-D4 operating point, reachable
from either axis
(
RS-20260821T192140Z-189fe392, finding-16).
The substitution is visible as a crossed pair of lines:
Mean score by search depth and chance resolution (shared cohort)
- 5 strata (approximate chance)
- 7 strata (exact chance)
- 7 strata, first 32 seeds (arm stopped, final)
Notes and sources
Cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring, 64 games per cell except the depth-5 seven-stratum arm, which was stopped by decision after the first 32 seeds (0xa51d1000-0xa51d101f) and is final. The stopped arm is drawn as its own dashed one-point series so the two sample sizes are never joined by a line (same convention as score-vs-depth). No per-mean confidence intervals are recorded anywhere for these arms; the bounds on the paired contrasts live in detection-floor-map. The d2s5 mean is the coordinator-supplied figure printed in finding-10's position-mode table and in the 2026-08-21 log ArmTable (the log path is not an accepted sourceRecord scheme, so the finding is cited). Evidence tier: development / public-development.
- docs/exploratory/finding-10-suite-validation.md
- docs/exploratory/finding-05-chance-strata.md
- docs/exploratory/finding-15-depth5-exact-estimator.md
- RS-20260821T205102Z-d89df4b5
Spec: web/content/figures/depth-chance-factorial.json · 4 source records · kind line
And the budget view of the same arms shows why neither axis wins on its own:
Mean score against logical work per move — the flat-topped budget frontier
- exact chance (7 strata)
- approximate chance (5 strata)
- exact chance, d5 M=1 (n=32, stopped, final)
Notes and sources
Cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring, n=64 per arm except the d5s7 point (32 of 64 planned games, stopped by decision, final), which is drawn as its own dashed one-point series so the two sample sizes are never joined by a line. Cache capacity differs across arms (60,000 entries for the recorded depth-4 comparators, 200,000 for the depth-5 arms, auto-sized 960,695 / 346,921 for the new reveal-sampling arms); capacity provably cannot change play but work per move is not comparable across capacities (finding-16 Limitations; RS-20260821T205102Z-d89df4b5 limitations). Equal-work pair: d3 M=6 (4,244,020; 376,442) vs d4 M=1 (4,956,614; 398,498) is statistically indistinguishable (paired -22,056 [-89,867, +46,009], RS-20260821T192140Z-189fe392 pairedD3M6MinusD4M1). The x axis is linear: the plan's log-scale recommendation is not expressible by the generator, and the d5s7 point dominates the range. Evidence tier: development / public-development.
- docs/exploratory/finding-05-chance-strata.md
- RS-20260821T192140Z-189fe392
- docs/exploratory/finding-15-depth5-exact-estimator.md
- RS-20260821T205102Z-d89df4b5
Spec: web/content/figures/score-vs-work-frontier.json · 4 source records · kind line
- Flow matters before spectacular chains. The task record repeatedly used roughly 2.4 numbered clears and 1.4 reveals per move as the region associated with stable long games. Treat these as diagnostic targets from limited runs, not proven universal thresholds.
- Static board potential is insufficient. Similar-looking boards can have very different futures depending on how reachable triggers and covered discs evolve across rises.
- State-value accuracy is not action-ranking accuracy. Training data must cover legal siblings or use an objective designed for relative action value. As of 2026-08-21 this is necessary but demonstrably not sufficient: a student given successor-closed coverage, every legal sibling, and exact search-value labels still ranked worse than a one-ply exact search.
- Score is heavy-tailed, and this sets a hard measurement floor. One
million-point game can coexist with a much lower average, and pairing does
not rescue small effects: before running a cohort, state the effect size the
mechanism predicts and check it against the detection floor
(
RS-20260821T205102Z-d89df4b5).
Each contrast of the depth factorial, drawn against the smallest effect its cohort could have seen:
Paired contrasts against their detection floors — every significant result is above its floor, every null below it
- paired mean delta (whisker: 95% lower bound)
- detection floor 1.645*sd/sqrt(n)
Notes and sources
All six contrasts from RS-20260821T205102Z-d89df4b5, cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring; the n=32 contrasts cover the cohort's first 32 seeds (d5s7 arm stopped by decision, final). Whiskers are one-sided 95% percentile-bootstrap lower bounds (20,000 resamples, Mulberry32 domain 0xb0075eed); upper bounds are NOT recorded for these contrasts in the record and are omitted, not zero. Detection floor = 1.645 x sd / sqrt(n), the smallest true effect whose one-sided 95% bound would clear zero (metrics.detectionFloorDefinition). The d4s7-d3s7 lower bound is carried as +26,468 from metrics.bootstrapVersusNormalApproximation; status.md and the 2026-08-21 log print +26,605, which is prose drift — the record and finding-05's interaction table both say +26,468. Evidence tier: public-development.
Spec: web/content/figures/detection-floor-map.json · 1 source record · kind bar
- D4 is useful but not the answer. It is a strong tactical fallback and teacher, yet its average is not close enough to qualify.
- The leaf evaluation prices what search cannot see. Refitting it toward a quantity the search already computes exactly — achievable clears over the next few moves — loses monotonically, and loses by destroying the upper tail rather than the lower one. A leaf that correlates weakly with short-horizon clear rate is behaving correctly, not failing.
- Cheap proxies for this game invert. Three independent times a short-horizon
screen has ranked configurations in the opposite order from complete games: the
suite-h9-v1scenario suite, a depth-3 screening run, and an eight-move achievable-clear label. Screen at the depth you intend to deploy, or do not screen.
Directions closed by exploratory evidence, 2026-08-20/23
Each entry rejects the exact configuration tested, at development or pilot tier. None of them is a proof that the underlying idea is impossible, and each names what would reopen it.
The screen and confirmation deltas behind these closures, with their recorded bounds:
Every screen and confirmation paired delta with its recorded 95% bound
Notes and sources
Whole-game paired deltas against each experiment's reference arm, corrected 17,000-point Hardcore scoring. Bounds are one-sided 95% bootstrap lower bounds; upper bounds are drawn only where recorded (the three reveal-sampling contrasts, CMA-ES, both survival-instinct arms). finding-03, finding-14, both learned-leaf arms and A900 record no upper bound — the whisker is absent there, not zero. finding-05's confirmation lower bound is printed +47,457 in the finding and +47,446.8 in RS-20260821T205102Z-d89df4b5; the figure uses the record (+47,447). A900's rejection rests on flat flow statistics and opposite-sign halves, not the score delta, which sits inside its 30,957 detection floor (RS-20260823T131226Z-16564ed9). Cohorts differ per bar (n carried per point; the survival-instinct arms share 128 fresh seeds, A900 256 fresh seeds); this chart assembles separate experiments, it does not pool them. Evidence tier: development / public-development screens.
- RS-20260821T205102Z-d89df4b5
- docs/exploratory/finding-08-learned-leaf.md
- RS-20260821T192140Z-189fe392
- docs/exploratory/finding-03-rollout-veto-17k.md
- docs/exploratory/finding-14-leaf-reweight.md
- RS-20260822T120736Z-662b39ca
- RS-20260822T233343Z-12becce9
- RS-20260823T131226Z-16564ed9
Spec: web/content/figures/screen-deltas-with-bounds.json · 8 source records · kind bar
The closures themselves, one dot per direction, each drawn against the detection floor its record states — the geometric difference between "rejected" and "not measurable":
Directions closed 2026-08-20/23 — headline paired delta with recorded bounds, and the detection floor where one is recorded
- headline paired delta (points)
- detection floor 1.645·sd/√n (where recorded)
Notes and sources
One dot per closed direction whose headline result is a points number, copied verbatim from its record; whiskers are the recorded bounds — one-sided 95% bootstrap lower bounds where that is all the record carries (both depth rows, reveal-construction A900, rollout veto, leaf refit), two-sided 95% where recorded (reveal M=2, CMA-ES, survival-instinct STRICT), and none for terminal utility, whose zero is EXACT (byte-identical games at four magnitudes from -1,000,000 to -50,000,000, 0-64-0; no statistical bound exists or is needed). Detection-floor markers (1.645*sd/sqrt(n)) are drawn only where the record carries one; no floor is recorded for reveal M=2, the rollout veto, or the leaf refit — markers omitted, not zero. Every closure rejects the exact tested configuration at development/pilot tier; none proves the underlying idea impossible, and each row's 'reopens if' condition stays in docs/research/status.md. The plan's 'depth beyond four plies' row is charted as its two recorded contrasts (s7 n=32 stopped/final, s5 n=64). Not chartable on a points axis, recorded here verbatim instead: compact afterstate model reproducing D4's ordering — top-1 0.375 against the 0.60 gate (RS-20260821T104500Z-77d21e90); the suite-h9-v1 benchmark as a strength measure — Spearman -0.257 (finding-10). status.md's table gained three further non-points closures after this figure's plan was written, also not chartable here: leaf-cost NNUE student top-1 0.296/0.3011 by half-fold against the 0.60 gate (RS-20260823T194142Z-946e3cd1), H-pool optimistic states tau -0.959 [cluster bootstrap -3.069, -0.390] (RS-20260823T205143Z-ead14c9d), and P-SOL G0 cheap-label proxy mean tau 0.370, LB95 0.283 (RS-20260823T225753Z-0fbd48c3). Cohorts differ per point (n on every marker); this chart assembles separate experiments, it does not pool them.
- RS-20260821T205102Z-d89df4b5
- RS-20260821T192140Z-189fe392
- RS-20260823T131226Z-16564ed9
- docs/exploratory/finding-03-rollout-veto-17k.md
- docs/exploratory/finding-14-leaf-reweight.md
- RS-20260822T120736Z-662b39ca
- RS-20260822T233343Z-12becce9
- docs/exploratory/finding-04-terminal-utility-saturated.md
Spec: web/content/figures/closed-directions-map.json · 8 source records · kind forest
| Direction | Result | Reopens if |
|---|---|---|
| Search depth beyond four plies | Not measurable: +23,367 (7 strata, n=32) and −8,624 (5 strata, n=64), both inside their detection floors (RS-20260821T205102Z-d89df4b5) | A design that can resolve sub-50k effects, or a mechanism predicting an effect above the floor |
| Stacking reveal sampling on top of the fourth ply | −41,950 at M=2 (RS-20260821T192140Z-189fe392) | A wider dose is tested; only 28.6% joint coverage was affordable at depth 4, and the dose that worked at depth 3 was 85.7% |
| Pricing the same-wave double hit on a solid gray (reveal construction) as a leaf term | Corpus: partial r −0.044 with 60% of live setups uncollected; in play, +3,204 over 256 fresh paired games, inside its floor (RS-20260823T131226Z-16564ed9) | A term that raises reveals per move at all; or a per-root counterfactual showing the uncollected setups were worth collecting |
| Harsher terminal (death) utility | Saturated: byte-identical play at every magnitude past the current value (finding-04) | Never, for this parameterization |
| A leaf-cost NNUE student reproducing D4's within-root ordering from successor-closed D4 values | Top-1 0.296/0.301 by half-fold against the 0.60 gate (RS-20260823T194142Z-946e3cd1) | A different architecture class at leaf cost, e.g. a cross-sibling set ranker; the self-play loop's leaf form stays blocked and its redesign is root-prior shaped |
| Optimistic states with fair labels (H-pool, the salvageable core of the oracle curriculum) | Stage D0: tau = −0.959 [−3.069, −0.390], with a degenerate denominator — 63/64 oracle games hit the 500-move cap (RS-20260823T205143Z-ead14c9d) | A D0 rerun with an uncapped horizon shows tau ≥ 0.25 and the oracle's action at least matching D4's under fair futures |
| Cheap M=1 continuation labels standing in for D3 N7M6 label semantics (P-SOL G0) | Guardrail kill: within-root orderings agree at only mean tau 0.370 (LB95 0.283) on 6 CRN-matched roots; the divergence is the reveal quadrature, not the engine (RS-20260823T225753Z-0fbd48c3) | Reopened 2026-08-24: E-FAST-M6 passed every equivalence gate (RS-20260824T010000Z-8f3e9b4f) at a realised 5.6x speedup (0.177 CPU-s/move in continuation duty), so a powered M=6 guardrail is now affordable; a full M=6 label corpus still costs thousands of CPU-hours and needs a P-SOL-3 design or a scale-out lease |
| 25-move rollout veto | −21,887 points per veto (finding-03) | A lower-variance rollout estimator |
| Refitting the leaf toward achievable clear rate | Monotone loss across six arms; fully-fitted vector −237,182 (finding-14) | A refit against remaining lifetime rather than achievable clears |
| Direct action override by the compact afterstate model | Override gate failed narrowly, and full training was harmful in one half-fold (RS-20260820T184500Z-63c0a8e2, RS-20260821T094500Z-1a7e3c55) | A materially larger student, or a stronger teacher than D1-continuation |
| Compact afterstate model reproducing D4's ordering | Top-1 0.375 against a 0.60 gate; exact D1 scores 0.486 (RS-20260821T104500Z-77d21e90) | A materially larger model; the claim is only established for compact evaluators |
The suite-h9-v1 scenario benchmark as a strength measure | Ranked policies backwards, Spearman −0.257 (finding-10) | A validated longer horizon; it remains usable as a diagnostic |
| A depth-5-distilled NNUE leaf refined by whole-game evolution inside the depth-3 search | Rejected at this budget: −106,964 paired on 64 held-out games (LB −146,580, floor 38,357), 14-0-50; the training-signal falsifier failed (0 of the last 10 generations above the fair control). Evolution did beat its own warm start by +35,375 (LB +16,899) (RS-20260903T025751Z-6577b33e) | A warm start that holds the teacher's ordering (the 0.441 top-1 probe is near the zero-leaf level), a larger teacher corpus than the 177 games the slow depth-5 teacher produced, or a continuation from the generation-60 population to see where the still-rising curve plateaus |
| The same leaf evolved 150 generations further with annealed mutation, until a preregistered plateau rule stopped it | Rejected at this budget: −68,441 paired on 64 fresh held-out games (LB −112,090, floor 43,193), 25-0-39; the continuation did beat the first run's candidate out of sample (+36,278, LB +9,085) and the curve levelled about 90,000 paired points below the fair control (RS-20260903T163321Z-733076b5) | A constant-sigma continuation from the same population to separate the plateau from the annealing; a warm start that holds the teacher's ordering; more games per candidate as sigma shrinks, so selection is not steered by noise |
| The same leaf evolved a further 150 generations with a 3.75x slower mutation decay, to test whether the annealing schedule caused the earlier plateau | Rejected at this budget, at the SAME plateau generation as before (149): −85,509 paired on 64 fresh held-out games (LB −128,482, floor 43,272); the slower decay produced a healthier training curve (rising population mean through all three 50-generation blocks, 26 of 150 best-candidate blocks above the fair control vs. 9 before) but its improvement over the immediately-prior candidate was NOT statistically confirmed (+13,572, LB −30,165, vs. the prior continuation's confirmed +36,278) (RS-20260904T090620Z-e5731bf0) | A much slower or non-exponential schedule, more games per candidate to see smaller differences as candidates converge, a larger population, or a warm start that holds the teacher's ordering rather than its value |
The two afterstate rows matter more than their tier suggests. The program's standing explanation for every failed learned policy was insufficient sibling coverage. The 2026-08-21 experiment supplied successor-closed coverage, every legal sibling, exact search-value labels, and completeness 1.0 — and the student still ranked worse than a one-ply exact search. That relocates the obstacle from the training data to the capacity of a compact board evaluator, and it makes "train a bigger student" a falsifiable next step rather than a hopeful one.
Open work
The next research step should be evidence-driven rather than another broad architecture sweep:
- Re-register any resumed SHA-locked experiment against the reorganized source tree and rerun cross-engine parity.
- Refit the search leaf against remaining lifetime, the quantity that correlates with score at r = 0.9995, rather than against achievable clears.
- Test whether student capacity is the binding constraint, since that is now the explicit claim on the table and the cheapest one to falsify.
- Test long-cycle features as bounded corrections to D4 before allowing them to control an entire game.
- Resolve the three recorded divergences between this simulator and the shipped game, which is an owner decision and not an agent decision.
Items 1, 4 and 5 are unchanged. Items 2 and 3 replace the AFBR-40 closure sequence, whose data-feasibility question was answered directly: closure was achieved and did not rescue the model.
AFBR-40 as originally proposed has no implementation, protocol, model, or measurements in this repository and must not appear in a list of attempted results. The action-free afterstate models that were built and measured on 2026-08-20/21 are separate, separately recorded, and are not AFBR-40.
For configurations and every retained entry point, use the
experiment index. For the unabridged chronological
record, use the experiment history. For alternatives and research
priority, use the strategy landscape and the staged
research roadmap. Future agents must use the standard
benchmark contract and machine-readable records under
research/; these do not retroactively upgrade the
historical evidence above.