Theories
Registered falsifiable claims, each with a mechanism and the criteria that would refute it.
The working record of the program: what is claimed, what was tested, what was measured, and the daily log. Agent-facing detail sits behind the bot-icon accordions.
On 2026-09-05 a learned evaluator beat the frozen fair leaf inside the same search for the first time, at screen tier. Lookup tables over every full row and full column of the board (ten million patterns per line), plus small windows, all conditioned on the rise clock, were trained by temporal-difference play on the Rust engine for four billion moves and placed as the leaf of the depth-3 seven-stratum fair search. On 256 never-read paired public-development games the tables scored 484,577 points and 140 moves against 314,438 and 93 for the identical search with the fair leaf: paired +170,139, bootstrap 95% lower bound +130,499, both halves positive, 167 wins to 89 (RS-20260905T215332Z-95d18a5a, approach page). On the same games the tables at depth 3 also beat the fair leaf at depth 4, the standing reference (377,803), by 106,775 with a lower bound of 62,574; that arm was diagnostic, not the preregistered comparator.
What this does and does not change. The evidence tier is SCREEN on public development seeds, opened once; it is not a qualification, no protected or final seed was opened, and the mean is under half the million-point target. The candidate was chosen at the best of twenty training-role validation points, so its screen margin is the honest number and its validation margin (+270,023) is not.
On 2026-09-06 the result replicated on a second, disjoint block. The same frozen tables (SHA-256 verified) played 512 never-read public-development games as the depth-3 leaf and scored 487,066 against 326,717 for the fair leaf in the same search: paired +160,349, bootstrap 95% lower bound +129,753, both halves positive, 330 wins to 182 (RS-20260906T040113Z-6ba93171). The same experiment trained a six-times-wider evaluator (2x4 and 4x2 window families added, 5.8 x 10^9 entries) on fresh training seeds until a 256-game validation margin stopped rising (a window-of-four plateau rule, which fired at 4.5 x 10^9 moves); that candidate passed the same gate on the same 512 games (481,869, paired +155,153, lower bound +126,819) and came out 5,196 behind the first tables with bounds from -40,535 to +29,158, so the preregistered scale verdict is inconclusive: no gain larger than about 35,000 in either direction. Played directly with no search the wide tables averaged 328,039 against 294,323 for the first tables on the same games, level with the fair leaf's depth-3 search; the extra scale improved the one-ply policy and left the leaf where it was. Both candidates beat the fair leaf at depth 4 on these games by more than 84,000.
Later on 2026-09-06 the first tables played one ply deeper. Unchanged and hash-verified, as the leaf of the reference depth-4 seven-stratum search they scored 516,155 against 465,675 for the same tables as the depth-3 leaf on 512 never-read public-development games from a fourth block: paired +50,479, bootstrap 95% lower bound +14,707, both halves positive, 284 wins to 228, a higher lower quartile (RS-20260906T171746Z-1623f833). On the same games the fair leaf's fourth ply was worth +49,599, and the per-game difference between the two depth steps is +881 with bounds from -40,932 to +42,867 (floor 42,550), so the preregistered interaction verdict is inconclusive: the learned leaf pays for depth about as well as the hand-written one, and no better or worse than a 512-game screen can see. At depth 4 the tables beat the fair leaf in the same search by 133,588 (lower bound 102,051); the depth-3 margin on this block was 132,707. The depth-4 search does about 36 times the depth-3 search's logical work per game. The first tables at depth 4 are now the candidate to carry forward; the next steps the record supports are a STANDARD-tier evaluation of that unchanged candidate on fresh development seeds, and a study of a leaf tuned for the search's use rather than for one-ply play (training on the boards the search evaluates, or removing the optimistic starting value from never-updated entries).
Late on 2026-09-06 a fourth experiment asked whether the leaf should change its mind as the board fills. A seed-free pass over the first run's saved accumulators found the frozen tables least trained on the fullest boards (0.98% of legal seven-high column patterns and 1.06% of five-or-more-disc top-row patterns ever updated, the touched full-line entries still at the optimistic starting value), so two repairs were preregistered against the frozen tables on a fifth never-read 512-game block (EX-20260906-ntuple-fill-conditioned-continuation-a9e5cbd3): five copies of every table keyed to a global fill bucket (occupied cells or tallest column), warm-started by copying the frozen tables into every bucket and trained on for up to two billion moves beside an unconditioned continuation as control; and two no-training edits that replace the 901,259,321 never-updated entries' starting value with zero or with the class mean. The occupied-cells candidate as the depth-3 leaf scored 506,494 against 485,455 for the frozen tables: paired +21,039 with bounds from -16,864 to +59,230 and a floor of 37,956, so the preregistered gate fails as a non-measurement; against the control continuation it is +15,093 and the control against the frozen tables +5,946, both inconclusive, and neither fill arm beat the control's best validation margin (RS-20260906T234914Z-a3fae1a9). The zeroed edit, with nothing trained, scored 514,897 at depth 3: paired +29,442 over the frozen tables with a lower bound of +5,813, changing only 194 of the 512 games, the one reading whose fixed verdict is supported. One ply deeper both repairs gave their gains back: the fill candidate at depth 4 was 22,631 behind the frozen tables (bounds -57,779 to +12,806) and the zeroed edit 10,592 behind (bounds -32,045 to +11,133), both inconclusive and both below their own depth-3 scores, while the frozen tables' fourth ply on these games was worth 36,500. The theory is assessed mixed, the frozen tables at depth 4 remain the candidate to carry forward, and the record now points at a leaf trained on the boards the search evaluates, and at a depth-aware price for never-updated entries, rather than at more buckets.
The frozen tables of both training runs, the three screens' per-game rows
and their analyses are published in the public research archive under their
run ids (https://data.drop7.dev/runs/<run-id>/ntuple-scale/..., digests in
the references; dataset records
DS-20260906-ntuple-scale-frozen-tables-ff977178
and
DS-20260906-ntuple-scale-wide-frozen-tables-c5eb52a6),
so another machine can reproduce the screens or continue training from them.
Through August the best dependable policy was fair depth-4 expectimax — a four-move look-ahead that treats the game's luck honestly. Even with its best chance model, its measured means sit under half of that target. This page shows where the gap is, what has been measured, and which directions the evidence has closed.
How score responds to search depth, under each of the two chance models:
All arms play the same 64-seed development cohort 0xa51d1000-0xa51d103f under corrected 17,000-point Hardcore scoring, except the depth-5 seven-stratum arm, which was stopped by decision after the first 32 seeds (0xa51d1000-0xa51d101f) and is drawn as its own one-point series so the two sample sizes are never joined by a line. Depth 2 was run only at seven strata. No confidence band is drawn on means; the paired contrasts with their bounds are in the companion bar figure. Evidence tier: development / public-development. The historical eight-game D3/D4 cohorts used 7,000-point scoring and are not shown.
Spec: web/content/figures/score-vs-depth.json · 3 source records · kind line
The same factorial as paired contrasts, which is the form the conclusions rest on:
Whisker is the one-sided 95% whole-game bootstrap lower bound; the upper end of the whisker is the point estimate itself (no upper bound is drawn). A contrast is significant when its lower bound clears zero: both stratum contrasts (7 minus 5 strata) do; neither depth contrast (5 minus 4 plies) does, and each sits below its own detection floor (47,052 at n=64, 107,988 at n=32). Cohort 0xa51d1000-0xa51d103f; the n=32 contrasts cover its first 32 seeds. Evidence tier: public-development.
Spec: web/content/figures/strata-5-vs-7.json · 1 source record · kind bar
The simulator and reference searches are mature enough to support reproducible research, but the strategy problem is unsolved. Corrected-score fair depth-4 expectimax is the strongest dependable reference found so far. Its average score across 64 games is 308,296 points, far below the goal of a one-million-point average.
Exploratory work on 2026-08-20/21 reproduced that reference on fresh seeds and completed the depth-by-chance-resolution factorial. Two results from it strongly suggest what to do next:
RS-20260821T205102Z-d89df4b5).
Choose experiments whose predicted effect exceeds the floor, or find a
lower-variance estimator than complete games.A fourth day, 2026-08-23, went to learned models and label economics — the reveal-construction probe in live play, the optimistic-state D0 gate, the leaf-cost NNUE student C0, and the P-SOL G0 label-semantics guardrail — and closed each of those four directions as tested; the research log for that day tells the story, and the closed-directions table below carries the headline numbers.
No candidate has qualified for the protected validation protocol. The frozen record marks both the protected and one-shot final cohorts as unopened. See the research status evidence for details.
Seven eras, one sentence each; every configuration and outcome below is tabled row by row in the experiment index.
The failure mode that recurs through eras 4-7 is easier drawn than described:
Mechanism diagram D1 of runs/RUN-20260823T191900Z-b9f8f80d/kimi-k3-figure-plan.md
("Diagrams (mechanisms, not charts)"). Hand-written, self-contained SVG; no measured
numbers are drawn.
The repeated failure mode named in docs/research/status.md §4 ("Learning
public-state values"): a model is trained on the outcome of the action that was
actually played, then deployment asks it to rank all seven legal columns — six
of which it never observed equally. This is failure class (iii) sibling
coverage / within-root discrimination of
docs/exploratory/audit-05-optimistic-curriculum.md §4, the largest class in the
census: 6 of 17 learned-policy experiments (experiments 5, 9, 11, 13, 14, 17).
docs/methodology.md,
information boundary).docs/research/status.md §7).fig-pt/fig-pop convention of the chart
generator): drill-downs on the root board (public-state definition and board
orientation), the labelled sibling, the unlabelled siblings, the deployment
fan, and the caption.docs/research/status.md "Directions closed",
included because it relocates the obstacle from coverage to capacity.docs/research/status.md §4 (sibling extrapolation paragraph) and §7;
"Directions closed" table (the successor-closed coverage row).docs/exploratory/audit-05-optimistic-curriculum.md §4 — class (iii)
definition and the counts table (6 of 17; experiments 5, 9, 11, 13, 14, 17).docs/methodology.md — the public-state definition.runs/RUN-20260823T191900Z-b9f8f80d/kimi-k3-figure-plan.md, D1.CSS variables with light-theme fallbacks (var(--fig-fg, #222),
var(--fig-muted, #888), var(--fig-accent, #2563eb), --fig-grid,
--fig-pop-bg, --fig-pop-border, --fig-cover, --fig-danger) so the diagram
renders standalone and adopts the console's dark theme inside .research-fig
(web/app/globals.css). Popovers are pure SVG/CSS (fig-pt + fig-pop), the
same convention as web/content/figures/score-vs-depth.svg; they need no
JavaScript.
Source: web/content/figures/diagrams/diagram-sibling-extrapolation.source.md
And every learned evaluator's ranking quality, against the exact search it tried to reproduce:
Panels differ across points and are named per category: the afterstate models and the fair D4 / fair D1 comparators are gated on the H40 D1-continuation panel; the D4-value student on a D4-ordering panel; the planner-distill student and its comparator on the fair-planner H5K256 panel; martingale-dual B0 and its comparator on the locked 477-root H200 panel; exact D1 / exact D2 on the historical panel of RS-20260821T104500Z-77d21e90. Compare within a panel, not across. Fair D1 pairwise and regret on the afterstate panel are not recorded, and exact D1 / exact D2 are recorded top-1 only, so those series points are omitted. Full-train pairwise 0.658 is from the record's summary prose; its per-half values are not recorded. Per-point labels are omitted to keep the dot grid readable; every popover carries the value and source. All eight students sit below their panel's fair-D4 comparator column. Evidence tier: pilot (afterstate and D4-value line); see each source for its panel definition.
Spec: web/content/figures/learned-ranking-metrics.json · 8 source records · kind dot
RS-20260821T192140Z-189fe392,
finding-16).The substitution is visible as a crossed pair of lines:
Cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring, 64 games per cell except the depth-5 seven-stratum arm, which was stopped by decision after the first 32 seeds (0xa51d1000-0xa51d101f) and is final. The stopped arm is drawn as its own dashed one-point series so the two sample sizes are never joined by a line (same convention as score-vs-depth). No per-mean confidence intervals are recorded anywhere for these arms; the bounds on the paired contrasts live in detection-floor-map. The d2s5 mean is the coordinator-supplied figure printed in finding-10's position-mode table and in the 2026-08-21 log ArmTable (the log path is not an accepted sourceRecord scheme, so the finding is cited). Evidence tier: development / public-development.
Spec: web/content/figures/depth-chance-factorial.json · 4 source records · kind line
And the budget view of the same arms shows why neither axis wins on its own:
Cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring, n=64 per arm except the d5s7 point (32 of 64 planned games, stopped by decision, final), which is drawn as its own dashed one-point series so the two sample sizes are never joined by a line. Cache capacity differs across arms (60,000 entries for the recorded depth-4 comparators, 200,000 for the depth-5 arms, auto-sized 960,695 / 346,921 for the new reveal-sampling arms); capacity provably cannot change play but work per move is not comparable across capacities (finding-16 Limitations; RS-20260821T205102Z-d89df4b5 limitations). Equal-work pair: d3 M=6 (4,244,020; 376,442) vs d4 M=1 (4,956,614; 398,498) is statistically indistinguishable (paired -22,056 [-89,867, +46,009], RS-20260821T192140Z-189fe392 pairedD3M6MinusD4M1). The x axis is linear: the plan's log-scale recommendation is not expressible by the generator, and the d5s7 point dominates the range. Evidence tier: development / public-development.
Spec: web/content/figures/score-vs-work-frontier.json · 4 source records · kind line
RS-20260821T205102Z-d89df4b5).Each contrast of the depth factorial, drawn against the smallest effect its cohort could have seen:
All six contrasts from RS-20260821T205102Z-d89df4b5, cohort 0xa51d1000-0xa51d103f, corrected 17,000-point Hardcore scoring; the n=32 contrasts cover the cohort's first 32 seeds (d5s7 arm stopped by decision, final). Whiskers are one-sided 95% percentile-bootstrap lower bounds (20,000 resamples, Mulberry32 domain 0xb0075eed); upper bounds are NOT recorded for these contrasts in the record and are omitted, not zero. Detection floor = 1.645 x sd / sqrt(n), the smallest true effect whose one-sided 95% bound would clear zero (metrics.detectionFloorDefinition). The d4s7-d3s7 lower bound is carried as +26,468 from metrics.bootstrapVersusNormalApproximation; status.md and the 2026-08-21 log print +26,605, which is prose drift — the record and finding-05's interaction table both say +26,468. Evidence tier: public-development.
Spec: web/content/figures/detection-floor-map.json · 1 source record · kind bar
suite-h9-v1 scenario suite, a depth-3 screening run, and an eight-move
achievable-clear label. Screen at the depth you intend to deploy, or do not
screen.Each entry rejects the exact configuration tested, at development or pilot tier. None of them is a proof that the underlying idea is impossible, and each names what would reopen it.
The screen and confirmation deltas behind these closures, with their recorded bounds:
Whole-game paired deltas against each experiment's reference arm, corrected 17,000-point Hardcore scoring. Bounds are one-sided 95% bootstrap lower bounds; upper bounds are drawn only where recorded (the three reveal-sampling contrasts, CMA-ES, both survival-instinct arms). finding-03, finding-14, both learned-leaf arms and A900 record no upper bound — the whisker is absent there, not zero. finding-05's confirmation lower bound is printed +47,457 in the finding and +47,446.8 in RS-20260821T205102Z-d89df4b5; the figure uses the record (+47,447). A900's rejection rests on flat flow statistics and opposite-sign halves, not the score delta, which sits inside its 30,957 detection floor (RS-20260823T131226Z-16564ed9). Cohorts differ per bar (n carried per point; the survival-instinct arms share 128 fresh seeds, A900 256 fresh seeds); this chart assembles separate experiments, it does not pool them. Evidence tier: development / public-development screens.
Spec: web/content/figures/screen-deltas-with-bounds.json · 8 source records · kind bar
The closures themselves, one dot per direction, each drawn against the detection floor its record states — the geometric difference between "rejected" and "not measurable":
One dot per closed direction whose headline result is a points number, copied verbatim from its record; whiskers are the recorded bounds — one-sided 95% bootstrap lower bounds where that is all the record carries (both depth rows, reveal-construction A900, rollout veto, leaf refit), two-sided 95% where recorded (reveal M=2, CMA-ES, survival-instinct STRICT), and none for terminal utility, whose zero is EXACT (byte-identical games at four magnitudes from -1,000,000 to -50,000,000, 0-64-0; no statistical bound exists or is needed). Detection-floor markers (1.645*sd/sqrt(n)) are drawn only where the record carries one; no floor is recorded for reveal M=2, the rollout veto, or the leaf refit — markers omitted, not zero. Every closure rejects the exact tested configuration at development/pilot tier; none proves the underlying idea impossible, and each row's 'reopens if' condition stays in docs/research/status.md. The plan's 'depth beyond four plies' row is charted as its two recorded contrasts (s7 n=32 stopped/final, s5 n=64). Not chartable on a points axis, recorded here verbatim instead: compact afterstate model reproducing D4's ordering — top-1 0.375 against the 0.60 gate (RS-20260821T104500Z-77d21e90); the suite-h9-v1 benchmark as a strength measure — Spearman -0.257 (finding-10). status.md's table gained three further non-points closures after this figure's plan was written, also not chartable here: leaf-cost NNUE student top-1 0.296/0.3011 by half-fold against the 0.60 gate (RS-20260823T194142Z-946e3cd1), H-pool optimistic states tau -0.959 [cluster bootstrap -3.069, -0.390] (RS-20260823T205143Z-ead14c9d), and P-SOL G0 cheap-label proxy mean tau 0.370, LB95 0.283 (RS-20260823T225753Z-0fbd48c3). Cohorts differ per point (n on every marker); this chart assembles separate experiments, it does not pool them.
Spec: web/content/figures/closed-directions-map.json · 8 source records · kind forest
| Direction | Result | Reopens if |
|---|---|---|
| Search depth beyond four plies | Not measurable: +23,367 (7 strata, n=32) and −8,624 (5 strata, n=64), both inside their detection floors (RS-20260821T205102Z-d89df4b5) | A design that can resolve sub-50k effects, or a mechanism predicting an effect above the floor |
| Stacking reveal sampling on top of the fourth ply | −41,950 at M=2 (RS-20260821T192140Z-189fe392) | A wider dose is tested; only 28.6% joint coverage was affordable at depth 4, and the dose that worked at depth 3 was 85.7% |
| Pricing the same-wave double hit on a solid gray (reveal construction) as a leaf term | Corpus: partial r −0.044 with 60% of live setups uncollected; in play, +3,204 over 256 fresh paired games, inside its floor (RS-20260823T131226Z-16564ed9) | A term that raises reveals per move at all; or a per-root counterfactual showing the uncollected setups were worth collecting |
| Harsher terminal (death) utility | Saturated: byte-identical play at every magnitude past the current value (finding-04) | Never, for this parameterization |
| A leaf-cost NNUE student reproducing D4's within-root ordering from successor-closed D4 values | Top-1 0.296/0.301 by half-fold against the 0.60 gate (RS-20260823T194142Z-946e3cd1) | A different architecture class at leaf cost, e.g. a cross-sibling set ranker; the self-play loop's leaf form stays blocked and its redesign is root-prior shaped |
| Optimistic states with fair labels (H-pool, the salvageable core of the oracle curriculum) | Stage D0: tau = −0.959 [−3.069, −0.390], with a degenerate denominator — 63/64 oracle games hit the 500-move cap (RS-20260823T205143Z-ead14c9d) | A D0 rerun with an uncapped horizon shows tau ≥ 0.25 and the oracle's action at least matching D4's under fair futures |
| Cheap M=1 continuation labels standing in for D3 N7M6 label semantics (P-SOL G0) | Guardrail kill: within-root orderings agree at only mean tau 0.370 (LB95 0.283) on 6 CRN-matched roots; the divergence is the reveal quadrature, not the engine (RS-20260823T225753Z-0fbd48c3) | Reopened 2026-08-24: E-FAST-M6 passed every equivalence gate (RS-20260824T010000Z-8f3e9b4f) at a realised 5.6x speedup (0.177 CPU-s/move in continuation duty), so a powered M=6 guardrail is now affordable; a full M=6 label corpus still costs thousands of CPU-hours and needs a P-SOL-3 design or a scale-out lease |
| 25-move rollout veto | −21,887 points per veto (finding-03) | A lower-variance rollout estimator |
| Refitting the leaf toward achievable clear rate | Monotone loss across six arms; fully-fitted vector −237,182 (finding-14) | A refit against remaining lifetime rather than achievable clears |
| Direct action override by the compact afterstate model | Override gate failed narrowly, and full training was harmful in one half-fold (RS-20260820T184500Z-63c0a8e2, RS-20260821T094500Z-1a7e3c55) | A materially larger student, or a stronger teacher than D1-continuation |
| Compact afterstate model reproducing D4's ordering | Top-1 0.375 against a 0.60 gate; exact D1 scores 0.486 (RS-20260821T104500Z-77d21e90) | A materially larger model; the claim is only established for compact evaluators |
The suite-h9-v1 scenario benchmark as a strength measure | Ranked policies backwards, Spearman −0.257 (finding-10) | A validated longer horizon; it remains usable as a diagnostic |
| A depth-5-distilled NNUE leaf refined by whole-game evolution inside the depth-3 search | Rejected at this budget: −106,964 paired on 64 held-out games (LB −146,580, floor 38,357), 14-0-50; the training-signal falsifier failed (0 of the last 10 generations above the fair control). Evolution did beat its own warm start by +35,375 (LB +16,899) (RS-20260903T025751Z-6577b33e) | A warm start that holds the teacher's ordering (the 0.441 top-1 probe is near the zero-leaf level), a larger teacher corpus than the 177 games the slow depth-5 teacher produced, or a continuation from the generation-60 population to see where the still-rising curve plateaus |
| The same leaf evolved 150 generations further with annealed mutation, until a preregistered plateau rule stopped it | Rejected at this budget: −68,441 paired on 64 fresh held-out games (LB −112,090, floor 43,193), 25-0-39; the continuation did beat the first run's candidate out of sample (+36,278, LB +9,085) and the curve levelled about 90,000 paired points below the fair control (RS-20260903T163321Z-733076b5) | A constant-sigma continuation from the same population to separate the plateau from the annealing; a warm start that holds the teacher's ordering; more games per candidate as sigma shrinks, so selection is not steered by noise |
| The same leaf evolved a further 150 generations with a 3.75x slower mutation decay, to test whether the annealing schedule caused the earlier plateau | Rejected at this budget, at the SAME plateau generation as before (149): −85,509 paired on 64 fresh held-out games (LB −128,482, floor 43,272); the slower decay produced a healthier training curve (rising population mean through all three 50-generation blocks, 26 of 150 best-candidate blocks above the fair control vs. 9 before) but its improvement over the immediately-prior candidate was NOT statistically confirmed (+13,572, LB −30,165, vs. the prior continuation's confirmed +36,278) (RS-20260904T090620Z-e5731bf0) | A much slower or non-exponential schedule, more games per candidate to see smaller differences as candidates converge, a larger population, or a warm start that holds the teacher's ordering rather than its value |
The two afterstate rows matter more than their tier suggests. The program's standing explanation for every failed learned policy was insufficient sibling coverage. The 2026-08-21 experiment supplied successor-closed coverage, every legal sibling, exact search-value labels, and completeness 1.0 — and the student still ranked worse than a one-ply exact search. That relocates the obstacle from the training data to the capacity of a compact board evaluator, and it makes "train a bigger student" a falsifiable next step rather than a hopeful one.
The next research step should be evidence-driven rather than another broad architecture sweep:
Items 1, 4 and 5 are unchanged. Items 2 and 3 replace the AFBR-40 closure sequence, whose data-feasibility question was answered directly: closure was achieved and did not rescue the model.
AFBR-40 as originally proposed has no implementation, protocol, model, or measurements in this repository and must not appear in a list of attempted results. The action-free afterstate models that were built and measured on 2026-08-20/21 are separate, separately recorded, and are not AFBR-40.
For configurations and every retained entry point, use the
experiment index. For the unabridged chronological
record, use the experiment history. For alternatives and research
priority, use the strategy landscape and the staged
research roadmap. Future agents must use the standard
benchmark contract and machine-readable records under
research/; these do not retroactively upgrade the
historical evidence above.
Registered falsifiable claims, each with a mechanism and the criteria that would refute it.
Preregistered protocols: candidate, comparator, cohort, metrics and gate, fixed before the data is read.
Recorded outcomes, each carrying its run validity, scientific outcome and evidence tier.
A dated account of what was tried each day, including what did not work.
Measuring instruments, harnesses and model probes: the directories that measure rather than play.
The methodology, the benchmark contract, the ledger and the agent contracts the research is bound to.