Offline gate: calibrated top-two near-tie override of fair D4 by the frozen afterstate model
At roots where fair D4's top two actions are near-tied (Q gap <= 500 points, the bottom ~37% of the top-two gap distribution measured on already-read development labels), the frozen iteration-3 distributional afterstate model can identify the better action more reliably than D4's own ordering, because near-ties are where D4's 4-ply estimates are most noise-sensitive while the model's calibrated H40 outcome distributions carry independent long-horizon information.
On this page
- Created
- Updated
The third pilot established that the learned board-scorer cannot replace the four-move search as the thing that picks the move. This experiment asks a much smaller question: could it still be useful in the one situation where the search is least sure of itself?
Often fair D4 rates its top two columns almost identically. Those near-ties are where a four-move estimate is most at the mercy of which futures it happened to sample. The candidate here is unchanged fair D4 with one conservative addition: at a near-tie — the top two within 500 points, a threshold fixed from already-read data before this experiment existed — consult the frozen third-iteration network, and swap to the search's second choice only if the network's advantage survives a resampling test. The comparator is unchanged fair D4. Nothing else about either player differs, and again no games were played: both were graded on how well they ordered columns at fresh held-out positions.
"Pass" required the override to cut average regret by at least 0.01 in each half of the held-out set, to fire on at least 5% of eligible positions, and to reproduce byte-identically across two runs.
What happened. A valid run with a fail outcome — but a narrow one, and
the record says so explicitly. The rule fired often (38.3% of positions were
near-tied; the override triggered on 37.0% of those), it was deterministic, and
it improved regret in both halves: by 0.0351 in one and 0.0075 in the other.
The frozen bar was 0.01 in each, so the second half missed by a quarter of a
percentage point and the gate fails. The record's own reading is that the
mechanism works as designed and that this is the first preregistered held-out
test here where a learned model's intervention improved on fair D4 at all, with
the checkpoint's ranking quality as the bottleneck. It also warns, in the
limitations, that the miss "should be read as a narrow failure, not as evidence
of no effect."
The run was valid; the outcome was fail, at the pilot (a small run to find bugs and project cost, not a strength claim) level. Of 7 preregistered checks, 6 passed and 1 failed.
The calibrated top-two near-tie override of fair D4 by the frozen iteration-3 afterstate model FAILED its frozen gate by a narrow margin, with genuinely positive signal. On 2,689 fresh held-out roots, 1,030 (38.3%) were near-tied at the frozen 500-point threshold and the override fired on 37.0% of those. On eligible roots the override reduced mean normalized regret versus unchanged D4 in both half-folds (half1 0.2484 vs 0.2835, +0.0351; half2 0.2343 vs 0.2418, +0.0075; pooled +0.0214), but the frozen criterion required at least +0.01 in EACH half-fold, and half2 fell short. On the 84 decisive eligible roots the improvement was larger (+0.0426: 0.0880 vs 0.1306). Label stability passed (decisive Spearman 0.814), model quantile calibration passed (0.861), the gate is deterministic (byte-identical repeated reports), and the corpus is successor-closed. Interpretation: the conservative-override MECHANISM works as designed (fires often, calibrated, positive in both halves) and this is the first preregistered held-out test in this repository where a learned model's intervention improved on fair D4 at all; the CHECKPOINT's ranking quality is the bottleneck, consistent with the model being undertrained (11 of 20 epochs) and trained on a weak D1-continuation teacher. Per the frozen failure action, no further override variants of this same checkpoint are opened; a retry requires new evidence such as a fully trained or stronger-teacher model under the same frozen rule.
Technical recordLimitations recorded with the result
- The evaluation target is the H40 phase-greedy-D1 continuation outcome - the same label family the model was trained to predict; D4 is compared against the same target, but the offline gate cannot measure real gameplay interaction, which only a SCREEN tier can.
- The model was undertrained (11 of 20 epochs) and used a weak D1-continuation teacher; both are documented as the likely bottleneck and motivate any retry.
- The near-tie threshold (500) and regret margin (0.01) were frozen choices; the half2 miss (0.0075) is close to the margin and the result should be read as a narrow failure, not as evidence of no effect.
- Single machine profile; FP32 on the shared-memory iGPU.
What it closes: no further override variants of this checkpoint without new evidence. What it leaves open: the same frozen rule applied to a fully trained model, or one taught by a stronger continuation player. And what it cannot speak to at all is real play — an offline ranking test cannot measure how these choices interact over a whole game, which only a paired gameplay tier could.
Technical recordThe registered protocol
- Hypothesis
- At roots where fair D4's top two actions are near-tied (Q gap <= 500 points, the bottom ~37% of the top-two gap distribution measured on already-read development labels), the frozen iteration-3 distributional afterstate model can identify the better action more reliably than D4's own ordering, because near-ties are where D4's 4-ply estimates are most noise-sensitive while the model's calibrated H40 outcome distributions carry independent long-horizon information. The override policy (D4, plus swapping to the second action only when the model's paired 256-scenario advantage has a positive 95% bootstrap lower bound) will show lower mean normalized regret than unchanged D4 on eligible fresh held-out roots.
- Arms
Arm Name Entry point Manifest Candidate d4-toptwo-override approaches/afterstate-learning/distributional-afterstate/override-gate.py– Comparator fair-d4 approaches/fair-expectimax/reference/fair-only-depth4.cppresearch/benchmarks/baselines-v1.json- Classification
- algorithmic
- Information boundary
- public-policy
- Benchmark tier
- PILOT
- Lifecycle
- completed
- Primary metric
- Mean normalized regret of the override policy versus unchanged fair D4 on override-eligible fresh held-out roots, against the scenario-mean H40 outcome ranking (K=256, phase-greedy D1 continuation), reported per origin-hash half-fold.
- Secondary metrics
- override rate among eligible roots (coverage)
- eligible-root count and near-tie rate
- whole-set mean normalized regret, top-1, and pairwise accuracy of the override policy versus D4
- the same metrics conditioned on decisive roots (spread > 20,000)
- label stability: decisive-root scenario-half (128 vs 128) Spearman
- quantile calibration of the frozen model on the fresh afterstates
- determinism: two identical gate runs must produce byte-identical reports
- Statistical unit
- whole-origin
- Uncertainty method
- Two deterministic held-out half-folds by origin-game hash (mix32 of the origin seed); the primary comparison must move the same direction in both halves. The override confidence rule itself is a paired bootstrap over the 256 aligned scenarios (1000 resamples, deterministic domain 0x4f565252 keyed on the public root hash only).
- Data role
- public-development
- Seed leases
SL-20260820T083000Z-5da70000
- Dataset references
DS-20260820-afterstate-h40-k256-9d31c47a
- Whole-origin split
- yes
- Reuse disclosure
- The model is the frozen iteration-3 checkpoint (trained only on seeds 0x5da70000-0x5da700ff, training-role). The near-tie threshold 500 was fixed from already-read iteration-2/3 comparator labels before this experiment's data existed. Evaluation roots come from fresh seeds 0x5da70300-0x5da7033f (64 origins), read for the first time by this experiment's single gate evaluation. No protected or final data is read.
- Pass criteria
- Corpus is successor-closed: every non-trivial harvested root has labels for every legal sibling under all 256 scenarios.
- Label stability floor: decisive-root scenario-half Spearman >= 0.5; below this the outcome is inconclusive.
- On eligible roots, override-policy mean normalized regret <= D4 mean normalized regret - 0.01 in EACH half-fold.
- Override rate >= 5% of eligible roots (a rule that never fires proves nothing).
- Nominal 76.5% quantile interval coverage of the frozen model on fresh afterstates within [0.66, 0.87] (sanity that the model is in-support).
- The gate script is deterministic: two runs produce byte-identical reports.
- All existing CHECK-tier self-tests pass before any label is inspected.
- On pass
- Preregister a SCREEN gameplay experiment: 32 paired public-development games, native fair D4 with the frozen override rule (model inference over stratified afterstate samples at near-tied roots only) versus unchanged fair D4, with per-game result rows and the standard cohort summary.
- On fail
- Record a valid negative (or inconclusive) result. A failure here closes the direct-override use of this model: no further override variants of the same checkpoint without new evidence.
- Gate fixed before controlled data
- yes
- Resources
Wall seconds 10800 CPU threads 24 Max host bytes 34359738368 Max GPU bytes 21474836480 GPU devices gfx1151 - Stop conditions
- Stop on any rules, information-boundary, legality, determinism, or parity failure.
- Stop corpus generation after 1 hour wall.
- Stop fair-D4 comparator labeling after 1 hour wall.
- Stop gate evaluation after 1 hour wall.
- Stop immediately on out-of-memory or on host available memory falling below 16 GiB.
- Stop on any read of a seed outside the named lease.
- Expected artifacts
runs/<run-id>/corpus-d/*.ndjson K=256 fresh held-out corpus with content manifestruns/<run-id>/d4-labels-d/*.tsv comparator rankingsruns/<run-id>/gate/override-gate-report.json (two byte-identical copies from repeated runs)research/datasets/<dataset-id>.json content-addressed corpus manifestresearch/results/<result-id>.json scientific assessment
- Amendments
Timestamp Before controlled data Reason 2026-08-20T18:45:00Z no lifecycle advanced preregistered -> completed after the run and result records were written; protocol content otherwise unchanged.
Technical recordResults recorded against this protocol
The calibrated top-two near-tie override of fair D4 by the frozen iteration-3 afterstate model FAILED its frozen gate by a narrow margin, with genuinely positive signal. On 2,689 fresh held-out roots, 1,030 (38.3%) were near-tied at the frozen 500-point threshold and the override fired on 37.0% of those. On eligible roots the override reduced mean normalized regret versus unchanged D4 in both half-folds (half1 0.2484 vs 0.2835, +0.0351; half2 0.2343 vs 0.2418, +0.0075; pooled +0.0214), but the frozen criterion required at least +0.01 in EACH half-fold, and half2 fell short. On the 84 decisive eligible roots the improvement was larger (+0.0426: 0.0880 vs 0.1306). Label stability passed (decisive Spearman 0.814), model quantile calibration passed (0.861), the gate is deterministic (byte-identical repeated reports), and the corpus is successor-closed. Interpretation: the conservative-override MECHANISM works as designed (fires often, calibrated, positive in both halves) and this is the first preregistered held-out test in this repository where a learned model's intervention improved on fair D4 at all; the CHECKPOINT's ranking quality is the bottleneck, consistent with the model being undertrained (11 of 20 epochs) and trained on a weak D1-continuation teacher. Per the frozen failure action, no further override variants of this same checkpoint are opened; a retry requires new evidence such as a fully trained or stronger-teacher model under the same frozen rule.
- ✓Successor-closed corpus (all legal siblings x 256 scenarios) — observed: 4,858,880 rows over 2,793 roots; completeness 1.0
- ✓Label stability floor: decisive-root scenario-half Spearman >= 0.5 — observed: 0.8144
- ✕Eligible-root override regret <= D4 regret - 0.01 in EACH half-fold — observed: half1 +0.0351 (passes); half2 +0.0075 (below the frozen 0.01 margin); pooled +0.0214
- ✓Override rate >= 5% of eligible roots — observed: 37.0%
- ✓Quantile interval coverage within [0.66, 0.87] — observed: 0.8606
- ✓Gate script deterministic (byte-identical repeated reports) — observed: two runs byte-identical after moving wall time out of the report
- ✓All existing CHECK-tier self-tests pass before any label is inspected — observed: SELFTEST OK (9 checks)
Technical recordRecorded metrics
- The evaluation target is the H40 phase-greedy-D1 continuation outcome - the same label family the model was trained to predict; D4 is compared against the same target, but the offline gate cannot measure real gameplay interaction, which only a SCREEN tier can.
- The model was undertrained (11 of 20 epochs) and used a weak D1-continuation teacher; both are documented as the likely bottleneck and motivate any retry.
- The near-tie threshold (500) and regret margin (0.01) were frozen choices; the half2 miss (0.0075) is close to the margin and the result should be read as a narrow failure, not as evidence of no effect.
- Single machine profile; FP32 on the shared-memory iGPU.
Agent contextHow to extend this record
To add a reader-facing explanation, write web/content/research/EX-20260820-d4-toptwo-override-gate-0bdb39a1.mdx; it renders above this record on the next request. The registered protocol itself is in the technical record above.
Record file: research/experiments/EX-20260820-d4-toptwo-override-gate-0bdb39a1.json, validated against research/schemas/experiment-v1.schema.json. Protocol hash: f23b2e04102038d5c281757e2ba72df689c52f8eef51d95a8c2b3502eccb63a7.
Registered by OpenCode / moonshotai/Kimi-K3.