Approaches

Expectimax search

Look a few moves ahead, take the best column on your own turns, and average over the discs the game might deal.

Fair expectimax reference (D3/D4)

The reference search looks four moves ahead, averages the sampled chance outcomes, and uses a hand-tuned board evaluator at the bottom.

completedrecorded

Expectimax searchfeatured

Separate reveal and next-disc sampling

Separates next-disc and hidden-value samples inside search. The change is worth about one extra move of look-ahead.

completedreproduced

Expectimax search

Exact search

Three attempts add independent chance estimates, depth, and voters to a search. None repairs a biased board evaluation.

rejectedrecorded

Expectimax search

Looking five moves ahead

Three attempts add a fifth move for selected columns, at row rises, or with fewer chance samples. None paid off.

rejectedrecorded

Expectimax search

Seven chance samples instead of five

The next disc has seven equally likely values, but the reference samples only five. This tests all seven.

rejectedrecorded

Expectimax search

Sparse expectimax

Look several moves ahead, but instead of considering every disc the game might deal, take a small fixed handful of representative ones, and always finish the depth you promised.

completed

Expectimax search

Turning the reference search's constants into dials

Makes death penalty, search depth, and chance samples configurable to test which settings improve survival.

completedreproduced

Heuristic evaluation

Score a board with a hand-written sum of visible traits, then play the column whose board scores highest.

Read the primer →

Heuristic evaluationfeatured

Paying the search for clears and reveals

Rewards clears and reveals inside search to test whether faster board flow improves survival.

rejectedrecorded

Heuristic evaluationfeatured

The fair leaf evaluator

The hand-written board evaluator every search in this family uses at the bottom of its look-ahead, and the tuning bench its coefficients came from.

completedrecorded

Heuristic evaluation

Accessible energy

Turn the board properties that separate a future-reading planner's positions from ordinary ones into a handful of readable numbers, and let a legal search use them.

rejectedrecorded

Heuristic evaluation

Adding a rise-cycle residual to the evaluator

Adds a small rise-cycle term to board evaluation and preserves the fitted weights for later work.

completedrecorded

Heuristic evaluation

Bringing back the old placement rewards

Restores immediate move rewards inside deeper search, then tests a reduced two-term version.

rejectedrecorded

Heuristic evaluation

Chain-reveal leaf substrate

Adds seven reveal-construction terms (aligned double hit, chain-to-crack, entombed high, and their danger-gated forms) on top of the frozen fair leaf through a memoised, generated search whose zero-weight arm is gated bit-identical to the unchanged search.

rejectedrecorded

Heuristic evaluation

Edge priority

Turn two claims from an experienced player into measurable evaluator changes by attacking high covered discs and distrusting edge columns.

Heuristic evaluation

Phase horizon

Judge a board by how it will look when the next row rises. The same board is more dangerous one drop from a rise than five drops away.

rejected

Heuristic evaluation

Tunneling

Dig a channel down beside the tall side of the board so that the dangerous gray discs high up can actually be reached and opened.

completed

Heuristic evaluation

Virtual ignition

Before choosing a column, imagine dropping a few imaginary discs onto the board to see which arrangements would go off like a firework, and prefer the boards that would.

rejected

Q-learning and value learning

Learn from past games how much each column is worth, so the player can rank moves without searching ahead.

Read the primer →

Q-learning and value learningfeatured

Denoised public value

Instead of asking one future what a board was worth, simulate 32 futures from it and average them.

completedrecorded

Q-learning and value learningfeatured

Klein-Friedmann linear Q-learning

Score each column with six hand-made numbers about what the drop does right now, learn the six weights from experience, and find out how much of Drop7 that can see.

completedreproduced

Q-learning and value learning

Double-DQN and continuation

Use textbook reinforcement learning to score each column from experience, reward survival, and add a short look-ahead.

completed

Q-learning and value learning

Long outcomes

Play each column forward for twenty-five moves under shared imagined futures, then learn from those outcomes. All five models ranked moves worse than a cheap exact search.

rejectedrecorded

Q-learning and value learning

Monte Carlo state value

Label every board with how many more moves the game it came from actually lasted, then only leave the safe move when the learned numbers agree it is better.

rejectedrecorded

Q-learning and value learning

One-ply Q as a pruning prior and a refit leaf

Fit a linear action value to the depth-4 search's own sibling values, then use it not to play but to decide which siblings the search may skip, and separately refit the leaf to the search's own value.

completedreproduced

Q-learning and value learning

Panel value

Builds complete training data by playing every legal column through the same imagined futures, then learns from it. Both models still ranked columns worse than the four-move search.

rejected

Q-learning and value learning

Rainbow-style Q-learning over patterns

Learn a score for every column with Atari-style training and a pattern table instead of a deep network. It beat random play, then lost to a one-move search.

rejectedrecorded

Q-learning and value learning

Sibling network on the 200-move panel

Start a network as an exact copy of the reference search, train it on a locked panel where every legal column was measured, and watch it get worse.

rejected

N-tuple networks

Learn the value of a board from small cell patterns, each with its own lookup table, trained over millions of self-play moves.

Read the primer →

N-tuple networksfeatured

Fixing the learning rule, and conditioning on the rise clock

Fixes an n-tuple update bug and adds rise-cycle context. The corrected learner remains too weak.

rejectedrecorded

N-tuple networksfeatured

N-tuple value with a two-rise look-ahead

The family's largest run trained a pattern evaluator on 50 million moves and searched two row rises deep. Search made it worse than trusting the network alone.

rejectedrecorded

N-tuple networks

Blending the learned value with the hand-written one

Mix the learned board value with the hand-written heuristic and see whether the combination beats either alone. It did not.

rejectedrecorded

N-tuple networks

Learning from every column, not just the one played

An n-tuple value learner whose every update looks at all seven legal columns under the same imagined luck, so that exploration noise cannot leak into what it learns.

rejected

N-tuple networks

Row and column lookup tables, learned from a billion moves

A lookup-table evaluator that reads every full row and full column of the board as one pattern, learns its numbers by temporal-difference play on the Rust engine, and replaces the hand-written leaf inside the depth-3 fair search.

completed

NNUE evaluators

A small neural network, cheap enough to run at every leaf of a search, that judges a board from its cells.

Read the primer →

NNUE evaluatorsfeatured

A learned survival estimate inside the reference search

Blends a learned survival estimate into the reference board evaluator and tests it under sampled and exact chance handling.

completedreproduced

NNUE evaluatorsfeatured

Structured NNUE

Give every cell a learned description and predict the game's remaining lifetime. Good predictions still did not produce good choices.

rejectedrecorded

NNUE evaluators

Chance-state NNUE

Judge the board at the moment just before the next disc is dealt, so the value never has to be split seven ways.

rejected

NNUE evaluators

Learned-guidance search

Let a small learned board evaluator decide where a deeper search should spend its time, while keeping the exact search as a safety net.

rejectedrecorded

Policy gradients

Train a network that picks columns directly, nudging it toward the choices that led to longer games.

Read the primer →

Policy gradientsfeatured

A PyTorch policy network, cloned then trained by playing

A small convolutional network copies a two-move search, then improves through 16,384 games. It finished about 40% short of its teacher.

rejectedrecorded

Policy gradientsfeatured

Learning a policy with explicit safety constraints

Learn a small correction to a simple search under hard safety limits. Every limit failed, so the run stopped before gameplay testing.

rejectedrecorded

Policy gradients

Copying the one-move search, in C++

Teach a policy network to imitate a simple exact search before self-play. It never imitated well enough to start.

rejectedrecorded

Policy gradients

Nudging a one-move search, from easy and hard starting boards

A learned correction trains on fresh games and difficult mid-game positions. It improved the simple search slightly but stayed far below the reference.

rejected

Evolutionary optimisation

Tune a player's weights by playing complete games, keeping the settings that scored best, and repeating.

Read the primer →

Evolutionary optimisationfeatured

An evolved NNUE leaf, distilled from a depth-5 teacher

A compact neural leaf evaluator for the depth-3 fair search, warm-started by distilling a depth-5 teacher's sibling-complete root values and then refined by an evolutionary search whose fitness is whole-game score itself.

rejectedrecorded

Evolutionary optimisationfeatured

Evolution

Let a search tune board-feature weights by playing thousands of games, keeping the best weights, and repeating.

rejectedrecorded

Evolutionary optimisation

Direct public policy

Skip the value model. Score each column with 25 weighted features and tune the weights against whole games.

rejected

Evolutionary optimisation

Evolved public policy

Give the weight search a larger phase-aware evaluator and optimise complete games directly, without imitation or proxy labels.

rejectedrecorded

Evolutionary optimisation

Evolving the leaf weights at the depth they are used

Evolves the reference evaluator on complete games, then tests the frozen winner once on unseen games.

Evolutionary optimisation

Machine-tuning the board evaluator

Let an optimiser adjust eight coefficients of the search's board evaluator by playing complete games, then freeze the winner and test it on games it never saw.

rejectedrecorded

Monte Carlo tree search

Grow the look-ahead only where it seems promising, guided by quick simulated playouts.

Read the primer →

Monte Carlo tree searchfeatured

Observable-state MCTS

Grow the look-ahead by simulation, but only ever let the search know what a player would actually be able to see.

rejectedrecorded

Monte Carlo tree search

PUCT

A complete but unrun tree search that starts from a prior about which columns are worth trying.

Monte Carlo tree search

TypeScript MCTS lab

A small Monte Carlo tree search you can run in a terminal and watch one decision at a time.

rejected

Rollouts and policy iteration

Judge a column by playing many moves forward with a fast policy, and let that overrule the search when the evidence is strong.

Read the primer →

Rollouts and policy iterationfeatured

Public rollout policy iteration

The cheap version uses fifteen shared futures of fifty moves for one improvement step over a simple search.

rejectedrecorded

Rollouts and policy iterationfeatured

Rollout veto

Keep the depth-4 search's move unless replaying every column forward for twenty-five moves says, across all seven imagined futures, that another one is better.

recorded

Rollouts and policy iteration

Judging a move by playing the game out

Score each column by simulating the next eight to twenty-four moves with a fast policy on identical imagined futures, instead of by looking a fixed few moves ahead.

rejectedrecorded

Rollouts and policy iteration

Letting a long look-ahead overrule the search

Plays every candidate 25 moves forward and overrides the reference only when one column is clearly better.

rejectedreproduced

Rollouts and policy iteration

Public regenerative B0

Eliminate weak columns cheaply, test the survivors against the reference, and override it only with strong evidence.

rejectedrecorded

Rollouts and policy iteration

Public survival rollout

Judge each column by survival over thirty-one shared futures of one hundred moves each.

rejected

Rollouts and policy iteration

Rollout

Judge a column by playing twenty-five more moves under several imagined disc streams instead of scoring only the resulting board.

rejected

Rollouts and policy iteration

Structural terminal veto

Keeps the four-move search in charge unless a long simulation favors its second choice. Two independent tests still finished behind the unchanged search.

rejected

Rollouts and policy iteration

Terminal policy iteration

The full-strength version uses 255 shared futures of 200 moves and changes the reference move only with 99% confidence in both score and lifetime gains.

rejected

Rollouts and policy iteration

Terminal rollout

Compare columns by playing each one to the end of the game. This faithful version was unaffordable.

Oracles, teachers and distillation

Let a planner that can see the future label positions, then train a player that cannot see the future to imitate it.

Read the primer →

Oracles, teachers and distillationfeatured

Compressing a legal planner into a network

Labels every legal move with an expensive public-information planner, then trains a small network to imitate it.

rejectedreproduced

Oracles, teachers and distillation

D4 root-Q clone

Train a tiny fast network to copy how the expensive depth-4 search orders the seven columns.

rejectedrecorded

Oracles, teachers and distillation

Depth-4 distillation

Tests whether a small model can imitate the reference and finds that a cheap exact search does better.

rejectedrecorded

Oracles, teachers and distillation

Expert iteration from the depth-4 search

Learn from the strongest search, play with the result, and repeat. After eight rounds, the learned policy was still less than half as good as its teacher.

rejectedrecorded

Oracles, teachers and distillation

Learning what a long-lived board looks like

Train a classifier to recognise boards from very long games and use it to guide play. It classified almost perfectly but chose worse moves.

rejectedrecorded

Oracles, teachers and distillation

Phase distillation

Train a fast network to copy a slow but reliable search, using both its chosen column and the values behind that choice.

rejected

Risk and survival objectives

Judge a move by its worst outcomes, or by how long the game keeps going, and see whether caution extends life.

Read the primer →

Risk and survival objectivesfeatured

Conservative fitted policy iteration

Learn the whole spread of how long a game might last, and only leave the safe move when the evidence says so.

rejectedrecorded

Risk and survival objectivesfeatured

Weighting the worst outcomes at the root

Instead of judging a column by its average outcome, judge it partly by its worst outcomes, and find that the resulting policy dies much sooner.

rejectedrecorded

Risk and survival objectives

Critical risk

Play normally on a healthy board, then switch to worst-case thinking when height, crowding, or covered discs put the game at risk.

Risk and survival objectives

Gray throughput

Clear numbered discs and open covered ones fast enough to keep the board from filling.

rejected

Risk and survival objectives

Risk sensitive

Ranks columns partly by their worst imagined futures. Every tested risk setting trails the baseline.

rejected

Risk and survival objectives

Survival instinct — never place a disc where it can't clear vertically

Adds one human rule to the depth-4 search by refusing risky drops above a disc's vertical safe zone, then tests whether the rule extends life.

Risk and survival objectives

Tail-survival CEM

Retunes an existing policy for survival milestones instead of average score while changing nothing else. The result barely moved.

rejectedrecorded

Afterstates

Score the board a move leaves behind, and build training data in which every column the player could have chosen was measured.

Read the primer →

Afterstates

A network that predicts how long you will live

Plans a neural evaluator that predicts remaining row rises so search can favor survival over immediate points.

Afterstates

Distributional afterstate ranker

Measures every legal column under aligned futures and trains one network to rank the resulting positions.

rejectedrecorded

Afterstates

Sibling advantage

Stop asking how good a board is. Replay all seven columns through exactly the same imagined future and learn only which one came out ahead.

rejected

Constructive planning

Plan a whole rise cycle toward a target board shape, so that later rises set off stacks prepared in advance.

Read the primer →

Constructive planningfeatured

Constructive spectrum

Plans a five-drop cycle toward a target board with spread-out heights, reachable covered discs, and shared triggers. It clearly beat the one-move-ahead reference.

completedrecorded

Constructive planningfeatured

Viability controller

Classifies each move as charge, dig, release, repair, or emergency, then picks the best move of that kind. It scored well below the one-move-ahead reference.

rejectedrecorded

Constructive planning

Cycle abstraction

Choose one of five plans for the whole five-drop cycle, then let a model decide which plan fits the board.

rejected

Constructive planning

Rise-boundary option QD

A bounded prototype commits to a plan for a five-drop cycle and keeps the best plan of each behavioural type. It has never played a game.

Constructive planning

Stacking a column for a later chain

Reward the search for building columns that are one quiet drop away from collapsing in several waves, and see whether deliberately stored energy makes games longer.

rejectedrecorded

Constructive planning

Vertical reservoir

Defines a promising vertical stack, tunes those preferences on complete games, and finishes behind the one-move reference.

rejectedrecorded

Determinized planning

Imagine several complete futures, plan each one as if it were certain, and average the plans.

Read the primer →

Determinized planningfeatured

Hindsight planner

Imagine seven complete futures, play each as if it were certain, and average the results. The policy is legal but misuses uncertainty.

rejectedrecorded

Determinized planning

Open loop

Decide a whole short sequence of columns in advance, judge it against many imagined disc streams at once, and play only its first move before planning again.