The scripted-round benchmark
The leaderboard seeks to fairly answer the question: given identical luck, which policy scores more?
Ordinary randomly seeded play cannot promise that. A policy that opens more gray discs consumes more random numbers and quietly receives a different disc sequence. The scripted-round format removes that problem by deciding every random outcome before the game starts.
Two predetermined tapes
Each round is a small JSON document with two tapes:
discs[m] is the visible disc at move m. Indexed by move
number, so every policy sees the same sequence no matter how it plays.
latentRows[r][c] is the hidden value that the covered disc in
column c of rise r will reveal. It is fixed before the game and stays with
the disc through gravity and rises.
Because rises always happen after every 5th move, latent row r is consumed by the rth rise no matter what the policies do.
A gray disc's value is therefore a property of the disc, not of the moment
it happens to be revealed.
Every covered cell in the opening row already has its value written down in latentRows[0]. Revealing it can only ever produce that value.
What a run records
Each policy and round pairing produces a complete game record with a final score, moves survived, censor flag at the 2,000-move cap, numbered clears and covered reveals per move, maximum chain depth, illegal choices (a legal fallback is played and the lapse is counted), wall time, and a SHA-256 trajectory checksum. The full move-by-move frames power the replay viewer. You can click any score from the leaderboard to watch the game.
Limits
Policies are also compared at fixed algorithmic work (bounded search budgets), not fixed wall time, so a faster machine does not make a policy look stronger. Policies that read state outside the strict public interface (level or move number) are flagged with an "extended state" badge.
Reproduce it
npm run bench # default policies, all 8 rounds
npm run bench -- --policies greedy,expectimax-d2 --rounds gauntlet-03
npm run bench -- --all # adds slow reference D3/D4
The output lands in web/data/leaderboard.json with one replay file per game
under web/data/replays/. Re-running the same configuration reproduces every
score and checksum exactly.