Policy gradients

The technique

Skip the value; keep a probability for each action, play by sampling, and after a good outcome raise the probability of what you did (lower it after a bad one).

Read the primer

Approaches using this technique

4 approaches, featured pages first. Each card is one theory of how to choose a column and what happened when it was tried.

Policy gradientsfeatured

A PyTorch policy network, cloned then trained by playing

A small convolutional network copies a two-move search, then improves through 16,384 games. It finished about 40% short of its teacher.

rejectedrecorded

Policy gradientsfeatured

Learning a policy with explicit safety constraints

Learn a small correction to a simple search under hard safety limits. Every limit failed, so the run stopped before gameplay testing.

rejectedrecorded

Policy gradients

Copying the one-move search, in C++

Teach a policy network to imitate a simple exact search before self-play. It never imitated well enough to start.

rejectedrecorded

Policy gradients

Nudging a one-move search, from easy and hard starting boards

A learned correction trains on fresh games and difficult mid-game positions. It improved the simple search slightly but stayed far below the reference.

rejected

Agent contextApproach directories in this group, by family

ntuple-rl