Training the Ultimate Loser

Defining successful losing

Shallow Red succeeds when its king is checkmated. Resigning, running out of time, and proposing illegal moves don't count. Reliability comes before speed: an engine that loses 95% of its games slowly is preferable to one that loses 85% quickly and draws the rest. Winning is the least desirable outcome.

Before training anything, I built a match system around python-chess. I distinguish a draw from an unfinished game. If an evaluation stops after 200 plies, that means I exhausted the testing budget; it doesn't mean the position is drawn.

Stockfish made the first results look easy

My first policies were simple. One chose a legal move uniformly at random. Another followed hand-written rules that encouraged hanging pieces and exposing the king.

Against Stockfish, both were checkmated in all 200 test games. Random play lasted about 34 plies on average; the sacrifice heuristic reduced that to about 30.

The problem is that Sotckfish wanted to win. It accepted free material and found attacks. Both sides were working toward the same final goal. They just disagreed about who should feel good about it.

This gave me a useful baseline, but it also taught me to be suspicious of a perfect loss rate against an opponent that was doing most of the work for me.

Teaching a network to imitate bad moves

I trained v1, the first neural model, on Stockfish evaluations in reverse. I generated 10,000 positions and evaluated every legal move with a small Stockfish search budget. I then trained a network to prefer the move Stockfish considered worst for the acting side.

The network sees a board as 21 planes of 8 × 8 values. These describe pieces and game state, with Black's perspective mirrored so both colors can share the same representation. The policy head scores 4,672 possible actions: 64 origin squares multiplied by 73 move patterns. Illegal actions are masked before selection.

The distilled policy lost to Stockfish in 24ish plies. Performing the reverse analysis live reduced that to about 18. Copying bad moves worked, and search improved it further.

For v2, I replaced the single preferred move with a ranking of every legal move. A soft probability distribution over that ranking lets the network learn that several sacrifices may be useful without treating them as equally good. I also experimented with a value head that estimated the eventual outcome.

At move time, the network proposed a shortlist and a reverse evaluator examined those candidates. Against Stockfish, this hybrid was pretty bad, usually losing in 16–18 plies. Against a random opponent, it got checkmated in only about 70% of the time.

Random opponents changed the objective in practice

A move can be terrible against Stockfish and ineffective against someone who doesn't notice why it's terrible. A random player may ignore a hanging queen, walk away from an exposed king, or accidentally leave Shallow Red with no legal moves.

For v3, I added 4,600 ranked positions generated against random replies to 5,227 reverse-Stockfish positions. I also tested weak conventional engines and opponents using simple material or capture rules.

The selected network has 32 channels, four residual blocks, and 115,634 parameters, including its value head. Its training run used the policy ranking objective with the value loss disabled; this engine doesn't rely on a learned value prediction to decide whether a move will eventually lose.

The network's main job is to order moves. The evaluator then considers its twelve highest-ranked legal candidates and lists every legal immediate opponent reply. Under a uniform-random opponent assumption, it can calculate exactly what fraction of those replies would checkmate Shallow Red. Other scoring terms reward checks and king pressure while discouraging draws and Shallow Red wins.

This combines an exact calculation about the next reply with heuristic estimates about what comes later. It doesn't calculate the exact probability of eventually losing the game.

I also added a low-material adjustment. Giving away everything can leave too little mobility and result in stalemate. Below a material threshold, the evaluator starts preserving pieces and king escapes instead of blindly continuing to sacrifice.

On a fresh 200-game random-opponent suite, this system lost 188 games by 600 plies, drew eight, left four unresolved, and won none. It also lost all 100 games against Stockfish in a sanity check test.

That combination of trained move ordering and tactical evaluation became research v3.

Making it playable in a browser

The research engine used Python, PyTorch, and expensive search. Serving that computation for every visitor would have been a very different product from a free browser game.

I built a lightweight TypeScript engine, then developed a smaller browser policy and an integer-quantized export. The browser policy has 24 channels, three residual blocks, and 37,633 policy parameters. Its artifact is ~40 KB (full engine is ~474KB). Move generation, neural inference, and tactical scoring can run on the player's machine.

The public engine and the research engine have evolved separately. One historical test of the lightweight TypeScript build recorded 254 losses in 300 random-opponent games, compared with research v3's 281. It also used different draw rules, so that difference measures the complete systems tested at the time. It isn't a clean measure of network quality.

The large planner described below belongs only to the research system.

Playing against someone who also wants to lose

Human players found it more interesting to try to lose to Shallow Red.

Two players with this objective can end up in very strange chess. They decline free pieces, compete to expose their kings, and avoid delivering mate. Sacrificing more material doesn't necessarily help when the other side refuses to take it.

I generated games against synthetic opponents with different losing strategies. Some avoided captures; others exposed their king, switched behaviors, or predicted Shallow Red's moves. This supplied legal, reachable positions without waiting for a large collection of human games.

Decisive games were rare, so I tried several ways to extract more learning signal from them.

Proofs and reinforcement learning had limited reach

Exact selfmate search asks a stronger question than ordinary losing play: can Shallow Red choose moves that guarantee its own checkmate against every legal opponent response?

That creates an AND/OR search tree. At a Shallow Red turn, finding one successful move is enough. At an opponent turn, every legal reply must still allow success.

I tested selfmate problems, independently verified proofs, mined earlier positions, explored tablebases, and expanded synthetic games through branching search. These methods found forced selfmates. A proof book could execute them correctly when the game entered a covered position.

Coverage was the problem. Ordinary games rarely reached those positions. Fine-tuning on them improved familiarity with known examples but did not reliably improve complete games on unseen position families. Those checkpoints stayed experimental and weren't promoted.

I also tried PPO RL from v3. In self-play, being checkmated earned positive reward, delivering checkmate earned negative reward, and quicker losses received larger rewards. Draws and 200-ply truncations were penalized. A KL penalty kept the policy from drifting too far from a frozen v3 reference.

A pilot on Modal generated 1,024 games and roughly 172,000 policy positions. Its completed checkmates became shorter, but there were fewer of them. Against a matched random-opponent suite, it lost 73 of 100 games while v3 lost 79.

That failed my reliability-first objective and was no bueno. Later experiments training only Shallow Red against frozen opponents also failed their promotion criteria. Reinforcement learning was part of the research, but it did not produce the selected neural checkpoint.

A planner found paths the policy missed

The more productive direction ended up being keeping v3 and adding a planner around it.

I first made weaker versions of v3 using the same neural weights. Some examined fewer candidate moves. Others removed the stalemate adjustment or replaced a controlled fraction of decisions with another legal move. Measuring these variants against ordinary opponents gave me a calibrated range of losing competence.

The planner watched the opponent's replies and activated after repeated agreement with a modeled policy. It then searched sequences of future Shallow Red decisions, keeping a small beam of promising alternatives. Hierarchical planning grouped several decisions into reusable chunks, allowing longer plans without expanding every possible line.

Cached plans continued only while the observed game matched the expected trajectory. When predictions failed, the engine could fall back to v3. An exact short selfmate check could also recognize a forced opportunity immediately.

Against one weaker v3 tier, a fresh 100-game comparison resulted in six baseline losses and 54 planner losses. Against ordinary random play, the guard stayed inactive and all games remained move-for-move identical to the baseline.

Learning the planner was harder than running it

I tried to compress that improvement into a cheap neural student. I trained on successful planner trajectories and later used DAgger: let the student play, ask the planner for corrections on positions the student actually visits, and train again.

The students failed to preserve the teacher's advantage on fresh games. Opponent history, plan state, and future coordination appear to matter in ways my training setup did not capture adequately. The result established a limitation of those particular distillation attempts; it did not establish that the planner can never be distilled.

I also explored competent-looking play and longer concealment strategies. The tested versions failed their performance or safety gates. The current promoted planner does not depend on a demonstrated ability to deceive human players.

Inferring an opponent within a constructed family

The strongest planner handles an unknown setting within a specific opponent family: v3 with a probability of replacing its normal decision with a different legal move.

I call that probability the corruption rate. The replacement mechanism is deterministic given the game context, which made experiments reproducible. After each observed reply, the planner narrows the interval of rates consistent with the opponent's behavior.

It receives neither the actual rate nor the opponent's configuration label. During planning, it considers the distinct replies possible across the remaining interval and chooses against the least favorable plausible continuation.

The promoted configuration searches twelve future Shallow Red turns with beam width four and three-turn planning chunks. This converted many deadlocks against corrupted opponents into losses.

The win that led to the win shield

One otherwise promising candidate won a game.

Its search already penalized moves that checkmated the opponent. But after a cached plan ended, the live game reached a position where every legal Shallow Red move delivered mate. By then, avoiding an immediate win was impossible.

The preventable mistake happened earlier: Shallow Red allowed its opponent to create that position.

I added a two-ply win shield, which checks every proposed move, including cached and fallback choices. It rejects an immediate opponent checkmate when an alternative exists. It also asks whether the opponent has a reply that would leave Shallow Red with only checkmating responses on its next turn. If so, it substitutes a safer move while one is still available.

This bounded check cannot rule out every deeper forced win. It directly addresses the failure I observed and records cases where no safe alternative exists.

The shield repaired the known game. In a fresh comparison across four corruption levels, each with 100 games per engine, the results were:

An additional 100-game random-opponent comparison was unchanged move for move: both systems lost 94 games, drew four, and left two unresolved. The planner and shield did not alter those games.

That shielded planner became my promoted research candidate, informally called v4 (although I guess it's analagous to v3 with a harness). Its neural checkpoint is still v3; the added capability lives in opponent inference, search, plan caching, and the shield.

What the 384 game comparison established

I then tested v4 directly against full v3. The engines played 192 fresh opening positions twice, swapping colors, for 384 games. Each opening began after six random legal plies, and each game had a 200-ply evaluation cap.

The original pass stopped five games on automatic fivefold repetition. I replayed those opening pairs with repetition and move-count termination disabled, preserving the original game IDs and seeds. Their move prefixes matched the original games. One former repetition continued into a v4 loss; four reached the cap.

The combined result was:

v4's successful losses took a median of 87.5 plies after the opening prefix. It lost 20 times as White and 18 as Black. The shield intervened 28 times across 17 games, and no protocol failures occurred.

What is learned and what is engineered

The neural network learned move preferences from labeled positions. The evaluator applies explicit tactical scoring. The planner uses an authored search algorithm and updates its estimate of the opponent during play. The shield performs a bounded check for dangerous wins. Calling all of this “the model” can hide where the improvement actually came from.

The early gains came from teaching a small network to rank bad moves. The later gains came from giving the engine a way to coordinate those moves around an opponent's behavior. Repeated attempts to replace that computation with additional training have not yet matched the planner.

None of this guarantees that a human can never lose to Shallow Red, but I think it would be insanely hard to do so.

I started with the idea that the worst chess engine should simply choose the worst move. The project keeps returning to a more specific question: which move is most likely to make this particular opponent finish the job?