Building a Chess Coach

White’s knight moves from f3 to e5, capturing a pawn in a position leading to checkmate. The game review marks it brilliant. But the useful coaching starts below the board: why does the move work, what does it threaten, and which tactical or positional features make it special?

This is the opening example from Anant Dole and Asbjørn Steinskog’s production chess coach at Take Take Take, the company founded by Magnus Carlsen. Its iOS and Android app lets people play friends and post about their games. After a game, the review combines automatic move annotations with generated commentary. Detectors identify relevant chess features; the language model turns those findings into an explanation of the move.

The coaching extends beyond individual moves. Insights such as accuracy in different game phases, current rating, and depth of knowledge in an opening become another layer of analysis. The coach uses them to identify opportunities for learning, connecting what happened in a game to what the player could improve next.

Chess has long exposed the difference between exploring possibilities and deciding which possibilities deserve attention. Claude Shannon’s Programming a Computer for Playing Chess—received in 1949 and published in March 1950—provides the historical starting point. Steinskog introduces Shannon as “the OG Claude” and uses his Type A and Type B strategies to frame the problem.

In Steinskog’s account, early computing limits made selective search look essential: a machine could not simply explore the entire game tree.

As computers became faster, increasingly powerful search carried chess engines far beyond those early limits. The landmark was Deep Blue’s match victory over Kasparov in 1997, against the reigning world champion under standard tournament controls; its first individual game victory had come in 1996.

Steinskog then moves to AlphaGo, where neural networks helped select lines in Go’s much larger search space, and AlphaZero, which extended the approach across Go, chess, and shogi. The Type A/Type B distinction is a useful historical lens, rather than an exclusive classification of modern engines. Language models arrived with a different capability profile: they could produce plausible opening moves, yet lose track of the game and hallucinate as play continued.

A clip of Magnus Carlsen commentating from the company’s Oslo office makes the limitation concrete. Grok chooses an early queen move to b6 in a poisoned-pawn line and subsequently loses badly. The commentary attributes the failure to its broader inability to play chess reliably, rather than simply to the opening choice.

The game came from Kaggle Game Arena, which evaluates models through gameplay. Its expansion to Werewolf adds a different challenge: models must navigate social deduction and deception. Chess, meanwhile, makes failures of sustained calculation particularly visible.

Reasoning models can work through moves to some extent, but Steinskog finds that their calculations still break down. That does not make the transformer architecture inherently unsuitable for chess. He describes a DeepMind transformer trained on millions of chess positions paired with Stockfish evaluations, learning chess evaluations instead of ordinary next-token language prediction, and reports grandmaster-level playing strength.

The crucial distinction is the training objective. A model trained to evaluate chess positions does not thereby acquire the ability to explain them in natural language. A language model that can discuss chess does not thereby become a reliable chess engine. The coach needs strong analysis and clear explanation, with an explicit boundary between them.

The game-review pipeline starts by running Stockfish over the whole game. Its analysis supplies the reference for the best move in each position. But the best move alone is not enough context for a lesson: the player also needs to understand threats, plans, tactical possibilities, and alternatives to what they actually played.

The next stage extracts those features. Tactical detectors look for forks, pins, and skewers. Positional detectors capture structural themes, including potential weaknesses such as doubled pawns. This gives the generator chess concepts to explain, rather than requiring it to discover those concepts from the board on its own.

Maia, from University of Toronto research, contributes a different signal: what a human is likely to play. Given a rating—for example, an online rating of 1500—it predicts a probability distribution over moves in the position.

Combining these signals lets the team distinguish move quality from human difficulty. A move can be Stockfish’s best choice while receiving a low predicted probability from Maia at the player’s level. The coach can then treat it as a move that may be hard for that player to find.

All of this becomes context for the language model. Its assigned job is deliberately narrow: translate the supplied information into English. The team does not want it independently inventing tactical explanations or calculating additional lines. Hallucinations still occur, but the architecture makes the intended source of every explanation clear: analysis first, language generation second.

In a game Steinskog played, his opponent advances a pawn to f5. Stockfish marks the move as bad. That verdict is useful for grading the game, but it leaves the player with an immediate practical question: what is the opponent trying to do?

The detectors identify a threat to trap the queen, including a bishop move to g5. They also identify the response: capturing the central pawn defends a square the queen can use to escape. The resulting explanation can acknowledge the threat while showing why it does not work.

A compact JSON representation of those supplied facts makes the separation concrete:

{
  "playedMove": "f5",
  "engineVerdict": "bad move",
  "threat": {
    "goal": "trap the queen",
    "continuation": "bishop to g5"
  },
  "response": {
    "action": "capture the central pawn",
    "effect": "defend an escape square for the queen"
  }
}

The field names here illustrate a context contract. The important content is the causal connection: the pawn capture protects the queen’s escape route. That is what turns an engine’s bad-move label into coaching.

The improvement loop begins when a player downvotes commentary. The report goes to Slack and to a running Claude Code session through a Claude Code channel. Channels are described here as a research-preview mechanism: an MCP server injects external events into an active session. That mechanism gets the report to the coding agent; the team’s custom triage skill defines what the agent should do with it.

The investigation follows a repeatable procedure:

1. Inspect the report. The commentary triage skill guides the agent through the position and the generated explanation.
2. Change the relevant cause. The agent can adjust the prompt, modify a detector, or create a new detector.
3. Regenerate commentary. Scripts let it run generation again with the proposed changes and inspect the result.
4. Ask for guidance. Questions return through Slack, so Steinskog can help resolve chess judgments even while away from his desk.
5. Submit for human review. When the result looks right, he asks for a pull request, reviews it in GitHub, and merges it.

The autonomous portion covers investigation and candidate changes. The described workflow retains human guidance and approval before those changes are merged.

The live demonstration starts with a deliberately weak report: the presenter opens the app on his phone and marks commentary as bad without first establishing that it is wrong. Slack receives the board position and the generated text. The same event reaches Claude Code, which invokes the triage skill and begins investigating.

The agent is running at high effort, so the presenters move on while it works. At this point, the visible result is a successful handoff into investigation, not a completed correction.

Post-game review has a different latency budget from background investigation. A player wants to step through moves and see what happened. An indefinite coach-thinking screen interrupts that interaction, even if the eventual answer is good.

The team targets under three seconds for generating coach feedback. Dole reports that Gemini 3 Flash typically delivers its first token in about one second, with average end-to-end latency of about three seconds in their application. Gemini 3 Flash therefore comes close to the interaction speed they want. These are application measurements; request sizes, routing, and reasoning settings are not specified.

Other reasoning models can produce useful analysis, but their completion times are less predictable. That makes them awkward for immediate move review. A planned chat-with-your-coach experience offers a different opportunity: users asking a question can reasonably expect to wait longer than users stepping through a finished game. The interface’s pace helps determine where additional reasoning is worth its cost in time.

Dole and Steinskog use their own chess calculation as the final quality check: they examine a position, work through how they would play, and compare that understanding with the generated response. Automated evaluations make those checks more repeatable.

The team uses 16 scenarios covering tactical patterns, blunders, and hallucination limits. The scenarios come from real games. A knight-fork case, for example, checks whether the model recognizes and mentions the tactic when given the context engine’s information. An LLM judge helps assess the responses. This evaluates whether the explanation uses the supplied chess facts, rather than merely whether it sounds fluent.

OpenRouter makes it easier to swap models as new releases arrive, including Gemini versions, GPT-5, and Claude models. The team compares output quality, then uses its own chess expertise to decide whether the result is actually good.

Dole reports these results for the team’s scenario suite:

GPT-5 Mini is described as less accurate, but the spoken latency comparison is unclear. Exact model versions, Claude’s thinking configuration, judge settings, and repeated-run details are not supplied, so the table describes this team’s evaluation rather than a general model ranking. The remaining failures motivate continued testing as models change.

Separating the data pipeline from language generation gives the team a way to improve chess understanding without asking the generator to do everything. Closing the feedback loop with agents then makes problems in either prompts or detectors easier to investigate and iterate on.

The context itself takes sustained work. Dole describes starting with a large JSON file, then pruning it step by step while checking how output quality changes. A clear context model does not emerge simply by including everything the analysis pipeline can produce; it develops through repeated decisions about what helps the explanation.

Automated evaluations provide feedback during that process, but subject-matter experts remain necessary. In this team, the builders are also strong chess players. In another domain, the people qualified to judge an explanation may need to be partners rather than the engineers building the system.

Before returning to the agent, the presenters invite attendees to play chess at the third-floor entrance around 3:45 PM, with QR signup or walk-up participation. They describe a simul in which one presenter plays four opponents on four boards, with extra time to walk between them. Winners receive wooden chess boards; if nobody wins, the two best players still receive sets. If everybody wins, they joke, they will have to buy more boards.

Back in the live session, the agent is still thinking, but it has posted an update saying it is investigating the position. Then it asks: “What specifically feels wrong about the commentary?”

The presenter admits that nothing is wrong and says the investigation will be closed. No completed fix or pull request appears in the demonstration. The useful result is the clarification itself: a downvote initiated an investigation, but it did not establish a defect. The loop leaves room for the agent to question the report before changing the system.