OpenThoughts: Data Recipes for Reasoning Models

What data lets a small model learn to reason? On competitive math tasks such as AIME, giving models room to generate long, step-by-step reasoning traces has accompanied striking improvements. That makes test-time scaling attractive—but it leaves a training question: which examples teach a model to use that extra thinking effectively?

DeepSeek-R1 made this question concrete. Ryan Marten emphasizes the supervised fine-tuning stage in its training pipeline: DeepSeek-V3-Base was trained on roughly 800,000 examples, including about 600,000 reasoning examples. Reinforcement learning helped produce the reasoning data, and further RL followed SFT. His description of R1 as an SFT model is therefore shorthand; that later RL also refined reasoning, while the small distilled models were trained with SFT alone.

Those strong small models suggested that the training method was accessible, but the data behind it remained insufficiently specified. A training recipe does not tell you which questions and reasoning traces to train on. Reconstructing that missing data recipe became the motivation for OpenThoughts.

For a specialized application, training your own reasoning model can serve performance, privacy, speed, cost, and control over the model’s future. Reasoning is another tool for solving a domain problem, and SFT and RL are complementary ways to develop it. The appeal of SFT here is its relative simplicity and effectiveness.

The research problem has three parts: how much data is necessary, which creation steps matter, and which choices work best within each step. OpenThoughts 3, announced with this talk, is the resulting dataset recipe. Marten presents it as state of the art at its June 2025 release.

The first comparison plots accuracy against dataset size on AIME competitive math, LiveCodeBench competitive programming, and GPQA Diamond science questions. To compare OpenThoughts data with the data released for Nemotron Nano, the team trained the same base model on each dataset. The original OpenThoughts paper identifies that shared base as Qwen2.5-7B-Instruct. Marten reports that the OpenThoughts recipe shifts the accuracy-versus-data-scale curve upward relative to the Nemotron Nano data recipe. This comparison concerns the training data, rather than differences between model architectures or starting checkpoints.

The next comparison concerns released models. Marten reports that the OpenThoughts-trained 7B model outperforms DeepSeek-R1-Distill-Qwen-7B, the model the project originally sought to reproduce. Against released Nemotron Nano, he reports better results on some benchmarks and comparable results on others. That second comparison does not hold the base model fixed: Nemotron Nano starts from Llama-3.1-8B-Instruct. These are release-time results across science, code, math, and held-out benchmarks, not a claim of universal superiority.

Moving right on a scaling plot means creating and training on more examples, with increasing expense. Moving the curve upward means improving what those examples contain. Dataset size and dataset recipe are separate experimental decisions: a better recipe seeks more accuracy at a given data scale, before committing to a larger run.

OpenThoughts decomposed data creation into a sequence of choices:

1. Source questions: identify candidate question collections.
2. Mix sources: decide which collections to combine.
3. Filter questions: select the questions worth answering.
4. Generate answers: sample reasoning traces from a teacher model.
5. Filter answers: test whether removing undesirable responses improves training.

After studying these stages, the team also compared teacher models to choose the generator for the final recipe. Treating each choice as an experiment makes it possible to discover which steps help, rather than assuming every additional curation step improves the dataset.

The broader Hugging Face organization shown in the talk contained over 5,000 datasets and almost 3,000 models; those are not the experiment count for this release. Marten reports roughly 1,000 experiments for the project itself. The strategy was to find measurable signal at the smallest practical scale, test many alternatives, and then scale the strongest candidates. Small-scale rankings can change at larger scale, so the final large run remains a test of whether the earlier findings transfer.

One of the most useful findings concerns repeated sampling. A training example is a question paired with a teacher response; it need not introduce a new question every time. Marten describes the following approximate fixed-size comparison:

In the comparison Marten describes, these constructions performed similarly. His 30,000-example illustration rounds the paper’s 31,600-example ablations; it should not be read as universal equivalence across domains or sampling settings. In particular, the paper’s math results do not establish that more traces always win.

The scaling opportunity is a separate point: sampling 16 responses per existing question produces 16 times as many examples as sampling once. If finding good questions is the bottleneck, multiple teacher attempts let the dataset grow without requiring an equally large increase in unique questions. That still costs generation and training compute, but it changes what must be sourced.

A model’s own benchmark strength does not determine its quality as a teacher. Marten compares this to a brilliant researcher who is a poor lecturer: solving a problem well and producing examples another model can learn from are different capabilities. In the OpenThoughts experiments, QwQ-32B was a stronger teacher than DeepSeek-R1, so the team switched to QwQ-32B for its recipe. The relevant outcome was the trained student’s performance.

Question provenance also proved less predictive than the team might have expected. Some of the best sources were entirely synthetic, outperforming tested sources drawn from forums or written manually by people. This is a finding about the selected sources, not a guarantee that synthetic questions are better. Its practical value is that synthetic generation can expand the supply of questions once a useful generation approach has been found.

Two question filters worked well:

• LLM difficulty labels: ask a language model to judge how difficult each question is, then retain the hardest questions.
• Response length: have a model attempt each question, then use the length of its response as a proxy for difficulty.

The second method assumes that harder problems tend to elicit more reasoning text. Length is a proxy, not a direct measurement of difficulty. In these experiments, both approaches worked better than the tested embedding-based methods and fastText classifiers—methods more familiar from pretraining data selection. Post-training data may therefore need different selection signals.

Two other intuitively appealing choices did not help. First, a smaller collection of high-quality sources performed better than a broader mix chosen to increase source diversity. Adding sources was not automatically useful simply because it made the dataset more varied.

Second, answer verification and filtering did not improve the general SFT and distillation recipe in the reported experiments. This result has an experimental qualification: the paper includes a 63,200-example unfiltered baseline compared with 31,600-example filtered sets, explicitly without controlling compute for that comparison. It does not establish that verification is useless under every budget or in every domain.

Marten offers a possible explanation: a teacher’s attempt at a difficult problem may contain useful reasoning even when its final answer is wrong. Discarding it on final-answer correctness alone could remove something the student can learn from. That is a hypothesis about why filtering may fail, distinct from the observed result—and it does not negate verification’s role in RL.

A specialized model does not require repeating the entire research program. Start with the published recipe, then use available compute to test a few alternatives at each stage. The important qualification is that the best choice may depend on the domain: OpenThoughts studied code, science, and math separately.

The distinction has a concrete explanation. Even a simple programming task can require many lines of code, so a long response need not mean difficult reasoning. An AIME answer is a compact integer; the final answer contributes little to the total response length. In that setting, length more closely reflects the reasoning preceding it.

When domain data is scarce, transform existing material into questions, then use those questions as in-context examples to generate more. This applies the synthetic-source finding to a practical constraint: a small collection of relevant material can seed a larger question set. Marten points to Curator, the team’s open-source library for synthetic-data generation.

Reliable evaluation is what makes recipe selection possible. Without a stable measure of whether a student improved, changes to sources, filters, or teachers become difficult to distinguish from noise. Evalchemy supports the team’s evaluations, including sharding and parallelism.

Small evaluation sets particularly need repeated sampling. Marten reports evaluating AIME’s 30 questions ten times and averaging the results to compare data strategies. Repetition does not add new problems, but it reduces the influence of a single stochastic attempt on the estimated score. The goal is a less noisy signal for deciding which recipe deserves the next training run.

Distillation is often treated as a way to approach a teacher’s performance, with RL reserved for exceeding it. The legal-domain experiment provides a narrower but useful counterexample: classification of Supreme Court decisions, specifically opinion ideology in the paper.

The procedure was straightforward:

1. Collect 2,000 unique questions.
2. Sample five teacher answers per question.
3. Verify the answers and discard incorrect responses.
4. Fine-tune a 7B student on the retained examples.

Here, verification mattered. The resulting student surpassed DeepSeek-R1 on the reported Supreme Court classification task. This is task-specific superiority, not a claim that the student became a generally stronger reasoning model. It also gives a concrete reason to retest the general recipe’s negative findings when moving into a specialized domain.

The prepared talk closes by handing over the materials needed to investigate the recipe: the paper, model weights, datasets, and code for data generation and evaluation. The released artifacts include the OpenThoughts3-1.2M dataset, OpenThinker3-7B weights, and the OpenThoughts data curation repository. OpenThoughts names the data project; OpenThinker names the trained model.

Marten credits a large team’s work over several months. The presentation’s closing QR code leads to a tweet collecting the project links, giving the audience a route from the results to the underlying work.

The first audience question asks what separates this training from regular supervised fine-tuning. Marten’s answer is direct: “This is just regular SFT.” The distinction lies in the training examples, not a special replacement for the supervised training procedure.

The follow-up asks how ordinary SFT can teach a model to think for longer. The supervised answers contain the thinking traces as well as the final outputs. Learning to generate those answers teaches the student to use its context window for a long reasoning sequence. The format itself is learnable through imitation; no separate mechanism for extended thinking is introduced in this explanation.

Why did the smaller QwQ-32B make a better teacher than DeepSeek-R1? The experiments establish the student outcome, but not its cause. Marten points to differences in reasoning-trace length distributions. Possible explanations include using more of the context window, producing more tokens or reasoning steps, and providing better-formatted responses. These remain hypotheses rather than a demonstrated account of the teacher advantage.

The team also tried Claude as a teacher and obtained poor results despite its strength as a model. Marten does not identify the version or give detailed comparison conditions, so the observation applies to that tested teacher, not the entire model family. What makes a response easy for a student to learn from remains an open research question.

The final question asks whether the project can identify the exact step where a reasoning chain goes wrong. It did not perform that fine-grained failure analysis. Marten points to literature on critical failure steps, then describes possible extensions: detect and intervene on erroneous steps during evaluation, or rewrite traces while creating the training dataset.

Those possibilities should not be confused with demonstrated improvements. The reasoning-trace modifications the team actually tried were not helpful. The talk ends with that boundary intact: selecting questions, sampling teachers, and evaluating students produced a useful recipe, while locating and repairing failures inside the traces still leaves substantial work to do.