Everything I Learned Training Frontier Small Models

What should a model retain—and what should it do exceptionally well—when it has to run on a phone or inside a car? Maxime Labonne approaches that problem as Liquid AI’s head of pre-training. The models he describes span 350 million to 24 billion parameters, with text, vision, and audio variants available on Hugging Face. At the time of the talk, Liquid had recently released a 450M vision-language model and updated its 350M text model.

On-device deployment imposes three connected constraints:

• Memory: The available hardware limits model size, which in turn limits how much knowledge the model can store.
• Task specificity: Limited capacity makes specialization valuable. Summarization or tool use can be a better target than general-purpose conversation.
• Latency: Interactive workloads need fast inference and enough throughput to keep the application responsive.

Small models are not simply scaled-down large models. Their architecture and training must allocate a limited budget toward the capabilities the deployment actually needs.

The smallest members of larger model families already use hybrid architectures. Gemma 3 270M combines sliding-window attention with grouped-query attention, or GQA. Qwen3.5-0.8B combines gated DeltaNet with gated attention. But Labonne’s first concern is not their sequence-processing blocks: it is the share of the model occupied by embeddings.

Labonne reports that embeddings account for 63% of Gemma 3 270M’s parameters and 29% of Qwen3.5-0.8B’s parameters. He calls the remaining budget the model’s effective parameters: the blocks available for reasoning and knowledge processing beyond the embedding layer. Embeddings still perform useful work, including representing a broad vocabulary; the design question is whether that vocabulary deserves so much of a tiny model’s memory footprint.

Labonne attributes these large embedding allocations to distillation from teachers with large vocabularies. Carrying that vocabulary into the student preserves a substantial embedding matrix even as the rest of the network shrinks. His proposed opportunity is to allocate more of the same memory footprint to the non-embedding network rather than inherit every architectural proportion from the teacher.

LFM2 also uses a hybrid architecture, combining short convolutions with GQA. Labonne reports an embedding share of about 19% for the LFM2 configuration shown. That leaves more of its parameter budget for the rest of the network. The team selected the architecture by implementing candidate operators on two target devices and profiling their actual performance.

The resulting gated short-convolution block was particularly attractive for latency. In the presented operator comparison, short convolutions cost less than sliding-window attention, gated DeltaNet, gated linear attention, and GQA. The important design step is to measure the operators where they will run: an appealing theoretical architecture is not enough if its kernels are expensive on the deployment hardware.

The CPU comparison covers an AMD Ryzen AI Max+ 395 and a Samsung Galaxy S25 Ultra; the slide labels the phone platform Qualcomm Snapdragon Gen4. It compares prefill, decode, and memory using llama.cpp, 4-bit quantization, and 2K input tokens. Labonne reports faster inference and lower memory use for LFM2, while explicitly noting that the compared models differ in size. These are deployment comparisons, not a controlled isolation of architecture at equal parameter counts.

The GPU comparison extends the argument to serving: Labonne reports strong aggregate throughput even at high concurrency. The companion measurements use an H100 SXM5 with BF16 and SGLang 0.5.9, 1,024 input tokens, up to 256 output tokens, and sustained concurrent requests. This measures serving throughput, which is distinct from the latency of one request.

The LFM2.5 recipe uses familiar stages: pre- and mid-training, supervised fine-tuning, preference alignment, and reinforcement learning. The surprising choice is the training budget. Liquid trained LFM2.5-350M, a 350-million-parameter model, on 28 trillion tokens. That is a training-token count, not a claim of 28 trillion distinct tokens.

That budget can look excessive through the lens of Chinchilla’s compute-optimal training study. But a fixed training-compute optimum is not a ceiling on useful training. Labonne reports that performance continues to improve as the number of pre-training tokens grows. He connects that observation to Roberts and colleagues’ Test-Time Scaling Makes Overtraining Compute-Optimal, which considers training and inference together, including inference sampling.

In Labonne’s application of those newer scaling laws, even 28 trillion tokens falls short of the proposed optimum for the 350M model. The practical implication is that more pre-training can remain valuable at very small scales. Small models also cost less to train than substantially larger ones, making continued training an accessible way to improve a model whose deployment size must remain fixed.

The next comparison is between the post-trained LFM2-350M and LFM2.5-350M, so it does not isolate the effect of additional pre-training. Labonne reports improvements across several capability categories:

For this model, the explicit priorities are data extraction and tool use. Leading code or mathematics performance is less important than being reliably useful at the tasks people will actually assign to a 350M model. Choose capabilities instead of aiming for average performance everywhere.

Small and large models can share the same post-training stages without using those stages in the same way. For supervised fine-tuning, Labonne recommends a narrow target. Taking an available model and fine-tuning it to call one particular function is an excellent fit: the narrower the task can be made, the more directly the training can develop the required behavior.

For preference alignment, Liquid uses an on-policy, length-normalized form of direct preference optimization, or DPO. Labonne values this stage for broad improvements in response quality: the model’s answers improve in ways that are not captured solely by benchmark scores.

Reinforcement learning is effective even at very small scales in Labonne’s experience, but its learning signal can be narrow. Using many environments and tasks helps the model generalize. Small models are also sensitive to their cold start: the SFT mixture should contain examples resembling the tasks the model will later face during RL.

When an RL task fails to train well, the remedy may begin before RL:

1. Check whether the SFT mixture contains similar tasks and examples.
2. Consider whether the task is too complex for the model’s current capabilities.
3. Add appropriate cold-start examples and repeat supervised fine-tuning.
4. Retry RL and check whether learning improves.

Poor RL progress is useful feedback about the earlier training mixture, not merely a reason to keep running the same optimization longer.

A doom loop occurs when a model repeats a sequence of words indefinitely instead of completing its answer. The recipe response shown in the talk shows that this is not confined to mathematical reasoning. Still, three risk factors compound: a tiny model, reasoning mode, and a task that is too difficult. A small reasoning model tackling very hard mathematics is especially exposed.

Liquid’s first intervention happens during preference alignment, beginning with on-policy data generation. The policy model—the model being trained—generates its own candidate answers, allowing the training data to expose its particular failures:

1. Start with roughly one million prompt samples.
2. Generate five rollouts per prompt using temperature sampling. Diversity increases the chance that at least one answer avoids looping.
3. Generate one additional rollout at temperature zero, which the team expects to expose looping more readily.
4. Send all six responses to an LLM jury for scoring.
5. Select the highest-scoring response as chosen and the lowest-scoring response as rejected.

The goal is for a looping response to become the rejected side of a preference pair. The flowchart also includes heuristic filtering after selection, although the talk does not specify those filters.

The selection step can be expressed directly in Python. It consumes scored rollouts; generation and jury scoring happen before this function. A tied best and worst score provides no preference signal, so the function returns no pair:

from collections.abc import Sequence


def select_preference_pair(
    prompt: str,
    scored_rollouts: Sequence[tuple[str, float]],
) -> dict[str, str] | None:
    if len(scored_rollouts) != 6:
        raise ValueError("Expected five sampled rollouts and one greedy rollout")

    rejected, worst_score = min(scored_rollouts, key=lambda item: item[1])
    chosen, best_score = max(scored_rollouts, key=lambda item: item[1])

    if best_score == worst_score:
        return None

    return {"prompt": prompt, "chosen": chosen, "rejected": rejected}

This selection does not itself detect repetition. It depends on the jury assigning worse scores to the failed answers so that preference training receives the intended signal.

Preference training teaches the model to favor better responses over its own looping failures. The second intervention uses reinforcement learning with verifiable rewards, supplemented by an n-gram repetition penalty. For a mathematics task, the reward process tries to extract the final answer. A rollout that never produces a final answer cannot receive a positive correctness reward. That already creates pressure against endless repetition.

The repetition penalty adds a direct cost for repeated sequences, while temperature sampling keeps RL rollouts diverse. Together, these mechanisms address both the training signal and the supply of candidate responses: loops are less attractive to optimize toward, and the model has more opportunities to produce a successful alternative.

For LFM2.5-1.2B-Thinking on difficult tasks across multiple benchmarks, Labonne reports a doom-loop rate of approximately 15–16% after mid-training. SFT barely changes the rate, despite having no doom-loop examples in its training data. DPO substantially reduces looping, and subsequent RL makes it almost nonexistent in his reported evaluation. The talk does not specify the full prompt set, decoding settings, output limits, or loop detector, so those stage comparisons should be read within that evaluation.

Labonne also reports a doom-loop rate above 50% for Qwen3.5-0.8B in reasoning mode under difficult-task use. He points to online complaints as well; a matching controlled comparison is not established here. His broader response is to optimize the architecture and the entire training stack for edge models, including failure modes that become more severe when capacity is small.

Agentic reinforcement learning offers another way to work within a small model’s capacity. Memory limits restrict stored knowledge; in Labonne’s account, that makes unsupported answers more likely. Web search changes the task from recalling every fact to finding the information needed for an answer. A tiny model can retrieve knowledge instead of relying entirely on what its parameters retain.

From his experience, small models can perform well on agentic tasks. The critical capability is then reasoning well enough to use tools reliably: deciding what to look up, interpreting the result, and continuing the task. Long context presents a related opportunity. A recursive language-model environment can let the model use Python to work with an input outside its immediate context, rather than requiring it to process everything directly in one pass. Labonne presents this as a route around weak long-context capabilities, not as a measured Liquid implementation in this talk.

This makes edge-model design interesting both scientifically and in production: limitations can be addressed through the surrounding system as well as through the weights. Labonne sees small-model agentic workloads as underexplored, with large models not necessarily the best fit for every deployment. At the time of the talk, Liquid was developing LFM3 and testing further architectural and training ideas.

The first audience question asks how to choose between small and large models in a workflow. Labonne identifies three reasons to use a local small model:

• Offline operation: A car cannot always depend on a reliable internet connection.
• Latency: Local inference can be attractive when response time is central to the workload.
• Privacy: Finance, healthcare, and other regulated settings can favor keeping processing on the device.

When the questioner clarifies that they mean his personal workflow, Labonne distinguishes making these models from using them: he builds them for other people and does not necessarily use them in his own workflow.

The final question returns to doom loops. Could a larger model first learn to avoid looping through RL, then pass that behavior to a smaller model through distillation—saving the student from repeating the anti-loop training stages? Labonne says experiments are needed. He suspects distillation alone would not completely solve the problem because it can resemble SFT, the stage that barely reduced looping in his earlier example.

He leaves room for the distillation formulation to matter, but offers no tested recipe. His expectation is that additional training batches would still be needed to prevent recurrence. A capable teacher may supply useful behavior; the small student still needs to be evaluated and trained against its own failures.