AI Engineer Europe 2026
Everything I Learned Training Frontier Small Models
Read the talk
Training Small Models for the Devices That Run Them
Memory limits, latency, and narrow tasks change how small models should be built—from embedding budgets and device profiling to preference training that suppresses repetitive loops.
From a talk by Maxime Labonne
Before you start: Familiarity with language-model parameters, token generation, and the distinction between pre-training and fine-tuning will help.
What fits on a phone or in a car?
What should a model retain—and what should it do exceptionally well—when it has to run on a phone or inside a car? Maxime Labonne approaches that problem as Liquid AI’s head of pre-training. The models he describes span 350 million to 24 billion parameters, with text, vision, and audio variants available on Hugging Face. At the time of the talk, Liquid had recently released a 450M vision-language model and updated its 350M text model.
On-device deployment imposes three connected constraints:
- Memory: The available hardware limits model size, which in turn limits how much knowledge the model can store.
- Task specificity: Limited capacity makes specialization valuable. Summarization or tool use can be a better target than general-purpose conversation.
- Latency: Interactive workloads need fast inference and enough throughput to keep the application responsive.
Small models are not simply scaled-down large models. Their architecture and training must allocate a limited budget toward the capabilities the deployment actually needs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Where the parameter budget goes
The smallest members of larger model families already use hybrid architectures. Gemma 3 270M combines sliding-window attention with grouped-query attention, or GQA. Qwen3.5-0.8B combines gated DeltaNet with gated attention. But Labonne’s first concern is not their sequence-processing blocks: it is the share of the model occupied by embeddings.
Labonne reports that embeddings account for 63% of Gemma 3 270M’s parameters and 29% of Qwen3.5-0.8B’s parameters. He calls the remaining budget the model’s effective parameters: the blocks available for reasoning and knowledge processing beyond the embedding layer. Embeddings still perform useful work, including representing a broad vocabulary; the design question is whether that vocabulary deserves so much of a tiny model’s memory footprint.
Labonne attributes these large embedding allocations to distillation from teachers with large vocabularies. Carrying that vocabulary into the student preserves a substantial embedding matrix even as the rest of the network shrinks. His proposed opportunity is to allocate more of the same memory footprint to the non-embedding network rather than inherit every architectural proportion from the teacher.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Choose operators on the target hardware
LFM2 also uses a hybrid architecture, combining short convolutions with GQA. Labonne reports an embedding share of about 19% for the LFM2 configuration shown. That leaves more of its parameter budget for the rest of the network. The team selected the architecture by implementing candidate operators on two target devices and profiling their actual performance.
The resulting gated short-convolution block was particularly attractive for latency. In the presented operator comparison, short convolutions cost less than sliding-window attention, gated DeltaNet, gated linear attention, and GQA. The important design step is to measure the operators where they will run: an appealing theoretical architecture is not enough if its kernels are expensive on the deployment hardware.
The CPU comparison covers an AMD Ryzen AI Max+ 395 and a Samsung Galaxy S25 Ultra; the slide labels the phone platform Qualcomm Snapdragon Gen4. It compares prefill, decode, and memory using llama.cpp, 4-bit quantization, and 2K input tokens. Labonne reports faster inference and lower memory use for LFM2, while explicitly noting that the compared models differ in size. These are deployment comparisons, not a controlled isolation of architecture at equal parameter counts.
The GPU comparison extends the argument to serving: Labonne reports strong aggregate throughput even at high concurrency. The companion measurements use an H100 SXM5 with BF16 and SGLang 0.5.9, 1,024 input tokens, up to 256 output tokens, and sustained concurrent requests. This measures serving throughput, which is distinct from the latency of one request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A 350M model trained on 28 trillion tokens
The LFM2.5 recipe uses familiar stages: pre- and mid-training, supervised fine-tuning, preference alignment, and reinforcement learning. The surprising choice is the training budget. Liquid trained LFM2.5-350M, a 350-million-parameter model, on 28 trillion tokens. That is a training-token count, not a claim of 28 trillion distinct tokens.
That budget can look excessive through the lens of Chinchilla’s compute-optimal training study. But a fixed training-compute optimum is not a ceiling on useful training. Labonne reports that performance continues to improve as the number of pre-training tokens grows. He connects that observation to Roberts and colleagues’ Test-Time Scaling Makes Overtraining Compute-Optimal, which considers training and inference together, including inference sampling.
In Labonne’s application of those newer scaling laws, even 28 trillion tokens falls short of the proposed optimum for the 350M model. The practical implication is that more pre-training can remain valuable at very small scales. Small models also cost less to train than substantially larger ones, making continued training an accessible way to improve a model whose deployment size must remain fixed.
The next comparison is between the post-trained LFM2-350M and LFM2.5-350M, so it does not isolate the effect of additional pre-training. Labonne reports improvements across several capability categories:
| Capability | Evaluation |
|---|---|
| Knowledge | GPQA Diamond |
| Instruction following | IFBench |
| Data extraction | CaseReportBench |
| Tool use | BFCL, tau2-bench |
For this model, the explicit priorities are data extraction and tool use. Leading code or mathematics performance is less important than being reliably useful at the tasks people will actually assign to a 350M model. Choose capabilities instead of aiming for average performance everywhere.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adapt the stages to limited capacity
Small and large models can share the same post-training stages without using those stages in the same way. For supervised fine-tuning, Labonne recommends a narrow target. Taking an available model and fine-tuning it to call one particular function is an excellent fit: the narrower the task can be made, the more directly the training can develop the required behavior.
For preference alignment, Liquid uses an on-policy, length-normalized form of direct preference optimization, or DPO. Labonne values this stage for broad improvements in response quality: the model’s answers improve in ways that are not captured solely by benchmark scores.
Reinforcement learning is effective even at very small scales in Labonne’s experience, but its learning signal can be narrow. Using many environments and tasks helps the model generalize. Small models are also sensitive to their cold start: the SFT mixture should contain examples resembling the tasks the model will later face during RL.
When an RL task fails to train well, the remedy may begin before RL:
- Check whether the SFT mixture contains similar tasks and examples.
- Consider whether the task is too complex for the model’s current capabilities.
- Add appropriate cold-start examples and repeat supervised fine-tuning.
- Retry RL and check whether learning improves.
Poor RL progress is useful feedback about the earlier training mixture, not merely a reason to keep running the same optimization longer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make repetitive failures into rejected answers
A doom loop occurs when a model repeats a sequence of words indefinitely instead of completing its answer. The recipe response shown in the talk shows that this is not confined to mathematical reasoning. Still, three risk factors compound: a tiny model, reasoning mode, and a task that is too difficult. A small reasoning model tackling very hard mathematics is especially exposed.
Liquid’s first intervention happens during preference alignment, beginning with on-policy data generation. The policy model—the model being trained—generates its own candidate answers, allowing the training data to expose its particular failures:
- Start with roughly one million prompt samples.
- Generate five rollouts per prompt using temperature sampling. Diversity increases the chance that at least one answer avoids looping.
- Generate one additional rollout at temperature zero, which the team expects to expose looping more readily.
- Send all six responses to an LLM jury for scoring.
- Select the highest-scoring response as
chosenand the lowest-scoring response asrejected.
The goal is for a looping response to become the rejected side of a preference pair. The flowchart also includes heuristic filtering after selection, although the talk does not specify those filters.
The selection step can be expressed directly in Python. It consumes scored rollouts; generation and jury scoring happen before this function. A tied best and worst score provides no preference signal, so the function returns no pair:
python
from collections.abc import Sequence
def select_preference_pair(
prompt: str,
scored_rollouts: Sequence[tuple[str, float]],
) -> dict[str, str] | None:
if len(scored_rollouts) != 6:
raise ValueError("Expected five sampled rollouts and one greedy rollout")
rejected, worst_score = min(scored_rollouts, key=lambda item: item[1])
chosen, best_score = max(scored_rollouts, key=lambda item: item[1])
if best_score == worst_score:
return None
return {"prompt": prompt, "chosen": chosen, "rejected": rejected}
This selection does not itself detect repetition. It depends on the jury assigning worse scores to the failed answers so that preference training receives the intended signal.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reward completion, penalize repetition
Preference training teaches the model to favor better responses over its own looping failures. The second intervention uses reinforcement learning with verifiable rewards, supplemented by an n-gram repetition penalty. For a mathematics task, the reward process tries to extract the final answer. A rollout that never produces a final answer cannot receive a positive correctness reward. That already creates pressure against endless repetition.
The repetition penalty adds a direct cost for repeated sequences, while temperature sampling keeps RL rollouts diverse. Together, these mechanisms address both the training signal and the supply of candidate responses: loops are less attractive to optimize toward, and the model has more opportunities to produce a successful alternative.
For LFM2.5-1.2B-Thinking on difficult tasks across multiple benchmarks, Labonne reports a doom-loop rate of approximately 15–16% after mid-training. SFT barely changes the rate, despite having no doom-loop examples in its training data. DPO substantially reduces looping, and subsequent RL makes it almost nonexistent in his reported evaluation. The talk does not specify the full prompt set, decoding settings, output limits, or loop detector, so those stage comparisons should be read within that evaluation.
Labonne also reports a doom-loop rate above 50% for Qwen3.5-0.8B in reasoning mode under difficult-task use. He points to online complaints as well; a matching controlled comparison is not established here. His broader response is to optimize the architecture and the entire training stack for edge models, including failure modes that become more severe when capacity is small.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move knowledge and context into tools
Agentic reinforcement learning offers another way to work within a small model’s capacity. Memory limits restrict stored knowledge; in Labonne’s account, that makes unsupported answers more likely. Web search changes the task from recalling every fact to finding the information needed for an answer. A tiny model can retrieve knowledge instead of relying entirely on what its parameters retain.
From his experience, small models can perform well on agentic tasks. The critical capability is then reasoning well enough to use tools reliably: deciding what to look up, interpreting the result, and continuing the task. Long context presents a related opportunity. A recursive language-model environment can let the model use Python to work with an input outside its immediate context, rather than requiring it to process everything directly in one pass. Labonne presents this as a route around weak long-context capabilities, not as a measured Liquid implementation in this talk.
This makes edge-model design interesting both scientifically and in production: limitations can be addressed through the surrounding system as well as through the weights. Labonne sees small-model agentic workloads as underexplored, with large models not necessarily the best fit for every deployment. At the time of the talk, Liquid was developing LFM3 and testing further architectural and training ideas.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
When to deploy locally—and what distillation may miss
The first audience question asks how to choose between small and large models in a workflow. Labonne identifies three reasons to use a local small model:
- Offline operation: A car cannot always depend on a reliable internet connection.
- Latency: Local inference can be attractive when response time is central to the workload.
- Privacy: Finance, healthcare, and other regulated settings can favor keeping processing on the device.
When the questioner clarifies that they mean his personal workflow, Labonne distinguishes making these models from using them: he builds them for other people and does not necessarily use them in his own workflow.
The final question returns to doom loops. Could a larger model first learn to avoid looping through RL, then pass that behavior to a smaller model through distillation—saving the student from repeating the anti-loop training stages? Labonne says experiments are needed. He suspects distillation alone would not completely solve the problem because it can resemble SFT, the stage that barely reduced looping in his earlier example.
He leaves room for the distillation formulation to matter, but offers no tested recipe. His expectation is that additional training batches would still be needed to prevent recurrence. A capable teacher may supply useful behavior; the small student still needs to be evaluated and trained against its own failures.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Model weights, architecture details, inference examples and tool-calling formats for Liquid's 350M text model.
Roberts and colleagues jointly optimize model size, training tokens and inference sampling under an end-to-end compute budget.
The original Chinchilla study of model size and training-data allocation under a fixed training-compute budget.
Architecture, model weights, thinking-mode benchmarks and deployment instructions for Qwen3.5-0.8B.
Further reading
Training-token budget, task benchmarks and deployment measurements for LFM2.5-350M.
Explains Gemma 3 270M's embedding budget and intended use in specialized, locally deployed applications.
- LFM2.5-1.2B-Thinking releaseArticle
Liquid's reasoning-model release describes its training approach and benchmark results.
Research on processing long inputs through an external environment and recursive model calls.
Updates since the talk
A July 2026 follow-up introduces a targeted training method for reducing repetitive loops in reasoning models.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hi everyone.
- 0:15
My name is Maxime Labonne. Uh, in this presentation, I want to talk about the lessons I've learned, uh, pre-training small models. Um, so for context, I work at Liquid AI as head of pre-training.
- 0:26
At Liquid, we mostly focus on edge models for on-device deployment. And as you can see here, we have models from three fifty million parameters to, uh, twenty-four billion parameters.
- 0:38
So this is very, very small. And, um, yesterday we released our new vLLM, uh, four, uh, fifty M and, uh, the week before we released the new version of the, uh, three fifty M model for text.
- 0:51
Um, so this is what we do. We work across text, uh, vision and audio and, uh, yeah, the models are available on Hugging Face if you want to try them out.
- 1:01
Um, in this presentation I want to talk about what separates small models and big models, and there are three main characteristics I want to talk about. So first of all, uh, the small models, they are memory-bound, um, because the hardware is, is what it is, right?
- 1:17
On a phone, in a car, et cetera. Uh, we can't really use super big models, uh, which is why we try to keep the, the size quite small. And because of that, we have low knowledge capacity compared to bigger models.
- 1:29
Uh, then the models are task specific, which is great because if you have a small knowledge capacity, you can at least focus on one thing very well. And so that means that they are usually not general purpose chatbots like ChatGPT.
- 1:42
They are a lot more narrow in terms of focus, and they can do something like summarization and tool use very, very well. Uh, so that's the second aspect. Uh, and the final one is that it's very latency sensitive, and that means that you need to have very, very fast, uh, throughput.
- 2:00
So all these characteristics are very important and we'll see in this presentation how they play with each other and how we can do better. Um, but the main lesson I want you to retain from this presentation is that small models are not just scaled-down versions of bigger models.
- 2:14
They also have their unique challenges, and we will see about how we do it, um, in this presentation. The first thing I want to talk about is the model architecture because there's a lot of interesting things that we can do here for edge models.
- 2:28
Um, I want to first talk about, uh, Gemma 3 two hundred seventy M and Qwen 3.5 zero point 8B. So these models are the smallest version of their respective family.
- 2:38
And you can see that both of them, they adopt a hybrid architecture. Uh, Gemma 3, uh, has sliding-window attention and GQA hybrid. Um, Qwen 3.5 has a new architecture with, uh, gated, um, DeltaNet and gated attention.
- 2:53
This is great because this is a lot faster. But what I'm interested in here is actually the embedding layer because if you look at, uh, the size of the embedding layer compared to, uh, all the parameters of the model, you see that actually Gemma 3 two hundred seventy M is mostly an embedding layer.
- 3:08
Uh, it's sixty-three percent of the total parameters. And even Qwen 3.5 zero point 8B it's, it's still like twenty-nine percent of the parameters. Um, so that's not super efficient because, uh, the effective parameters, the parameters that are really used for reasoning, for knowledge capacity and all that stuff, um, are not the embedding parameters.
- 3:29
Um, they are-- it's the rest. So the effective size is actually a lot smaller and it means that you could squeeze more reasoning and, uh, more performance from the same memory footprint.
- 3:39
Um, and the reason why they do that is because they use distillation, uh, to train the models. Um, so they distill these models like those are the, uh, student models and they have teacher models with a huge vocabulary, uh, sizes and this is why, um, we have these super big, um, embedding layers.
- 3:59
All right. Uh, let's talk about the LFM2 architecture now. As you can see, the LFM2 architecture is actually not that different, uh, in terms of, um, just layers. We also have like a hybrid architecture and this time we have short convolutions and, uh, GQA.
- 4:16
Um, and I want to talk a bit about-- Well, first you can see that, um, the embedding layer is actually a lot smaller compared to the others. It's like nineteen percent of the parameters.
- 4:25
So we have more effective, uh, parameters which is great. And I want to talk about how we created this architecture and we did on-device profiling. So instead of doing more like theoretical work, uh, we decided to say like, "Okay, let's try to really implement it on the target hardware."
- 4:42
So we had two target hardware here and, uh, we wanted to see how it performs in real life to be able to optimize, um, the architecture, find the right operators here.
- 4:53
And the thing that we found is this gated short convolution block that you can see here. And why this is nice, it's because it's very, very fast. Uh, the short conv are a lot faster than all the alternatives that you can see here, uh, compared to sliding window attention from Gemma 3, the gated DeltaNet from Qwen 3.5,
- 5:13
uh, gated linear attention and, uh, grouped-query attention. You can see that, uh, the cost ratio is really in favor of short conv which is great because we said that, um, this is very latency sensitive.
- 5:23
So this is exactly what we want. So this is quite theoretical, but if we look in practice, um, and we profile, uh, the inference of these models, you can see here on the two CPU, um, the AMD Ryzen, um, Max plus three nine five and the, uh, Samsung Galaxy S25 Ultra.
- 5:42
Um, all these models don't have necessarily the same size, but it gives you a rough picture and you can see that the short conv really allow LFM2 architecture to be a lot faster and also use, uh, less memory.
- 5:54
Um, so that's great. And you can see also GPU. It's not really just for CPU, but also on GPU you can see that it has a lot of, uh, throughput, uh, even at very high, uh, concurrency levels.
- 6:07
All right. Let's talk a bit about training now. So, [clears throat]
- 6:12
the LFM2.5 training recipe is quite similar to what you can five-- find, uh, elsewhere in terms of stages. We have pre and mid training on twenty-eight trillion tokens. We have, uh, supervised fine-tuning, preference alignment, and reinforcement learning.
- 6:28
And here you can see I'm talking about twenty-eight trillion tokens, and I said that we released a model of, um, three fifty M parameters last week. Um, so yeah, we pre-trained a three fifty million parameter model on twenty-eight trillion tokens.
- 6:43
If you're familiar with, uh, Chinchilla scaling laws, that might sound a bit weird because we're supposed to be compute optimal at like, I, I don't know, like maybe one billion, not, not even one, uh, trillion parameter, I mean.
- 6:56
Um, but it's actually not the case and we see that the performance still, um, grows when you scale the number of pre-training tokens. And there was a super interesting paper by Roberts et al., uh, published last week about the test time scaling laws.
- 7:11
And you can see here how LFM2.5-350M compares to their new scaling laws. You can see Chinchilla scaling laws here and the new ones that they proposed, uh, here. And actually, we did not pre-train the models on N of tokens.
- 7:25
We should pre-train even more, uh, to be, uh, optimal according to their laws. But this is cool because more pre-training works, and it works even at the smallest scale, which is great because these models are a lot cheaper to train than, uh, much bigger models.
- 7:40
And here you can see a comparison. It's not just for pre-training, it's like post-training, uh, models. And, um, you can see that the LFM2.5 model is, uh, significantly better than the, the previous version, LFM2-350M on a lot of different benchmarks.
- 7:57
So you have, uh, knowledge with GPQ Diamond, you have instruction following with IF bench, you have case report bench, which is data extraction, and also a lot of tool used with PFCL and, and Tao two bench.
- 8:09
Um, with this model, it's only three fifty million parameters, so what we wanted to do is we wanted the model to be very, very good at data extraction and at tool use.
- 8:19
And the rest, if it's not the best model at-- in code, it doesn't matter. Like, people don't use it that way anyway. Same for math. I think it's really nice to try to target some capabilities and not try to be like average on everything.
- 8:31
All right. Let's talk exactly about that. Post-training, um, small and big models, what the difference. Um, we have pretty much the same stage, so this is not really in terms of stages that we see a difference.
- 8:43
It's more about how you do it. So for supervised fine-tuning, it's better if you're actually quite narrow and you focus on some task. Um, it's true for general purpose post-training, but it's also true if you do fine-tuning.
- 8:55
So you can take one of these models on Hugging Face and just fine-tune it for your use case. And for example, you have a use case where you have a, a particular function that you want to call.
- 9:04
This is great. This is a excellent use case. Like, the more narrow you can, um, you can find it or design it, uh, the better it is. Then we have preference alignment.
- 9:14
So during post-training, we have our, um, own, uh, on-policy length normalize direct preference optimization algorithm that we quite like. And preference alignment is very nice because it brings you general improvements.
- 9:27
It's not just about benchmarks. It's really like overall, after preference alignment, the model is better, it sounds better, and this is really nice to be able to, um, just improve it overall.
- 9:39
And finally, we have reinforcement learning. And reinforcement learning is extremely efficient, even at very small scale. It's a really, really important technique that we use everywhere. And the main thing is that it's very narrow in terms of focus, so you want to have like as many environments, as many tasks as possible and make sure that, um, you
- 9:59
generalize well thanks to this. Um, and then for small models in particular, they're quite sensitive to cold start SFT data. So if you have a particular task in reinforcement learning, it's always good to have similar samples and a similar task in your supervised fine-tuning mixture.
- 10:18
And this is good feedback that you can see during reinforcement learning. Something doesn't train very well, it's probably because you are missing some, uh, cold start SFT data. Maybe the task is too complex.
- 10:28
There are different reasons. But, um, you can try to start again, uh, from the supervised fine-tuning, uh, stage, add your data, and then see if it improves anything.
- 10:41
All right. But there's a new problem, uh, with small language models that you might have encountered even with bigger ones, and this is Doom looping. So the problem with Doom looping is, as you can see here, it's going to start repeating a sequence of words over and over and over and over again, and it just like never
- 10:57
stops. Um, so this is a problem all the time, but it's particularly a problem if you have small models, if you have reasoning models, and if you have complex tasks.
- 11:07
If the task is basically too complex for the model. Hopefully, this recipe is not too complex for this model, but this can happen anywhere. And you have the three of them at the same time.
- 11:17
So if you have a tiny reasoning models on like super difficult math task, this is the perfect recipe to have a lot of Doom loops. Um, so this is a unique challenge that you find with small models, uh, to give you a concrete example.
- 11:33
And here I can talk about like how we solve it. The first thing is that we solve it during the preference alignment stage, and in particular for the data generation part that we do.
- 11:45
So here you can see the pipeline that we use, uh, to do the data generation, the on-policy data generation for preference alignment. So we start with prompts for like one million samples to give you a rough idea, and then we use the policy model, the model that we want to train, with temperature sampling, and we just generate
- 12:04
five rollouts. Because we use temperature sampling, these rollouts tend to be a lot more diverse, and we expect that not all of them will, uh, have Doom loops. At least one should not Doom loop, right?
- 12:17
And on the other hand, we generate just one extra, uh, rollout with a policy model with temperature zero. And this one we think that it's going to Doom loop And then we give everything to LLM jury to score all the, um, rollouts.
- 12:33
We pick the best one. The one with the highest score is the chosen answer, the one with the worst score as the rejected answer. And the idea is that if we have some Doom loop here, the response for the Doom loop will be rejected.
- 12:46
So we will train the model during preference alignment to not Doom loop. And this is quite effective. So this is solution number one. And then we have solution number two, and this one is about using reinforcement learning with verifiable rewards, and we add a bit of n-gram repetition penalty.
- 13:05
Uh, but you can see that with reinforcement learning with, um, verifiable rewards, it's a very nice way to actually, um, solve this issue. Because if you have a question, like a math question like this one,
- 13:20
you are going to try to extract the final answer. If you do not have a final answer, uh, you won't get a positive reward. So this is already being taken care of during, um, reinforcement learning with verifiable rewards.
- 13:35
But on top of that, you can add a bit of repetition penalty to make sure that, um, you are going to generate more, like less Doom loops in general.
- 13:45
And same thing, we also use temperature sampling here, so the rollouts are also quite diverse, and it just is less likely that you are going to get like a lot of Doom loops, uh, all the time.
- 13:57
So this is the, the second, uh, solution, and that allowed us to, uh, really reduce the Doom loop ratio. So this is a real example with, uh, LFM 2.5 1.2B thinking, which is a small model.
- 14:11
It's a reasoning model. And on top of that, we threw really hard task at it. So you can see that after mid-training, the Doom loop ratio that we calculated across like a lot of benchmarks was about fifteen percent, um, or even sixteen percent.
- 14:27
And then after SFT, it, it barely moves. Like SFT is not the right stage to fix this. Um, we didn't have Doom loop, uh, examples during the SFT stage, but it's not enough, uh, to get rid of this issue.
- 14:40
Um, after DPO, so that was a first solution, um, it really reduces quite a lot, and you can see that after reinforcement learning the problem is almost non-existent. If today you try to do the same thing with, um, Qwen 3.5-0.8B in reasoning mode, you will see a lot, a lot, a lot of Doom loops, like over fifty
- 14:59
percent of Doom loops. Just something that people complained about online, and that also shows that the Qwen 3.5, like this tiny model, is just a scaled-down version of bigger models.
- 15:09
And this is not the approach that we're taking here at Liquid. We want to say, "Okay," like the edge models, they are their own thing. And, um, this is also a way to just optimize the entire architecture, the entire pre-training stack to make sure that, uh, we treat them as, um, as best as possible.
- 15:28
And finally, I want to talk about, uh, next stage, next steps for, uh, all these, uh, small models with agency reinforcement learning. The final characteristic I didn't mention here is about, uh, being memory bound.
- 15:40
If you're memory bound, it means that you have low knowledge capacity. If you have low know- low knowledge capacity, it means that you're going to hallucinate a lot. But a nice way to solve this issue is just providing like web search tools to the model.
- 15:53
If you have a tiny model that is able to Google everything that you, um, throw at it in terms of like knowledge questions, you're going to have like much, much better performance, uh, than if you just rely on the, uh, base models.
- 16:07
And same thing with, um, a, a lot of problems that you can, uh, throw at the model. I think that from experience, these tiny models are actually very good at agentic task, and this is how we should use them.
- 16:19
Um, it doesn't matter if, um, they don't have the knowledge capacity of big models. What they truly need is really good reasoning capabilities to make sure that they are able to use these tools in a reliable manner.
- 16:34
And another point that I haven't mentioned here is that small models are also not very good at long context capabilities. But it's okay, because if you have like a recursive, um, language model environment, then you can use Python and, and like basically take a shortcut, uh, to, to solve this issue.
- 16:52
So most of the issues that you find with small language models can actually be fixed in different ways. It just requires more creativity. It just requires thinking about this problem, not like you would think about it, uh, from a bigger model perspective.
- 17:06
Uh, but everything about this is fixable. All right, so in conclusion, some takeaways. Um, I hope I convinced you that edge models have unique challenges, and they are actually interesting from scientific point of view and also production point of view.
- 17:21
Um, if you combine them with agentic tools, they tend to perform really, really well, and this is something that is currently underexplored. We talk about agentic workloads with really big models, but it's not necessarily, um, the best use case.
- 17:35
It's not necessarily the best fit all the time. And, uh, yeah, finally, we're working on LFM3, and we have like a, a ton of crazy experiments and, uh, ideas to try.
- 17:45
So, um, come work with us if you're interested, uh, in this space. Thank you everyone. [audience applauding]
- 17:58
Yes.
- 17:59
Um, can you share a bit about how you use these models in your workflow and how, how you make the decision about when to use a big model or a small model?
- 18:07
Yeah, this is a, a good question. So the question is like how we use this model in the workflows and, uh, how we decide between small models and big models.
- 18:15
So the, the main idea here is that you will try to use the small models when you don't have a internet connection, for example. So in-car deployment is a good example of that because y- you can't have like a reliable internet connection, so it makes sense.
- 18:31
Uh, latency is also a big one. If you have a workload that is very latency sensitive, uh, small models running locally are always going to be better. And another one is, is privacy.
- 18:42
If you use a regulated environment, um, if you work in finance or healthcare, this is also a good one.
- 18:47
Yes. Like in, in your specific workflow is the question. Like do you use-
- 18:50
It-- I make the models, so I... Like, I make the models for other people.
- 18:56
Okay.
- 18:56
But, uh, like in my workflows, like not necessarily. Yes.
- 19:00
Have you tried for Doom looping, have you tried like... You, you, you talked about how Qwen 3.5 is a scaled down version of a bigger one.
- 19:07
Yep.
- 19:07
Have you seen, uh, the work that you do with RL and all this, uh, re- reduction of Doom looping on a bigger model, uh, be able to be distilled into a smaller model without having to re- redo the steps?
- 19:19
It's a good question. I think we need to do some experiments to see like if, uh, just distilling from a bigger model translates well in terms of Doom looping.
- 19:27
I would say no. I, I don't think so because I think it would be too close to SFT. It depends like how you do this distillation. If you stop K and you have like enough, uh, K, maybe this is, uh, good enough.
- 19:40
Um, but I, I think that it would not completely be solved, and you would still need like several batches to make sure that it doesn't happen again.
- 19:48
Okay.
- 19:48
Right. Yeah. All right. I, I think I should quit, but, um, I, I'll be around if you, you have oth- other questions. Thank you very much.
- 19:55
Thank you. [audience applauding] [outro music]