AI Engineer Europe 2026
You Might Not Need 50 Diffusion Steps
Read the talk
You Might Not Need 50 Diffusion Steps
Making diffusion generation usable means reducing latency without losing quality. Quantization, caching and step distillation attack different parts of that problem—and can be combined.
From a talk by Ziv Ilan
Before you start: Basic familiarity with diffusion denoising, model inference and GPU memory will help; no prior experience with distillation is required.
From good images to usable latency
Why does generating a good image still take too long for an interactive application? A diffusion model repeatedly denoises an image or video rather than emitting one token at a time. Ilan describes typical generation as taking 20–50 denoising steps. That repeated computation is the starting point for the latency problem.
Models such as FLUX.2, LTX 2.1 and Google’s Nano Banana have made image and video generation useful across more applications. But a developer or enterprise needs more than an impressive output: generation must be fast, scalable and supported by dependable serving infrastructure. Ziv Ilan, who introduces himself as working with frontier model builders in NVIDIA’s Paris AI Labs team, frames diffusion serving as less mature than the autoregressive LLM ecosystem around vLLM. Some of that ecosystem’s optimization ideas can transfer, but they need to fit diffusion’s computation pattern.
The target is real-time image and video generation for robotics world models, computer games and content creation. Latency matters both before the first image appears and while producing high-quality 720p or 1080p content. Three techniques address different parts of that cost: quantization, caching and distillation. They are a selected toolkit, not an exhaustive list of diffusion optimizations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make each computation cheaper
Quantization is the simplest entry point in this presentation, followed by caching and then distillation. That teaching order differs from the order Ilan describes for deploying a fully optimized model: distill first, then quantize, then apply caching.
| Approach | When precision is reduced | Practical consideration |
|---|---|---|
| Post-training quantization (PTQ) | After training | Simpler adoption; preserving generation quality can be difficult |
| Quantization-aware training (QAT) | Accounted for during training | Requires a training process that accommodates reduced precision |
Diffusion introduces a particular constraint: image and video quality must survive the precision reduction. Ilan also describes these models as attention-heavy, limiting the gains from quantization relative to LLMs and VLMs. Even so, lower precision is an accessible way to exploit features of Blackwell and newer hardware.
The FLUX.2 work with Black Forest Labs illustrates a second choice: static versus dynamic scaling. Static scaling establishes ranges ahead of execution; dynamic scaling computes ranges as needed so they can adapt to the values being processed. NVFP4 has multiple scaling levels: per-tensor scaling may be static or dynamic, while per-block scaling is computed dynamically for blocks of 16 elements. The important distinction is when a scale is determined, not whether every model parameter receives a fixed range upfront.
There are two adoption paths. TensorRT-LLM VisualGen provides an open-source implementation to work from. Partner pre-quantized checkpoints on Hugging Face offer a more direct loading path when subsequent fine-tuning or LoRA adapters are unnecessary. Quantization reduces memory requirements, potentially bringing a model within reach of consumer or lower-end data-center GPUs, and can improve inference performance as well. The slide pairs BF16 and NVFP4 images with these two ways of running quantized FLUX.2.
Attention remains an optimization target in its own right. Ilan points to recent FP4-attention research from Hao Lab as an example of work aimed at this expensive part of diffusion models. Improving the rest of the model’s arithmetic does not eliminate the need to address attention cost and quality.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reuse work that has barely changed
Autoregressive serving makes extensive use of the KV cache, preserving attention keys and values from previously processed tokens. Diffusion does not append one token on every denoising step, so that reuse pattern does not transfer directly. Its opportunity is different: successive denoising steps can contain sufficiently similar computation that some work need not be repeated.
TeaCache provides a concrete example. It estimates changes in model outputs using timestep-modulated inputs as a proxy, then uses that estimate to decide whether expensive computation can be reused. It does not first calculate the complete output merely to discover that calculating it was unnecessary. Ilan presents this as a broad reuse decision over the pixel or latent space, before contrasting it with more granular, chunk-based approaches.
For chunk-based reuse, consider the room in which he is speaking. Much of the audience sits still, watching the screen, while the presenter moves. The audience region is a candidate for reuse; the presenter’s region needs recomputation. A threshold controls how much change is acceptable before cached work must be refreshed. This classroom analogy explains selective reuse, rather than the specific internal algorithm of TeaCache.
A small Python function makes that decision boundary explicit. Given estimated changes for the audience and presenter chunks, it returns a proposed action for each; it does not perform a denoising step.
python
def plan_chunk_updates(change_estimates, threshold):
return {
chunk: "reuse" if change < threshold else "recompute"
for chunk, change in change_estimates.items()
}
change_estimates = {"audience": 0.01, "presenter": 0.20}
plan = plan_chunk_updates(change_estimates, threshold=0.05)
# {"audience": "reuse", "presenter": "recompute"}
These teaching values show why the same threshold can preserve one chunk’s cached computation while requesting fresh computation for another.
The threshold is a quality decision as well as a performance decision. Reusing too aggressively can significantly damage the generated image. Evaluate the resulting content, not just the saved computation. In the talk’s VisualGen interface, caching is enabled with a flag and configured with a threshold; the current beta documentation exposes cache_config, including cache_backend and teacache_thresh. Ilan also names vLLM Omni and SGLang Diffusion as serving options with caching support.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reduce steps instead of parameters
Distillation often brings to mind DeepSeek and the transfer of capabilities from a large teacher into a smaller student. The diffusion objective here is different: keep the parameter count, but reduce the number of denoising steps. The student learns to produce a good image or video with fewer repeated evaluations of the model.
Ilan gives the example of reducing 50 denoising steps to four, eight or sometimes one while aiming to preserve quality. This is not simply a request to stop an ordinary sampler early: the student is trained for the shorter generation process.
Ilan cites potential performance improvements of 10×–200×, without specifying a benchmark baseline, hardware configuration or quality metric. The objective is to make high-quality real-time generation possible, and he reports that a GTC demonstration in San Jose used two distillation techniques to achieve real-time generation. Streaming output would make these models usable in a broader range of interactive applications.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn the teacher’s path or its output distribution
Both main distillation families use a teacher and a student, but they differ in what the student must reproduce.
| Family | What the student learns | Constraint on the path |
|---|---|---|
| Trajectory-based | The teacher’s denoising trajectory | Follow the teacher’s progression |
| Distribution-based | The teacher’s output distribution | May take a different route |
Matching a distribution does not require retracing the teacher’s exact denoising path. It gives the student freedom to learn a shorter route to the desired output distribution.
Ilan describes distribution-based approaches as more common and higher quality at the time of the talk. The two families can also be combined: he cites a recent FastVideo release for a hybrid approach that preserved quality while making training more stable.
That training requirement explains why distillation comes last in the presentation. It is a post-training process whose success depends on suitable data and reliable convergence. It requires additional compute, time and expertise, with recipes still evolving through research. Both open-source and closed-source model builders are applying these methods, but selecting a recipe remains more demanding than enabling an inference optimization.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Organize distillation at model scale
NVIDIA FastGen is infrastructure for this work, not another distillation algorithm. Its open-source repository brings multiple techniques into a framework for experimenting with and scaling post-training. Ilan cites video diffusion models with 20B, 30B and 40B parameters, and anticipates models reaching hundreds of billions. At those scales, sharding work across GPUs becomes part of the training problem. FastGen structures that machinery so practitioners can concentrate on output quality and the recipe they want to use.
Training data is optional in the illustrated workflow. Open-source data can provide a useful starting point, but a specialized target distribution changes the decision: if the application’s data differs substantially from general-purpose material, Ilan recommends using the application’s own data for fine-tuning. The purpose is to make the optimized model work for the intended distribution, not merely for a generic demonstration.
The benefit can be lower compute requirements as well as lower latency. Ilan reports near-real-time or real-time video generation at GTC on one Blackwell B200 GPU, depending on output quality. The talk does not specify resolution, frame rate or a quality evaluation for that result. This is an inference demonstration; it does not establish the hardware required to train the distilled model.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Diffusion within frames, autoregression across them
Further opportunities may come from architectures that combine diffusion with autoregressive generation. Ilan mentions Transfusion and autoregressive diffusion as developing directions, then describes a frame-by-frame mechanism: diffusion generates a frame, and successive frames are generated autoregressively. That structure could bring more autoregressive optimization techniques into image and video generation, although he characterizes the area as still research-driven.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Compose the optimizations incrementally
These optimizations can be combined. Distillation reduces the number of steps; quantization reduces the cost and memory footprint of computation; caching avoids some repeated work. Multi-GPU execution and context parallelism offer additional options. Choosing one does not require abandoning the others.
For incremental experimentation, Ilan proposes a practical sequence:
- Start with quantization. If it meets the application’s needs, stop there.
- If more improvement is needed, consider multi-GPU execution, context parallelism and caching.
- Take on distillation when the remaining latency target justifies the more demanding post-training work.
This sequence starts with the easiest intervention to assess. It differs from ordering the stages of a complete optimization pipeline, where distillation can precede quantization and caching. Ilan presents distillation as the most impactful option, but also the one requiring the greatest investment.
The available open-source tooling includes support for the Wan, FLUX.2 and LTX2 model families, with more support being added. The invitation is to try these implementations and contribute improvements, helping diffusion serving approach the maturity already available in the LLM and vLLM ecosystem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What hardware and data does distillation need?
An audience question brings the discussion back to access: if GB200 systems are difficult to obtain, what compute is needed for fine-tuning or distillation, and how large should the dataset be? GB200 is not a requirement. Ilan names Hopper/H100, B200 and B300 as possible hardware choices. Distillation requires less compute than pre-training, but it still needs substantial resources; his answer does not promise that an arbitrary model can be distilled on a single instance.
Model size changes the requirement. Ilan points to smaller video generation models with 2B or 4B parameters as requiring less compute. A single-GPU inference result should therefore not be used as a training budget: the size of the model and the chosen post-training process still determine the resources needed.
On data, the first requirement is knowing how to evaluate the result. Compare what a general-purpose dataset produces against what data specific to the intended application produces. Ilan reports that general demonstrations work without a special dataset, while a domain such as protein generation calls for more specialized data. He gives no numerical dataset-size requirement. The actionable starting point is an evaluation that can reveal whether general data is sufficient for the application—or whether its target distribution requires a different training set.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Current beta guide to supported diffusion models, quantization, caching and multi-GPU inference.
Training-free diffusion caching implementations that estimate when transformer computation can be reused.
Open-source infrastructure for experimenting with diffusion distillation methods and few-step generation.
Video generation framework with inference optimizations, fine-tuning, distributed training and distillation workflows.
Further reading
Explains NVFP4 scaling, TeaCache and combined inference optimizations for FLUX.2, with code examples and benchmark conditions.
Hao AI Lab's study of quantization-aware training for preserving quality with FP4 attention.
Introduces trajectory-based, distribution-based and causal distillation, including a Wan2.1-14B training example.
Updates since the talk
May 2026 release of an LTX-2 video generation and editing application, with deployment instructions and streaming architecture.
Read the complete timestamped transcript
- 0:00
[upbeat music] Okay.
- 0:17
Hope everyone are awake after lunch, and nice to meet you all. I'm Ziv. I'm in the AI Labs team in NVIDIA based out of Paris and working with different, uh, frontier model builders across a lot of domains.
- 0:32
Of course, diffusion is one of them, and we'll hear about a couple of examples of, of work we do with them. Um, we have only twenty minutes, so obviously it's kind of a mix between going deep and going very high level.
- 0:44
Each of these topics will probably be a full day or full, um, conference to cover. So I, I'll try to cover everything I can within this timeframe, but feel free to reach out afterwards, either through LinkedIn or I'll stay here a few minutes afterwards.
- 1:00
So without further ado, diffusion models, I assume everyone are here knows about it. Anyone doesn't know what VideoGen, ImageGen, how they work on a high level, denoising? Perfect. Okay.
- 1:12
The idea is that, of course, unlike autoregressive, um, architectures, LLM, the idea is that you have a lot of iterations to denoise the image or the video, usually between twenty to fifty steps.
- 1:26
And, um, we see, I think in the last year, an influx of very good high-quality models, both for image generation, whether it's FLUX.2, um, video generation, um, LTX 2.1, um, Google with the Nano Banana and, and the later generations.
- 1:44
And we do see a lot of more practical use cases for that. And the main challenge once we have some interesting use cases is how to make it actual usable, right?
- 1:54
We know that it's cool to generate videos or to generate images, but now if we talk about a developer context or a enterprise context, this should be, uh, fast.
- 2:06
Okay? We want it to be mature, we want it to be scalable, and these are usually the challenges that, uh, are hard to solve as this ecosystem is, is not as mature as the autoregressive LLM, vLLM ecosystem.
- 2:19
Okay? So we try to borrow a lot of the concept we see work very well for LLM, and we gradually kind of distill them, if I'll steal this, um, termin-terminology, into the world of diffusion models.
- 2:31
Okay? We'll cover a few of the topics here, but again, it's every day we see more and more research, um, in this domain, and I expect this world to be even more mature in the next AI engineer.
- 2:44
Use case enel-- enablement, real-time image, real-time video is obviously the, the Holy Grail, okay? Imagine how many new use cases where it's, uh, world models for, for robotics, for, um, you know, computer games, for content generation.
- 2:58
It opens a lot of new avenues for companies and developers to use it. And the big challenge together is, of course, the latency. Okay? It takes a lot of time to get a first image and then to obviously get a high quality, okay, if we talk about 1080p or 720p, uh, content out there.
- 3:15
And to bridge this gap, I'll talk about three concepts. Of course, it's not, um, covering all the, uh, the ways you can optimize your VideoGen, ImageGen models, but I'll touch on quantization, caching and distillation.
- 3:29
It's not necessarily the, the order you'll deploy it yourself, okay? Usually, you'll start a distillation, then do some, uh, quantization, then some caching. But I started from the simple to the more complex.
- 3:40
Okay? Simple is usually quantization, okay? For those of you who tried it in LLMs, concepts are quite similar. Okay? Then we'll talk about caching and distillation. When we talk about quantization, we have two approaches, okay?
- 3:55
Post-training quantization and quantization-aware training. Um, I'd say in many cases, of course, we do want to use the more simple approach like PTQ, but we know that at least for the, to maintain the image quality, the video quality, it's a little bit more complex with the diffusion models.
- 4:14
Okay? Uh, we also know that, um, these type of models are more attention heavy, which means that the impact of doing quantization is not as impactful as the LLMs, vLLMs.
- 4:25
But it is s-still quite a, a low-hanging fruit when we talk about taking advantage of the more advanced, um, features of Blackwell, for example, and, and more modern, uh, compute.
- 4:38
Uh, in this example, just the work we did with Black Forest Labs on FLUX.2, you can see that, um, using usually dynamic quantization, okay, you don't wanna use... We can use static, which means that we compute all the range of all the, uh, different parameters upfront, deploy it, and use this static range for the quantization.
- 4:57
In this case, we use dynamic approach, okay, which means that some of the range will, uh, be computed on the fly. Okay? Again, to make sure that the distribution is in line with the different, uh, data distribution that you'll probably want to use when running these models.
- 5:13
It's something that you can either do it yourself. Okay? We recently released a good example in our TensorRT-LLM VisualGen repository, open source. You can start using it and see how it goes.
- 5:25
What we also try to, to do to, again, help the community to adopt it, is also to help our partners to do quant-- pre-quantized, uh, checkpoints. So you can just go to Hugging Face, load the, the quantized checkpoint, and start using it.
- 5:39
Okay? If you don't need to fine-tune or to do some LoRA adapters afterwards, it's something, again, it's quite handy, and you can already see the impact. Of course, when we talk about quantization, the impact is both on the memory.
- 5:51
Okay? It will require less memory, which means you can run it on lower-end GPUs, whether it's consumer GPUs or lower-end data center GPUs. But, um, also something that will help you in the performance.
- 6:05
Okay? So this is one part of the toolkit. Again, a whole world sitting behind it to make sure that it's something that is effective. Just today, I've seen one of the latest, uh, research coming from Hao Lab, um, about Attention FP4, which again, as I mentioned, attention is quite heavy for this kind of model.
- 6:24
So we do try to follow up with the latest research and make sure it's accessible for your devel-- you as a community.
- 6:34
Okay. When we are talking about the second stage, caching, okay? KV cache is something that, you know, anyone that worked a little bit about in-- with LLMs, with autoregressive models, it's, it's something that everyone talk about how to, um, to use it efficiently, how to offload it, et cetera, et cetera.
- 6:51
Again, it's a whole world. With the characteristics of diffusion models, it's not the same way, right? We don't, we don't generate a token every time, so it's harder to use these kind of techniques when we talk about denoising steps or, you know, getting, again, making sure that we use the computation we had before in the way we'll
- 7:14
generate future images or future videos. There are some-- TeaCache is one example. Again, it's not, um, a very strong example, but it's a good example to understand the concept.
- 7:23
Okay? While we are doing denoising steps, right? We talked about twenty to fifty steps. There are areas between the denoising steps that are pretty much the same, okay? So we don't necessarily need to recompute them.
- 7:36
What TeaCache is doing is, okay, if, if there's-- there was a minimal change or very small change between the denoising steps, it compares it, and you underst-- okay, now I don't need to, um, to recompute for the next denoising step.
- 7:51
Okay? So it's more general. Okay? It, it does it for the entire pixel space or latent space. Okay?
- 7:58
More modern techniques of caching will do it in a more chunk based. Okay? Imagine that, I don't know, now we are in the classroom here. Most of you audience are sitting, staring at the, the screen, so nothing much changes.
- 8:09
But I try to be a little dynamic, so I'm, uh, you know, you still wake up and follow me, and which means that this chunk of the video doesn't necessarily need to...
- 8:19
You guys don't need to recompute. I do need to. Okay? So we'll isolate just th-this chunk and recalculate that. Okay? Of course, you can define the threshold, um, and this is something that actually makes a lot of impact.
- 8:32
We provided here some good examples of the expected boost you can get from using this, um, but make sure that you try it, of course, and you maintain the quality.
- 8:42
Okay? Caching is something that if you don't do it the right way, can have quite a significant impact on the quality of the image. Okay? And as content creators, again, world models, et cetera, it's something you wanna make sure that, uh, you maintain while you get the boost.
- 8:58
Okay? Um, so that's caching. Again, I encourage you to read more about different techniques. This is something that is already available in the TensorRT-LLM VisualGen I mentioned. Just a flag you enable, and you set up the threshold, uh, you can experiment with it.
- 9:13
But also it's available, you know, in vLLM Omni, as GLAN Diffusion and other serving, um, libraries.
- 9:22
Distillation. Okay? A-and this goes to the you don't necessarily need fifty steps. Distillation is something we've seen, again, um, I-I'd say probably the big bang for distillation was during the DeepSeek first release, how they managed to distill from a very big model to much smaller models and get, um, you know,
- 9:42
I would say acceptable quality, but with a much, with a much smaller, uh, model. In diffusion, the goal is not to get to a smaller model, okay? You'll still have the same number of parameters.
- 9:53
This is more about step distillation, okay? Training the model, the student model, to generate as good quality images or videos, but by using much less steps. Okay? Instead of fifty steps, going to four steps, eight steps, in some cases one, one shot, okay?
- 10:12
And maintaining the quality. Okay? And this is the big challenge. And imagine if you are able to reduce this significant number of steps, but maintain the quality, it's something that can give you ten x, two hundred x improvement in performance.
- 10:26
And if you go back to the real-time generation, this is something today, it's probably the only way that it can get us there in good quality. Okay? Um, there is the next one, I think there's some demo we did in the last, uh, GTC conference a couple of weeks ago in, in San Jose, uh, with two different,
- 10:46
um, distillation techniques, um, and we got to a real-time generation. Okay? And this is something, again, that everyone are looking for, all the AI labs, and I'm sure also the bigger players, because this is, um, this means that we can actually get to, again, streaming something that will open a lot of new use cases.
- 11:07
Okay, so how do we get it? Okay, we are-- When we talk about distillation, we always have a teacher model and a student model. Currently, we have two main approaches when we talk about distillation.
- 11:17
Okay? One is trajectory-based, which means we'll try the student-- try to teach the student how to follow the trajectory of the denoising steps as the teacher is doing. Okay?
- 11:29
And the second is dis-distribution based, which means we'll only look at the output distribution. Okay? We want the, uh, student to get to the same point at the end, but we'll let the student understand how to get there, okay, and not by following the exact trajectory.
- 11:45
Okay? The more common, I would say, and better quality technique these days is distribution-based, and we also see a lot of ways that can be combined. These techniques can be combined.
- 11:55
Um, in the last fast video release, they actually managed to do kind of a hybrid approach that maintained the quality but also got to a more stable, um, um, training.
- 12:06
The challenge and why I kept it to the last is that distillation usu-usually it's a post-training technique, okay? Which means that you Let's say if you do want it to work with your data, it's something you'll need to use some data for that technique and, and you want it to converge in a good way, right?
- 12:21
'Cause otherwise it will just, um, you know, garbage in, garbage out. Okay? So it will require more compute, it will require more time, also more proficiency. Again, as it's an exploratory still or research driven, uh, domain, there's a lot of different techniques out there, and we expect more to come.
- 12:40
But we are starting to see more mature techniques coming and some very good examples shown in, again, in with the latest open source, um, models. Of course, um, closed source model builders are also using this approach.
- 12:56
So FastGen is something that came out of our, um, NV research group. Okay? It's an open source repository. You can go... There's a lot of different techniques there. It-- Okay, it's not a tech- distillation technique or method, but the idea is that because it's so complex when we talk about, you know, large models, okay, a lot of
- 13:12
the new video diffusion models are twenty, thirty, forty B, uh, parameters, and we expect it again to get to hundreds of billions of parameters. It requires post-training, it requires scale sharding all across different, uh, GPUs.
- 13:27
So to manage all of this, we came with FastGen as a way to structure this process for you and enable you to focus only on the quality and of course, you know, fine-tuning the exact recipe that you wanna use.
- 13:40
Like you can see, there's an optional training data here. If you're not using, um... Well, you can always use open source data and it will work up to a point, right?
- 13:50
And, and we're actually happy about the results there. Um, but if you want it to work for or if your use case have very specific data distribution, then we recommend you to use your own data for the fine-tune.
- 14:03
Um, some of the results quoted here, okay, um, you know, the speed up, it's actually something we got, not just the speed up doesn't come only in time, it's also in using much smaller, uh, much, um, less compute, I would say, to get to real-time.
- 14:20
Okay? We got again, in, in GDC, as I mentioned, we got to one GPU of Blackwell B200 to generate near real-time video or real-time, again, depends on the quality of the output.
- 14:32
So it does something that we highly recommend you to look into if you wanna get to this point. Okay? We do expect, again, a lot of the other autoregressive techniques to come and gradually, um,
- 14:46
be relevant for the video dif- video generation and image generation. We also see a lot of new model builders working kind of a transfusion or autoregressive diffusion approach. Okay?
- 14:57
So you use the diffusion to generate, uh, um, um, a frame, sorry, but then it generates frame a-after frame in a autoregressive manner. So again, we expect a lot more of these techniques to get into this domain, but it's still a lot of research, uh, driven.
- 15:13
So make sure, again, this is one very good example you can take a look at. And I think the best value about it is all of it is incremental, okay?
- 15:21
You can use this plus this plus this. You don't necessarily need to decide, okay, I'm doing only distillation or only quantization or only context parallelism or... Again, there's a lot of different techniques out there, and they're all incremental.
- 15:34
Okay? So you can start with quantization, as I mentioned, which is the easier approach. If it's good enough for you, stay there. If not, okay, let's move to now multi-GPU, maybe do some context parallelism, maybe add some caching techniques.
- 15:47
Okay? And then last and the most impactful, that's the distillation. And again, hope to see a lot of you trying it, uh, and getting into the real-time performance. Now, try it yourself.
- 16:02
Okay. All of it are open source resources that you can use. We have added support for the open source, uh, models as well, whether it's the One Family, FLUX.2 Family, LTX2 Family, and other ongoing.
- 16:17
So hopefully you'll be able-- we, we'll be able to see also you guys contributing to this, um, and making video diffusion as good as we see with LLM/vLLM. Um, I think I'm almost at time, so if there is maybe one, two questions, happy to try and answer.
- 16:35
If not, um, we can let you one minute of breathing. [clapping]
- 16:48
On, on average, what would you say is like, are there requirements for you to fine-tune this model? Because access to GB200s are not that easy right now. And, uh, in terms of dataset, how big are the datasets that you've seen work well with some of these models?
- 17:03
Okay, so the question was about the compute needed for that, and then the dataset needed for that, just for everyone to hear. Um, what's good about distillation is that you don't need, um, GB200, right?
- 17:15
You can do it with Hoppers, you can do it with, uh, H100, H100, B200, you know, B300 if you... So it's not necessarily that you need very big compute as you do for, uh, pre-training, uh, but you still need to compute.
- 17:31
Okay? Uh, so it's not something you just, you know, take your, um, I don't know, just one instance and start doing it. Of course, it depends on, on the size of the model, right?
- 17:40
If your model is small, you have video generation models that are very small, two B, four B parameters, so this will require obviously much less compute. On the data front, I think it's very important to make sure that,
- 17:53
one, you know how to evaluate. Okay? So you can understand what's different if I just use just a general purpose dataset versus your specific data, uh, requires for your use case.
- 18:03
And in such cases, we have seen, um, differences. So for the more general demos, we don't do-- use any special dataset, and it works well. But again, if it's something that, I don't know, protein generation or something around that, that it will require, you know, something more specific.
- 18:20
I'm at time, I think. But yeah, until they'll kick me out.
- 18:25
Any one other question there? Okay. We can afterwards, I think. Thanks, everyone. [clapping] [outro jingle]