Text Diffusion — Brendan O'Donoghue, Google DeepMind

Talk transcript

1. 0:15

   Some people are still filtering into the room. But, uh, it's mostly intro stuff for the first couple of slides, so they won't miss anything. Uh, okay. Welcome everybody. My name's Brendan.

2. 0:23

   I'm, I'm, uh, a research scientist at DeepMind. I'm talking today about text diffusion, which is kind of a more forward-looking research area, uh, at, at DeepMind. So you're probably familiar with image and video diffusion, which is kind of state-of-the-art for these modalities right now, where you, you know, you take ground truth, say image, you add noise to

3. 0:42

   it in training, and then you train a neural network to remove that noise gradually. And then at inference time, you just initialize the, the picture with pure noise, and then you iteratively refine out the noise to recover back to, you know, whatever image or video or audio or whatever you're looking for.

4. 1:00

   And the principle is essentially the same for text, for text diffusion, where you start with a clean, uh, sequence of tokens, like so a clean sentence or something like that, and then you gradually add noise.

5. 1:12

   You corrupt it somehow. There- there's lots of different ways to do that. You can do it in a continuous or discrete way, but let's just say discrete for now, which would in this case just mean adding random tokens or replacing tokens with other random tokens.

6. 1:23

   And you do that for a bunch of different noise levels, and you train the neural network to try to fill in, to try to correct the mistakes basically in the text.

7. 1:32

   And then at inference time, you initialize the sequence of tokens to just pure noise, like pure random discrete tokens from the vocabulary, and then you iteratively refine through that to fill in the information in the order that the neural network wants to do to recover back to, say, a clean sentence.

8. 1:49

   Uh, and then in practice, so we, you know, I saw- we sh- I showed you some GIFs here of what it looks like for images. You get very similar looking outputs for text where it kind of, uh, you know, starts off all noisy and then gradually like fills in the text and you- and you get clean, relatively

9. 2:03

   clean outputs at the end. Okay. So y- the, the team I'm on, we, uh, we had a, a research demo release o- one year ago now, uh, called Gemini Diffusion, which was a variant of a Gemini model which did text diffusion instead of autoregressive next token generation.

10. 2:19

    And that was like a research preview that was open to about 100K people. Uh, uh, and, you know, we're still, you know, keep, keep posted for new, new developments in that direction, uh, upcoming soon.

11. 2:32

    Um, and we did have some good numbers at the time, but again, it's a year ago, which is like, you know, prehistoric times in this field. Uh, our kind of main comparator model was Gemini 2.0 Flashlight at the time 'cause that was the architecture we were branching from, the text diffusion model, and we basically had very similar

12. 2:48

    quality across the board there. Um, mostly, you know, a l- a little bit of advantage in code, a little bit of disadvantage in some other areas, but kind of relatively similar performance at much better latencies.

13. 3:00

    Um, but again, yeah, this is a year ago, so I wouldn't fixate too much on these numbers. Okay. So what's the difference between autoregressive generation and diffusion? So in, in the standard vanilla Gemini, Gemma, you know, GPT, whatever, um, generation of text, you do this, you know, you have some context that comes in and you wanna generate

14. 3:17

    some response to that, and the model does it one token at a time. So it generates the first token and then condition on that generates the next token and so on.

15. 3:24

    Whereas in diffusion, you have the, you know, a context will come in, whatever that is, and it'll initialize like un- like I mentioned, like a long sequence of tokens, could be hundreds, could be thousands, could be shorter, depends on the model, uh, to be random noise, and then it iteratively refines that canvas to remove the noise over

16. 3:41

    the course of a few denoising steps. So rather than one token at a time, it does the entire block together but over a couple of iterations. So it's not just one pass, it does multiple passes, but it gets to attend to the future tokens and so on.

17. 3:55

    So it's a kind of a different way of generating text. Uh, um, so that obviously has some pros and cons. So the main pro that people really like and probably is the biggest advantage that text diffusion models have is that it's, it's just faster inference.

18. 4:09

    It just generates faster tokens per second 'cause it makes much better use of the hardware, the TPU and the GPU. And I have some slides on that to explain why.

19. 4:17

    Um, but some other advantages are it can do bidirectional attention within the, within this canvas of tokens. So, you know, uh, autoregressive models can only attend to the past.

20. 4:26

    They have causal attention within their, you know, their transformer. Whereas the, a text diffusion model is not restricted by that. It can attend to the future. And that has some i-interesting properties, like it can do self-corrected generation based on future tokens.

21. 4:39

    So it could like, you know, do some reasoning, see that it got the answer incorrect, and then go back and fix the reasoning and do it again. I have, I have a demo of that.

22. 4:47

    Um, because this, this process is, is iterative, it does a number of steps to, to, to respond. It- that means a model can actually do adaptive computation. It, it, it turns out that you can train the model to spend more time on harder problems and less time on easier problems.

23. 5:02

    And like, like diffusion models in general, you can do things like in-place editing where you say like, "Fix the last tokens and give me the prefix that corresponds to those tokens," and stuff.

24. 5:11

    Um, but the main disadvantage it has and the reason why it's not kind of used everywhere right now is lower throughput for large batches. So autoregressive models are slow, but you can have a big batch of queries together and then if you push that through the neural network to, uh, on the GPU, each individual user is slow

25. 5:31

    but the- you make good use of the TPU by doing that and so you can serve a lot of queries and so you keep your costs down. You can serve a lot.

26. 5:38

    Whereas since text diffusion does m-multiple forward passes on the same data, it, you know, hits a compute threshold basically earlier and it, it's even though it's lower latency for any one user, it tends to be, you know, lower throughput overall, so higher cost to serve.

27. 5:53

    And right now, you know, if you have, if you've played with Claude recently, uh, you'll know that they have some throughput concerns so, uh-

28. 6:02

    People really care about throughput right now, and so no one's landing text diffusion into any of these big models primarily because of that disadvantage. It's just too expensive to serve, even if it is much lower latency.

29. 6:13

    Okay. So just leading into why does it have lower latency, unless you-- in case you're not familiar with kind of the architecture of how GPUs and TPUs run today.

30. 6:20

    So in a, in a GPU, there's like a, a tensor core which does these like big matrix multiplies. It's very efficient, has a lot of flops or hops or, you know, whatever.

31. 6:29

    Uh, and then the memory that sits on the TPU, GPU, this, this, uh, HBM, that's where the weights and the activations and everything are stored, and it has to transfer over from the memory all the weights and the activations in the KV cache into the tensor core in order to do the computation.

32. 6:46

    And so it has to flow through this bandwidth channel, and that bandwidth channel is very tight. It turns out that the, that both GPUs and TPUs have a lot of flops and not that much bandwidth.

33. 6:56

    It's quite hard to... It's expensive to put bandwidth onto these chips, and it's easy to put flops. So because of that ratio, you can, uh, you, you know, you can...

34. 7:04

    If you do the more flops you do for each streaming amount of data you put through, the better, right? So,

35. 7:12

    so ba- uh, when we're serving, uh, an autoregressive model, we- these, these, these chips are memory bound. They're basically bottlenecked by this bandwidth. Um, so when you do autoregressive next token generation, for each token, you're doing one token at a time, let's say batch size one.

36. 7:27

    You have to stream over the entire neural network and all the KV cache and everything to get one token, and then you do it again for the next token, and so on.

37. 7:35

    Whereas for text diffusion, you're generating, say, two fifty-six tokens. You still stream over everything. But if you can do that less times than the number of tokens, this iterative refinement process, then you'll get a speed up.

38. 7:49

    So if you can do, say, twenty-four passes to generate two fifty-six tokens, you'll be doing ten times fewer memory transfers than an autoregressive model. And if you are truly memory bound, then you'll be ten times faster, something like that.

39. 8:02

    So that's, that's the real reason. That's the hardware reason why, uh, text diffusion models are much lower latency than autoregressive models. Okay. So, you know, we had this Gemini diffusion demo.

40. 8:13

    M- maybe some of you got access to it, uh, last year. And, you know, that was able to hit, you know, something like two thousand tokens a second, um, pretty consistently, depending on the length of the query.

41. 8:22

    Obviously, it depends. The longer sequence it's generating, the less it's prefill dominated, and so you can really lean into these very long sequences of very fast tokens. But if you're only generating one token, for instance, you'll be just dominated by the cost of the prefill.

42. 8:35

    And the tokens per second number that was reported on this webpage was incorporated prefill and everything like that. So this was two thousand tokens a second. That's genuine raw tokens that you would receive in your web browser.

43. 8:46

    Okay. So that was the kind of a whirlwind tour of text diffusion and its main advantage, which is latency. But I wanna dig in a little bit into some of the other advantages that text diffusion has, which are kind of a little bit less talked about in the literature, um, but I think are pretty cool, and this

44. 9:00

    is why I'm excited about it. So at, at, at the, at I/O last year, Google I/O last year, they showed this demo for the text diffusion model, which is, you know, it's this really easy prompt.

45. 9:10

    But, uh, you know, lots of models actually make mistakes on it. So the prompt is, could you go to next slide? What is the square root of eighty-one times two-thirds squared plus, you know, blah, blah, blah?

46. 9:20

    And I think the answer is thirty-nine to this problem.

47. 9:23

    And so, you know, you pass that into the model, and you ask the, you ask the Gemini diffusion to respond to that. And after one forward pass, these are the tokens it's generated.

48. 9:33

    So one forward pass through the model, it's starting to respond. Now, it's doing this iterative refinement process, so one forward pass is, is not all it's going to do.

49. 9:41

    But after one forward pass, it has this. So it has answer equals, and then it says sixty. It's not correct, but that's what it's guessing for now. And then it starts to do the reasoning.

50. 9:50

    So it has solution, calculate the square root of eight- of eighty-one, and so on. After two forward passes, it's changed sixty to forty-nine, and it's gotten a little bit further into the reasoning.

51. 10:00

    So it's gotten like, you know, five steps into the reasoning. Two squared equals four. And some of the blue tokens are kinda still gonna change. And then after three forward passes, it's actually gotten all the way through the reasoning.

52. 10:12

    So it gets, it gets, and it gets the answer correct at the end. Th- thirty-six plus three is equal to thirty-nine. And it's gone back and fixed the original response to say thirty-nine.

53. 10:20

    So it's, uh, it had a mistake twice, sixty and a forty-nine. But once it finished the reasoning, it was able to return back and fix the mistake that it made at the start.

54. 10:30

    Now, it's gonna do a couple more forward passes in order to fix, you know, some of these tokens that aren't quite right in the text. But overall, that's basically the structure of the output that it'll return.

55. 10:38

    And this is, you know, this is a, a property that text diffusion models have, this ability to do bidirectional reasoning. So to, to not only see the past but also see the future that it's going to utter, it's gonna respond, and also to use that information to do self-correction.

56. 10:52

    So it made a mistake, but it was able to, you know, had another forward pass. It was able to go through and fix that mistake. At the time, you know, much, much bigger models than the one we were serving made a mistake for this problem.

57. 11:04

    So both Cha- ChatGPT 4o, which was new at the time, and Gemini 2.5 Flash, which was brand new at the time, both made an error on this p- on this exact problem.

58. 11:12

    So you give them the exact same prompt, and then they would say, you know, remember the answer is thirty-nine. The GPT 4o said forty 'cause that's the best guess it can do at that one token.

59. 11:21

    It went through the reasoning, and then it said thirty... It did manage to figure out it was thirty-nine. It said, "I made a m- Sorry, I made a mistake.

60. 11:26

    It's thirty-nine, not forty." Uh, Gemini 2.5 Flash also made a mistake, said forty-two, and then it, it actually just stuck to its guns and never changed it and said, "Thirty-six plus three is forty-two."

61. 11:35

    So it like inc- incorporated the error into its reasoning later. Uh, and these are way bigger models than the Gemini diffusion model. So it's, it really is a property, a flaw of autoregressive models that the text diffusion models don't have.

62. 11:47

    Um, and you know, you can fix this with modern reasoning thinking models, uh, but you know, then you're just kind of punting the, the problem into something else. But Anyway, okay.

63. 11:56

    So that's, that's, that's one advantage, which is bidirectional reasoning, self-correction. Another one is what I, I hinted at before, which is dynamic computation. So you can, you can give the model more time at inference, more forward passes.

64. 12:07

    You give it a bigger budget, and it can just do better. It's not exactly monotonic, but it is roughly monotonic that the quality across every eval basically just continues to go up because it can...

65. 12:17

    It gets... E-even if the, even if the solution is almost entirely clean and correct, it gets to look at it and see that it made a mistake and then fix it.

66. 12:25

    So you get this, you get this nice kind of curve where you always see as the number of denoising steps, which is the forward passes go- increases, you get, you go- overall the, the, uh, quality gets higher.

67. 12:35

    And these are just six coding evals that we monitor internally.

68. 12:39

    On top of that, a slightly different concept is it, is the model can do adaptive computation, which is that you can allow the model, you train it in a way to determine itself when it is finished, and then for easy responses, it can use a little bit of compute, and for harder responses, it can take longer.

69. 12:56

    So here's just three examples from, from the Gemini diffusion model, which is what are the first one hundred digits of pi? This is actually a hundred tokens. It looks like a short response.

70. 13:04

    It's actually a hundred tokens. And it only takes four steps to do that because the model i- it's an easy promp- it's an easy response 'cause you've just memorized the hundred digits of pi.

71. 13:12

    You can just output it. Whereas, uh, you know, four, uh, an autoregressive model in f- the same time would have only done four tokens.

72. 13:19

    Um, so that's a very easy one. Slightly more challenging is to write a little bit of code. So that takes, you know, 18 forward passes, uh, to generate fizz buzz.

73. 13:27

    And then some- something more complicated is explain quantum mechanics in a single paragraph, and that took 31 denoising steps. It just took its time, just for whatever reason, decided to spend longer on those ones.

74. 13:36

    And so the model naturally gets to decide, I guess, to determine when it's going to finish and return the response. And, and typically, we see that harder evals take more time.

75. 13:47

    So th-this was, this is a year ago now, so the evals are kind of old school. But, um, the, you know, on the higher- on the harder end is GPQA diamond, which for th-the model size we were targeting was quite a hard eval, and that took a long time for it to respond to those ones.

76. 14:03

    Whereas on the other end are like MBPP, which is mostly basic Python programs. It was very, it was very easy to re- you know, took a very little time for it to respond to these things.

77. 14:11

    And this was entirely determined by the model itself. Just easier problems, easier prompts it could respond to quickly, and harder ones it decided itself to spend more time reasoning.

78. 14:20

    Okay. So that's, that's another property, which is the dynamic and adaptive computation. Lastly is the kind of i- fast in-place editing. So diffusion models in general have this very nice property where you can so take an image, like in this example, you know, cut something out of it or whatever, or give it a little prompt, and it'll

79. 14:41

    fill it in, and you can use the context, uh, uh, that, that you haven't cut out to fill in the, the piece you've cut out correctly. And so you, you can use that for like clever image editing, things like that.

80. 14:54

    Um, and the becau- the reason it can do that is because, uh, you know, it's not autoregressive. There are autoregressive image generators, right, which go left to right, top to bottom, like raster order, generating pixels.

81. 15:03

    You know, but diffusion doesn't work like that. It'll just see the entire image and then start to denoise it. And because of that, because it gets to see every pix- every pixel gets to see every pixel, it can fill in the missing information and, and do it in a way that's kinda consistent whatever prompt you're giving.

82. 15:20

    Uh, so we can do something similar. So I have a couple of demos here. Uh, can you see that? Yeah. So this is just some code, and you say, "There's a bug in this code.

83. 15:28

    Can you fix it?" And it just, it'll just make the edit in the correct place. Like, it won't... You, you can barely see that, but it's just did a little fix here of the indices.

84. 15:35

    Uh, and you can say, uh, things like, "Can you add documentation?" It'll go in. It's not re- it's not just one by one generating all the tokens. It's doing a clever editing procedure to, to actually fill in the, uh, the correct edits here.

85. 15:47

    You can do that with, you know, more general text. Like, you can take a story and then say, "Add a middle paragraph." And because it can see the, the first and the, and this third paragraph, it can fill in the paragraph in a way that's consistent with the, the rest of the story, for instance.

86. 16:02

    And this is just in-place editing, basically. Um, okay. So that's, uh... Those are some of the advantages. Uh, don't have a lot of time. Uh, m- the biggest advantage, like I mentioned, is this low latency, and, you know, we, we really lean into that.

87. 16:17

    And, uh, I just wanna show you a couple of demos of some of the things that people internally have built to kind of show what the advantage of low latency can give you.

88. 16:25

    So it's not just the same thing faster. It can really unlock some new, some really new a-applications. So in your own work, you're all AI engineers, uh, it'd be interesting to see when, when the next diffusion model comes out from our team, what the low latency could unlock, uh, and kinda what new applications can be built.

89. 16:43

    So here are just some demos. So this is Wikipedia. Let me just pause it, actually. This is Wikipedia where everything is generated on the fly, even the HTML. So e- this, this is actually being...

90. 16:53

    Oops. This is actually being generated by the, by the model on the fly. So the- it's a webpage with the HTML and the text and everything being generated on the fly.

91. 17:02

    So it looks like regular Wikipedia, and when you click on it, it's, it's... The, the latency is low enough that it can just fill in the page as if it was a real Wikipedia page.

92. 17:10

    Um, so that's kind of, uh, Wikipedia generated on the fly by a, just a very low latency model. We have a similar thing where we did it for Reddit.

93. 17:18

    So now all the responses to your posts will be by bots, if they weren't already. Um, [chuckles] and it's generating fake comments. Not the... So, uh, the Gemini diffusion model was not an image-generating model.

94. 17:31

    So this demo links in the... which was our start at state-of-the-art image generator model at the time, which I think was Juno. Was it? It was before NanoBanana. So it's the two of these models working together to fill in the webpage.

95. 17:43

    So it's the image generation model is a little slower, um, but you can see that it's like, you know, you can, you can invent any Reddit you want, sharks in this case, and it'll generate the page with the text.

96. 17:55

    The images follow a minute, a minute later, and then you can interact with this website as if it was a real website with, you know, real users and so on.

97. 18:02

    Just being, uh, the entire... All the comments, all the images, all the HTML, everything is being generated entirely on the fly here. This is being generated by the model.

98. 18:10

    Um, I love this one. This is my favorite one. This is an operating system also being entirely generated on the fly. So every click here is generating the next page of the operating system.

99. 18:22

    So it looks like a real operating system, but it's all being generated by the model on the fly, responding to every click. So every time you enter like the README, it generates the text, but it also generates, you know, the, the web pages you can do.

100. 18:32

     C- you know, go back to, to desktop and, and so on. I think this...

101. 18:37

     Yeah. Okay. Um, and then this is a, this is a demo from someone on Twitter who used the,

102. 18:44

     who used the Gemini Diffusion API, uh, or sorry, not the API, the web page, to do some v- vibe coding with his voice. So I, I really like this one.

103. 18:55

     Create a to-do app. Add 10 random to-dos.

104. 19:05

     Allow to-dos to have a completed state. Mark four random to-dos as completed.

105. 19:14

     Allow me to sort to-dos by name and by state.

106. 19:23

     All right, let's see if this works. Sort by name, sort by state. Testing. Enter. Was added to the bottom. We'll try deleting a few. Let's add one. Everything's working.

107. 19:31

     Please convert this to dark mode. And this was literally 15 seconds of work.

108. 19:40

     Okay. So that was, uh, yeah, it was someone outside of our team, so he, he could say that. Uh, yeah, the vibe coding by voice. But, but just in general, we think that, you know, low latency models can really unlock some new experiences, uh, for users and new products, and so we're excited to see what people will

109. 19:54

     do when the next generation comes out. Okay. And on that note, thank you very much. [audience applauding]

110. 20:05

     Questions? Yeah.

111. 20:06

     Yeah. Uh, did, did you use the same corpuses of text to pre-train, uh, Vision models as for the autoregressive models? Also, how, how does it work for, uh, reinforcement learning with human feedback?

112. 20:19

     So you use the same data too?

113. 20:22

     Yeah. Yeah. So yeah, we use all the same data. Yeah. I mean, the algorithms have to change a bit, but we use all the same data. Yeah.

114. 20:28

     And also maybe, uh, can you distill these models? Are there any techniques for doing that?

115. 20:33

     You can distill them. Yeah. Yeah. There... I'm not sure if there are any published ones.

116. 20:37

     Can you share more about it? [laughs]

117. 20:38

     I don't, I don't think so. Maybe there's some externally, but...

118. 20:41

     W- w- will we have in the near future like general available viability of this?

119. 20:45

     There's a, the, uh... Yeah. So we're gonna release something soon. Yeah.

120. 20:51

     Cool.

121. 20:52

     Yeah. Yeah.

122. 20:53

     Yeah. So as you scale the model like more parameters, more data, do you find like the adaptive computation increases like number of steps increases with like bigger models for even like easy prompts or?

123. 21:04

     Uh, yeah. So there's a... The bigger models tend to require less steps for the same output. So they kind of, you know, even if the model get, is getting bigger and the FLOPS per forward pass are getting bigger, they tend to reduce the, the forward passes they need.

124. 21:22

     So it's kind of a, it... You kind of have some sort of a diminishing cost of serving even the biggest models. Yeah.

125. 21:29

     So the next question, do you... How do you price the, uh, price? Does tokens, does it matter for the diffusion model?

126. 21:38

     Uh, I don't know. We haven't got to that yet. I'm a research scientist. [audience laughing] Uh,

127. 21:43

     yeah.

128. 21:45

     How do you define the size of the answer? Because for-

129. 21:48

     Yeah

130. 21:48

     ... images, you expect that I have this image and I have the output and-

131. 21:53

     Yeah

132. 21:53

     ... frame. How do you do that in-

133. 21:55

     Mm

134. 21:56

     ... coding or text?

135. 21:57

     So, so there's a few different ways. The easiest way is to just fix some, some window length and then just iterate on that. So it's like autoregressive, blockwise autoregressive.

136. 22:07

     It's kind of the standard way to do it. But you can have like a, a head that'll predict the length of the response and stuff like that if you wanted.

137. 22:13

     Yeah.

138. 22:13

     Yeah. But, but you have to fix the outcome in order to produce a length model to, to work.

139. 22:19

     No, it still can generate unlimited text, uh, text, but it's just if you fix a window length, then it just does that window length autoregressively if it needs to generate, uh, many, many windows of text.

140. 22:31

     Yeah.

141. 22:32

     You might have already mentioned this, but when you gave that example of the denoising steps being different ranges for different, you know, uh, causing the problems-

142. 22:39

     Yeah

143. 22:40

     ... could you ahead of time set like a, a limit on the denoising-

144. 22:44

     Yeah

145. 22:44

     ... steps that you, that you require for a problem so you can-

146. 22:46

     Yeah

147. 22:47

     ... almost just understand what your latency is gonna be ahead of time?

148. 22:50

     Mm-hmm. Yeah, yeah. These are all with a limit, but it just finishes with earlier than the limit.

149. 22:55

     All right. Okay.

150. 22:55

     Yeah.

151. 22:56

     If you, uh, if you had multiple windows, like you just said, going through, can, can they then go back and attempt to prove this window?

152. 23:03

     Yeah.

153. 23:03

     Or is that kind of as the window goes, it's-

154. 23:05

     It's, uh-

155. 23:06

     ... set in stone

156. 23:06

     ... yes. It's, it's... I mean, uh, you could potentially, but for, for us, we just set it in stone and-

157. 23:10

     Right

158. 23:10

     ... continue.

159. 23:11

     And it just moves on.

160. 23:11

     Yeah. Yeah. Lots of questions here.

161. 23:15

     You mentioned it's, it can be slow on large, larger batches. Would it make sense to have like a hybrid between autoregressive diffusion where it's autoregressive of like a token at a time or like a chunk at a time?

162. 23:24

     Yeah, yeah. That's how it works. Yeah.

163. 23:26

     Oh, it sort of works.

164. 23:26

     Yeah.

165. 23:27

     Okay. Thanks.

166. 23:28

     So it starts to prefill with autoregr- uh, autoregressive style, and then it goes into diffusion mode or?

167. 23:35

     Oh, uh, no. So it's... Prefill is the, the same. It's just so you've got some context and you pre- you prefill. Uh, and then after that, the generation step is typically in blocks of some fixed size, like 512 or 1,000 or 32 or whatever you want.

168. 23:49

     And then that's autoregressive. Yeah.

169. 23:53

     I, I get that you're doing the denoising inside like a bi-directional embedding space. But how do you... What is the process of getting back the token IDs? Is it like, uh, Latent diffusion instead of-

170. 24:06

     So you can do that. Um, the easiest way to do it is to just have like, uh, logits at the top, just vanilla prediction head at the top.

171. 24:16

     Well, so th- this is like, this is all like discrete diffusion, right? So it's always tokens in, tokens out. Like if I show you the-

172. 24:24

     For every step.

173. 24:24

     For every step, yeah. So if I go back here. It's always like a discrete corruption process, and then you fill in a discrete token back in. So it's always...

174. 24:33

     So you're always in a discrete space. But you can do it in latent spaces, but-- and people have done that. But most of the, most of the text diffusion models and literature is discrete diffusion today.

175. 24:45

     Yeah.

176. 24:46

     Is there only a Gemini Diffusion?

177. 24:48

     There might be one day. [laughs] Yeah.

178. 24:52

     Yeah, maybe what, what's your outlook like? Do you think one of the architectures will win in the future or will they have different use cases or?

179. 25:02

     I think for now they have different use cases. So l- you know, if you think about what a low latency model provides that's like, that's worse throughput. So what's that trade-off?

180. 25:12

     Is on-device applications. So we, we are in a couple of on-device applications already, um, in within like the Alphabet ecosystem. So robotics, things like that, where you wanna run a, a model on the device itself.

181. 25:25

     So your phone or a robot or whatever. And then you... But you want it to be low latency and you're not batching with thousands of other queries like Gemini being served in the server side.

182. 25:33

     So you want the lowest latency model. Quality isn't really any-- They're the same quality basically. So then you may as well pick the low latency one 'cause you don't have the throughput concerns.

183. 25:43

     What do you mean like in the, in the future like can we get quality up to the par with, with the current like protein models?

184. 25:51

     I-- Quality isn't the concern. It's the throughput, uh, for serving in a big batch setting.

185. 25:55

     Okay. Got it.

186. 25:57

     Yeah. Yeah. Yeah.

187. 25:58

     So currently you are getting more or less the same quality of front-end models in your research?

188. 26:02

     Yeah. Yeah.

189. 26:03

     Okay. So what about reasoning tasks? Is it possible to differently verify what we want in some way?

190. 26:09

     Sorry?

191. 26:10

     What, what about reasoning tasks? Is it possible to differently verify-

192. 26:14

     Yeah. Yeah. Yeah. You can do RL. Yeah. You just need to change the algorithm. Yeah, but you can still do it. Yeah.

193. 26:22

     Yeah.

194. 26:23

     Is there much scope for combining diffusion and also aggressive text models?

195. 26:28

     Yeah, you can do that. Yeah.

196. 26:31

     Has your team done that? [laughs] [laughs]

197. 26:34

     Yeah. [laughs] [laughs] Yeah.

198. 26:38

     Can you like initialize with a smaller model so you don't have to start from noise, so you can have to do less forward passes?

199. 26:43

     Um, the, the problem with that is if you train it with noise, it expects noise. You'd have to train it with the small model as you like the output and that just adds complexity.

200. 26:55

     So we don't usually do that. Yeah.

201. 26:59

     Is there a paper like several versions back that you can actually go and implement and learn more about it?

202. 27:05

     Um, well, not from, not from our team, but yeah, there are-- there's a bunch of literature out there, yeah.

203. 27:09

     Okay.

204. 27:10

     Yeah.

205. 27:11

     Which are respectively composed.

206. 27:13

     Yeah. You get the idea, I think.

207. 27:16

     Yeah.

208. 27:17

     Yeah. Any other questions? There was a lot of questions there. Gotten through them all. That's good. Oh, one more. Okay.

209. 27:25

     All right. This is a random one. What happens if you ask it to generate noise?

210. 27:29

     Um, I don't know. [laughs] I don't think we've tried to do that. [laughs] [laughs]

211. 27:37

     Probably would work. It would probably do something.

212. 27:39

     Yeah.

213. 27:41

     Yeah.

214. 27:43

     Probably.

215. 27:43

     Cool. Okay. Thanks, everybody. [clapping] [outro jingle]