Let's go Bananas with GenMedia

Talk transcript

1. 0:00

   [upbeat music] Good morning, everyone.

2. 0:16

   Thank you for being here so early in the morning and, um, to be able to [laughs] be the few ones who pass security to, uh, to be, actually be here.

3. 0:25

   Um, I'm, uh, I'm Guillaume, and I will talk... tell you about, uh, GenMedia in general. Um,

4. 0:34

   like, uh... so this is, this is my life as, um, as Nano Banana sees it. Um, basically, I, I joined Google six years ago. I used to be a video game producer before.

5. 0:47

   Um, I initially worked on Stadia, the streaming game company that, uh, product that, uh, that we killed, as so many other product. Um- [laughing] And, uh, and I've been at DeepMind for two years now.

6. 1:03

   Uh, and, uh, as I worked as a, um, what we called a developer advocate. So if you're not familiar with what a developer advocate, basically my, my job is to make sure that, uh, when-whenever we release things or whatever we release, uh, you guys, the developers, have everything you need to work with, uh, our products.

7. 1:22

   So you, you need documentations, you need code samples, you need, um, [clears throat] demos. Uh, now there's are new things that you need as well, like skills and, uh, prompt guides and, and things like that.

8. 1:35

   So making sure that you can, you can start right away and it works. Um, and on the other, on the other side, when I'm, uh, talking internally, um, um, that's the reason why it's advocate, because I'm advocating for the developers and trying to bring some, uh, let's say, common sense to the internal teams and making sure that

9. 1:54

   what we, what we release makes sense, uh, in the, in the real world, and it's not, uh, entirely developed, uh, for Google by, by Google. Um, a, a very good example of that is the Imagine models.

10. 2:08

    Uh, when we released... when we had, like Imagine and Nano Banana, the... each model has its, has-had its own set of API, which doesn't make any damn sense. Like you...

11. 2:20

    like a normal developer should be able to just swap the model name and it works. Uh, and I've, I've, I fought quite a long time for, for that. [laughing] I never managed to win.

12. 2:31

    But in the end, I think the Imagine brand is... doesn't exist anymore, so I kind of win by default. [laughing] Um, but yeah, you see, you see the kind of, uh, the kind of work I, I have to do on a daily basis.

13. 2:44

    Um, as I said, I'm mostly working on the GenMedia models, so we, like, um... and that's why the talk is about, uh, GenMedia. So, uh, if you, if you look at the world, your phone and a-anything like medias are everywhere.

14. 2:58

    So, uh, you have like images, videos, sounds, uh, everywhere. So it's really core in our world. And, um, and that was at the core of what DeepMind is, is building with, uh, with, uh, with our models.

15. 3:11

    Um, we... like you, you, you're starting to, uh, to hear with, uh, with Lucant's new startup, uh, about world model, but that's, uh, exactly what we have been trying to build at Google for, uh, since the beginning.

16. 3:24

    Um, and our vision of a world model is that it's, uh, it's a model that, as its name imply, understands the world, but meaning it can ingest, uh, as many modalities as possible.

17. 3:34

    So sound, videos, audio, uh, sensors, uh, whatever kind of all five senses and then, um, uh, outputs or like, uh, yeah, talks in, in different mo-- as many different modalities as possible.

18. 3:49

    So audio, text and, and, and so on and, and much more in the, in the future. Um, and we, we tend to, to have like specific models. Uh, we have all image generation models.

19. 4:01

    We have all video, uh, generation models. But deep down the, the goal is really to have like one models that encompass all of that. It's just that for release purposes it's easier to ship s-- like specific models than to, uh, only always update the main model and like have risks of, uh, breaking something else at the same

20. 4:19

    time. But like, uh, a quick, uh, a quick story, like when we released Gemini 1.0, it was two years ago. It, it seems... it feels like it was like years ago, like eons ago.

21. 4:32

    But, uh, the first, uh, Gemini model, Gemini 1.1, was meant to be multimodal because all of our models have always been multimodal. But since, I guess the testing was not finished or whate-whatever reason it wasn't there yet, um, they, they removed the, uh, image understanding, the multimodal understanding i-inputs from the model.

22. 4:53

    So the 1.1 was not multimodal, and then 1.5 came, and then this one was multimodal in, and I think that was the first one that was, uh, that was doing that.

23. 5:01

    And which once again, is crazy. Like it's, it was a year and a half ago, and now you don't imagine working with a model that is not a multimodal one.

24. 5:10

    Um, but still, when you were using that o- that one very often you were giving it an image and it was telling you, "Oh, I'm sorry, I'm just a, uh, LLM.

25. 5:18

    I can't, I can't deal with images." And that's just because some of the training that was ma- that was added at, at the end of 1.0 that was, "Oh, you know how to deal with image, but if you are asked, don't use it."

26. 5:30

    Though some of that training was still remaining in 1.5 and coming up for, from time to time. So that was, uh, yeah. That ki- it was kind of annoying for- [laughing] ...

27. 5:38

    until we, we shipped, we switched up to, um, to 2.0.

28. 5:43

    Um, so as I said, we don't only have the Gemini models in, um, at DeepMind, so we have, uh, all of the image, video, music generations. We have a couple of other very specific models.

29. 5:54

    We have the robotics one that is kind of a multimodal one as well because the vision is very important for, for robotics. Uh, we have a bunch of agents that we are shipping.

30. 6:04

    I think this year is going to be the, the year of agents. The real one. Last year was everybody talking of agents. This year is a- Is a year where we are actually going to build agents.

31. 6:14

    And we have the open models, and I forgot to update the slide because it's not Gemma 3 anymore, it's Gemma 4 since last week. And then bunch of, uh, research-y models like AlphaEvolve, uh, AlphaGenome, WeatherNext and so on that are very, like, s- very specific for research purposes.

32. 6:34

    Um, just on the, on, on GenMedia, we've shipped things, uh, on average, uh, more than every month. Uh, on, on the whole, if you just take all of DeepMind, we are shipping things every five days, uh, on average.

33. 6:51

    Someti- some weeks we're shipping two or three things. And if we had all of the small, small features, we are shipping multiple times per week, so it's, uh... That's why I'm, I'm, l- I and most of my colleagues are very busy because we always have, like, new things to, uh, to document and talk about.

34. 7:07

    Um, so very quickly about the new things that we shipped recently. Um, NanoBanana, we shipped NanoBanana 2, uh, which has, uh, new, uh, spec ratios, uh, from, uh, of, uh, like 520p pixels to, to 4K.

35. 7:27

    Um, it has search grounding as, uh, as you a- all know. But it also have, like, uh, image grounding. So if you can ask it to search for images on the web and use those images as grounding so that it knows, uh, uh, like, yeah, it has better knowledge about what, what things look like.

36. 7:44

    It's very useful for, uh, architecture stuff, for example, for, uh, animals and s- and things like that. Uh, there's a lot of

37. 7:54

    rules in total that I, I was not able to, uh, to all figure out. But, for example, buildings, they have to be old enough, uh, otherwise there are legal reasons we can't use the images.

38. 8:04

    So, um, that's-- But it's still, uh, very useful when, when it works. Uh, we have the Video models. You've heard about Video 3, Video 3.1. We, um, we just released last week, uh, Video 3.1 Lite, which is our cheapest model.

39. 8:22

    Uh, I think it's five cents per second, so forty c- forty cent for, for one video, which is very cheap. Uh, and the idea is that you can iterate on, um, on, on the, on the prompt that way and then upscale afterwards.

40. 8:36

    Um, and we have Lyria, which is our music generation model that we, um, that we released, uh, two weeks ago. Um, and with which you can create either, uh, thirty-second clips or full songs of three minutes.

41. 8:52

    I will, I will show you demos of all of that afterwards. That's the point of the, [chuckles] of the session anyway. And, uh, but, uh, there's also another Lyria model that, uh, people don't know about, that is called Lyria Real-Time, and that actually my, my favorite model.

42. 9:06

    Um, and this one is basically you, you, you create music as well, but it's not a, uh, a diffusion model like users where you just give a prompt and you get, uh, something out of it.

43. 9:17

    It's, uh, it's a predict model, so which means it's, uh, it's, it's basically a live model. So it creates music, and it's continue to create music, uh, in real-time until you stop it.

44. 9:27

    And you can just send new prompts, and it will, like a DJ, mix and, like, swap to the, to the new music in real-time, and, uh, real-time two seconds.

45. 9:35

    Uh, but, uh, but then, uh... And that's, that's quite fun to play with. Uh, if we have time at the end, I will, I will show you a demo.

46. 9:43

    Um, but as I said, uh, this is meant to be a workshop, so the goal is for you to, uh, to play with, uh, with the models and for me to show you codes instead of slides.

47. 9:52

    So, um, this is the content that we're going to use. Um, that's this one as well, if you can read what I wrote. Um,

48. 10:06

    and if that doesn't work, just tell me.

49. 10:20

    Are you all in? No.

50. 10:24

    Not yet.

51. 10:25

    No, it doesn't work?

52. 10:29

    No.

53. 10:30

    Oops. Let me, let me try double-check that the link works.

54. 10:37

    Enjoy. Yeah.

55. 10:42

    Uh, no. Uh, I have the title. [chuckles] [chuckles]

56. 10:48

    Cool. Um, are you all in? Okay. So, uh, full disclosure before we start, uh, this is using GenMedia models, which means they are all paid models. So, uh, running the, the notebook is going to cost you something like one dollar.

57. 11:05

    Uh, and you can just keep the video generation because that's, uh, most of the, of the cost of that, uh, of that one dollar.

58. 11:14

    So, um, the... Yep. Uh, can zoom in a bit.

59. 11:23

    Um, so the, the goal of that, uh, that content I prepared is to illustrate a book using the different, uh, GenMedia models. So the, what we are going to do is that we are going to, uh, to take a book that, uh, is, uh, is an open source one that I took on the Gutenberg, uh, lib- uh,

60. 11:44

    online library. And basically, we are going to use, uh, Gemini to come up with prompts and then the GenMedia to create, uh, the content for the prompts so that we can-- we will have images of the characters, imaging of the scenes, uh, videos of those, uh, those things and, and so on.

61. 12:02

    Um, and, uh, and that, uh, that example comes from what we call the cookbook. So we have, uh, we have this GitHub repo where we are posting examples of, uh, like both quick start guides to explain how to use a new model or how to use a new feature and also exam-- full, like, full-fledged examples like this

62. 12:21

    one on how to go further and to mix different features into, uh, into content. So if you, if you- Looking for ideas, that's a, that's a good place to, to check, uh, what you can do with the models.

63. 12:34

    So, um, basically introduction, uh, billing, um, so let's start. So the first thing is that you need to install the SDK. Um, you need the latest one because of im- uh, music generation that was, uh, shipped last week, uh, two weeks before, so actually you need the next one.

64. 12:56

    Um, so it's starting. And I should have done that. Um, and you will need an API key if you don't have an API key from, uh, from AI Studio, and you need...

65. 13:10

    the other side, you need that API key to be a paid one. Um, if you don't have a paid API key, you can still use the image generation examples by using the, uh, Nano Banana 1 model, which has a free tier.

66. 13:23

    So you can just swap the model we are going to use for, for this one.

67. 13:29

    And why is that so slow? I think.

68. 13:38

    Small, neat ears and [REDACTED:physical_attribute].

69. 13:42

    No.

70. 13:42

    It was the water rat. Then the two animals-

71. 13:45

    I think it's good

72. 13:46

    ... regarded each other cautiously.

73. 13:49

    Okay. So then I'm just loading the, uh, the API key. Um, and I'm creating the client, uh, the GenAI clients. And you can see there, I added these parts that you, you...

74. 14:01

    we don't add in all of our examples. That is basically the auto retry, uh, thing because, like, if we are using Nano Banana 2 at the moment, so the...

75. 14:11

    especially in the evening when the US wakes up, uh, the model can be overloaded. So that's, uh, that's basic- That part basically says that it's going to be automatically retrying, uh, five times after two secs.

76. 14:25

    Um, a bunch of imports and then, uh, we are going to select all of the models that we are going to use. So, uh, in this case, um, uh, 3.1 flash image preview, which is Nano Banana 2, uh, Gemini 2.

77. 14:40

    2.5 flash. No.

78. 14:41

    Wonderful day.

79. 14:42

    Let's use-

80. 14:43

    You know I love-

81. 14:44

    ... 3 point... 3 flash instead. Uh, Lyria clip to create, uh, 30-second cl- uh, music clips and, um,

82. 14:54

    and the TTS model, this one, the pro one. Up.

83. 14:59

    So and I added this checkbox that should have been false by default, but I made a mistake yesterday evening, uh, just so that you can... y- y- if you...

84. 15:08

    you, you won't be able... you should not be able to run the notebook by mistake if you don't want to pay. Um, then just, uh, I'm just setting limits here because, uh, uh, like a book can be, can have a lot of chapters, lots of characters, and that can be quite long to, uh, to, to generate.

85. 15:24

    So that's why I set up limits on how man- how many we want, uh, for the demo purposes and cost, uh, purposes as well.

86. 15:32

    Look ahead.

87. 15:33

    So, um, as I said, we are going to use, uh, an open source book, uh, that's, uh, Will... uh, The Wind, uh, in the Willows, uh, from Kenneth Grahame that I don't remember reading, but I think in the UK it's quite well-known.

88. 15:48

    Um, so basically I'm just downloading it from the Gutenberg project and, uh, and here I'm using, uh, f- client file upload, which is, um... You, you might know that we have basically two ways of using the Gemini A- the, the Gemini models.

89. 16:08

    We have the AI Studio Gemini API way, and you have Vertex. And the main difference between the two, and I can... maybe that's a good time to switch to this slide.

90. 16:20

    Up. Um, but, uh, we... y- you know, at Google, we like to basically create multiple products that are doing the same thing and confuse our users. That's kind of our, uh, motto.

91. 16:33

    Um, so that we're doing the same with, with AI. So we have a bunch of, uh... But actually when you think about it, it's, it c- it can make more sense.

92. 16:41

    So on the, on the left, we have what we call the cons- consumers app. So it's a fully developed, uh, app f- for... that are easy to use for anybody who is not technical.

93. 16:53

    So you can do plenty of things with the, with Gemini, but you, you are, you're lacking c- as a developer, you might be frustrated because you're lacking control about which models, which features, which parameters are, are being used.

94. 17:04

    And on the other end, we have, uh, Vertex AI, which is, uh, the exact opposite. It's meant for enterprise, so you have a lot of control. You can control on which data center it runs.

95. 17:14

    You can, uh, you have control o- about your buckets, who has access to what and, and so on. The only thing is that it comes with like... uh, with great powers comes great responsibilities.

96. 17:24

    So you... it's, it can be a pain for people to start there. So that's why we have the Gemini Developer API that are kind of a middle ground for, for developers, uh, where you can just create an API key and then start using the, um, the models right away.

97. 17:39

    We... which comes with s- security, uh, risks as well, that if your key leaks everybo- uh, anybody can, can just use it,

98. 17:47

    uh. Um, and, and AI Studio is like, is, is the kind of the, the same vein that's it's, uh, it's meant to be a place where you can test a model and, uh, and play with them, uh, as easily as possible.

99. 18:01

    And the good thing is that we have the same SDK between Vertex AI and Developer API, so you can s- swap from another, one to another. So there's no wrong place to start playing with the models because you can always change.

100. 18:13

     Um, and then we can skip that. And I can go back here because actually what I was going to say is that we actually have a few differences between when you're using Vertex and when you're using, uh, the Gemini API.

101. 18:26

     Um, because, uh- Like, the, the goal of the Gemini API is basically to hide all of the complexity from Vertex. And one of those complexity is creating buckets, creating ACLs for the buckets, giving rights and, and, and all of those things.

102. 18:40

     So the Gemini API have, has that, uh, that API that calls that is called file upload. And basically what it does is that you upload a file and then it's easily ac- accessible from the model.

103. 18:52

     Um, so I upload the file and then I c- I will use, uh, I will use what we call chat mode. So basically, uh, what it does is that it, it chains requests and it keeps the history of it, uh, so that it's, uh, it's easier to, to keep all of the context.

104. 19:09

     And in this case, it's going to be good because we are going to feed the whole book to, uh, to the model, uh, thanks to the, the large context window.

105. 19:17

     And that's why, uh, it's going to be useful in, in, in our case because, um... I'm going to run that while I talk. Um, it's, uh, it's going to be useful because we...

106. 19:29

     Like, for image generation it's, it's always a good idea to know what's, what have been generated pre- uh, previously, so it c- it keeps the same styles and, and things like that.

107. 19:38

     Um, I'm also going to use structured output so that we... Like, we have a structure about what the, the model outputs, and we, we can, we can talk the same language, which is going to be very simple.

108. 19:47

     I want it to generate prompts, and I want the prompts to have a name, uh, so that we know it's, if it's a chapter, a character, or something, and the prompt in itself.

109. 19:57

     So I'm just creating like, uh, initializing the, the chat clients, uh, with the response type, uh, JSON, the scheme that I, that I'm providing, and something that we shipped yesterday that is a service tier priority.

110. 20:11

     So don't do that yourself. I think you should remove that line. Um, what, what it does is that it basically, uh... It's, it's... Yeah, we shipped that last week.

111. 20:23

     So we have, uh, three service tiers. You have the normal one. You're paying the normal price. You're, like, uh, in the queue with everybody else. And we have another one that is called Flex, and that's basically I don't care if that takes a long time, but I want to pay, uh, to, to pay less.

112. 20:38

     So you're going to have a f- a 50% discount, but your request can be, uh, can be delayed and, and so on, up to a few minutes. And on the other end, you have priority, where you are going to pay twice the price, but at the same time you're, you're kind of guaranteed that it's going to be

113. 20:55

     fast because you will go- you will have the fast track like, uh, in the airport or anywhere, anywhere else. So yesterday I added that, and I'm, I'm going to use that to be certain that it works well for me.

114. 21:05

     But for you, you might want to save a few bucks and not add that here. How much more expensive? So twice. Twice. Okay. Yeah.

115. 21:15

     Um, and I'm, I'm not sure it works with View anyways, which is the most expensive ones, uh, of the model. And, uh, not with Lla yet, so. Okay. Yeah.

116. 21:24

     Yeah. Um, so anyway, so I'm starting with, uh, I've created the ch- this chat, and what I'm going to do is to send it a first message that basically says, uh, "I'm feeding you the, the whole book.

117. 21:37

     Uh, I don't need to do anything with w- with the book yet, but you have it. It's in your context and the instructions we, we'll follow."

118. 21:46

     And then we are going to define the style. Uh, usually I just set nothing and then I let, uh, Gemini come up with its own style. Uh, but then, but I'm getting tired of having exactly the same styles or same style always, so let's write something.

119. 22:03

     Um, uh, a colorful, um, building block style. Um,

120. 22:18

     well, let's, let's go with that. Let's see what's... how it goes. So I'm just defining a style. And by the way, if you're using, uh, Colab, this, uh, Colab has those things that they called, um, they call magics, and that's kind of nice to, to create those notebooks and to have the, like, forms that people can fill,

121. 22:40

     and that's just, uh, that fits into the, into the code.

122. 22:45

     Um, then some system instructions, uh, to, to, to, uh, to direct the model, uh, like to, uh, into the s- the kind of images that we want. 'Cause the problem I had at the beginning when I was working on those examples is whenever you ask a portrait image and the model knows it's about a book, it tends

123. 23:06

     to create, like, cover pages and to add a title, and that's, uh, I didn't want those styles, or to create, uh, ones with different panels, and I didn't want that either.

124. 23:15

     So, uh, just, that's just what the system instructions are about.

125. 23:20

     And then, like, we can start, uh, working and basically, uh, I'm going to ask it to create prompts for each characters of the book. So, uh, can you describe the main characters, only the adults?

126. 23:32

     Uh, actually you... we could remove that because it's, it was just from the beginning of Nano Banana when, uh, you could not generate, uh, kids' images in Europe, but it's not, it's not true anymore.

127. 23:42

     Uh, so, um, you can, you can create image from nothing, uh, with kids, but you can't, uh, edit images with kids. That's, uh, that's a current limitation in Europe.

128. 23:57

     But anyway, here's all, uh, all prompts for, for each character, and then we can move to, uh, creating, uh, the, the, the images for, for each of them. And what I'm going to do is that I'm going to create another chat just for the images, 'cause I don't want to mix the, the text and the, and the

129. 24:17

     images output. But I want it to be a chat so that it will have all of the history of the previous images it, uh, it created. Um, so I'm, I'm setting up with the responsibility image, the aspect ratio we want, the system instructions we decided, uh, did, and the style, and priority as, as I said before.

130. 24:36

     Um, and here we go. So here's the mole. That's nice.

131. 24:46

     Here's the, uh, water rat. And that's actually way faster now that I'm using priority, so this works. [laughs]

132. 24:57

     Um, I, I could have done it, uh, in a better way and, like, make all of the calls asynchronous. That would have, wouldn't have worked with, uh, with chat mode, but that would have been a way to, to make it faster.

133. 25:09

     Um, I think it's good enough. The, the toad. Oof, way bigger than the car for some reason.

134. 25:17

     It's a child book. Yeah. Uh, and then Mr. Badger. Um,

135. 25:25

     yeah, so as you see, like, this is mainly a demonstration of what you can do. Like, uh, every time I run it, I- I'm thinking like, "Oh, if I was to optimize it, there is plenty of better ways to do, to do that."

136. 25:38

     Um, but... And, uh, b- um, the badger, and then the author will be at the end. Um, one of the thing I'm doing in the code, if I go up, is that I'm also saving all of the generated image from the character in a, in an array, and I will explain afterwards why I'm doing that.

137. 25:58

     And that's, once again, I'm just appending them one after another. It- in, in real world, I would, I would save them in a, in a better way. Um, [laughs]

138. 26:10

     so, and up, and the author is here. So now we can do- we can move to the next phase, which is basically illustrating the book. So, uh, same thing, like I'm going to ask, uh, the chat, uh, okay, "For each chapter, give me a prompt to illustrate the- what's happening.

139. 26:25

     It should be a single image, not a multi-tiered one." Uh, I'm trying to force it to describe the character again, even though, uh, there I will have the images as references, just because it's, uh, it's, it's still better.

140. 26:39

     So let's go with that. That should be quite fast. Um, well, the same thing, like the, the, the way it works with chat, it's, uh, it's keeping an history that is basically all the previous messages, and it's sending back the history to, to the model every time we make a, we make a call.

141. 27:01

     Um, which can be, like in this case, since it's res- it's basically resending the book all the time to the model. So that's, that's the reason why it, it can be quite slow.

142. 27:10

     Um, we actually, uh, released new API a few month ago that are called the Interactions API. Um, let's run that while I talk. Um, and the, and the main difference between the, the new API and, uh, and the old ones is that the new APIs are going to be stateful and stateless.

143. 27:29

     And what, what, what it changes is that every time you, you make a call, you get an interactions ID, and you can reuse that interactions ID in future calls, and it will recover all of the, all of the context directly from the server.

144. 27:44

     So you don't, you don't need to re-upload the same context again and again and again at every turn of the conversation. And it's also making it easier to, um, to, uh, to fork the discussion.

145. 27:57

     Uh, for example, like you create, uh, you want to create a song, and with, with images and, uh, with a cover image. You can create the lyrics with one model, and then you, you, you, you fork it and in, on one end you create the c- images, and the other end you, you create the, uh, the song.

146. 28:14

     You had a question. Uh, how long do you store the, the session? Huh? How long do you store the session? Is it- Um, that's a good question. I think it's two days, something like that.

147. 28:27

     Um, it's still, it's still in, uh, in preview, so, uh... But, uh,

148. 28:33

     there is good chances at, at IO we make it the default API. Um, but I, yeah, I don't... I, I'm not using it enough to, yeah, to know. Uh, one of the cool thing it does as well is that since, uh, it knows that it's context that you're going to reuse, it's auto ca- automatically caching it as

149. 28:50

     well, so it makes it cheaper to, to run. But even though those, th- the normal API are also doing the same, but, um. So we have our chapter. So the third chapter is next to the river.

150. 29:03

     So the, the mole and the, and the b- and, uh, forgot his name, characters are together. Then it's on the road with the, with the toad,

151. 29:16

     and then in the forest in the snow. Oof, scary. [laughs]

152. 29:23

     So, see, and that's, uh, and, uh, and basically what I, what I did there is that I used the fact that we were using the, the history to, uh, to, to trust the model to have all of the, all of the previous images of the character so that, uh, it would remember, uh, how they look like and

153. 29:43

     what's, uh, and how to, uh, create new, um, images of them.

154. 29:49

     Um, but, uh, there are actually better ways to do that. So I tried another way, uh, which is basically to create a new, um, a new, uh, structured output that is basically the name of the chapter, the prompt, but also the list of the characters that are appearing in this chapter, and that's actually what I was doing

155. 30:10

     if I was wanted t- to do it at scale with like, uh, more than a few char- uh, characters. And basically, I'm, um, I'm going to ask the model to, uh, to give me a prompt for each chapter, but also to give me the, uh, to have, uh, thanks to the list of characters, I will only give

156. 30:29

     it as references the, the right images for the,

157. 30:34

     for the chapter. So I'm going to basically get the same thing here.

158. 30:41

     Up. And so the first, in the first image, that should be the mole and the water rat. In the third one... In the f- second one, Mr. Toad, mole, water rat, gray horse.

159. 30:51

     And the, and the third one, mole and water rat again. Um, so I created a very dirty, uh, script that basically, uh, search through the s- the character images that we, that we saved earlier so that I can give it a list of characters, and it was, uh, going to, to give me the list of images to,

160. 31:11

     uh, to, to give to the model. Uh, did I run it? Yeah. And, and basically I'm going to do the same thing and go to, and generate images for each chapter, but this time, instead of on relying on chat mode, I'm going to use generate content, so the, like, unary call.

161. 31:27

     But I'm going to pass it the images that are, uh, from the characters that are specifically in this, uh, in this chap- in this chapter image.

162. 31:39

     So it should give slightly better context to the model, uh, of about what to, um, what to, uh, what to display and how to show them. Um,

163. 31:51

     let's see if it's works better. I think if...

164. 32:01

     And, uh, like, like I said, like, if I wanted to do it at scale, I would, I would, uh, have a lot of improvement I would do, uh, about that.

165. 32:10

     Uh, and one of them would be that I think I would generate more than one image per, per character. I think I would have, uh, been one portrait image and then one full, uh, full, uh, full body image, and then maybe from the side, from the, from the back, and ask the model to tell me which exactly,

166. 32:24

     which, uh, like how they are going to be displayed so that I can give, uh, give the, Nano Banana exactly the, the, the reference we need for the, for the, uh, for the generation.

167. 32:37

     So here they are. Um, and see, it's more or less the same, to be honest. But, uh,

168. 32:52

     except that this time, I don't know why it's, uh, it seems to be attacking the mole for some reason. [laughs]

169. 32:58

     I don't think that's what the story is about. It's that, uh- [laughs] You, you, you might know better than me. [laughs]

170. 33:04

     Um, we can, we can check the prompt. What the, what does the prompt it says? La, la, la.

171. 33:11

     No, it doesn't say anything about attacking the... No, it's rescuing him. [laughs] [laughs]

172. 33:21

     Yeah. Yeah, that's, uh, that's basically how you can, uh, you can use the, um, uh, use the model. Uh, for, for those who came, the content I'm showing, you can, you can op- you can open it there.

173. 33:36

     And, uh, it's, uh, it's a collab that is about taking a book and creating, uh, images and videos to illustrate the book, uh, and the characters. So we went through, um, creating prompts for each character and then generating images for each of those characters, and then creating new prompts for each chapter, and then, uh, creating, uh, images

174. 33:58

     for the chapters, uh, using the reference that we have from the, from the images. If you can't see what I wrote, it's goo.gle/cookbook-illustration.

175. 34:12

     Um, so, uh, and then we're going to try to move to the next phase with the, with videos now. Uh, so we are going to use, uh, Veo to generate, uh, videos of those, uh, of those images.

176. 34:27

     So I'm going to swap to the bigger model because I can pay for it. You can, you can skip to the, stay to, uh, stay with the cheapest one if you don't want to spend too much.

177. 34:37

     Um, and basically I'm going to, uh, just do exactly, like, take the last chapter, take the last image, uh, and, and basically send it the same prompt and the, and the reference image.

178. 34:50

     So that's what I do here. So when I pass an image to Veo, that's basically use... It's going to use it as the first frame for the video. Um, and funnily enough, like, most of the,

179. 35:03

     most of the, like, a lot of the training for video generation model is actually image generation because I think the most important part of, uh, generating a video is generating the first frame so that it knows where to start with and then what to do with it.

180. 35:17

     Um, so-

181. 35:20

     So what's the best video model? Is it-

182. 35:22

     It's, uh, the one that doesn't have light or fast, so.

183. 35:27

     So fast is just a faster version of the same-

184. 35:29

     Yeah. That's, uh, the-

185. 35:31

     Even more expensive, but faster.

186. 35:33

     Yeah.

187. 35:34

     Okay.

188. 35:34

     So Veo 3.1 is, uh, is the main Veo model, and the other ones are basically, uh, like, smaller versions of it that are running fa- uh, slightly faster and, like, basically doing less, less, uh, generation terms.

189. 35:49

     And yes, I said I was going to repeat the questions, and I forgot. The questions was which one is, uh, faster, the, the mo- the better model among the three.

190. 36:00

     Um, I guess the doors just opened. [laughs] Um,

191. 36:07

     it's still very early. So up we can see where it goes.

192. 36:12

     Wait, but it innovates this image.

193. 36:14

     Let's get out of this dreadful place. [laughs]

194. 36:16

     Oh, thank you, Water Rat. I thought I was done for.

195. 36:19

     Yeah, it saved. And, and there were sounds. I don't know if we... Can we have more sounds so we can see what-

196. 36:26

     Yeah.

197. 36:28

     Quick, give me your hand. We must get out of this dreadful place. [laughs]

198. 36:33

     Oh, thank you, Water Rat. I thought I was done for.

199. 36:37

     That's, that's not that bad, except the wrong character is speaking. That's a problem with basically using the same prompt when you, when you create the videos and the, and the audio, be- and the, and the image because it doesn't have the extra content about what exactly is meant to be happening afterwards.

200. 36:51

     Ah, okay.

201. 36:52

     So that's why I add another... I added another example that is basically we're going to do the same, except we are going to add one more step that is basically asking Gemini to come up with a new, um, a new prompt just for the, just for the video and to explain what's, what's happening afterwards.

202. 37:11

     So, um, that's what I did. I'm going to animate this, the, the CC, this chapter image, and can you create a prompt with view about what's happening in the next few seconds after the initial image?

203. 37:22

     Um, and I'm passing it the ima- the last image that it knows exactly where to start. Um, up.

204. 37:32

     And while it runs, since, you know, we have yet again new people who came, if you want to follow on your laptop, that's, uh, that's a link to open the, the collab I'm showing.

205. 37:41

     So goo.gle/cookbook-illustration. And, uh, what we're doing, what we are doing at the moment is illustrating, um, the book that is called, uh, Will of the Willow, something like that.

206. 37:56

     Um, and, uh, and creating images and then now videos to illustrate what's happening in the book. So we came up with, um, with a prompt that is like, "In a colorful building block style, the Water Rat in his, uh, round brown face and blue jersey lowers his silver pistols and offers a reassuring pat to the mole's shoulder.

207. 38:18

     The mole in his black velvet smoking suit ex-, uh, exhales a puff of white plastic vapors in relief, and as his pink snout twitches,

208. 38:27

     they turn and begin to walk together," blah, blah, blah, blah, blah. What's, what's interesting in the prompt is that I feel like it's, uh, it, it got the, the, the style of the book.

209. 38:36

     So the prompt is kind of written in the same style, it's the same old-ish style of, uh, of, of speaking as well. Um, and we can see that it's realized that, uh, um, it's, um, it's not attacking the mole, it's just saving it and then, and, and so on.

210. 38:54

     And, um, and also what's, uh, what we can see from there is that, uh, you remember I set, I set some system instruction at the beginning saying that, uh, whenever it describe characters, it should always try to describe li- uh, what they look like and all.

211. 39:08

     So it added what... how they are dressed all the time, so, uh, which also helps with the character consistency. So let's see if this one is better. [grunts]

212. 39:27

     I think it's better, except there's no, there's no discussion. [laughs]

213. 39:35

     So you, you can try with yourself, just be careful. As I said, uh, earlier, video generation can be expensive, so don't, don't try it like a hundred times, uh, until you get it.

214. 39:46

     Uh, it can be, it can be ca- ca- expensive. Then, um, we have this new, uh, Lyria model that we shipped two weeks ago that is our new music generation model.

215. 39:58

     And the, basically, as I said, it's just a, a model to which you can give a prompt, and it will come up with a, with a song. And, uh, and you have two different models.

216. 40:08

     You have the clip model that is creating forty-second, uh, musics. Uh, like I think it's for, for four cents per song. And you have the full song model that is creating, uh, up to three minutes, uh, songs, and it costs like twice as much to eight cents per, uh, per music.

217. 40:27

     Um, for the sake of the demonstration here, I'm using the clip model, uh, so the fastest one. Um,

218. 40:34

     so we are going to do, to use exactly the same, the same trick as before. So we are going to have Gemini create the, uh, the prompts for the music generation.

219. 40:45

     So, uh, so I'm asking it to create, uh, instrumental songs for each chapter and to create, uh, to create them for Lyria. Keep the consistency between the chapters, uh, but at the same time highlighting the what, uh, what's specific in each chapter so that, uh, like, I don't want like three times or four times the same, the

220. 41:05

     same song. I want different ones for, for each song. And, and by the way, something that I forgot to say is that as, as you can imagine, like, we have like multiple models, we have multiple teams inside of, uh, inside of DeepMind, but actually, uh, all of them are working together.

221. 41:22

     And the big part of the training data for the GenMedia models is, uh, is being made with the help of Gemini. Uh, so, so basically, all of our GenMedia models are trained with like prompts that are written by Gemini.

222. 41:36

     So that's also why Gemini is quite good at, at creating, uh, the prompts for, for the, for the GenMedia models because, yeah, it's, uh, they, they are, they have been trained to, uh, to listen to him very well.

223. 41:47

     So, uh, it's, uh, that's, that's why these kind of, uh, of tricks of having Gemini write the prompts for you, uh, works quite well.

224. 41:56

     Um, and in any case, deep down, there's always a bit of rewriting of your prompts that are being done by the GenMedia models before it's actually, uh, starting to generate.

225. 42:05

     Just because otherwise, when people are sending like one-liners, they're... they... the, the models won't, won't be, won't do anything interesting with a, with a one-liner. S- And, um, so usually the longer your prompt, the more interesting it's going to be and the more likely it's g- it's going to be following what you're, um, what you're asking for.

226. 42:27

     So I think I talked a lot because it's, uh, like the, uh, music generation is actually quite very fast, so we can see the different songs. [classical music]

227. 42:43

     I think that fits with a pastoral suit, uh, with spr- uh, representing spring and flow- and flowing waters. [classical music]

228. 42:55

     And see next one, the open road. [upbeat music] Feels more adventurous, yes?

229. 43:10

     And then the dark forest. Let's see. [tense music]

230. 43:27

     Well, and you can see, you can see the prompt here, like, it, it comes with which, uh, which, um, instruments to use, how to use them. Um, what's, uh, what's interesting with the Lyria model is that it's actually...

231. 43:39

     You-- Everything is managed in the prompt. You don't have that... You don't r- you don't have parameters at all at, at the moment. So if you want the song to be a certain duration, you can just ask in the prompt.

232. 43:49

     If you want the song to be using w- a certain scale, you can ask it in the prompt. If you want a certain BPM, you ask in the, in, in the prompt as well, and the model is really good at understanding everything you ask in the prompt.

233. 44:01

     And you can ask... Like, it doesn't make sense for thirty-second songs, uh, much, but, like, if you- you're building a longer song, you can, you can say, "During the first thirty second, that's what I want the songs to be," and then it switch to something else after.

234. 44:16

     Uh, or you can say, "This is the intro, this is the outro, this is the," uh, always forget how to say that in English, but the, the part that, uh, that is coming up multiple times in the song so that it knows exactly how to...

235. 44:28

     Chorus?

236. 44:31

     Chorus.

237. 44:31

     Um, this knows, uh, that this part needs to be repeated, and that it needs to be the same thing and the same model. And if you want, uh, lyrics, you can either provide the lyrics or, um, or, uh, or just, uh, let it in-invent lyrics.

238. 44:49

     So let's say we are going to change that. We are going to create, uh, songs with chapter. Um, add lyrics

239. 45:00

     to describe what's happening up in the chapter.

240. 45:11

     Up. And let's see how it goes with lyrics this time.

241. 45:18

     As I said, music generation is quite fast, like, uh, especially the thirty-second model takes a few sec to, to generate things. The, the longer, uh, part here is actually sending, sending the book again to the model and to ask it for, um, for prompts.

242. 45:38

     Uh, I think it didn't... I think it didn't work here. [upbeat music]

243. 45:47

     Mole and work flung-

244. 45:50

     I think it does work.

245. 45:52

     Mole flung down his brush and ran to the sun.

246. 45:59

     Away from the cleaning and work to be done.

247. 46:06

     And see this one. [upbeat music]

248. 46:09

     Toad in his caravan of yellow and red.

249. 46:15

     Dreamed of the dusty high roads ahead.

250. 46:22

     And, like, uh, you can see that the, the theme of the songs is quite the same. That's, uh, kind of the, the price we have to pay for using Chatnode, because it remembers the prompt it did before.

251. 46:31

     So we ask for new prompts, but it still has a memory of, uh, what it did before. So I think it's kind of encoded to kind of use the same kind of prompt and describe the scene the same way.

252. 46:42

     Uh, but still, it's, uh, it's very funny to, to work with the lyrics. And you... As you can see in the prompt, it's basically just, like, adding the lyrics in the prompt, and the model understand that this, these are the lyrics I need to, I need to play and then add in the song.

253. 46:57

     Um, and you can also say that this, this specific, uh, part, uh, this, uh, is, is set at exactly this moment in the, in the song. This part is, uh, is, uh, is, is, um, is being set another moment.

254. 47:11

     And if you check the output of the Lyria model, you actually have the whole lyrics with the times. So you can create a karaoke app or something like that using what you get from out of the model.

255. 47:22

     Let's, let's try the third one.

256. 47:24

     Into the wild wood where the shadows are deep, and evil faces through the hollow trees peep. Mole is in terror and lost in the snow, till Ratty arrives with his pistols aglow.

257. 47:48

     Yeah, we could do, uh, we could do musical with that.

258. 47:53

     Um, so that's, that's for music generation. And then, um, we have, uh, we also have a text, uh, generation models, and I'm going to show you something, uh, very fun.

259. 48:05

     I guess you all know about the all text-to-speech mo- uh, model because everybody loved the, uh, Notebook LM integration that can create, um, uh, a podcast. And, uh, and that's actually great that you can select two different voices, so you have two, two characters talking with each other.

260. 48:22

     But I'm going to show you a trick that, uh, with which you can actually create something that is basically, uh, you, you can add more characters than actually two, uh, when you're creating, uh, discussions with the TTS model.

261. 48:35

     So here's what I'm going to do. Uh, I'm going to ask the model to extract a specific dialogue from the, from the book, just because I didn't want to copy-paste it.

262. 48:44

     Um, so I told it that it starts with small, neat ears and, and, uh, [REDACTED:physical_attribute], and ends with his ears in the air. Um, but then I asked it to write it as a play, so that it's basically a transcript of wh- which, which, uh, what which character should be, uh, should be saying.

263. 49:02

     And the, and the trick is that I'm asking it to create a specific style of the, of way of speaking for each character, even though it's going to be using the same voice.

264. 49:11

     And to, and to write the transcript that way. So when it's a narrator, I'm going to use one specific voice, and when it's all of the other character, it's going to be the same voice for all of them.

265. 49:22

     Um, so nar- so narrator is saying something, and then character is saying something, and then write the style between parentheses, and that's what is going to tell the model how to s- how to, how to talk.

266. 49:33

     And that's, that's something that you can also use to say, "Oh, is i-i- it is saying this part very, uh, like whispering. And then this part is like, has a lot of emotion in it," and you can, you can play with the way the character talks.

267. 49:47

     But I'm, I'm going to, uh, use that to ask the model to come up with very different ways f- from the same voice to, to, uh, to talk when each character is talking, and that's actually creates, uh, the feeling of a- actual different voices for, for each of them.

268. 50:05

     So, and then I'm going to pass that to the TTS model and to, uh, and to ask it to, to read it, basically. Uh, one of the trick, and like I got ...

269. 50:16

     I, I lost f- 15 minutes about, because of that yesterday evening. So this... Well, you cannot just give it the text to read. You always have to start with, "Read this text," or something like that.

270. 50:28

     Otherwise, it, uh, for some reason, it doesn't know that it needs to read the text that is giving it, gi- given to it. Um, and, uh, this is, uh, very complex.

271. 50:38

     Uh, I think there's no simple way to, to set it up. But basically what I say is, "Speaker narrator is using the v- the voice, uh, Sulafat, and character is using Fenrir."

272. 50:49

     And this is all text. So narrator talks, then the character, the first character talk, and that's, uh, is going to have long poetic pauses. And then the second one is breathless, uh, and, uh, and very stutter.

273. 51:04

     And you can see that we can, we can guess which character is which one because it's the same, uh, way of speaking that is reused for each of those lines.

274. 51:15

     And it's still running. The problem, uh, with the TTS model is that it's a very good model, but it's not a very fast one. Um, the reason for that, it's a-

275. 51:24

     Small neat ears and thick silky hair. It was the water rat. Then the two animals stood and regarded each other cautiously.

276. 51:35

     Again. Again.

277. 51:35

     "Hello, Mole." "H- hello, Rat." "Would you like to come over?" "Oh, it's all very well to, to talk." The rat said nothing, but stooped and unfastened a rope and pulled on it, then lightly stepped into a little boat, which the mole had not observed.

278. 51:58

     The rat sculled smartly across and made fast. Then he held up his forepaw as the mole stepped gingerly down. "Lean on that. Now then, step lively." The mole, to his surprise and rapture, found himself actually seated-

279. 52:18

     They're not very lucky. They are both turning very slowly

280. 52:18

     ... in the stern of a real boat. "This has been a, a wonderful day. Do you know, I've never, never been in a boat before in all my life."

281. 52:31

     "What? Never been in a ... You never... Well, I... What have you been doing then?" "Is it-" [laughs] "Is it so nice as all that?"

282. 52:43

     Yeah. We can stop that. But like, you, you, you can, you can see, you, you couldn't guess that we are using the same voice, uh, for, for the two characters, uh, being like, say, say, clearly we're, uh, steered into different directions.

283. 52:54

     And you can use this trick to actually, uh, create like multiple characters, multiple voices for those characters and, uh, and make it seems like seamless for, for, for users.

284. 53:05

     As I said, uh, earlier, this is meant to be just demonstration on how to do it. If you want to d- if you were to do, to do it like, uh, at scale, that's ex- that...

285. 53:16

     I would not do this kind of like create, uh, trick like that. I would actually create a, a full transcript with the actual names and then keep on the side a prompt for each character.

286. 53:27

     And maybe sometimes you still want to, uh, you st- you don't want to talk all of them exactly uh, uh, the same way because sometimes they still need to be excited even though they, they talk very slow and, and so on.

287. 53:38

     But still, that's, uh, that's, uh, that's just to show how good the, the TTS model is at creating, uh, different voices. And even though I asked it to force an accent, it didn't f- it didn't do it.

288. 53:50

     But you can also play with like this character as an Irish accent. This one is Singlish. This one speak f- uh, with a German accent or whatever. And that's, uh, that's also a very easy way to, uh, to create different feeling, uh, about the, about the character for, of, uh, with us- using the same voice.

289. 54:08

     Um, we're nearly at time for the, for the questions. But like, uh, just to finish, like, uh, we, we used a very large context window of the model, uh, to, to fit it to full book and fit it multiple times the full book because with chat we, we just, uh, upload it all the time.

290. 54:25

     But since it's, uh, it's a multi- multimodal in model, you can ... You just... It's, uh, it work with all the things than just text. So you can just fit it like an audiobook.

291. 54:36

     Uh, you can fit it a, uh, video as well. You, like you can play with that and not just, uh, get, uh, get limited to, uh, to text to, uh, to be straight on things.

292. 54:45

     So this is another example with, uh, with another book, which, which one, which is, uh, The Adventures of Chatterer the Red Squirrel. And, and we are basically going to do the same thing.

293. 54:56

     I'm going to run all of it at the same time.

294. 55:00

     Uh, and this time, I basically ask it, uh, somewhere, uh, to use a style that is futuristic, science fiction, utopia, saturated neon lights. So it's going to be not the kind of squirrel you are expecting.

295. 55:14

     No. Um, yeah. And while it runs, I think, uh... Oh, I said I was going to show you, um, so we also have, like, I think you all know about, uh, AI Studio, but, uh, in AI Studio we have a gallery with lots of, uh, example apps that we are building, and I wanted to show you.

296. 55:34

     Uh, it's going to be in GenMedia. Uh, as I told you, we have the Lyria model that is creating musics, uh, songs. We, but we also have the... Uh, no, not this one.

297. 55:47

     Let's, let's go with it. Ah. Up. This one's better. We also have the Lyria real time, um, model I was talking about. And, and basically, you are asking it to create, uh, to, to make music that is post-punk tunes and neo-soul at the same time. [electronic music]

298. 56:07

     But you can say, "Okay, I want more K-pop,

299. 56:11

     and slightly more drums." I don't know what post-punk is, so I don't want it.

300. 56:23

     And see, you can hear the music changing.

301. 56:27

     And let's go with something, like, more chill.

302. 56:46

     So I think that, as I said, that's my favorite model because I think it's underused and I, I, I... Like, there's plenty of things I, I can imagine doing.

303. 56:53

     Like I, as I said, I come from the video game industry, so one of the things I would have tried is can you create music in real time for the player depending on, like, where you...

304. 57:03

     in which region they are. In, are they in the forest? Are they jumping? Are they cooking? Are they fighting? Uh, how much HP do they have? And so, so, so the music could change in real life, in real time.

305. 57:16

     And so that's, uh, yeah. Those are the kind of things you can, you can try. And, uh, for some reason, the link is not there, but there's another very cool example, uh, our colleagues who are working on this model made.

306. 57:27

     It's basically, uh, you are in space, and each planet is a prompt, and you can move through the planets, and depending on which planet you are close to, the music changes.

307. 57:37

     So you can just move around, uh, the planets. And sometimes there are weird things happening because, like, Christmas songs is just next to Viking metal. So, uh- [laughs] ... the, the mix can be, it can be quite funny.

308. 57:50

     Um, so that's it for the, for the presentation. I have some time for questions now.

309. 57:59

     Up. [applause] Thank you. [laughs] Yeah, and for those who arrived too late, like, you can, you can check the, the content afterwards. And as I said, uh, we have this, this cookbook that is basically a GitHub repo where we are adding, uh, the quick starts on how to use the models, uh, uh, kind of some, some tricks, and also

310. 58:23

     examples of, uh, like, uh, more complex AI, more, more complex things you can build when you are mixing different capabilities and models.

311. 58:32

     I have a question. I, I don't know if you can answer it, but-

312. 58:34

     Yeah, I think for questions we need mic, so that it's, I don't need to repeat them. [laughs]

313. 58:42

     One second.

314. 58:45

     You fade out, uh... I know where it is.

315. 58:48

     One, two, one, two, one, two, one, two.

316. 58:52

     So, so thank you first of all very much for this nice demo. It was a lot of fun to follow along. I have a question. Um, in our company, we, um, we offering to all em- employees, um, also some of, of models, and I think we are still on Nano Banana 1 because we can only offer models

317. 59:14

     hosted in the U- in Europe. And basically, all the new models are still-

318. 59:18

     Yeah

319. 59:18

     ... in preview, so we don't have access to it. Do you know if this will change?

320. 59:23

     So, uh, the short answer is no.

321. 59:25

     Okay.

322. 59:26

     Uh, but I, like, I was expecting the question because every, like, uh, it's, it's a pain for everyone in, uh, in Europe. As I, as I said, my job is to bring the feedback from the developers and to try to make things change.

323. 59:37

     So that's, that's part of the, one of the fights I'm fighting at the moment, uh, so that we, uh, we have some, some ways to offer better, a better, uh, exit for, for, or better ways to use a model for, for people in Europe.

324. 59:49

     'Cause, uh, in Europe, we care about, uh, like data privacy and data sovereignty and all of that. So I know it's a, it's a problem. Um, so the, the, the core of the p- problem in a way, it's, uh, the rule that, uh, at Google Cloud that, uh, every preview model is only available on, uh, global endpoints.

325. 1:00:09

     Um, so that's, uh, is unlikely to change. But what we are going to try to change is to release, uh, the model in global accessibility, uh, faster. The, the problem we have, we had with Nano Banana 2 and the Pro and, uh, and, uh, like Gemini 3.3 as well, is that we release models too quickly back to

326. 1:00:31

     back. And so instead of like having, uh, Gemini 3 going GA, we released Gemini 3.1, and so we, like, kind of reset the, the pre- the, the preview counter.

327. 1:00:41

     Uh, so that's, uh... We, we need to, we need to make, to do something about that. But yeah, I, I know. I, I hear you. It's, uh, it's kind of my P Zero thing that I want to change, um.

328. 1:00:52

     Thank you.

329. 1:00:58

     And, like, have you, have you run the notebook at the same time?

330. 1:01:01

     Yes.

331. 1:01:03

     Which side were you... Oh, wow. [laughs] [laughs] Did someone else run it at the same time and with a different style and that, uh... Or maybe a different book?

332. 1:01:15

     Um.

333. 1:01:16

     No?

334. 1:01:16

     I, I looked for Frankenstein.

335. 1:01:18

     Oh.

336. 1:01:20

     And choose like retro ga-

337. 1:01:23

     Sure

338. 1:01:23

     So, so I did the Frankenstein book, and choose like a retro gaming style, and it was quite interesting. So, um,

339. 1:01:32

     yeah.

340. 1:01:33

     Yeah.

341. 1:01:33

     So the character looked very video game-like.

342. 1:01:40

     Oh, yeah. [laughs] And, um, yeah, the, the main difficulty with, uh, with books like Frankenstein is that sometimes the model is not going to be, uh, allowing t- to, uh, things that are too, like, let's say, graphic that could be happening in the book.

343. 1:01:57

     So, uh, it can be a bit toned down. Or worse, it's not going to, uh, to accept to, uh, to show the image. Um, yeah. That's why I settled with kids books for the- [laughs] ...

344. 1:02:08

     for the example. That's easier. [laughs] Ex- except when I was not able to, uh, make, uh, images of c- of, uh, childrens, which was also another limiting factor.

345. 1:02:19

     Um, yeah. I can show you other... I don't know what, uh, page afterwards is going to show. I can show you an other cool demos that we have related to GenMedia.

346. 1:02:30

     Um, in the meantime, if, if you have questions, just, uh, raise your hand and, uh, and we can... Up.

347. 1:02:38

     Uh, up. If we're going back here. See, that's, uh,

348. 1:02:46

     whatever. Uh, that's the, uh, futuristic neon style of a version of, uh, of Chatter, the, the squirrel. [laughs]

349. 1:03:05

     Up. Um. I can show you, um, a bit more about Lyria because it's new and, uh, you likely already know everything about Nano Banana. Um,

350. 1:03:20

     up. So as I said earlier, when you work with, um, with Lyria, the... you always get, uh, two outputs. Well, if you, if you set the modalities to be, uh, they're audio and text, you will get two outputs, and the first output is going to be the, the lyrics, and the second output is going to be the

351. 1:03:39

     music. Um, and that's actually one of the few model where it's very interesting to use, uh, streaming. So when you do generate content here, you can use generate_content_underscore stream.

352. 1:03:52

     And what it does that you receive the first part first, and then the second part afterwards. So you get the lyrics first, so if you want to do something, uh, according to the lyrics, like creating an image or, uh, like giving a- this song a title, then you can do it while the music i- is, uh, is

353. 1:04:09

     still generating, and you don't have to wait for the full output to be, to be there.

354. 1:04:14

     So you get the, you get the lyrics and you get the timing. So the... this, um, this sentence is going to be said at the beginning, and then after 4.8 sec, it's going to say something else, and so on.

355. 1:04:25

     So you can, you can hear-

356. 1:04:26

     The air is still and cold up here. The mountain tops are sharp and clear. And then a streak of gentle gold.

357. 1:04:40

     So yeah. And then you can do-- and you can provide the same thing. And here it's only, like, you can see m- the last one, it's providing when it starts and when it ends because it wants to have, like, 1.2 second without, uh, without being said at the end.

358. 1:04:56

     Um, up. Uh, you can also create images from, uh, music from images. So that's one of the thing that I forgot to do in, uh, in today's demo. Like, I should have also gave the, the, the images from the, from the chapter so that it would have been used, uh, as a reference to create the first image.

359. 1:05:14

     So I use this picture of, uh, like, a grocery list for, for making a pot-au-feu, and then it will come up with a song about doing a pot-au-feu. [dramatic music]

360. 1:05:32

     What was the prompt again? An epic song with opera voices about this quest.

361. 1:05:39

     See? It's becoming epic.

362. 1:05:43

     The scrolls are dry. The ink is ancient. The list is long. The soul is patient. Beef shank and ribs, the holy prize.

363. 1:05:59

     In the cellar where the shadow lies. The carrots and the turnips weep.

364. 1:06:07

     And that's so also nice that you can use multiple voices as well.

365. 1:06:10

     The destiny is at the gate.

366. 1:06:12

     I'm going to skip ahead a bit.

367. 1:06:13

     The fate or chance. The time is gone. The celery wilts, the celery wilts. The end is near. The final moment of the quest is here.

368. 1:06:29

     Bouquet garni, bouquet garni.

369. 1:06:31

     That's the chorus I like. [laughs]

370. 1:06:34

     The cloves, the cloves, the salt.

371. 1:06:37

     So, uh, but then, um, and, oh, I, I talked about the interactions API earlier, about there being all new, uh, ways of using the API. So that's an example using those.

372. 1:06:46

     So it basically was the same as, uh, the current API. Um, so you give a model, you give an input, and you get re- responsibility modalities. Uh, yeah. Phillip, who worked on that, is not there, so I can say it.

373. 1:06:59

     I would not have renamed, uh, uh, content to input because it's going to confuse everybody, but that's how it is. Um, and then you get the output and that's, um...

374. 1:07:12

     And if we can check in the output, I think, no. Uh, yeah. We don't, we don't see it here. But there's, uh... And then it works basically the same way, except you get this, uh, interactions ID that you can use to, uh, to chain things

375. 1:07:27

     Um, and then that's, uh, yeah, that's one with images and prompting. Um, what you can do as well is you can use the BPM part. So you can... You, you want a song that is very fast or very slow, so that's, that introduces [upbeat music] And I even t- uh, told it to use a reset accelerando illusion, so it

376. 1:07:47

     gives the, uh, the illusion that the music is getting faster and faster and faster. [upbeat music]

377. 1:07:53

     So if you want, uh, music for when you do your sports routine, that's, uh, how you do it.

378. 1:08:00

     And as I said, you can give like, a specific time. So the first 10 second are going to be fast acoustic guitar, and then it goes into piano for 20 sec- for 10 more sec, and then it's full band afterwards, so [upbeat music]

379. 1:08:21

     Actually, it's not following it. [laughs] Okay, forget, forget what I just saw... showed [laughs] [laughs]

380. 1:08:32

     For some reason. And then, but the easiest way is like this. You can use the, the... You can give the structure. So that's how I want my intro, that's how I want my verse, that's what I want my outro, and that's 30 second song, so it's not...

381. 1:08:45

     Like, you, you won't have the chorus, but you can also add the chorus and, uh, the bridge and so on. So, um-

382. 1:08:51

     The darkness breaks, the shadows fall.

383. 1:08:55

     Where is it about? Ah, yes, the sunset

384. 1:08:57

     Hear the dawn's triumphant call. A golden light, a glorious sight. Chasing night with Heaven's might.

385. 1:09:13

     Our hearts resound for-

386. 1:09:15

     Yeah. I need to make made a better, like, longer song, I think, for this example. Um, but you can also chain all of that into, uh... Like, everything together.

387. 1:09:26

     So this is a full song where, uh, from, from the s- uh, first two second I have an intro. I, I can tell exactly which, uh, which scale to use, how intense it's going to be, and then it move to another verse, and so on and so on, and that's, uh, how you can get something very, uh,

388. 1:09:43

     complex. [slow romantic music] So it starts very, very slow, and then if we move,

389. 1:09:52

     the drums, uh, and bass started and so on, so we should be in this part. It's still laid back.

390. 1:10:00

     And, and it's starting to be, yeah, to add grooves and so on. So if you, if you want to create complex things, it's better to use a full song model because it's, uh...

391. 1:10:13

     The, the, the short one is taking shortcuts to actually build something that is interesting in 30 seconds.

392. 1:10:20

     And as, as I already showed, you can provide the lyrics, so it's, uh, creating a song about Nano bananas.

393. 1:10:25

     Yellow peel, a tiny sweet. The Nano banana, a tropical treat. But wait, it hums, it starts to create. Switching into AI mode.

394. 1:10:37

     Um, but, uh, and, and what's, what's funny as well is that you can use it to create things that are basically not, not, uh, songs. So you can, uh, you can ask it to create, uh, music, but without...

395. 1:10:49

     With very calm, uh, music in the background and just some- something reading a text or, or, or something like that. So this... And this one I'm using the reasoning capabilities of the model to...

396. 1:10:59

     And its, its knowledge about what Shakespeare is doing. And so to, to create a text that is basically, uh-

397. 1:11:05

     The fleeting candle-

398. 1:11:06

     ... something that looks like Shakespearean

399. 1:11:07

     ... of my life burns low. And all my glories to the grave must go.

400. 1:11:21

     Oh, heavy grace to be at last at peace.

401. 1:11:26

     And...

402. 1:11:29

     Grant my weary soul a sweet release

403. 1:11:33

     And I didn't really tell it to read, so that's why there's still music, but you can, you can really steer it into not having background music at all and, uh, and just, uh, say things.

404. 1:11:43

     And you... It works also in different languages, so you can, uh, you can just ask it to, uh, to, to, uh, to create songs in all languages that you want.

405. 1:11:51

     Sometimes there are a few words that are not pronounced the right way. It's still, uh, it's, it's still g- getting better. Uh- [singing] And this one I try to ask Gravit to, um, to use two different languages in the same song and s- and since it's the same as, uh, TTS and live models that we have, it's really

406. 1:12:08

     good at switching, um, switching the, the, the language in the middle of the, of the generation. [singing]

407. 1:12:18

     And once again, I'm using the model knowledge of things because it's basically trying to explain how Google Thought is working, uh, in music. [singing]

408. 1:12:28

     Ole!

409. 1:12:29

     Um, and instrumental. You, you saw it. So yeah, there's plenty of, uh, very cool things to, uh, to build with, uh, with the Lyria models, so give, give it a try.

410. 1:12:37

     It's, uh, it's very easy to use and, uh, because everything is in the prompt. So, yeah.

411. 1:12:45

     Cool.

412. 1:12:47

     Do you have any other question? No? What do you want, uh, Paige to show you afterwards?

413. 1:12:57

     Ooh.

414. 1:12:57

     She has, she has seven more minutes to prepare, uh, something new in her, in her presentation. [laughs]

415. 1:13:02

     True. Uh, and also just for, uh, for clarification, I'm not sure if this was announced to the, uh, to all of y'all, um, but there are some electrical issues in the building.

416. 1:13:12

     Um, so, uh, there, uh, y'all are like the lucky valiant few that made it here earlier this morning. Um, the, uh, most of the attendees were not allowed into the building.

417. 1:13:24

     Um, and so, uh, the, uh... We'll do the session that's coming up at 10:40, but we'll also be bringing back everybody in the afternoon who was going to be presenting in the morning to do kind of like a, a whistle stop tour of all of the Google DeepMind things for the a-

418. 1:13:42

     So if you would prefer to come back for the afternoon workshop, you can. It'll just be at 1:00 PM.

419. 1:13:53

     So, and that was the example I was talking about.

420. 1:13:56

     Ooh.

421. 1:13:56

     It's just, I don't know where Christmas songs are, but, uh-

422. 1:13:59

     How can you try this, um, by your own?

423. 1:14:02

     You just search for Space DJ, and it's available online. [upbeat music]

424. 1:14:10

     Can you move the, to put the songs, uh, higher? [upbeat music]

425. 1:14:22

     She chants this, so it, it could be nice. [singing]

426. 1:14:26

     So the model is not meant to do voices, but it can do some kind of like, uh, vocalizations like that.

427. 1:14:34

     Uh. [singing] It was not that bad at, uh, using the same voice and switching the style of music and everything.

428. 1:14:55

     Yeah. And you have an autopilot, so it just moves around and, like, creates music, uh, until you stop it. [singing]

429. 1:15:04

     Yeah, Southern rock. I think Nashville songs. Yeah. [singing]

430. 1:15:19

     Yeah, it's not moving fast enough. Let's move s- [laughs] Let's move to another place. [singing]

431. 1:15:33

     [REDACTED:origin] hip-hop. [rapping] Turn table, turn tables. [rapping]

432. 1:15:51

     [REDACTED:origin] hip-hop, I guess. [laughs] No, no [REDACTED:origin] here? [laughs]

433. 1:15:58

     Um, but yeah, that's, uh, that's a very cool model, and, uh, the only thing is that the session ends after 10 minutes so that you- so you have to run it, uh, at, at the time.

434. 1:16:08

     But, uh, like, in theory it should work. [rapping] That's...

435. 1:16:17

     It's just missing a search button. Speed metal. [metal music]

436. 1:16:30

     So I will stop with that. [laughs] But yeah. Give it, give it a try. It's a really cool model to play with.

437. 1:16:37

     Um, okay. So yeah, I guess that's it. I will still be around if you have other questions, things you want to discuss that you didn't want to be on camera.

438. 1:16:49

     So yeah. [audience cheering] [audience applauding] Thank you. [upbeat music]