Gemma, DeepMind's Family of Open Models

Talk transcript

1. 0:00

   [upbeat music] All right.

2. 0:15

   Uh, hi everyone. It's Pol here. Uh, so I'm super excited to give this talk because just seven days ago, we released Gemma 4. Uh, so before this conference, who here has heard about Gemma already?

3. 0:28

   Okay. So most of you. Great. So Gemma is Google DeepMind's, uh, family of open models. Open models means that these are models that you can, uh, take, you can download, you can run in your own infrastructure, your own devices, you can fine-tune for your own use cases.

4. 0:42

   So about a year ago, we released Gemma 3. Back then, Gemma 3 were the most capable open models that could fit in a single consumer GPU. So, uh, we designed models from one billion parameters all the way to twenty-seven billion parameters.

5. 0:54

   And back then in LM Arena, it was a very strong model. So you see here like different open models, uh, under LM Arena scores, and those small dots at the bottom represent how many H100s or A100s you would need just to be able to load the models.

6. 1:09

   So, uh, this is again Gemma 3. That's from one year ago. Uh, but you can see that even if it's a model from a year ago, it's a tiny model or a relatively small model that is extremely capable.

7. 1:20

   But yeah, so last week, uh, we released Gemma 4, and this is my first conference talking about Gemma 4, so very excited about that. So Gemma 4 is the family of most capable of open models that, uh, Google has released ever.

8. 1:31

   These are models that go from, uh, two billion parameters all the way to thirty-two billion parameters. Uh, these models have very different capabilities, so I'm going to talk a bit about these different things.

9. 1:41

   And if you are wondering what's the E there, I also explain that in a second. So the smallest two models can run in an Android phone, in an iOS, in a iPhone phone as well, even in a Raspberry Pi.

10. 1:52

    These are really small, small models that are multimodal, have reasoning, can do like very cool on-device agentic things. Then there's a MoE, a Mixture of Experts model that's, uh, like super fast, high...

11. 2:04

    uh, very low latency. A model that you can do, uh, that can do very cool things. And then you have the 31B. That's the most, uh, intelligent model, the most capable.

12. 2:13

    So when you want like the most raw intelligence, you would use this large model. But even the 31B is a model that can run in a consumer GPU. So all of these models have been in, uh, developer-friendly sizes, which is quite important to us.

13. 2:27

    So let me show you a couple of demos,

14. 2:31

    assuming the videos load. Uh, so there's a lot happening here. So let me, uh, begin with the one at the right. That's, uh, an application where you have Gemma running directly in an Android phone where you can pick different skills.

15. 2:44

    So pretty much here you have a full agentic setup where the model is picking maybe like a skill to play the piano, and then you have Gemma playing the piano, right?

16. 2:53

    Uh, the one at the left is Gemma live coding, also on device. Uh, this is again airplane mode, no API calls, fully running in a phone. And the example in the middle is, uh, in a laptop computer.

17. 3:04

    We have twenty instances or ten... sorry, ten instances of Gemma running in parallel. Uh, each of them is doing a different SPG. And in a couple of seconds you are going to see like, uh, ten SPGs generated by different agents, all of these running on device with llama.cpp.

18. 3:19

    And even then it's like a hundred tokens per second. And there you can see the SPGs that were generated by the ten different, uh, Gemma models. Uh, Gemma is a good coding, uh, model.

19. 3:29

    It can do agentic stuff. It can do coding. It can do even Android, uh, app development. Uh, and again, all of this offline. So, uh, the LM Arena scores are quite nice.

20. 3:39

    Uh, here you can see like bunch of different models. X-axis is how many billion parameters the model has. Y-axis is the LM Arena score. And I know like LM Arena is not the perfect benchmark, but it does give you like some proxy of how much the community likes the model for general use cases like conversations and so

21. 3:57

    on. Uh, and Gemma has like a nice kind of a mix between being friendly and like, uh, helpful and at the same time being very capable. Uh, and you can see like this corner at the top left, that means that these are very small models that are very capable, which is quite exciting.

22. 4:14

    Uh, it's been exciting to see how the models have progressed over the last two years. So last year it was Gemma 3. Two years ago, it was Gemma 1.

23. 4:22

    Uh, sorry, yeah, Gemma 2. And you can see like for a bunch of different things, uh, the models have get, get, getting better and better without going, uh, bigger.

24. 4:30

    Which for me is quite exciting because if I think where we'll stand in a year from now or in two years from now, uh, I do think we'll have extremely capable models running directly in our own devices, in our own pockets.

25. 4:42

    Uh, I'll skip the benchmarks. Uh, but yeah, what is exciting is that, uh, Gemma can fit in a desktop computer, it can fit in a laptop, it can fit in a phone.

26. 4:51

    Uh, I saw yesterday or two days ago that someone put llama.cpp in a Nintendo Switch, and they are using llama.cpp, uh, to try Gemma directly there. So I don't know how things will be in a couple of years, but [audience laughing] uh, I'm excited for it.

27. 5:06

    Uh, something that we heard a lot with the previous Gemma versions is... wa-was that the license that we had was not great. Like, people wanted a proper open source license.

28. 5:14

    So with Gemma 4, we changed our license to an actual Apache 2 license that gives you control to, uh, pretty much you have the flexibility of the Apache 2 license.

29. 5:23

    So, uh, that's quite nice as well. Now, uh, you have probably heard about Mixture of Experts. That's the 27B model, uh, 26B model. You have heard about transformer sentence models, but you have probably never heard about the E here.

30. 5:39

    So E2B stands for, uh, effectively two billion parameters. So actually Gemma E2B has more parameters. It has four billion parameters or so. And it has a new novel kind of architecture, uh, called per-layer embeddings.

31. 5:51

    Uh, that was something that we released, uh, summer of last year. So there's this small block at the bottom, and the TLDR here is that pretty much there is like a embedding kind of a per each layer, as the name indicates.

32. 6:04

    And it works more of a-

33. 6:07

    Pretty much as a lookup table rather than a computation that you need to do. So pretty much this is an extremely fast thing. You don't need to have this in the GPU.

34. 6:14

    You can have this in the CPU. You can have this in the disk. And this is a architecture decision that is really optimized for on-device, like mobile use cases.

35. 6:23

    So that's why the smallest models that can run in an Android or in an iPhone are using this E2B, uh, or E4B architecture. So even if the model is five billion parameters, you actually just load two billion parameters into the GPU, and then the rest can be, like, much lower memory because you are not doing any of

36. 6:39

    the matrix multiplications that you would usually do with a transformer architecture. And this can be done by leveraging llama.cpp with a simple flag, override tensor, and then you move the per-layer embeddings to CPU or even to disk, and it should work quite well out of the box.

37. 6:55

    Uh, a couple of other exciting things. The smallest models can do multimodal understanding for images, for videos, and even for audio. So you can do speech recognition. You can do speech to translate the text.

38. 7:08

    So I can speak in Spanish, and the text can be, uh, transcribed to, I don't know, French. Uh, and then the larger model can do, like, extremely capable multimodal understanding.

39. 7:18

    So, uh, videos, uh, fine-grained details. Uh, I actually have a couple of examples in here. So, for example, it can do things such as pointing where the llama is in the picture.

40. 7:29

    Uh, it can, uh, do object detection, so it can detect different objects in a picture. And what is cool is that this model is heavily multilingual. So Gemma 4 has a...

41. 7:39

    Well, it was trained with over one hundred and forty languages, and it uses the tokenizer, uh, that is based on Gemini as well. So pretty much all of the multilingual research that powers Gemini, uh, is also enabling Gemma.

42. 7:50

    Uh, the tokenizer piece is quite interesting because independently of the raw capabilities of Gemma, this tokenizer was designed for multilingual use cases, uh, and we took lo- lots of care with it.

43. 8:02

    Uh, which is interesting because if you want to fine-tune Gemma for a different language for which there are, uh, low digital resource languages, so let's say like an indigenous language in Peru, Quechua, or I don't know, one of the official languages in India, you can pick the model, you can use, uh, your data, you can train the

44. 8:19

    model. And independently of the raw capabilities of Gemma, just because of the tokenizer decisions, things tend to work quite well out of the box. So then you can mix the multilingual with multimodal capabilities.

45. 8:31

    So for example, here to get the text, uh, or an explanation of, uh, an image with Japanese text, and that's quite cool.

46. 8:43

    Uh, so we released the model a week ago. Uh, just last, uh, yesterday, we got to ten million downloads just for Gemma 4-based models. There are over one thousand models based on Gemma 4 already, so quantizations or fine-tunes by the community.

47. 8:58

    Over five hundred million downloads of the whole Gemma family. So what is very cool for me is that Gemma is not just about always some model that you can use, but it's more about enabling the ecosystem to build on top of it.

48. 9:10

    And that's what the community has done over the last few days. Uh, it was top of at Hugging Face. People have been building, like, cool examples. The Onslaught people have been doing, like, full repository audits using Gemma.

49. 9:21

    People are putting Gemma, like, in, in all kinds of devices and exploring all of the capabilities, which is quite nice. And all of this is not done just by us.

50. 9:27

    We collaborate with an open source ecosystem. We work with Onslaught, MLX, Ollama, Hugging Face, vLLM, SGLang, and pretty much we want to ensure that when we launch a new tool, both for Gemini and for Gemma, people can leverage the capabilities out of the box, right?

51. 9:41

    Like, they should not need to switch to, uh, Keras if they want to fine-tune Gemma. Like, if they are fine-tuning with, uh, Hugging Face transformers, they should be able to do that.

52. 9:50

    So for us, it's very important and critical to be where the community is, and that's why really shout out to all of those of you that are working in the open, uh, source ecosystem, that are contributing to, uh, different tools, uh, maintainers of all of these repositories, because it's really a way to enable the ecosystem to do

53. 10:05

    amazing things. Uh, another part that I like about Gemma is all of the product integrations that we can do. So Android Studio, uh, I don't know if anyone here is an Android developer, but Android Studio has, like, a, a agent mode where you have a agent that helps you write code and develop.

54. 10:20

    And there's a offline mode now where you can have a llama.cpp or a Ollama or vLLM-powered, uh, uh, system in which you have Gemma, uh, helping you write code, uh, for Android development.

55. 10:33

    And we did include some Android-related datasets and benchmarks while training Gemma, so it's actually a very capable model for Android development. So I talked a bit about how many, like, people are fine-tuning and about how many people are sharing.

56. 10:45

    So let me share a bit about the, the Gemma numbers. Uh, so this number is, uh, outdated. This is from last week. Now we have five hundred million downloads, as I mentioned.

57. 10:54

    And in total, Gemma has over one hundred thousand, uh, models. So again, uh, maybe you just want to use Gemma out of the box, like open models may work great for you.

58. 11:03

    But maybe you want to, uh, improve the capabilities. Maybe you want to change the style in which the model is talking with the users. Maybe you don't want a conversational model, right?

59. 11:12

    Maybe you just want a model that can predict certain thing in your own context. Uh, or maybe you just have too many GPUs at home, and you just want to burn them.

60. 11:21

    Uh, I don't know what's your reason, but you can fine-tune models for many cool things. So Google has done a couple of what we call official Gemma variants. We did a ShieldGemma, which is a family of word-rate models.

61. 11:32

    Those are great for production use cases where maybe you don't want users to put, uh, let's say toxic images or toxic text that does not match the policies that you have set up.

62. 11:43

    Uh, so ShieldGemma is the family of models that allows you to do that. But then there are also other kind of use cases. So for example, for medical use cases, we have released MedGemma, which is a multimodal Gemma, uh, three-based model for different, uh, medical tasks.

63. 11:57

    So radiology, uh, X-ray, uh, chest X-ray understanding, and a bunch of other things. And again, these are open models. You can use them, and you can also fine-tune them even more if you have, like, a even, uh, more niche kind of use case.

64. 12:13

    So that's what Google has done. But the community is also doing cool things. So for example, there is AI Singapore. It's a group that is training models for, uh, Southeast Asian languages.

65. 12:23

    Uh, there are a bunch of them, and they have been building quite a bit of research with open models to push even further the state-of-the-art capabilities in terms of multilinguality.

66. 12:33

    Or another example is SARVAM. Uh, so in India, uh, there are many official, uh, languages, and there is this effort by the government. They are investing in a couple of, uh, big startups to train national models.

67. 12:45

    So, uh, this is more on the sovereign AI and official, like, languages, uh, point of view. But people are doing, like, very interesting stuff on the multilingual side of things.

68. 12:56

    Apart of that, there is quite a bit of other, like, cool research happening. So there was this paper we released in December of last year about how some researchers from DeepMind were able to use Gemma 3 to propose some, uh, cancer therapy pathways, which was actually taken to an actual lab.

69. 13:11

    And they were able to validate that the pathways that were proposed by this, uh, Gemma-based model were able to actually, uh, lead to actual results that could be validated.

70. 13:21

    So that was quite exciting because it's not just about, uh, having your assistant or chat-chatting with, uh, yeah, your... I don't know, like doing role-playing and whatnot. It's also about building models that can be used for actual things that help the community for many different things.

71. 13:36

    So, uh, be that like finance or be that, uh, I don't know, like legal reviews, uh, offline use cases where you don't want your data to leave your servers.

72. 13:46

    Uh, if that's like for offline modes, if you're in, in, uh, I don't know, in the subway, if you're in an airplane and you need to use AI for something.

73. 13:54

    If you want to have a Chrome extension that has, uh, Gemma in there and help you understand what is in your screen. If you want to do on-device control.

74. 14:03

    The open models are getting there. And for me, that's quite exciting because if you compare where we are now versus how we were like one year ago, two years ago, open models now can do very cool, very interesting, highly agentic, complex tasks entirely on device, entirely in your phone.

75. 14:19

    Uh, so I really like recommend all of you to just spend like one hour in the next two weeks, just play with open models, uh, the latest open models, and try to understand which are the capabilities.

76. 14:30

    Of course, there are many things for which you will want to use a API-based model. If you want like the most raw intelligence, you will go and use like Gemini or, uh, your model of choice.

77. 14:40

    But if you want to have things on device, there are many exciting things that you can already do. Uh, and for me, what is more exciting is I don't know how things will be in six or twelve months from now, but I think we are heading towards a very exciting direction where people will be able to have

78. 14:54

    extremely capable open models i-in their own devices that are customized for their own use cases with their own data, uh. So yeah, uh, please try the models, build something, and share that.

79. 15:07

    All right. Thank you. [audience clapping] [outro jingle]