AI Engineer Europe 2026
How Google DeepMind is researching the next Frontier of AI for Gemini — Raia Hadsell, VP of Research
Read the talk
Beyond language models: embeddings, weather forecasts and interactive worlds
Raia Hadsell traces three directions for frontier AI: representations that retrieve across modalities, forecasts that model uncertainty, and generated worlds that respond and remember.
From a talk by Raia Hadsell
Before you start: Basic familiarity with neural networks and vector similarity will help with the embedding example; the forecasting and world-model mechanisms are explained as they arise.
A research path from philosophy to neural networks
What comes next for intelligence when the research agenda extends beyond language models? Raia Hadsell approaches that question through a career that began outside computer science. At the time of the talk, she describes almost 13 years at DeepMind and a role as a UK AI ambassador, helping connect government, academia and industry.
Her undergraduate work in the 1990s was in philosophy of religion. In the 2000s, she moved into computer science and completed her PhD with Yann LeCun, studying convolutional networks and neural networks for robots. In the following decade, she joined a DeepMind of roughly 30–40 people, working on Atari, Go, StarCraft and robotics. She now helps lead approximately 1,200 scientists and engineers across 10 labs. That trajectory connects the three research areas that follow: learning representations, predicting physical systems and building environments in which agents can act.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Find the root problem
The immediate questions include which architectures should come next for Gemini, which problems AI can uniquely address, and how advances will change human and robotic intelligence as well as artificial intelligence. The research target is broader than a better model in isolation: people and the systems around them change as the technology changes.
Hadsell describes the approach as looking for root nodes. A foundational unsolved problem can support many downstream applications once it is resolved. Partners help both identify those problems and carry the resulting capabilities into practical uses. The selection criterion is also a responsibility: building AI for humanity’s benefit means choosing problems worth solving. Embeddings, weather forecasting and world models are three examples of this agenda that are not directly about language models.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recognize the concept, not just its presentation
Suppose you hear the name Jennifer Aniston, see her picture, watch a video of her or recognize her voice. The sensory inputs differ, but the person you recognize is the same. Hadsell introduces the neuroscience analogy of a Jennifer Aniston cell: not necessarily one neuron, but a small set of neurons responding to a particular person, thing or place. In her account, that representation is robust across modalities and supports rapid recognition, retrieval and comparison.
An embedding model aims for a related property in an artificial neural network. It encodes inputs into representations whose relationships remain useful when the presentation changes. The important output is not a newly generated description; it is a representation that lets a system compare inputs and retrieve relevant material.
Contrastive losses train those relationships, encouraging related examples to have compatible representations and distinguishing them from unrelated examples. Hadsell connects this to her PhD work on Siamese neural networks, an early approach to learning comparisons between inputs. The resulting capability complements generative AI: sometimes the next operation should generate something new, and sometimes it should retrieve something that already exists.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One semantic space, adjustable vector sizes
Gemini Embedding 2 brings that idea to a model derived from Gemini and described by Hadsell as fully omnimodal. A unified semantic space removes the need to build separate representations for audio, vision and text and then map them together afterward. The intended benefit is an end-to-end representation that preserves information otherwise lost at those joining steps.
Hadsell describes a single vector representing up to 8K text tokens, 128 seconds of video, 80 seconds of audio or a full PDF, for retrieval, querying and agentic logic. For implementation, the March 2026 launch announcement specifies 8,192 text tokens, 120 seconds of video and PDFs of up to six pages; it does not establish the spoken 80-second audio limit. Those published limits should govern input preparation rather than the broader spoken account.
Matryoshka Representation Learning, or MRL, makes the representation’s size adjustable. The same network learns nested representations, so a shorter prefix can support an inexpensive retrieval pass and additional dimensions can support a more expressive comparison. Hadsell’s example starts with 256 dimensions and then expands. This means using more coordinates from the learned representation—not recovering coordinates that were discarded when only a short vector was stored.
A retrieval implementation can therefore keep full vectors available while using their first 256 coordinates to shortlist candidates. This Python example takes already-computed MRL embeddings, normalizes the chosen coordinates for cosine similarity, and reranks the shortlist using the full representation:
python
from math import sqrt
def cosine(a: list[float], b: list[float]) -> float:
if len(a) != len(b):
raise ValueError("Vector dimensions must match")
norm_a = sqrt(sum(x * x for x in a))
norm_b = sqrt(sum(x * x for x in b))
if norm_a == 0 or norm_b == 0:
raise ValueError("Cannot compare a zero vector")
return sum(x * y for x, y in zip(a, b)) / (norm_a * norm_b)
def retrieve(
query: list[float],
documents: dict[str, list[float]],
shortlist_size: int = 20,
limit: int = 5,
) -> list[str]:
if len(query) < 256:
raise ValueError("Expected at least 256 dimensions")
if any(len(vector) != len(query) for vector in documents.values()):
raise ValueError("All embeddings must have matching dimensions")
if not 0 < limit <= shortlist_size:
raise ValueError("Require 0 < limit <= shortlist_size")
shortlist = sorted(
documents,
key=lambda key: cosine(query[:256], documents[key][:256]),
reverse=True,
)[:shortlist_size]
return sorted(
shortlist,
key=lambda key: cosine(query, documents[key]),
reverse=True,
)[:limit]
Only shortlisted documents receive the full-dimensional comparison. The richer pass cannot rescue a candidate excluded by the first pass, so shortlist size remains a retrieval-quality choice.
The broader aim is a shared semantic space that is both useful and economical to search. Hadsell describes the model’s quality as state of the art, without presenting a numerical embedding benchmark in this talk. Its role in the research agenda is clear: strong retrieval is a companion capability to strong generation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Learn to predict the atmosphere
The weather work began with a concrete question from a scientist at the UK Met Office: could AI predict rainfall better than physics-based models? Hadsell took the question back to DeepMind. Atmospheric simulation is difficult, but the team had a substantial learning substrate: four decades of global weather data. That made forecasting a tractable neural-network problem even though the underlying physical system remained complex.
GraphCast predicts atmospheric conditions around the globe. Hadsell describes a horizon of up to 15 days; the original GraphCast release documents a 10-day horizon. Its architecture uses a spherical graph neural network, which she illustrates as a structure surrounding Earth and representing the atmosphere from the surface into the lower stratosphere.
The model takes atmospheric state as input and predicts the next state autoregressively: predictions become inputs for subsequent forecast steps. Hadsell describes roughly 100 atmospheric variables, with wind speed, temperature and humidity as examples. The globe and three maps on the slide make the scope tangible: this is a prediction of a changing global field, not a single temperature or a text answer about tomorrow’s weather.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From a hurricane track to a distribution of risks
The Hurricane Lee demonstration follows a storm through the Atlantic: it pauses, turns north, accelerates and reaches Nova Scotia. Hadsell introduces it with a late-2024 reference, but the documented event was in September 2023. The animation shows nine days of the graph model’s predicted movement.
For Hurricane Lee, Hadsell reports accurate landfall guidance nine days ahead, compared with six days for the best physics-based models. The value of that particular result is operational: three additional days before a major hurricane’s landfall can materially change preparation time.
The next model, GenCast, retains a mesh-based approach but makes the forecast probabilistic. Weather is chaotic, so the goal is not only to identify a likely outcome. A useful forecast must also describe the tails of the distribution—the less likely outcomes that may carry severe consequences. Producing possible futures makes that uncertainty available for weather prediction and decisions about risk.
Hadsell summarizes GenCast’s accuracy as winning 97% of approximately 1,300 comparisons. The published evaluation is more precise: GenCast outperformed ECMWF ENS on 97.2% of 1,320 variable-and-lead-time targets, using 2019 test data after training through 2018. This measures performance across evaluation targets, not the percentage of individual forecasts that were correct.
The reported eight-minute runtime on one Cloud TPU v5 is for one 15-day ensemble member, not the full ensemble. Hadsell contrasts this with hours on a large supercomputer for conventional forecasting. The computational difference matters alongside accuracy: a probabilistic forecast requires multiple members to represent alternative futures, so the cost of producing each member affects what can be run operationally.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the cyclone a direct prediction target
The next change concerns what the network is trained to predict. Hadsell introduces FGN, which she expands as Functional Generative Network. Rather than predicting the weather and then attaching a cyclone detector as post-processing, this model incorporates cyclone recognition, categorization, trajectory, wind speed and eye formation directly into training.
| Approach | Prediction path |
|---|---|
| Weather forecast plus detector | Predict atmospheric conditions, then detect cyclones |
| Direct cyclone training | Train for cyclone properties within the network |
The distinction is where the application’s target enters the system. In the second approach, identifying and characterizing the cyclone is part of what the network learns to do, rather than only an interpretation of its output afterward.
Hadsell reports that the US National Hurricane Center has already used the model and is enthusiastic about the advantage it offers. Her account does not specify whether that use is experimental or part of routine operations. Worldwide adoption remains a hope for the coming years.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate the environment, not just train the agent
World models emerge from a different strand of DeepMind’s history: Atari, Go, StarCraft and MuJoCo-type environments for robotics. Those settings let researchers study an agent’s actions and their consequences. Over time, the question widened from how to train the agent to how to create an effectively unbounded supply of environments for it.
Genie 1 provided an early version of that capability. A requested world became a short 2D platformer environment that responded to left and right controls and allowed the player to jump around. Its worlds lasted only a few seconds, but their variety and responsiveness justified scaling the data, improving the method and training again on 3D games.
Genie 2 extended the idea into 3D. Its environments were interactive, but interaction was too slow to be real time, and their appearance did not yet reach realistic, high-definition quality.
| Model | Generated environment | Limitation described in the talk |
|---|---|---|
| Genie 1 | Varied, controllable 2D platformers | Lasted only a few seconds |
| Genie 2 | Interactive 3D worlds | Slow interaction; limited realism |
The progression separates several requirements that can otherwise collapse into the word “generation”: diversity, responsiveness, dimensionality and visual quality each have to improve.
Hadsell then introduces Veo 3 as a turning point before moving to the newer world demonstrations. She does not give a Veo-to-Genie integration recipe; the emphasis shifts to what the generated environments now let a person do.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A body inside the generated scene
The first request is a world in which the user walks down a muddy lane in Kent. It resembles somewhere near Hadsell’s home. Looking down reveals a body, and moving forward affects the water. Hadsell interprets this as more than reproducing the appearance of a lane: the model also produces aspects of engaging with that environment.
The next example switches from walking to skiing. The first-person view includes gloved hands and ski tips beneath a snowy slope and distant peaks. The change illustrates the breadth of interaction the system is being asked to support: a generated world needs to respond to how a participant moves through it, not merely offer different scenery.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Extend a video into a world that remembers
The next demonstration begins with an artist’s short video. Its on-screen label changes from Original to Genie as Genie 3 continues the scene into an environment the user can explore. The artist supplied only the first few seconds; the generated continuation allows flight through the world.
Interaction includes bouncing off a structure, moving away and returning to it. In the selected frame, an orange aircraft approaches a large blue ring over a green landscape. The significant behavior is the ability to revisit the structure: an explorable environment needs previously encountered places to remain available as the viewpoint changes.
The memory test becomes more explicit with the prompt, “I'm an origami lizard in an origami world.” Hadsell describes running in one direction for a minute, returning to the starting point and finding the environment as it was before. This example tests spatial consistency alongside visual quality and control. The Genie 3 announcement describes consistency over a few minutes, so the demonstration supports a bounded memory capability rather than indefinite persistence.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Change the world while someone is inside it
The final demonstration adds another kind of control: prompting the world during interaction. It begins with a walk along Camden Canal in London, near the DeepMind office. Hadsell then supplies additional prompts and changes the environment repeatedly while remaining inside the experience. The world is no longer determined solely by the request that started it.
That suggests a different form of multiplayer interaction: one person could adversarially prompt another person’s world, altering the experience as they play. Hadsell presents this as a possible new gaming format. The educational possibility follows from the same capability—entering a world in order to learn about it, with the environment itself becoming part of the learning experience. Both are prospective applications of the demonstrated control.
Hadsell closes by pointing to a talk the following morning from her colleague Omar about Gemma 4, returning explicitly to language models after an agenda built around retrieving, forecasting and interacting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Introduces unified embeddings for text, images, video, audio and PDFs, with launch limits and links to developer tools.
Explains nested embedding dimensions and retrieval that uses smaller vectors for shortlisting before richer reranking.
Explains GraphCast's global forecasting approach and the historical Hurricane Lee example.
Introduces ensemble forecasts and explains the reported accuracy comparison and per-member computation time.
Describes real-time generated environments and the model's announced resolution, frame rate and consistency horizon.
Further reading
Introduces Weather Lab and experimental cyclone forecasting evaluated on historical storm observations.
Read the complete timestamped transcript
- 0:00
[upbeat music] Our next speaker is VP of Research at Google DeepMind.
- 0:19
Please join me in welcoming to the stage Raia Hadsell. [upbeat music] [clapping]
- 0:34
Hello, everyone. Wonderful. Uh, what a lovely full room and good smiles. I heard the, uh, dig on Google there at the end. [laughing] I, I d- I did catch that.
- 0:44
Um, so my name is Raia Hadsell. Uh, I've been a part of, uh, DeepMind, uh, for the last, uh, almost 13 years, and I'm very happy to have AI engineer come here to London.
- 0:58
I'm also, uh, very proud this year to be, uh, a UK AI ambassador, so I help the government, um, academia, industry sort of bridge those, those gaps. Um, and, uh, yes, I'm [REDACTED:origin] by birth, but I've been here for long enough, um, that I can count myself among the proud [REDACTED:origin] as well.
- 1:20
So I'm going to talk a little bit about frontier AI and the future of intelligence.
- 1:26
Uh, to start, a little bit of a, of a longer, um, introduction to who I am. Uh, it's good to be as old as I am. You get to look at this by the decades.
- 1:35
So in the '90s, I did my undergraduate degree in philosophy of religion. Um, was definitely not a, not a computer scientist yet. Um, uh, but I really enjoyed it, before you ask.
- 1:50
Uh, uh, yes, I learned a lot, and I'm glad that I did it. And no, it hasn't been very useful since. [laughing]
- 1:57
Um, in the 2000s, I did a bit of a pivot, uh, moved into computer science, uh, after some, some good advice from those close to me, and spent my PhD years in New York City working on, uh, convolutional networks, neural networks for robots, uh, with, uh, with Yann LeCun.
- 2:17
Uh, just a lot of fun. Um, I then in the 2010s made the decision to join a small group of, uh, curious, um, scrappy, uh, individuals working at DeepMind.
- 2:32
Um, it was a group of about, um, uh, 30, 40 people at the time. And, uh, we spent the rest of that decade working on things like Atari video games, uh, Go, StarCraft, and some robotics.
- 2:46
Um, a lot of fun. And, um, uh, now I am a VP, uh, within DeepMind. I help run about a group of about 1,200 scientists and engineers, um, across 10 labs.
- 3:03
Um, um, and, uh, we're working on a lot of different things. I'll tell you about three of those.
- 3:11
Um, so first, uh, frontier AI is, uh, an area where, um, we really are trying to make sure that we are staying in the front. So we're thinking about what are the next architectures that we're going to use for Gemini?
- 3:27
What are the next problems that only AI, uh, can really address? Um, and how are we going to build the future of intelligence? And that's thinking not just about artificial intelligence, but it's create the future of human intelligence as well, and even robotics intelligence as well.
- 3:47
Um, we are all on this, uh, journey together, and I think that it's important to think about how, how humans change as well as the technology. Um, our approach, we look for root nodes.
- 3:59
You know, we're not gonna waste time on the leaves. We're gonna really find for a big problem space that hasn't been solved, what are-- how deep can we go?
- 4:08
Find the deepest problems and solve those in order to then enable, um, a lot of downstream stream impact. Um, we partner, you know, really with the world. I really think about it very broadly and think about who are the partners that can help us find those root nodes and solve those problems, and also atta- you know, bring
- 4:29
it to the, to the leaf nodes, and solving problems that are worth solving. Um, the motto or the mission of DeepMind, um, is to build AI responsibly, uh, for the benefit of humanity, so I really take that seriously.
- 4:43
We want to build pro- build-- uh, solve problems that are worth solving. Um, all right. So, uh, we work in a lot of different areas within frontier AI, um, in DeepMind.
- 4:54
These are sort of some of the different, uh, categories. I'm not gonna tell you about all of them, so you can, uh, just, uh, maybe keep those a mystery.
- 5:04
Um, but I'll just pick out a couple. So first, in advanced models, um, I actually wanted to, uh, bring up, uh, an embeddings model. So the theme of this talk overall is things that are not directly language models.
- 5:18
Um, and in the modeling space, I wanted to talk about embedding models. To start that out, I'll ask if anyone knows what a Jennifer Aniston cell is.
- 5:31
Aha, I got a few neuroscience-- neuroscientists in the room. So this is actually a concept from neuroscience, um, where we've discovered that, uh, there are not just a single cell, but a small number of neurons that will encode for a specific thing, as in a specific person, and that those combinations of neurons that only activate for that
- 5:55
one person or that one thing or that one place. Those cells are actually very robust. They, uh, activate regardless of modality. Um, and this is used by the brain very, very, very fast retrieval for recognition, um, and for comparison functions.
- 6:14
Uh, so that means that when I say the name Jennifer Aniston, or if I showed you a picture or a video, or if you even heard her voice, if you knew her, if you were enough of a fan, then those-- all those different modalities lead to the same set of cells activating.
- 6:30
Um, so we want that in a, uh, artificial neural network, um, for the same reason. We want fast retrieval, recognition, and comparison. Um, and so we can trade what's called an embedding model, uh, in order to encode for those concepts, in order to be more robust, um, to different, uh, different ways the information can be presented, um,
- 6:55
and to be very, very good at sort of understanding what is the comparison between these different activations. Use contrastive losses. One of the reasons why I like this space is that because I did my PhD work in part looking at Siamese neural networks, which was an early way of understanding what is a contrastive loss function.
- 7:17
Um, and so these, these embedding, uh, uh, functions are really critical companion to generative AI. Sometimes we want to generate, sometimes we want to retrieve.
- 7:33
Um, so the group at Google has been working on this for a long time, and just recently we've actually released Gemini Embeddings two. So this is exciting to me because it really is sort of the, the ideal.
- 7:44
It is fully omnimodal. Um, it uses, uh, uh, it's derived from Gemini. Um, so it's got sort of that level of, of knowledge and understanding of the world. And it is-- and it allows extremely, extremely good retrieval.
- 8:02
Um, uh, in a little bit more, more detail then, um, why is it good that it is unified and multimodal? It means that you don't have to have different steps to try to bring things together and ma-map them together.
- 8:17
You can be truly end to end and not lose information by trying to combine audio information, visual information, text information together. Um, so you can get a single vector that represents text, uh, uh, up to eight thousand, eight K tokens, um, uh, a hundred and twenty-eight seconds of video, eighty seconds of
- 8:41
audio, um, and a full PDF. And together that can give you a lot of information. You can then use that, um, to be able, uh, to use it for retrieval, um, uh, for, for, for querying, um, for agentic logic and other things.
- 9:00
Um, we also use something called the matri- Matryoshka Representation Learning, MRL, um, and that allows us to have dif-- be able to have the same network but represent different, uh, dimensions.
- 9:13
So for instance, you could start out doing a retrieval, uh, using only, uh, two hundred and fifty-six dimensions for your embedding, and then you can expand that to get to more expressiveness.
- 9:26
Um, so, uh, this also-- this gives us-- we can demonstrate that we have-- this allows us to have a unified, um, semantic space, um, and really state-of-the-art quality. Uh, so, um, uh, just something that's come out recently that I think should-- it doesn't get talked about quite as often as language models, but it's really important as that,
- 9:48
uh, companion, I think. All right. Next, I wanted to quickly talk about another thing that is not a language model. This is not a language model at all. There's no language involved.
- 10:00
Um, and this is work that we've done on the weather. Uh, in London, it rains a lot. Um, and a few years ago, there was a, um, uh, a informatics, uh, scientist at the Met Office, Meteorological Office for the UK, the UK's weather agency, that said, "Can you predict better rainfall than our physics-based models, um, using AI?"
- 10:24
And I said, "I don't know. Interesting problem. Let me take this back to the team." Took this back to the team, um, at DeepMind. We started working on this, and we discovered, yes, you know what?
- 10:35
Predicting the weather, even though it is a very, very hard problem using a physics simulation of the atmosphere, is actually quite tractable for neural network models, given that we have forty years of data, forty years of global data on what the weather is.
- 10:56
Um, so a couple of years ago, we came out with GraphCast. Um, GraphCast, uh, predicts the, uh, predicts the state of the atmosphere up to fifteen days out, um, everywhere on Earth and for many different variables.
- 11:12
And this uses a spherical graph neural network. Uh, can think about this, uh, uh, en-encompassing the Earth and having nodes that go all the way from the surface of the Earth all the way up into the lower stratosphere.
- 11:27
Um, and we actually feed in and then predict in an autoregressively a hundred different atmospheric, uh, variables, for instance, uh, wind speed, temperature, and humidity, as shown here. Um, and this worked very well.
- 11:42
Here's a, a quick example. We were excited to see this, um, in late twenty twenty-four. This is Hurricane Lee, sort of comes into the Atlantic, pauses for a moment, and then takes a, takes a, a turn to the north, speeds up, and makes landfall on Nova Scotia.
- 12:00
Um, the total-- this total video is a- uh, nine days' worth, so that's how far the, the hurricane moves. And this is actually the output of the graph neural network.
- 12:11
That's its prediction. And the prediction that it made is accurate nine days out of where that landfall is going to be.
- 12:20
In comparison, the best gold standard models, um, uh, that are, that are physics-based were only accurate six days out as to where that landfall was going to be. When you're talking about a major hurricane hitting land, three days is really important.
- 12:39
Um, so with this we said, "Okay, this is important, and we're going to keep on pushing the science." So the team developed the next model. We called this GenCast, and the difference here is that this model, while also based on a mesh, um, is probabilistic, and it has a higher accuracy and a higher efficiency.
- 12:58
The weather is fundamentally chaotic, and we wanna know what's happening, um, on the tails, and so having a model that's probabilistic i- allows us to do that, allows this to be operationalized and used for actual weather prediction.
- 13:12
Um, GenCast also was more accurate, so when we compared it to 1,300 gold standard, uh, benchmarked weather forecasts, then this was more accurate 97% of the time, and it was also, uh, could be-- we could produce that 15-day forecast in eight minutes on a single chip instead of hours on a very large supercomputer.
- 13:37
So much different sort of space of the solution that we are proposing. Um, and just this last year, this team is relentless. They're constantly coming up with new models, um, and so the latest one is called FGN, uh, Functional Generative Network.
- 13:53
This directly predicts cyclones. Rather than predicting the weather and then having to add on a cyclone detector as sort of post-processing, this actually incorporates the, the categorization, the recognition of cyclones, their trajectory, their wind speed, and the formation of the eye directly into the network.
- 14:13
We train for that, which means that it's much better. Um, so this has already been used, um, in the US by the National Hurricane Center, um, and they are very, very excited by how much, um, of an advantage this now gives.
- 14:26
Um, so, uh, this, this will, uh, hopefully be used worldwide in, in the coming years.
- 14:33
All right. Lastly, I wanted to use the last few minutes to talk about, again, something that is not s- uh, language model based, so this is world models. Um, and this actually came out of work that DeepMind has done on games and simulation for a long time.
- 14:51
Um, we've been working on Atari, on Go, on StarCraft, um, and then on, you know, MuJoCo-type environments for robotics, because we wanted to understand, um, agency and the environment.
- 15:05
We started focusing more and more on not just the training the agent, but creating the in-- an, an infinite environment.
- 15:14
You know, when we-- when I did work on, uh, locomotion here, um, then... Oh, that's not playing well.
- 15:25
Maybe this will play. I'm gonna jump forward to Genie 1. So we wanted, uh, so this is Genie 1. It could only run for a few seconds, but you could say, "Hey, I want this type of a world," and then you could produce this little platformer 2D game environment where you could jump around for a few minutes,
- 15:43
and it would actually respond to whether you hit the, the left or the right, and it could produce a reasonable diversity of different-looking platformer type of worlds. This was enough to say, "Hmm, we might have something here.
- 15:56
Let's scale up. Let's scale up the data, um, and train again. Improve the method
- 16:03
and train again, now on 3D games." Then we produced Genie 2. Genie 2 is, uh, is interactive, but it's not yet real time, so you need to go awfully slowly.
- 16:16
Um, um, and it can, uh, produce 3D environments, but it couldn't do anything that was really real-world type of quality, uh, and, uh, uh, hi- more higher definition.
- 16:29
Um, so we were working on that, and then along came VO3. [audience cheering] All right.
- 16:38
Now [laughs] All right. Well, now I am out of time, but I will still take another, another minute or two to just show, show you these. Um, so this is saying that-- telling Genie, "I want a world where I'm walking down a muddy lane in Kent."
- 16:58
Um, uh, this looks not far from my house. The fun thing here is that you look down at yourself, and you realize that you actually have a body. You're actually interacting with the world.
- 17:07
It's a little bit odd to, uh, to know what's coming out of this model. It's really understood not just the appearance of a lane in Kent, but actually what it takes to engage with that, make the water move, and to walk forward.
- 17:22
Um, of course, it's not just scenes that are walking. We can very happily, uh, ski, um, uh, and so you can create an environment where you can engage with the world in so many different, different ways.
- 17:39
Um, here's an example where it says Original there. That's-- we started this, we prompted this with a fragment of video, and now it's changed to Genie, Genie 3. So this is an artist.
- 17:52
He made those-- that-- those first few seconds, and then we used that to prompt Genie and bring this world to life. He was so tickled to see that we could take a little snippet of his world that he had laboriously created and bring that to life in a way that means that you can fly through it.
- 18:07
You can bounce off of this thing, and it remembers that, oh, here's that, here's that weird structure there, and go back to that and fly through there. Um, so these environments are not only diverse and interactive.
- 18:21
High quality. They also have, uh, memory. So the prompt here was, "I'm an origami lizard in an origami world," [laughs] and this is what you get. And we use this as a nice little test that I can spend, you know, uh, I can spend a minute running in one direction, run back, uh, to the start, and everything is
- 18:40
exactly as it was at the beginning, um, because we have a really good memory. Working in these environments gives us consistency and control.
- 18:50
Um, and lastly, we have, uh, we're able to prompt this world as you're in it. So that means that, um, while I'm in a world that might be a little bit boring.
- 18:59
Here I am, you know, this is a world saying I'm walking down the [REDACTED:location] in London here,
- 19:07
uh, near the DeepMind office. Well, what happens if I prompt it at the same time? Then what happens? [laughing]
- 19:15
Ah. I've just changed the world that I'm in.
- 19:19
Can change it again. There we go. Immediately the world is, is, i- is different. And one more time just for fun. [laughing]
- 19:33
I love the idea of a new form of gaming where I could be adversarially prompting your experience of a world. It just creates a whole different sort of, um, entertainment, a whole world, a whole new frontier, um, that I think can be really amazing.
- 19:52
Not just for entertainment, but for education as well. Um, the ability to be able to go into a world in order to learn about it, I think is incredibly powerful, um, and may well be something that, that, that we see more and more of.
- 20:06
Um, and, uh, with that, I will, uh, say thank you. And just a quick call-out that tomorrow morning, um, my, uh, colleague Omar is going to talk about Gemma 4, which is a language model. [laughs]
- 20:19
All right. Thank you. [applauding] [upbeat electronic music]