# Speech-to-Speech Model Research at Google DeepMind — Valeria Wu Fon & Tom Ouyang, Google DeepMind

Valeria Wu Fon, Google DeepMind · Tom Ouyang, Google DeepMind

[Talk page](https://ai.engineer/talks/18Um2VjMM_g-speech-speech-model-research-at-google-deepmind) · [Video](https://www.youtube.com/watch?v=18Um2VjMM_g)

## Speech-to-Speech Model Research at Google DeepMind

Valeria Wu Fon and Tom Ouyang explain how joint audio, video, and text training supports versatile voice agents—and why reasoning, response timing, language use, and visual output must work together.

### A transcription model still leaves the conversation to build

[0:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=12s)

Adding voice to an agent can look straightforward: put automatic speech recognition (ASR) before the agent and text-to-speech (TTS) after it. But recognizing words and speaking an answer leave much of a conversation to the surrounding system. [Valeria Wu Fon](https://www.linkedin.com/in/valeriawu/), Gemini’s speech-to-speech product lead, and [Tom Ouyang](https://research.google/people/tomouyang/), a speech-to-speech engineer, introduce their work through that gap. The intended applications span everyday questions in Search Live and Gemini Live, as well as enterprise voice agents exposed through cloud and API products. [1:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=93s)

In Ouyang’s account, speech recognition before roughly 2018 usually required a chain of specialized components: feature extraction, acoustic modeling, pronunciation modeling, language modeling, and a second pass that rescored candidate transcriptions. Moving toward end-to-end neural systems let a model learn the mapping from acoustic input to text, reducing the amount of domain knowledge needed to assemble that chain.

The scope of the learned task remained narrow. Audio went in; a transcription came out. Translation, replies, descriptions of a speaker’s emotion or pace, and the use of images still required additional system work. Collapsing the recognizer’s internal pipeline therefore did not create a general conversational model. Every new capability could bring another integration problem.

### Pre-training teaches relationships between audio, video, and text

[2:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=153s)

**Gemini** changes where those relationships are learned. Rather than merely attaching audio embeddings to a text model, Ouyang describes pre-training on interleaved multimodal examples. A single training example can begin with a text instruction, continue through a sequence of video and audio, and require an output that depends on both. The modalities meet during the stage that supplies the bulk of the training data. [3:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=183s)

Consider the bedtime-story example. The instruction asks for a summary; the following video and audio supply the story. The expected output includes a written summary and timestamps for interesting events. To satisfy that target, the model must turn its understanding of what it hears and sees into text, while retaining where relevant events occurred.

Different training tasks exercise different directions through the same foundation:

- **Video captioning:** Audio and video both inform the caption, so the text can reflect information carried by either signal.
- **Speech recognition and synthesis:** Examples connect audio to text and text to audio.
- **Agentic tasks:** Task-oriented examples join those modality conversions in a unified token embedding space, giving the model a foundation for understanding how audio, video, and text relate.

What does the bedtime-story task require the model to connect? The diagram follows the example from instruction through sensory input to its output target. The summary and event timestamps depend on the audio and video together: training asks the model to express both the story’s meaning and where events happen. This is a picture of the training task, without specifying an encoder design or tokenization scheme.

The bedtime-story training example
A text instruction establishes the task; interleaved audio and video supply the story; the output expresses its meaning and event timing.
- Text instruction — Ask for a bedtime-story summary.
- Interleaved video and audio — Supply the story as a sequence of multimodal inputs.
- Multimodal model — Learn to connect audio and video understanding with text output.
- Written summary — Express the story’s content in text.
- Event timestamps — Annotate where interesting things happen.
- Text instruction → Multimodal model (Defines the requested task)
- Interleaved video and audio → Multimodal model (Supplies audio and video context)
- Multimodal model → Written summary (Expresses understanding in text)
- Multimodal model → Event timestamps (Locates interesting events)

### Translation must begin before the utterance is complete

[4:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=243s)

Live translation puts those learned relationships under a timing constraint. A listener can specify English as the desired output while friends speak Spanish, Italian, or Chinese. The model must translate as they speak. An offline translator has the full utterance available before producing a result; the streaming system begins with incomplete input. [4:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=273s)

Ouyang reports that the team finds streaming translation quality about as good as offline systems, though the talk does not specify the evaluation conditions or numerical results for that comparison. The task combines several demands: switching among languages without knowing them beforehand, preserving the source speaker’s voice, understanding multiple speakers, tolerating noise, and producing output in real time. He attributes much of that capability to pre-training, making the application almost a prompting task.

Translation is one mode of the shared foundation. With an instruction to translate and incoming audio, the model produces streaming translation. With image and audio input and a question about the image, it answers. An embodied agent can add a face and tools that show information. The proposed advantage is reuse: different prompts call on relationships already learned by one model. [5:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=333s)

### More thinking can make a conversation slower

[6:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=370s)

Wu Fon organizes the research around three objectives that must coexist in the same product:

- **Conversational:** Low latency, quick responses, and a natural conversational rhythm.
- **Intelligent:** Task completion, instruction following, and reasoning that help the model accomplish something useful.
- **Multimodal:** Inputs can include video, screen sharing, and PDFs alongside audio, while outputs can go beyond speech.

[6:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=370s)

Internationalization runs through all three. Wu Fon says she believes the majority of Gemini users are non-English speakers, so the goal includes making these capabilities work across the languages customers use. A system that speaks fluently in one locale still has work to do before it can serve the intended audience.

The objectives interfere with one another. Increasing the thinking budget gives the model more time to reason before answering or calling a tool, and Wu Fon reports that this improves intelligence evaluations. But that work delays the first audio response. The user experiences the delay as a pause in the conversation, which can undermine its natural rhythm. The research challenge is to improve useful reasoning while keeping the exchange responsive; the talk presents this as ongoing work. [7:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=459s)

### From a multilingual meeting to a sofa question

[8:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=519s)

The first demonstration returns to translation. The live model is described as supporting 70-plus languages, and a Google Meet example shows the interaction: enable speech translation, select the desired language, and let several participants converse. The exchange moves through weather in Shanghai, a family visit to a park, and a birthday dinner at a restaurant in Sweden. [8:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=519s)

The timing matters as much as the translated content. Wu Fon highlights that translation begins shortly after a participant starts speaking, reducing the feeling of a rigid, turn-by-turn relay. The model starts catching the new speech directly, so participants do not have to treat every interruption as a disruption to a separate translation cycle.

The next example moves into Search Live, which Wu Fon says uses the same speech-to-speech model as the developer-facing Live API. In the sofa demonstration, real-time video and audio let the user ask a question without first describing the furniture. A tool call then brings up relevant search cards, adding a visual route to more information alongside the spoken exchange. [11:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=672s)

Localization also changes the answer itself. Wu Fon describes the response as Spanish localized for Spain, with the furniture term mid-century left in English because that term is commonly used in Spanish. The useful behavior is selective: preserve a familiar borrowed expression while speaking the surrounding answer in the user’s language. Translating every word would miss that usage.

### A roadside agent needs accurate identifiers and selective attention

[12:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=734s)

The roadside-assistance demonstration gives the same model a different job. A driver has blown a tire and pulled over. The agent begins by asking for a name and policy number. When the driver cannot supply the policy number, the agent changes the information it requests: a registration plate and postcode can support the lookup instead. That change keeps the task moving despite a missing identifier. [12:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=734s)

A truck horn interrupts the situation, and the driver reacts before providing the requested details. The agent subsequently says it has found the record, warns the driver to stay clear of the road, and asks whether the vehicle is a blue MINI Cooper F series. The driver confirms. The observable progression is from an unavailable policy number to alternate identifiers, then a retrieved vehicle description and confirmation. The demonstration stops there; it does not show assistance being dispatched.

This task makes two requirements especially visible:

- **Alphanumeric accuracy:** Plates, postcodes, and addresses carry task-critical characters. A plausible-sounding conversation is insufficient if the identifiers used to locate a record are wrong.
- **Proactive audio:** The model decides when external input warrants a response or interruption. Background noise or another person talking should not automatically make it stop speaking or cut itself short.

[13:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=801s)

Here, proactive includes knowing when to leave the conversation alone. The model must distinguish relevant input from sound that happens around the user. Wu Fon connects that requirement to the places voice agents will actually be used: on trains, during walks, and on the go. Quiet-office assumptions would make the model’s conversational timing poorly suited to those environments.

### Visual presence adds another output to coordinate

[13:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=831s)

The final demonstration adds a visual presence. Wu Fon introduces a pilot with Citi at Cloud Next that supports customized real-time avatars, ranging from hyperrealistic humans to cartoons. The experience uses the same speech-to-speech model and combines low-latency conversation with multilingual lip syncing. An avatar gives the spoken interaction another output whose timing must remain coordinated with the exchange. [14:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=861s)

In the example, a user asks about progress toward a daughter’s college fund. The agent replies about the goal, mentions a possible opportunity, and then acknowledges that another person has joined, offering congratulations on college acceptance. Wu Fon uses the demo to bring together multimodal input and output, tool calling for relevant user information, conversational fluidity, and internationalization. It is presented as an early convergence of those capabilities in a pilot.

The closing ambition is broader than giving an assistant a good voice. Wu Fon’s forecast is that AGI will be spoken. For the team, that means one promptable model must let a user move among translation, taking action, brainstorming, and rambling. Each mode changes what a useful response looks like: translation follows someone else’s speech, an agent advances a task, and an open-ended exchange follows the user’s thought. The intended product must switch among those modes while keeping its understanding, timing, language use, and outputs working together. [15:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=956s)

## Key takeaways

- End-to-end speech recognition simplified audio-to-text modeling, but translation, responses, speaker interpretation, and visual context still required additional system work.
- Interleaved audio, video, and text pre-training teaches cross-modal relationships that a shared model can draw on for translation, visual questions, and speech generation.
- Reasoning quality and conversational speed can conflict: more thinking before an answer or tool call delays the first audio response.
- Useful voice behavior includes selective localization and selective attention: keep familiar borrowed terms when appropriate, and avoid treating every background sound as a conversational interruption.
- The longer-term goal is a single promptable model that can switch among translation, task execution, brainstorming, and informal conversation while coordinating multimodal input and output.

## Resources

- [Tom Ouyang’s research and publications](https://research.google/people/tomouyang/) — A route into Ouyang’s earlier work on language modeling and multilingual input, including keyboard decoding under latency and memory constraints. These publications provide background on input-system tradeoffs rather than specifications for the speech model.

## Related talks

- [Milliseconds to Magic: Real‑Time Workflows using the Gemini Live API and Pipecat](https://ai.engineer/talks/fWY1FQwpWkY-milliseconds-magic-real-time-workflows-using) — A practical companion topic for moving from the shared speech model to real-time application workflows.
- [Engineering voice agents: Latency, quality, and scale](https://ai.engineer/talks/N7b1PJc7SFc-engineering-voice-agents-latency-quality-scale) — Continues the engineering questions raised by the tradeoff between useful answers and conversational latency.
- [Realtime Conversational Video with Pipecat and Tavus — Chad Bailey and Brian Johnson, Daily & Tavus](https://ai.engineer/talks/ujt0da9Z29Q-realtime-conversational-video-pipecat-tavus) — An adjacent topic for adding visual presence to spoken interaction.

## Slides

### Streaming speech translation

Google DeepMind’s live model supports streaming translation in 70+ languages. A Google Meet video introduces how people speaking different languages can hold a live conversation.

[8:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=515s) [Slide says the live model supports streaming translation in more than 70 languages; the video area is blank.](https://assets.ai.engineer/sites/aiecode2025/sha256/96077ed80bf0d8a97ce107a48660f79e5b6ebb35b72313abe8c5ea59b371b6ae.webp)

Our live model powers streaming translation that supports 70+ languages

Confidential - Google DeepMind
Proprietary & Confidential

Interface text:
Play

### Turn on speech translation

The Google Meet demo starts by selecting Translate speech; the speaker says users then choose a language for real-time translation during a conversation.

[9:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=545s) [A meeting toolbar highlights the Translate speech control.](https://assets.ai.engineer/sites/aiecode2025/sha256/0391e557228cede13fabb2d2edd57d27b734e6bd7f97828d1881abd4f16e2207.webp)

Sequences shortened. Results may vary. Check responses for accuracy. Availability may vary by country.

Interface text:
Captions
Translate speech

### Choose a translation language

The demo shows English selected as the language to hear. The speaker says translation then happens in real time as multiple speakers converse.

[9:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=550s) [A language selection screen shows English as the language for hearing translation.](https://assets.ai.engineer/sites/aiecode2025/sha256/d6b668e1e2918ec84e4159d2b5ff8a04056e48b29c8258306244f3a7d56a5463.webp)

What language do you want to hear translation in?

English

Interface text:
Back
Next
Sequences shortened. Results may vary. Check responses for accuracy. Availability may vary by country.

### Enable translation for others

The demo offers Translate me to opt into translation for this and future meetings. Meet says translations use a voice like the speaker’s and no voice data is saved; the feature is experimental.

[9:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=555s) [A meeting screen explains speech translation and offers a Translate me button.](https://assets.ai.engineer/sites/aiecode2025/sha256/e3dc33dfa14c95f2c1da40ebde50fddededf90ec66af41a0b831a00b43dcbb4e.webp)

Let Gemini translate what you say for others

Meet can translate what you say, in a voice like yours, for people who speak a different language. This is an experimental feature, so translations may not always be correct. By selecting Translate me, you agree to be translated for this and future meetings. You can change your preferences later in General settings.

Google uses your voice only to produce the translation. No voice data is saved. Learn more about speech translation and our Meet Terms

Interface text:
Not now
Translate me

Sequences shortened. Results may vary. Check responses for accuracy. Availability may vary by country.

### Speech translation in a call

In a three-person call, the user's tile shows “Translating for others” as the participants converse. The presenter describes the demo as real-time translation in a multi-speaker setting.

[9:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=577s) [Three participants appear in a video call with a Translating for others indicator.](https://assets.ai.engineer/sites/aiecode2025/sha256/f8f6bd613868f7da938c94f7bb23dcd730d4605400387f5592bc2a6f63dcc750.webp)

Interface text:
Cassie
Anna
You
Translating for others
Video call
3
Sequences shortened. Results may vary. Check responses for accuracy. Availability may vary by country.

### One speech-to-speech model for conversational frontiers

The same speech-to-speech model powers everyday conversations in Search Live and enterprise voice agents through the developer API. The slide presents a video for each use case.

[10:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=621s) [Slide comparing Search Live for everyday help with developer API voice agents, with two video placeholders.](https://assets.ai.engineer/sites/aiecode2025/sha256/fa384d85b5ded29595b58bb35046a687270aeb4f38a64ef61ad09fb51d4c0a51.webp)

One speech-to-speech model for every conversational frontier

For everyday help, in every language
in Search Live

For enterprise grade voice agents on
our developer API

### Search Live chair demo

A user asks about a chair in Spanish while streaming video and audio. Search Live displays a search card for more information; the speaker says the response is localized to Spanish as used in Spain.

[11:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=674s) [Phone shows a chair and search cards beside Spanish text asking about the chair and offering further information.](https://assets.ai.engineer/sites/aiecode2025/sha256/17aa6609b57f3f1b226d693c9e5d7cabd27246b076ac3c8a8d2fbdf5d1ad44c0.webp)

¿Podrías decirme qué estilo de sillón es este?

¿Hay algo más que te gustaría saber sobre él?

Interface text:
9:30
3 sitios
Sillas de estilo Boucle para tu hogar
Sofás y Sillones
Silenciar
Video
Transcripción

### Speech-to-speech applications after the Search Live demo

One speech-to-speech model supports Search Live assistance, illustrated by a Spanish chair query, and enterprise voice agents through the developer API. The enterprise demo is queued in a video player.

[11:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=716s) [Applications slide with the chair demo visible on the left and a video player on the right.](https://assets.ai.engineer/sites/aiecode2025/sha256/1aff6bcf5ef6ce96ad9a05f9ad602628ef3815afda30372592e8a7adab9300fd.webp)

One speech-to-speech model for every conversational frontier

¿Podrías decirme qué estilo de sillón es este?

¿Hay algo más que te gustaría saber sobre él?

For everyday help, in every language in Search Live

For enterprise grade voice agents on our developer API

Interface text:
G
[illegible]
0:00 / 2:20
9

### From voice to multimodal output

A Citi Sky pilot preview illustrates a visual avatar. The slide describes styles from hyper-realistic humans to cartoons, with natural movement and low-latency multilingual lip-syncing powered by the same S2S model.

[12:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=722s) [Slide describing customizable avatars and lifelike movement alongside a Citi Sky video preview.](https://assets.ai.engineer/sites/aiecode2025/sha256/08967074757dbc7be9db503a2ae9f3e24c43081ceb3e71708a9aea3a76b76e91.webp)

From voice out to multimodal out: True conversational AGI requires more than voice, it demands a visual presence

Customizable Style supports personalized avatars ranging from hyper-realistic humans to cartoon appearances

Lifelike Realism with natural movement, and low-latency, multilingual lip-syncing

All powered by the same S2S model

Citi SKY℠
Every Question. Every Moment.
ASK MORE

Interface text:
Confidential - Google DeepMind
Introducing Citi Sky℠
Citi
Watch on YouTube
& Confidential

### Roadside assistance voice agent

Ace responds to a driver with a blown tyre by asking for a name and policy number. When the driver cannot provide the number, Ace offers to look them up using their registration plate and postcode.

[12:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=741s) [A roadside-assistance chat begins with a driver requesting help after a flat tire.](https://assets.ai.engineer/sites/aiecode2025/sha256/75846a77f422cadee75eed2bc1e5e093aad0dcc0c437ef8780f20f64f1b98ca9.webp)

Ace

Hi I've blown a tyre and had to pull over to the side of the road Can you help

Hello, you've reached Auto Transform Insurance Roadside Assistance. My name is Ace. I understand this is stressful, but we'll take care of everything. To get started, could I please have your name and your policy number?

Illustrative demonstration built with Gemini 3.1 Pro, powered by Gemini 3.1 Flash Live.

Interface text:
Ask something...

### Citi Sky avatar demo

In the Citi Sky demo, a user asks about his daughter’s college fund. The avatar replies that he is tracking well and mentions a new opportunity.

[14:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=888s) [Citi Sky branding surrounds a phone displaying a conversational avatar.](https://assets.ai.engineer/sites/aiecode2025/sha256/91df79a9c8b586ad620417a3ce2109c311d199703c8613387f2e67735c22eba5.webp)

citi SKY℠

Every Question. Every Moment.
ASK MORE

Interface text:
Introducing Citi Sky℠
Citi
www.youtube.com – To exit full screen, press esc
Chat
Avatar
1:14 / 2:31
More videos
YouTube

### Citi Sky avatar

In the demo, the Citi Sky avatar responds to a question about a daughter's college fund, saying the savings are on track and identifying an opportunity to reach the goal sooner.

[14:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=892s) [A phone displays the Citi Sky avatar beside the phrase “Every Question. Every Moment. Ask More.”](https://assets.ai.engineer/sites/aiecode2025/sha256/1b95b86799695de5b1d96c59a0d6bff119ccf6db58ca8a0901b26d5a1334064a.webp)

citi SKY℠

Every Question. Every Moment.
ASK MORE

Interface text:
Chat
Avatar
Ask me anything
Beta Demo

### Citi Sky participant view

The avatar greets Lisa as she joins Jackson and congratulates her on her college acceptance. A thumbnail shows two participants.

[15:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=901s) [The avatar appears above a thumbnail showing two participants.](https://assets.ai.engineer/sites/aiecode2025/sha256/865428291b43dacc9714122d9dfb741b6b9134095dd61c1c168e2d58eb96c6d9.webp)

citi SKY ℠

Every Question. Every Moment.
ASK MORE

Interface text:
Chat
Avatar
Ask me anything
Beta Demo

### Citi Sky Spanish-language response

In the Citi Sky demo, the avatar responds in Spanish that technology and communications services have performed strongly so far this year.

[15:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=924s) [The avatar appears with a Spanish-language response displayed on the phone.](https://assets.ai.engineer/sites/aiecode2025/sha256/ff09bb0e2b1bf280a349104cca27bf6d3e1ae94a1a698d7423b99c796f32b066.webp)

citi SKY ℠

Every Question. Every Moment.
ASK MORE

Chat
Avatar

Sí, los servicios de tecnología y comunicaciones han mostrado un desempeño sólido en lo que va del año.

[illegible]

Interface text:
Introducing Citi Sky℠
Citi
1:47 / 2:31
More videos
YouTube

### AGI will be spoken

The speakers envision a single promptable speech-to-speech model that lets users switch among translation, taking action, brainstorming and other conversation modes.

[15:50](https://www.youtube.com/watch?v=18Um2VjMM_g&t=950s) [A black slide states, “AGI will not be typed, it will be spoken.”](https://assets.ai.engineer/sites/aiecode2025/sha256/ba0fbe9d5b604b3d4157c354d1977df73cf0ef403a4d187206d0e535fa68724b.webp)

AGI will not be typed,
it will be spoken

Confidential - Google DeepMind
Proprietary & Confidential

## Full transcript

[0:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=1s) [music]

[0:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=12s) >> Hi everyone. Thanks for coming to our

[0:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=14s) talk. We're going to talk a lot about

[0:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=16s) how we are planning on voiceifying the

[0:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=18s) agentic future with speech-to-speech

[0:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=20s) research here at Google DeepMind. My

[0:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=22s) name is Valeria. I'm the product lead

[0:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=24s) for the speech-to-speech model in Gemini

[0:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=25s) and Tom.

[0:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=27s) >> Yeah, my name is Tom. I work on

[0:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=28s) speech-to-speech as an engineer and

[0:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=30s) Gemini. So, yeah, great to be presenting

[0:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=32s) our work.

[0:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=33s) >> Cool.

[0:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=35s) So, before we start diving into the

[0:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=37s) history of Google research and audio and

[0:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=39s) the latest research we've been doing, I

[0:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=41s) just want to take a step back and really

[0:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=43s) kind of like think about why this is

[0:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=45s) important to us in the team. And the

[0:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=47s) main reason is that voice is the most

[0:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=49s) natural way for humans to interact with

[0:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=51s) both the physical and the virtual world.

[0:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=53s) And we're already seeing so many

[0:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=55s) applications that are starting even

[0:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=56s) within the own Google products. We see

[0:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=58s) voice being used to ask questions and

[1:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=61s) you know, do homework or info seeking or

[1:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=63s) EDU in search live and Gemini live. All

[1:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=65s) the way to these same models being

[1:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=67s) deployed in cloud and the API for

[1:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=69s) enterprise or voice agent use cases.

[1:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=71s) Should I use this as a

[1:13](https://www.youtube.com/watch?v=18Um2VjMM_g&t=73s) microphone?

[1:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=75s) Hello. Okay.

[1:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=76s) Um

[1:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=78s) So, because of that, we only think that

[1:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=79s) the number of applications is going to,

[1:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=81s) you know, exponentially increase over

[1:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=83s) the next few years and we believe that

[1:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=85s) speech-to-speech models are the way to

[1:26](https://www.youtube.com/watch?v=18Um2VjMM_g&t=86s) go when we want to build robust

[1:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=88s) universal voice agents. And with that,

[1:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=90s) I'll hand it off to Tom.

[1:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=93s) >> All right. So, of course, like one way

[1:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=94s) to create a voice agent is to just add

[1:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=97s) an ASR like speech-to-text model in the

[1:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=99s) pipeline and then a text-to-speech model

[1:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=100s) on the other end, right? So, of course,

[1:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=102s) that's where a lot of the history of

[1:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=103s) speech come from. Of course, Google has

[1:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=105s) been working on this for a long time.

[1:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=106s) I'm going to give you a little bit of a

[1:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=107s) historical overview of what speech

[1:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=109s) modeling, especially speech-to-text

[1:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=111s) automatic

[1:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=112s) speech recognition, typically looks

[1:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=114s) like, right? So, up until around 2018,

[1:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=116s) this usually involved a lot of different

[1:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=117s) components. You have um you know of

[1:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=119s) course feature extraction that's fairly

[2:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=121s) general and then you have all these

[2:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=122s) different pieces like acoustic modeling,

[2:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=124s) pronunciation modeling, language

[2:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=126s) modeling, a second pass rescoring that

[2:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=128s) allows you to go from the audio input to

[2:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=130s) a text transcription. Right? And of

[2:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=132s) course like around 2018, these moved

[2:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=134s) more and more towards end-to-end

[2:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=135s) systems. You don't have to have so much

[2:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=136s) domain knowledge. You can actually have

[2:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=138s) mostly the the neural model learn this

[2:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=141s) pattern and mapping between acoustic

[2:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=143s) inputs and text. But these aren't really

[2:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=145s) the end-to-end models that we think

[2:26](https://www.youtube.com/watch?v=18Um2VjMM_g&t=146s) about when we think about LLMs. They're

[2:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=147s) only doing kind of one thing, which is

[2:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=149s) speech to a transcription of that

[2:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=151s) speech. They're not responding, they're

[2:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=152s) not translating. If you wanted to have

[2:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=154s) the model tell you about the tone or the

[2:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=156s) emotion or the speed of the speaker. If

[2:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=158s) you wanted to bias it towards words or

[2:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=160s) much less like images, those are all

[2:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=161s) things you have to build yourself as

[2:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=163s) part of the system and there's really

[2:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=165s) really a barrier to how easily you can

[2:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=166s) scale these systems. So,

[2:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=169s) fast forward to now, which is kind of of

[2:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=171s) course the LLM era, right? Like you

[2:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=172s) know, the first LLMs were mostly text,

[2:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=174s) but even there I think you could kind of

[2:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=175s) hack audio embeddings into these text

[2:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=177s) models and it kind of worked, but now of

[2:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=179s) course for a long time now, Gemini

[3:00](https://www.youtube.com/watch?v=18Um2VjMM_g&t=180s) models have been very natively

[3:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=182s) multimodal. So, what does that mean? It

[3:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=184s) means that when we train these models in

[3:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=185s) pre-training, which is where the bulk of

[3:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=186s) the data comes from, these are

[3:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=188s) multimodal interleaved examples, right?

[3:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=190s) So, the bottom uh diagram here gives you

[3:13](https://www.youtube.com/watch?v=18Um2VjMM_g&t=193s) kind of one example of what that might

[3:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=194s) look like. So, this is a task where

[3:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=196s) you're asking the model to summarize a

[3:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=198s) bedtime story and there's a text prompt

[3:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=199s) in the beginning, but then there's this

[3:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=201s) sequence of video and audio inputs that

[3:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=204s) the model gets

[3:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=205s) and then of course like what you expect

[3:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=207s) the model to do here is produce um both

[3:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=209s) the summary and also annotate timestamps

[3:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=211s) for where interesting things happen and

[3:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=213s) so forth. So, this example is teaching

[3:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=215s) the model to translate its understanding

[3:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=217s) of the audio and the video into text.

[3:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=219s) You might have other examples in

[3:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=221s) pre-training that ask you to caption a

[3:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=223s) video. So, you might have a video that

[3:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=224s) has audio and the model is learning to

[3:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=227s) bias towards both the video and the

[3:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=228s) audio signal to caption this well. And

[3:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=231s) of course like there's limitless like

[3:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=233s) YouTube videos with captions that you

[3:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=234s) can train these models on. Other models

[3:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=235s) might actually try to generate audio

[3:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=237s) from the video from the text, right? So,

[3:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=239s) you can have ASR, TTS, or any

[4:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=241s) combination of these plus all of these

[4:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=242s) sort of agentic tasks all kind of

[4:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=245s) learned under one unified token

[4:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=247s) embedding space. So, this becomes a

[4:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=249s) foundation for a lot of what we want to

[4:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=250s) do in audio because we already have a

[4:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=251s) model that understands audio, video,

[4:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=254s) text, and how these things relate and

[4:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=256s) transition from one to the next.

[4:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=259s) So, very quickly, like one of the

[4:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=261s) applications that this enables that

[4:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=262s) we've launched recently is live

[4:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=264s) translation, right? And this is a kind

[4:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=265s) of application that kind of only works

[4:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=267s) when you have all of these capabilities

[4:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=269s) working within the same model. You have

[4:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=270s) basically state-of-the-art translation

[4:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=272s) quality. Even though this model is

[4:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=273s) translating basically as the user or

[4:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=275s) speakers are speaking, you kind of ask,

[4:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=277s) "Hey, I I speak English. There's maybe

[4:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=279s) friends who are talking in Spanish and

[4:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=281s) Italian and Chinese." And it's

[4:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=282s) translating all of them to your language

[4:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=283s) as they talk. Um and then we're we're

[4:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=285s) finding is the translation quality for

[4:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=286s) this like streaming real-time

[4:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=288s) translation is about as good as you

[4:50](https://www.youtube.com/watch?v=18Um2VjMM_g&t=290s) would get with offline systems, right?

[4:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=292s) Where you kind of know the full

[4:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=293s) utterance

[4:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=294s) um from the very beginning. So, that's

[4:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=296s) something that has been classically very

[4:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=298s) hard to do with these cascaded systems,

[5:00](https://www.youtube.com/watch?v=18Um2VjMM_g&t=300s) but with LLMs, it actually just

[5:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=303s) a lot of it comes out of the

[5:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=304s) pre-training. So, of course, to do this

[5:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=306s) task it needs to do multilingual

[5:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=307s) switching because you could be

[5:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=308s) translating across different languages.

[5:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=309s) You don't know what those languages are

[5:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=310s) beforehand. It needs to preserve the

[5:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=312s) speaker voice of the source speaker and

[5:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=315s) be able to understand multiple speakers,

[5:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=317s) be robust to noise, and of course, like

[5:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=319s) do all this in real time, right? So,

[5:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=321s) again, it would be very hard to try to

[5:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=323s) engineer this, but then with the LLM and

[5:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=325s) Gemini models, this almost becomes a

[5:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=328s) prompting task.

[5:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=329s) And on that, like, I know, in this

[5:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=331s) diagram we're saying, "Hey, at the top

[5:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=332s) with these models, if you prompt it to

[5:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=334s) do the speaking like this streaming

[5:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=336s) translation task,

[5:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=338s) and you give it the audio, it will

[5:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=339s) produce the streaming translation

[5:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=340s) output, right? The same model, if you

[5:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=342s) ask it to act like an agent and respond

[5:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=344s) to maybe image and audio input, maybe

[5:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=347s) asking questions about that image, it

[5:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=348s) will give you an answer. And finally,

[5:50](https://www.youtube.com/watch?v=18Um2VjMM_g&t=350s) like very well we'll show examples of

[5:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=351s) this, you can also have it create this

[5:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=353s) embodied, you know, virtual agent that

[5:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=355s) has a face, that has, you know, things

[5:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=357s) and tools that it can show you, and it

[5:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=359s) will produce this sort of embodied agent

[6:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=361s) experience. So, with that, I'm going to

[6:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=363s) give it to Valeria to talk more about

[6:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=364s) the North Star and some of the key demo

[6:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=367s) products that we built.

[6:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=370s) >> Yeah, so to create this type of kind of

[6:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=372s) universal, versatile, uh kind of

[6:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=374s) model/product,

[6:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=376s) there's like three vectors that we think

[6:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=377s) about when we do research and product

[6:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=379s) for these models. Um and they also come

[6:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=381s) with some challenges, so I'll like walk

[6:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=383s) you through some of them. So, at the

[6:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=385s) core of a speech-to-speech model, the

[6:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=387s) first thing that people usually think

[6:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=388s) about is that it has very

[6:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=389s) conversational, right? It's low latency,

[6:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=391s) it's very conversational, very snappy,

[6:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=393s) very natural. But, I think within our

[6:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=395s) team, we really don't only want this

[6:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=396s) model to sound nice. We also have two

[6:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=399s) pillars at the top that we also really

[6:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=400s) care about like pulling all together

[6:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=402s) into one model, which is intelligence

[6:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=404s) and it being multimodal. So, when we

[6:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=406s) talk about intelligence, we talk about,

[6:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=408s) you know, task completion, instruction

[6:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=409s) following, reasoning, like capabilities

[6:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=411s) that the model needs to have natively in

[6:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=414s) order to complete tasks and to like do

[6:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=416s) things uh that have high customer

[6:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=418s) satisfaction, for example. And on the

[7:00](https://www.youtube.com/watch?v=18Um2VjMM_g&t=420s) other side of the Venn diagram, we also

[7:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=422s) have the idea that these models should

[7:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=424s) be very multimodal, both in audio in and

[7:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=426s) audio out, right? So, sometimes a user

[7:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=428s) doesn't only want to input audio in and

[7:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=431s) have that be the start of the

[7:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=432s) conversation. We need video, your screen

[7:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=434s) sharing, PDFs, whatever you would want

[7:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=436s) the model to interpret and understand,

[7:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=438s) we should be able to stream it in and

[7:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=440s) also produce output out of it. So,

[7:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=442s) that's kind of like the trifecta of

[7:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=443s) which we think about speech-to-speech

[7:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=445s) models. Um and I want to add a caveat

[7:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=447s) about ITNN. I think actually the

[7:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=449s) majority of our Gemini users are

[7:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=450s) non-English speakers. Um so, we put a

[7:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=452s) big focus on having and making sure that

[7:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=454s) all these capabilities work within not

[7:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=457s) only, you know, EN-US, but all the

[7:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=459s) languages that our customers care about.

[7:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=461s) Um

[7:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=461s) of course, this also our North Star also

[7:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=464s) becomes one of the biggest challenges in

[7:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=465s) our research because

[7:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=467s) once you move one of the knobs, it's

[7:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=469s) very easy for the other knobs to kind of

[7:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=471s) like mess up, right? Like a very quick

[7:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=473s) example, oh, how do we increase

[7:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=475s) intelligence in the model? Well, you can

[7:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=477s) turn thinking high or like the thinking

[7:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=479s) is high as possible to have the model

[8:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=481s) think a lot before calling a tool or

[8:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=483s) answering a question, which in eval's it

[8:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=485s) does show that it does improve the

[8:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=487s) model's intelligence. But when what does

[8:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=489s) that do to latency, right? And time to

[8:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=491s) first audio and the naturalness of the

[8:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=492s) conversation? So, within the Gemini

[8:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=495s) team, we're really trying to push

[8:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=496s) forward research initiatives that can

[8:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=498s) kind of blend in the three of them

[8:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=500s) without really sacrificing any of those

[8:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=502s) by a lot. Um,

[8:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=504s) but in the meantime, we're going to show

[8:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=505s) you some of the demos that we think are

[8:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=507s) hinting at how our speech-to-speech

[8:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=509s) model can combine all these three into

[8:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=512s) really cool application. So,

[8:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=514s) the first one is you kind of saw this as

[8:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=516s) a preview, but our live model, as you

[8:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=518s) know, uh, powers streaming translation

[8:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=520s) that supports 70+ languages. So, this is

[8:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=522s) a little bit around the core of

[8:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=524s) conversation quality and IT&N efforts

[8:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=526s) that we have in the team. I'll play a

[8:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=527s) quick video on how this works on Google

[8:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=529s) Meets to help two people or maybe

[8:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=531s) multiple people that are speaking

[8:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=533s) different languages still have a live

[8:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=534s) conversation.

[8:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=537s) Oh.

[8:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=538s) One sec. Okay.

[9:00](https://www.youtube.com/watch?v=18Um2VjMM_g&t=540s) >> Okay, let's turn on speech translation.

[9:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=544s) >> So, here the user can simply select the

[9:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=546s) language that they want the translation

[9:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=548s) to happen in and then the rest will be

[9:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=550s) done in real time in multiple speakers

[9:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=552s) kind of having a conversation back and

[9:13](https://www.youtube.com/watch?v=18Um2VjMM_g&t=553s) forth. So, I'll show you a snippet of

[9:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=555s) this video of what happens.

[9:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=560s) >> It's great to see you both. Cassie,

[9:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=562s) how's the weather in Shanghai?

[9:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=565s) >> It's nice to meet you. The weather here

[9:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=567s) is really nice, sunny and bright. I

[9:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=569s) spent all day Saturday in the park with

[9:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=571s) my family.

[9:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=575s) >> That sounds wonderful.

[9:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=577s) Anna, you mentioned last week that you

[9:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=579s) were celebrating your birthday. How was

[9:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=580s) it?

[9:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=581s) >> Oh, that was fantastic. I had dinner at

[9:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=585s) my favorite restaurant with some

[9:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=586s) friends. If you visit Sweden, you must

[9:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=589s) try this restaurant. It was

[9:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=592s) >> So, um a lot of things happening. Not

[9:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=594s) only there's, you know, real-time

[9:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=596s) translation. Whoops.

[9:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=598s) How do I

[9:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=599s) back to Okay.

[10:00](https://www.youtube.com/watch?v=18Um2VjMM_g&t=600s) Um not only there's real-time

[10:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=602s) translation that is happening, uh but we

[10:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=604s) see that it's in a multi-speaker

[10:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=605s) setting, you know, low latency, like

[10:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=607s) right after the user starts speaking,

[10:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=609s) the translation kicks off, so that it

[10:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=610s) doesn't feel like it's really turn by

[10:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=612s) turn and robotic and you're afraid to

[10:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=614s) interrupt because, you know, once you

[10:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=616s) start speaking, the model will start

[10:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=617s) catching your translation directly. So,

[10:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=619s) now I'll show some other applications

[10:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=621s) where we see that we have one single

[10:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=623s) speech-to-speech model for many

[10:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=624s) conversational frontiers. So, two

[10:26](https://www.youtube.com/watch?v=18Um2VjMM_g&t=626s) products that I want to highlight here

[10:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=628s) are the same model that we power search

[10:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=630s) live for everyday conversations in any

[10:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=632s) language. It's the same model that we

[10:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=634s) use to power developer experiences in

[10:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=636s) our live API. As you'll see in the

[10:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=638s) videos, both use cases are extremely

[10:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=640s) different, but because we try and make

[10:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=642s) this this model or this speech-to-speech

[10:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=644s) model very promptable and universal, you

[10:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=647s) can see how it adapts to the different

[10:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=648s) use cases. So, let me just play the

[10:50](https://www.youtube.com/watch?v=18Um2VjMM_g&t=650s) first video.

[11:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=672s) So, to highlight in this video, many

[11:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=674s) things are happening. First, the

[11:15](https://www.youtube.com/watch?v=18Um2VjMM_g&t=675s) multimodal, you know, it's real-time uh

[11:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=677s) video and audio streaming in, so that

[11:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=679s) you don't have to actually describe what

[11:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=680s) the sofa looks like. But then the

[11:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=682s) multimodal out, you see that there was a

[11:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=684s) tool called made to pull up the right

[11:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=685s) search cards if the user wanted to know

[11:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=687s) more information. And then finally, the

[11:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=689s) ITN component, you know, we're

[11:31](https://www.youtube.com/watch?v=18Um2VjMM_g&t=691s) localizing the speaker's Spanish to

[11:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=692s) Spain Spanish. And also, when we do the

[11:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=696s) kind of responding in Spanish, we're

[11:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=697s) also smart enough to know that some

[11:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=699s) words are better to be left in English,

[11:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=701s) like the mid-century type of word. It

[11:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=703s) didn't translate it directly to Spanish

[11:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=705s) cuz it knew that the term was commonly

[11:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=706s) known in English. Sorry, in Spanish. So,

[11:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=709s) this is one example of how users are

[11:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=711s) using search live for like info-seeking

[11:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=713s) questions. Now, let me show a very

[11:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=715s) different demo on how we do this for

[11:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=717s) conversational voice agents.

[12:02](https://www.youtube.com/watch?v=18Um2VjMM_g&t=722s) Oh.

[12:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=724s) Sorry.

[12:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=734s) >> Hi, I've blown a tire and had to pull

[12:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=737s) over to the side of the road. Can you

[12:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=738s) help?

[12:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=740s) >> Hello, you've reached Auto Transform

[12:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=743s) Insurance Roadside Assistance. My name

[12:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=745s) is Ace. I understand this is stressful,

[12:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=747s) but we'll take care of everything. To

[12:29](https://www.youtube.com/watch?v=18Um2VjMM_g&t=749s) get started, could I please have your

[12:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=750s) name and your policy number?

[12:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=752s) >> Uh my name is Richard Jenkins. I I don't

[12:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=754s) actually have my policy number to hand.

[12:39](https://www.youtube.com/watch?v=18Um2VjMM_g&t=759s) >> No worries, Richard. I can look you up

[12:41](https://www.youtube.com/watch?v=18Um2VjMM_g&t=761s) with your registration plate and

[12:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=762s) postcode instead. Could you provide

[12:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=764s) those for me, please?

[12:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=766s) >> Whoa, that was a bit close. Um yeah, my

[12:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=769s) registration plate is BD21

[12:54](https://www.youtube.com/watch?v=18Um2VjMM_g&t=774s) XYA

[12:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=776s) and uh my postcode is SN48ZX.

[13:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=783s) >> Policy details BD21

[13:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=786s) XYA. Thank you. For your safety, please

[13:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=789s) stay clear of the road. I've found your

[13:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=791s) details and I see you're in a blue Mini

[13:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=794s) Cooper F-Series. Is that the vehicle

[13:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=796s) you're in?

[13:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=796s) >> Yeah, yeah, that that's the vehicle.

[13:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=800s) >> Got it.

[13:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=801s) >> Okay, so I'll pause it here, but as you

[13:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=803s) can see, very different things are

[13:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=804s) happening under the hood with the same

[13:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=805s) model. You know, it's more about

[13:27](https://www.youtube.com/watch?v=18Um2VjMM_g&t=807s) alphanumeric accuracy for complex kind

[13:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=810s) of like postcode numbers or addresses.

[13:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=812s) There's like this feature that we have

[13:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=814s) called proactive audio, which is the

[13:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=816s) idea that the LLM knows when or when not

[13:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=818s) to respond to an external input. So, for

[13:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=820s) example, if you're in a conversation and

[13:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=822s) someone else is talking or there's

[13:43](https://www.youtube.com/watch?v=18Um2VjMM_g&t=823s) background noise happening in the back,

[13:45](https://www.youtube.com/watch?v=18Um2VjMM_g&t=825s) the model knows to not stop or cut

[13:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=827s) itself short because there's external

[13:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=829s) noise happening. Because we realized

[13:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=831s) that the majority of these conversations

[13:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=833s) are not happening in an office room with

[13:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=835s) like no noise, you know, in a sealed

[13:56](https://www.youtube.com/watch?v=18Um2VjMM_g&t=836s) environment. They're happening on the

[13:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=838s) go, on the train, while you're on a

[13:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=839s) walk. Um and that's the type of

[14:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=841s) experiences that we want to facilitate

[14:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=843s) with the speech-to-speech model.

[14:05](https://www.youtube.com/watch?v=18Um2VjMM_g&t=845s) Um lastly, um another thing we're really

[14:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=847s) excited about is not only voice out, but

[14:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=849s) also multimodal out. We believe that,

[14:12](https://www.youtube.com/watch?v=18Um2VjMM_g&t=852s) you know, the true AGI conversational

[14:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=854s) frontier will also require visual

[14:16](https://www.youtube.com/watch?v=18Um2VjMM_g&t=856s) presence. So, we were excited to launch

[14:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=858s) our first kind of pilot demo with City

[14:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=860s) um in Cloud Next, which supports

[14:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=862s) customized real-time avatars.

[14:24](https://www.youtube.com/watch?v=18Um2VjMM_g&t=864s) Um and you can kind of like

[14:26](https://www.youtube.com/watch?v=18Um2VjMM_g&t=866s) personalize anything from a

[14:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=868s) hyperrealistic human to a cartoon

[14:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=870s) appearance and everything in the middle.

[14:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=872s) And powered by the same speech-to-speech

[14:33](https://www.youtube.com/watch?v=18Um2VjMM_g&t=873s) model we've been showing, it allows to

[14:35](https://www.youtube.com/watch?v=18Um2VjMM_g&t=875s) have, you know, low-latency,

[14:37](https://www.youtube.com/watch?v=18Um2VjMM_g&t=877s) multilingual lip-syncing, and a really

[14:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=878s) kind of like uh ongoing fluid

[14:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=880s) conversation that has visual presence.

[14:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=882s) So, let me show you a demo that also

[14:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=884s) brings our Venn diagram together and

[14:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=886s) what we're excited about.

[14:48](https://www.youtube.com/watch?v=18Um2VjMM_g&t=888s) >> All this is reminding me of my

[14:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=889s) daughter's college fund. How are we

[14:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=891s) tracking on that?

[14:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=895s) >> You're tracking well, Jackson. I've also

[14:57](https://www.youtube.com/watch?v=18Um2VjMM_g&t=897s) identified a new opportunity that may

[14:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=899s) get you there even sooner.

[15:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=901s) Oh, and I can see Lisa just joined you.

[15:03](https://www.youtube.com/watch?v=18Um2VjMM_g&t=903s) Hi, Lisa. She must be so excited about

[15:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=906s) her college acceptance. Congratulations.

[15:08](https://www.youtube.com/watch?v=18Um2VjMM_g&t=908s) It's wonderful to see your savings goals

[15:10](https://www.youtube.com/watch?v=18Um2VjMM_g&t=910s) coming to life.

[15:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=911s) >> Excelente. Las cosas han estado tan

[15:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=914s) inestables últimamente.

[15:18](https://www.youtube.com/watch?v=18Um2VjMM_g&t=918s) >> Sí. Los servicios de tecnología y

[15:20](https://www.youtube.com/watch?v=18Um2VjMM_g&t=920s) comunicaciones han mostrado un desempeño

[15:23](https://www.youtube.com/watch?v=18Um2VjMM_g&t=923s) sólido en lo que va del año.

[15:25](https://www.youtube.com/watch?v=18Um2VjMM_g&t=925s) >> That's good news.

[15:26](https://www.youtube.com/watch?v=18Um2VjMM_g&t=926s) >> I always joke that the user's audio in

[15:28](https://www.youtube.com/watch?v=18Um2VjMM_g&t=928s) Spanish is worse than the audio model

[15:30](https://www.youtube.com/watch?v=18Um2VjMM_g&t=930s) speaking back, but um this is kind of

[15:32](https://www.youtube.com/watch?v=18Um2VjMM_g&t=932s) just to show how like our Venn diagram

[15:34](https://www.youtube.com/watch?v=18Um2VjMM_g&t=934s) of combining, you know, multimodality in

[15:36](https://www.youtube.com/watch?v=18Um2VjMM_g&t=936s) and out, you know, tool calling to like

[15:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=938s) pull up the relevant examples from the

[15:40](https://www.youtube.com/watch?v=18Um2VjMM_g&t=940s) user, and also conversational fluidity

[15:42](https://www.youtube.com/watch?v=18Um2VjMM_g&t=942s) with ITNN are starting slowly to come

[15:44](https://www.youtube.com/watch?v=18Um2VjMM_g&t=944s) together in these types of demos that

[15:46](https://www.youtube.com/watch?v=18Um2VjMM_g&t=946s) we're excited to keep pushing the

[15:47](https://www.youtube.com/watch?v=18Um2VjMM_g&t=947s) frontier of.

[15:49](https://www.youtube.com/watch?v=18Um2VjMM_g&t=949s) Um so with this parting thought, I guess

[15:51](https://www.youtube.com/watch?v=18Um2VjMM_g&t=951s) last

[15:52](https://www.youtube.com/watch?v=18Um2VjMM_g&t=952s) kind of thought that we have for you is

[15:53](https://www.youtube.com/watch?v=18Um2VjMM_g&t=953s) that we believe that AGI will not be

[15:55](https://www.youtube.com/watch?v=18Um2VjMM_g&t=955s) typed, that it will be spoken. Um and

[15:58](https://www.youtube.com/watch?v=18Um2VjMM_g&t=958s) for it to be spoken, there's a lot of

[15:59](https://www.youtube.com/watch?v=18Um2VjMM_g&t=959s) things that need to work together in a

[16:01](https://www.youtube.com/watch?v=18Um2VjMM_g&t=961s) single promptable, versatile model that

[16:04](https://www.youtube.com/watch?v=18Um2VjMM_g&t=964s) allows a user to switch between all the

[16:06](https://www.youtube.com/watch?v=18Um2VjMM_g&t=966s) sorts of conversation modes that we're

[16:07](https://www.youtube.com/watch?v=18Um2VjMM_g&t=967s) looking at, right? From translation to

[16:09](https://www.youtube.com/watch?v=18Um2VjMM_g&t=969s) taking action to brainstorming to

[16:11](https://www.youtube.com/watch?v=18Um2VjMM_g&t=971s) rambling, and we truly believe in the

[16:13](https://www.youtube.com/watch?v=18Um2VjMM_g&t=973s) power of these speech-to-speech models

[16:14](https://www.youtube.com/watch?v=18Um2VjMM_g&t=974s) to achieve that like seamless switching.

[16:17](https://www.youtube.com/watch?v=18Um2VjMM_g&t=977s) Um so we're excited to push the frontier

[16:19](https://www.youtube.com/watch?v=18Um2VjMM_g&t=979s) on that. So if you're excited or want to

[16:21](https://www.youtube.com/watch?v=18Um2VjMM_g&t=981s) learn more, please come talk to us, and

[16:22](https://www.youtube.com/watch?v=18Um2VjMM_g&t=982s) thank you so much for coming.

[16:38](https://www.youtube.com/watch?v=18Um2VjMM_g&t=998s) >> [music]
