# Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff

Thor Schaeff, Google DeepMind

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/xfPVBzDYjH4-your-agent-hear-you-now-building-live) · [Video](https://www.youtube.com/watch?v=xfPVBzDYjH4)

## Can Your Agent Hear You Now? Building Live Voice Agents with Gemini

Thor Schaeff connects live conversation to image generation, translation and music, then explains the stateful WebSocket session behind Gemini’s voice and vision agents.

### Expressive speech is only the opening act

[0:12](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=12s)

The opening voice sounds ready to host the conference: “Welcome one and all to the grandest stage on Earth.” But when the audience guesses which model produced it, Gemini Live is the wrong answer. The theatrical welcome comes from **Gemini Text to Speech**. Thor Schaeff, who works on developer experience at Google DeepMind, starts with a useful distinction: generating expressive speech and conducting a live conversation are different capabilities.

Gemini TTS accepts inline text instructions to steer delivery and can stream its speech output. Schaeff describes its latency as lower than before, without giving a measurement. The rest of the session moves beyond delivering prepared words: an agent will listen to a developing story, invoke image generation, translate a live microphone feed and turn a spoken music request into a generated track.

The first enabling change is faster image generation and editing with **NanoBanana Lite**. That makes an illustrated conversation more plausible: the user can keep talking while the application adds pictures to the story. Schaeff introduces his recently “vibe-coded” Gemini NanoBanana story canvas as a way to try that combination in front of the room.

### A story changes direction while its illustrations retain context

[3:58](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=238s)

The story starts with a small prompt: Taipei in the 1980s. Gemini Live expands it into a rainy night market, neon reflected on wet pavement, passing scooters and a young woman hiding a glowing object under her jacket. Schaeff then changes the direction: a new kind of drink has emerged. The continuation follows the woman to a stall where a vendor shakes a metal cup and serves a milky drink with tapioca pearls. Bubble tea becomes the next story beat; this is generated fiction, rather than a historical demonstration.

The illustrations come from a separate capability connected through a tool call. Gemini Live handles the conversation, while the application uses the Interactions API to request an image from NanoBanana Lite as the story develops. Schaeff puts the actual image-generation time at roughly two to three seconds in this demo. That is short enough to accompany the exchange, but it remains a noticeable generation step rather than an instantaneous visual response.

The harder problem is continuity. Independently generating each picture would leave the application asking for a new scene without the same explicit connection to the preceding result. The **Interactions API** provides a stateful continuation: the first image request has no previous interaction ID; the next request supplies the previous interaction ID so generation can use that earlier context.

The useful code comparison is between those first and subsequent requests: where does the previous interaction ID enter? Schaeff points to the first image’s visual feeling and characters, then describes carrying them into subsequent generations through the linked interaction. The ID gives the next request continuity with earlier context; the demo does not establish a guarantee that every character or stylistic detail will remain identical.

![In the Interactions API call, `previous_interaction_id` receives the prior ID when one exists and `undefined` otherwise—the visible branch that distinguishes a continuation from the first request.](https://ai.engineer/api/talk-essay-media/6b5d35b724c1cc6a31e79e41d412d42ea887705b9620a34f0e22f527d4e11000)

In the Interactions API call, `previous_interaction_id` receives the prior ID when one exists and `undefined` otherwise—the visible branch that distinguishes a continuation from the first request.

```typescript
const interaction = await ai.interactions.create({
  model: 'gemini-3.1-flash-lite-image',
  input: prompt,
  previous_interaction_id: previousInteractionId || undefined,
});
```

There are two kinds of state to keep straight here. The live conversation carries the unfolding narrative, including the user’s intervention about a new drink. The image-generation continuation carries prior generation context. Connecting those capabilities lets the story advance while giving its illustrations a basis for staying recognizable.

### One microphone feed, different listening languages

[6:47](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=407s)

The next demo turns the room into a multilingual listening session. Schaeff creates a session for **Gemini 3.5 Live Translate**, asks the audience to join through a QR code and enables his microphone. Each listener selects a preferred language on their phone. The useful comparison is between the shared speech input and the listeners’ output choices: someone choosing German can listen to a German rendering of the same talk while Schaeff continues speaking to everyone.

![The broadcast session lists German, Turkish and Hindi as separate translations, each with an active listener.](https://ai.engineer/api/talk-essay-media/80f7dd0929c3a7531ca36084cce10441e62d5a0f80ba839947befe44544d9bee)

The broadcast session lists German, Turkish and Hindi as separate translations, each with an active listener.

The timing depends on recognizing enough context to begin translating. Schaeff describes the output as near real time: the model listens to the ongoing speech and starts producing the desired language once it recognizes what he is talking about. This gives the translation some context before it commits to an utterance. The recording supplies no numerical translation delay or accuracy evaluation.

The language choices operate on two sides of the exchange:

- **Speaker input.** The model automatically detects the language being spoken, so the presenter can change languages without manually choosing a new source language.
- **Listener output.** The listener chooses the language they want to hear. That preference can remain different from the presenter’s current language.

Schaeff proposes switching to German and also checks whether anyone is listening in Chinese. The point is to let the presenter and audience use their own preferred languages within the same event.

A listener confirms receiving audio and responds positively when asked how it sounds. Headphones matter in this setting: the translated stream needs to be audible alongside the presenter’s amplified original speech. The demo therefore combines a model capability with a practical delivery arrangement—microphone input, a joinable session and personal audio output. Schaeff presents Live Translate as available through the Gemini API for integration into applications.

### A radio DJ turns conversation into a music-generation request

[10:13](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=613s)

The radio demo borrows a familiar interaction: phone the host and ask for a song. Here, **Gemini 3.1 Flash Live** plays the DJ, and a callable function gives it access to **Lyria 3** through the Gemini API. The conversational model can gather the request and keep the exchange going; the music model supplies the generated song.

The DJ opens by asking for a genre and a lyrical direction. Schaeff requests high-energy reggaeton about the AI Engineer World’s Fair in San Francisco. The agent asks a narrower follow-up: should it include particular technologies or big names, or aim for a festival atmosphere? “Just surprise me, please” leaves those remaining creative choices to the agent.

The DJ announces generation, and the resulting song combines English and Spanish lyrics about San Francisco, neural networks, algorithms and the fair. The request’s genre and subject have become a concrete media result. After playback, the DJ returns to its conversational role and asks what to play next. That return matters: generating the artifact is one action inside the continuing exchange.

Schaeff is candid about the business value of an endlessly enthusiastic jukebox: the application itself may not be what a company needs. Its components are more transferable:

- **Conversational input.** The agent gathers intent through ordinary speech and asks a follow-up before acting.
- **A callable capability.** Music generation gives the agent something useful to do beyond answering. Schaeff proposes CRM lookups or backend ticket creation as other possible tools; those actions are not demonstrated here.
- **Multilingual interaction.** The experience can accommodate multiple languages, with the generated song providing a playful example of mixed-language output.

The implementation decision is to give the live agent access to a capability that serves the application’s purpose.

### Inside the Live API: a stateful session with native audio

[14:05](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=845s)

Gemini 3.1 Flash Live is accessed through the **Live API**. An application creates a stateful WebSocket session, then sends text, audio and video frames over that connection. The model returns audio chunks that the application can stream to the user, together with a transcript of what was said. This is an ongoing session carrying live inputs and outputs, rather than a separate request for each finished recording.

Where does text sit in that exchange? Schaeff distinguishes this native audio model from a cascading voice pipeline: speech understanding and response generation do not require an intervening text representation in the model’s path. The transcript is still available to the application, but it should not be confused with the medium through which all of the model’s audio intelligence operates. The diagram separates the incoming modalities from the two kinds of output returned through the live session.

The session supports several behaviors that make conversation usable:

- **Language switching.** Schaeff reports 90 supported languages in the preview and connects native audio processing with fluid multilingual responses.
- **Barge-in.** A user can interrupt the model rather than waiting for a long response to finish. The overly talkative DJ is the obvious example.
- **Tool calls.** The agent can invoke capabilities such as the music generator instead of limiting itself to spoken answers.
- **Transcription.** The application receives text alongside the audio response.

These are capabilities of the conversational experience; implementing them still requires the application to handle the live media exchange.

Real-time media can become difficult to scale even when the model interface is straightforward. Schaeff points to LiveKit and Pipecat as partners that help developers get started with Gemini Live. The session does not develop their transport or deployment designs, but it makes a useful separation: choosing the conversational model does not by itself solve the application’s media infrastructure.

What travels through the live session?
Text, audio and video frames enter a stateful WebSocket session. Audio chunks and a transcript return to the application; the transcript is an output, not a required intermediate stage in the native audio path.
- Application inputs — Text, live audio and video frames.
- Stateful Live API session — A WebSocket connection carries the ongoing exchange.
- Native audio model — Understands speech and produces spoken responses without requiring a text cascade.
- Streamed audio — The application plays returned audio chunks.
- Transcript — Text of what was said is also available to the application.
- Application inputs → Stateful Live API session (Send inputs over WebSocket)
- Stateful Live API session → Native audio model (Deliver live inputs)
- Native audio model → Stateful Live API session (Return response audio and transcript)
- Stateful Live API session → Streamed audio (Stream audio chunks)
- Stateful Live API session → Transcript (Provide transcription)

### Try the voice, add vision, then recover a language switch

[16:36](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=996s)

Google AI Studio provides the shortest route to trying the interaction before building an application. Schaeff opens its Live interface, selects Gemini 3.1 Flash Live and chooses the Puck voice. The same interface also offers Live Translate. This lets a developer explore the model’s behavior through a conversation rather than starting with a media client.

Another option enables grounding with Google Search. Training alone does not supply live information such as today’s weather; Search gives the model a way to look information up. Schaeff describes this as a built-in function call. He explains the option before the demonstration, but the ensuing exchange tests hearing, vision and languages rather than showing a search result.

The vision test moves from a broad question to an inspectable detail. Asked whether it can see him, the assistant describes a black shirt, black hat and blue background. Asked what the sweatshirt says, it answers that “Gemini” appears in yellow letters. Comparing the camera input with that answer would show what detail the assistant extracts: the task has moved beyond acknowledging video to reading particular lettering.

The next question connects vision to language. Schaeff asks in German what the word means, and the assistant answers “Zwillinge”—twins. The conversation has moved from reading lettering in the camera view to translating that lettering, without starting a separate translation application.

The attempted move to Mandarin is less smooth. The assistant continues with a German response, and Schaeff remarks that his Mandarin “didn’t work out so well.” The precise cause is unclear, so this moment supports a narrow lesson: the intended language switch was not reliably understood in that exchange. He repairs it with an explicit request to speak Mandarin, which the assistant acknowledges.

Schaeff then explains the final Mandarin exchange for the audience: he asked what the model had eaten, and it replied that it did not need to eat because it was a model. The session ends with a small but useful demonstration of recovery. A missed conversational cue does not have to end the interaction; an explicit language request can redirect it, and the same live session can continue. AI Studio is his suggested starting point for trying those capabilities before connecting them to an application’s tools and media infrastructure.

## Key takeaways

- The story canvas connects Gemini Live to NanoBanana Lite through tool calls. Passing the previous interaction ID gives later image requests access to earlier generation context.
- Live Translate separates the automatically detected speaking language from each listener’s chosen output language, beginning translation as enough context becomes available.
- A live agent becomes more useful when conversation can invoke a concrete capability. The DJ gathers a music request, calls Lyria 3 and returns to the conversation after the song.
- Gemini 3.1 Flash Live uses the Live API’s stateful WebSocket session for text, audio and video inputs, returning audio chunks plus transcription. Native audio processing does not require the transcript to serve as an intermediate text stage.
- Multilingual capability still benefits from clear conversational control: when the attempted Mandarin switch faltered, an explicit request let the exchange continue.

## Slides

### Build agents with the Gemini API

Thor Schaeff introduces a session on building agents with the Gemini API.

[0:17](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=17s) [Title slide naming Thor Schaeff, with a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/5424a31fef01aef02a5175bb0b4643ebfd80560e91f7731a56d33c3df6581ef4.webp)

Build agents with the Gemini API

Thor Schaeff | @thorwebdev

### Build agents with the Gemini API

Thor Schaeff introduces a talk on building agents with the Gemini API.

[2:00](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=120s) [Title slide with Google DeepMind, Thor Schaeff and a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/f9b7a89e384e38530fa2b2bc5a97db9d732d494e7987224b982c2bae6e528948.webp)

Build agents with the Gemini API

Google DeepMind

Thor Schaeff | @thorwebdev

### Thor Schaeff

Thor Schaeff works on developer experience at Google DeepMind, including the Gemini API and Google AI Studio. He invites API users to share feedback.

[2:03](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=123s) [Presenter introduction with Thor Schaeff's portrait, Google DeepMind role and a QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/f48d4e1f175299d4cca9af8ff41fa21591629cfec300bab9ac9d81e0fcec47fd.webp)

Thor Schaeff
Developer Experience, Google DeepMind
@thorwebdev

### Get started with just a few clicks

The slide maps a path from getting an API key in Google AI Studio and experimenting on the free tier to setting up billing and managing spend.

[2:35](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=155s) [Steps for getting started with Google AI Studio and the Gemini API, alongside an interface screenshot.](https://assets.ai.engineer/sites/aiecode2025/sha256/cd0cad17450f85ee627546ec42bcec39dd7154618e9699cf4d23ba31d37ca54f.webp)

Get started with just a few clicks

1. Visit Google AI Studio (ai.studio)
2. Get your API key
3. No setup required
4. Generous free-tier
5. Start experimenting
6. Set up billing
7. Manage spend

Interface text:
Google AI Studio
Playground
History
Build
Apps
Gallery
Dashboard
Documentation
Search
What's new
Get API key
Settings
Explore Google models
Models
Agents
Featured
Test out our most advanced and newest models.
Code and Chat
Build chatbots, agents, and code with Gemini 3.
Image Generation
Create and edit images with Nano Banana and Imagen.
Video Generation
Generate videos with Veo models, our state of the art video generation models.
Speech and Music
Real-time
Start typing a prompt to see what our models can do
Run settings
Get code
Gemini 3.1 Pro Preview
[illegible]
System instructions
Optional tone and style instructions for the model
Temperature
1
Thinking level
High
Tools
Structured outputs
Edit
Code execution

### Gemini API inputs and outputs

The diagram maps inputs including instructions, text, code and media into Gemini, with outputs including text, media, function calls and code.

[2:59](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=179s) [Diagram connecting multiple input types to Gemini and its output types.](https://assets.ai.engineer/sites/aiecode2025/sha256/159d503e854d5662784aef51cc21b46b7e314e46d7b8ce0b0270e3cca7724db2.webp)

Gemini API inputs and outputs

Inputs:
system instructions
text
code
image
audio
video
speech
API information
URLs

Outputs:
text
image
audio
video
function call
code

### Story Canvas: Taipei narration and illustration

After the user sets the story in 1980s Taipei, Story Canvas narrates a night-market scene and begins generating an illustration.

[4:27](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=267s) [Story Canvas displays a Taipei story prompt and a partially generated narrator response.](https://assets.ai.engineer/sites/aiecode2025/sha256/7f85a0b3b3199be6e7762319d4367817ff42d58a089c7eab112b7985bcb4eca0.webp)

YOU
We're in Taipei in the 1980s.

NARRATOR
The neon lights of Taipei's night market pulsed with an almost magical energy, reflecting off the wet pavement and the chrome of passing scooters. Rain had just passed, leaving a humid mist that hung in the air. Amidst the throng of shoppers and vendors, a young woman clutched a small,

Interface text:
Story Canvas
Connected & Ready
Disconnect
CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%
INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]
Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.
Start voice session
Add the next plot point...
Send
localhost:5173

[4:31](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=271s) [Story Canvas displays a Taipei night-market narration while an illustration is generating.](https://assets.ai.engineer/sites/aiecode2025/sha256/e542e93a671b12686be29e89a4e2967a1a4ab10873488f85ea88091212b0845d.webp)

YOU
We're in Taipei in the 1980s.

NARRATOR
The neon lights of Taipei's night market pulsed with an almost magical energy, reflecting off the wet pavement and the chrome of passing scooters. Rain had just passed, leaving a humid mist that hung in the air. Amidst the throng of shoppers and vendors, a young woman clutched a small, strangely glowing object under her jacket.

Gemini is illustrating “A bustling Taipei night market in the 1980s, ne...”

Interface text:
Story Canvas
Illustrating scene...
Disconnect
CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%
INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]
Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.
Start voice session
Add the next plot point...
Send
localhost:5173

### First night-market illustration

Story Canvas illustrates the narrated Taipei night market with a woman holding a glowing object among the crowds.

[4:35](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=275s) [A generated night-market image shows a woman holding a glowing object.](https://assets.ai.engineer/sites/aiecode2025/sha256/3d18ae959afd7d91d98d31e011f94447967edf7beac3f6d641cdb6b9d17b18a8.webp)

and vendors, a young woman clutched a small, strangely glowing object under her jacket. [Interrupted]

Interface text:
Story Canvas
Connected & Ready
Disconnect

CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%

INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]

Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.
Start voice session
Add the next plot point...
Send

Gemini Voice Library
Story Canvas – Gemini [illegible]
Live Translate
[Public] Live Jukebox | Goog[illegible]
Ask Gemini
localhost:5173

### Story Canvas: bubble-tea narration

A prompt about a new drink in 1980s Taipei steers the story to a stall, where the narrator describes a customer tasting bubble tea.

[4:59](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=299s) [Story Canvas displays a new prompt and narration about bubble tea in 1980s Taipei.](https://assets.ai.engineer/sites/aiecode2025/sha256/b008f81f10f913cc7ee0ca52c5e5bc0b4ed40c643d11d230a1b97eb78a999564.webp)

YOU
Something very exciting happened in 1980s Taipei, a new kind of drink emerged.

NARRATOR
She paused at a stall where a man was vigorously shaking a metal cup, creating a frothy, milky concoction. He handed a cup to a customer, who took a long sip with wide eyes, savoring the strange tapioca pearls at the bottom. This was the birth of bubble tea, and its sweet, chewy magic was spreading like wildfire.

Interface text:
Story Canvas
Connected & Ready
Disconnect
CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%
INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]
Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.
Start voice session
Add the next plot point...
Send
localhost:5173

### Bubble-tea illustration

Gemini Live triggers image generation through the Interactions API. The bubble-tea scene continues the earlier night-market image using a previous interaction ID to carry context across generations.

[5:20](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=320s) [A generated night-market image shows a woman drinking bubble tea beside a man.](https://assets.ai.engineer/sites/aiecode2025/sha256/9770ba16cabdb4467950fd293798cf2f3f43ef1bb8b107de3c941cd5529af8a3.webp)

bubble tea, and its sweet, chewy magic was spreading like wildfire.

Interface text:
Story Canvas
Connected & Ready
Disconnect
CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%
INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]
Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.
Add the next plot point...
Send
localhost:5173

### Story Canvas demo

Story Canvas shows a woman holding a glowing object in a night market. Schaeff explains that passing the previous interaction ID preserves context to keep characters and visual style consistent across image generations.

[6:23](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=383s) [Story Canvas displays a generated image of a woman holding a glowing object in a night market.](https://assets.ai.engineer/sites/aiecode2025/sha256/971c9e0f2399cc953fa6b0bd7b907c7d8f6ee3bc5075bed3421ae6db79b4b967.webp)

Story Canvas

and vendors, a young woman clutched a small, strangely glowing object under her jacket. [Interrupted]

CANVAS SETTINGS
Live Model
Gemini 3.1 Flash Live (Default)
Narrator Voice
Puck (Warm)
Playback Volume
80%

INSTRUCTIONS
Story Direction
You are a creative fantasy narrator and interactive storytelling partner. We will write a story together taking turns. You keep your turn focused on a single engaging plot point (2-3 sentences), keeping the narrative [illegible]

Speak or Type: Click “Connect Live” first. Talk using your microphone or type in the chat to add plot points.

Add the next plot point...
Send

Interface text:
Connected & Ready
Disconnect
localhost:5173
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox | [illegible]
Ask Gemini

### Live Translate demo

Schaeff creates a live translation session and enables his microphone. Attendees choose their languages, and the dashboard shows active listeners for each.

[7:24](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=444s) [Live Translate session creation page with a masked input and a creating indicator.](https://assets.ai.engineer/sites/aiecode2025/sha256/4b447d6c49b642c8fb9b9d2965293a2e59724cf55340ec8cf02d8467410a44fb.webp)

Live Translate

Broadcast your voice. Attendees choose their language. Translation spins up on demand.

[redacted]
Event ID (optional, e.g. weekly-sync)
Creating...

01 Speak into your microphone — your audio goes live
02 [illegible]

Interface text:
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox | [illegible]
Ask Gemini
live-translate-[illegible].us-central1.run.app

[8:11](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=491s) [A list of active translation languages and listener counts above an End broadcast button.](https://assets.ai.engineer/sites/aiecode2025/sha256/c061d2cdd243167ed546e684f953cfbe6b6161b6b96b6a841e44bbf6b6ec50a0.webp)

Korean — 1 listener • active
Basque — 1 listener • active
Marathi — 1 listener • active
Spanish — 1 listener • active
Japanese — 1 listener • active
Portuguese (Brazil) — 3 listeners • active
Russian — 2 listeners • active
Javanese — 1 listener • active
Chinese (Traditional) — 2 listeners • active
English — 1 listener • active

End broadcast

Interface text:
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox | [illegible]
Ask Gemini
live-translate-[illegible].us-central1.run.app

### Gemini 3.5 Live Translate

Schaeff says Live Translate is available through the Gemini API for use in applications.

[9:51](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=591s) [Gemini 3.5 Live Translate title on a pale blue background.](https://assets.ai.engineer/sites/aiecode2025/sha256/4e7dcb75bdb1a3a1c4beb51c6cbebc0c87b592f5767e3e69977eba96c68da0c2.webp)

Gemini 3.5 Live Translate

### Gemini 3.1 Flash Live

Schaeff introduces Gemini 3.1 Flash Live as a real-time conversational model used in the Gemini app.

[10:03](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=603s) [Gemini 3.1 Flash Live title beside a dotted microphone graphic.](https://assets.ai.engineer/sites/aiecode2025/sha256/fead636b086ea88ee17ceb636aba13efa17ef79ad7eb2a38b4b42c7ffc9469aa.webp)

Gemini 3.1 Flash Live

### Live Jukebox demo

Live Jukebox puts Gemini Live in the role of a radio DJ: listeners request a song, and the agent calls Lyria 3 to generate music on demand.

[10:34](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=634s) [Live Jukebox interface with a turntable, empty session log and sharing QR code.](https://assets.ai.engineer/sites/aiecode2025/sha256/b1df2e6765cf87192574328220e5c1296bfc6f94ad4cd57387a2b311b99ce3f2.webp)

Live Jukebox
POWERED BY GEMINI LIVE & LYRIA

STUDIO A
SESSION

SESSION LOGS
0 TRACKS
No tracks generated yet.
Wish for a song to begin.

READY TO JAM
TALK TO DJ
Text the DJ your request...

LIVE MUSIC GENERATION POWERED BY LYRIA

go.thor.bio/lyria-3-live-jukebox

### Live Jukebox demo

The Gemini Live DJ takes a request for high-energy reggaeton about the AI Engineering World’s Fair and uses Lyria to generate a track. The session log shows it ready to play.

[11:18](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=678s) [The jukebox shows the DJ responding and a track in the session log.](https://assets.ai.engineer/sites/aiecode2025/sha256/f15e4a3eafb71da04bf37e4127d7532ba6a304ddb2c0efa22c0a2186bfe15aba.webp)

Live Jukebox
POWERED BY GEMINI LIVE & LYRIA

STUDIO A SESSION

SESSION LOGS
1 TRACKS
High-energy reggaeton track capturing the electrifying atmosphere of the AI Engineering...
[illegible] · Cutting Vinyl...

DJ IS RESPONDING
STOP THE DJ
Text the DJ your request...

Interface text:
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox
Ask Gemini
aistudio.google.com/apps/[illegible]
Remix
Device

[12:09](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=729s) [Live Jukebox interface with a virtual record player and a generated track in the session log.](https://assets.ai.engineer/sites/aiecode2025/sha256/013041dc7ddc3013d1572015af6ac957ef47284bb5d36f374f6517e49725075a.webp)

Live Jukebox
POWERED BY GEMINI LIVE & LYRIA

STUDIO A SESSION

SESSION LOGS
1 TRACKS
High-energy reggaeton track capturing the electrifying atmosphere of the AI...
[illegible] · Ready to Play

READY TO JAM
TALK TO DJ
Text the DJ your request...

Interface text:
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox
Ask Gemini
aistudio.google.com/apps/[illegible]
Remix
Device

### Live API overview

An app sends text, audio, and video to Gemini over a WebSocket. The Live API streams audio back along with a transcript.

[14:06](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=846s) [Live API enables real-time voice and vision interactions; a diagram shows an app exchanging media with the API over WebSocket.](https://assets.ai.engineer/sites/aiecode2025/sha256/3b3466080ed0d59c69bc6b5601cbc533d2c802d8bc8fca6e8578acfb61a7ee79.webp)

Live API

The Live API enables low-latency, real-time voice and vision interactions with Gemini.

App → Live API: Text, audio, video
Live API → App: Text, audio
Connect via WebSocket

### Live API key features

The Live API supports conversation in 90+ languages, interruptions, tool calls, and transcripts of both sides of an audio interaction.

[15:35](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=935s) [Key features include multilingual support, interruption, tool use, and audio transcription.](https://assets.ai.engineer/sites/aiecode2025/sha256/a0812235c0e726627ebfa8c8b2245d003b94f061f87ac3db5e79023da3d822d2.webp)

Live API - Key features

Live API offers a comprehensive set of features for building engaging voice agents

Multilingual support: Converse in 90+ supported languages.
Barge-in: Users can interrupt the model at any time for responsive interactions.
Tool use: Integrates tools like function calling and Google Search for dynamic interactions.
Audio transcriptions: Provides text transcripts of both user input and model output.

### Live API partner ecosystem

Partners including LiveKit and Pipecat help developers get started with the Gemini Live API as they build real-time conversational and multimodal applications.

[15:56](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=956s) [Partner ecosystem slide showing integration logos including Firebase AI Logic, LiveKit, Pipecat, VisionAgents, and Voximplant.](https://assets.ai.engineer/sites/aiecode2025/sha256/18a9225af171baf21959e9b71ca70df6b5d8e8af50b683399e08638116cb4019.webp)

Live API -
Partner ecosystem

Streamline development with partner integrations

Firebase AI Logic
LiveKit
Pipecat
software mansion.
VisionAgents
voximplant

### Try Gemini Live in AI Studio

Google AI Studio offers a way to try Gemini Live: talk to the model, share a webcam or screen, and adjust settings such as the voice.

[16:23](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=983s) [AI Studio Live access slide with a QR code, talk and sharing controls, and model settings.](https://assets.ai.engineer/sites/aiecode2025/sha256/78fbfd2e017173972da744e25b8632a7dda89b53cfbcfe56ee6c1e11af5f7e85.webp)

ai.studio/live

Talk to Gemini live

Talk
Webcam
Share Screen

Interface text:
Google AI Studio
Playground
Build
Dashboard
Documentation
Search
What's new
Get API key
Settings

Playground
Run settings
Get code

Gemini 3.1 Flash Live Preview
gemini-3.1-flash-live-preview
Our low-latency, audio-to-audio model optimized for real-time dialogue with acoustic nuance detection, numeric precision, and multimodal awareness.

System instructions
Optional tone and style instructions for the model

No API Key
Switch to a paid API key to unlock higher quota and more features.

Voice
Zephyr

Media resolution
258 tokens / image

Thinking level
No Thinking

Start typing a prompt
Tools

### Live Translate setup

The Live Translate playground offers microphone or tab audio input, with English selected as the target language. The model card describes low-latency speech-to-speech translation for 70+ languages.

[17:09](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1029s) [Google AI Studio Live Translate playground with English selected as the target language.](https://assets.ai.engineer/sites/aiecode2025/sha256/bbde36deb5f7b724e775830f3c3dcff0d15d5ced16614f8d95c08b17ae9f1135.webp)

Try the Live API
Talk
Share Audio From Tab

Gemini 3.5 Live Translate Preview
gemini-3.5-live-translate-preview
A real-time speech-to-speech translation model delivering low latency translation for 70+ languages.

System instructions
Optional tone and style instructions for the model

Target language
English
Echo target language

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
Run settings
Get code
Model used to generate response
[redacted]  PRO

### Choosing a live model

The model picker shows Live Translate alongside other models. Schaeff says he will select Gemini 3.1 Flash Live for the demo.

[17:12](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1032s) [Model selection panel showing Gemini 3.5 Live Translate Preview.](https://assets.ai.engineer/sites/aiecode2025/sha256/c691d5953f6aedac3bc3e2c24263673a303cdc6c7d45ae36fda5844d8f021aa6.webp)

Model selection
Search for a model or agent
All
Featured
Gemini
Live
Images
Video
Audio
Music
Agents
Gemma

[illegible]
coding tasks.
All context lengths • Input: $1.50 / Output: $9.00
Knowledge cut off: Jan 2025
Release date: May 19, 2026

Gemini 3.5 Live Translate Preview  New
gemini-3.5-live-translate-preview
A real-time speech-to-speech translation model delivering low latency translation for 70+ languages.
Audio • Input: $3.50 / Output: $21.00
Knowledge cut off: Jan 2025
Release date: Jun 9, 2026

Gemini 3.1 Flash Lite  New
gemini-3.1-flash-lite
Our most cost-efficient model, optimized for high-volume agentic tasks, translation, and simple data processing.

Interface text:
Google AI Studio
Playground
Try the Live API

### Live API playground

The Live API playground offers Talk, Webcam, and Share Screen controls. Thor selects the Puck voice and points out the option to enable grounding with Google Search.

[17:19](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1039s) [Live API playground with Talk, Webcam and Share Screen controls and the Puck voice selected.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c4aabbe148a08aafa426148e2a3461c7bb9dc85746014516194768af99c997d.webp)

Try the Live API
Talk
Webcam
Share Screen

Run settings
System instructions
Optional tone and style instructions for the model

Voice
Puck

Media resolution
258 tokens / image

Thinking level
No Thinking

Session Context

Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search

Start typing a prompt
Tools
Run

Interface text:
Google AI Studio
Playground
Explore
Playground
History
Build
New app
My apps
Gallery
Manage
Dashboard
Documentation
Get code
[redacted] PRO
Gemini Voice Library
Story Canvas — Gemini Live
Live Translate
[Public] Live Jukebox | Goog[illegible]
Google AI Studio
Ask Gemini
aistudio.google.com/live?model=gemini-3.1-flash-live-preview

### Live audio conversation

Thor asks whether the model can hear him, and it responds aloud that it can.

[17:56](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1076s) [A live audio exchange shows a user asking if the model can hear them and the beginning of the model's reply.](https://assets.ai.engineer/sites/aiecode2025/sha256/1e6d764d1a6742fafe1d0edfddda7d8723e5b05e69cc52e55af8d44a2bd38e9c.webp)

User
0:00 / 0:03
Hey, can you hear me?

Model
Yes, I can hear you perfectly. How can

Stream is live

Interface text:
Google AI Studio
Playground
Explore
Playground
History
Build
New app
My apps
Gallery
Manage
Dashboard
Documentation
[redacted]  PRO

Run settings
Get code
System instructions
Optional tone and style instructions for the model
Voice
Puck
Media resolution
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search

Start typing a prompt
Tools
Select camera source
Run

### Live webcam conversation

In a live webcam exchange, the model describes the presenter’s clothing and background, then reads “Gemini” on his sweatshirt.

[18:08](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1088s) [The Live API playground shows a webcam feed and a response describing the person visible in it.](https://assets.ai.engineer/sites/aiecode2025/sha256/8ccd7daa3ae7fead0794ace163f9afc00926369b2315375630dd75d15c7989c6.webp)

Can you see me?

Model
Yes, I can see you clearly. You're wearing a black shirt and a black hat, sitting in front of a blue background. Anything specific you wanted to show me?

Gemini

Interface text:
Google AI Studio
Playground
Explore
Playground
History
Build
New app
My apps
Gallery
Manage
Dashboard
Documentation
0:00
0:00 / 0:05
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
Stream is live
Start typing a prompt
Tools
Run
[redacted]
PRO
Browser tabs: Gemini Voice Library; Story Canvas — Gemini Live; Live Translate; [Public] Live Jukebox | Goog[illegible]; Google AI Studio
Ask Gemini
Address: aistudio.google.com/live?model=[illegible]

[18:18](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1098s) [Google AI Studio identifies the word Gemini on the presenter’s sweatshirt.](https://assets.ai.engineer/sites/aiecode2025/sha256/5e67eb762b5499d916de6ef89357c55f769f937d4fe1500769a1e185fa595607.webp)

Yeah, what does it say on my sweatshirt?

Model
I can see the word "Gemini" printed in yellow letters. Anything else you'd like me to identify?

Gemini

Interface text:
Google AI Studio
Playground
Explore
Playground
History
Build
New app
My apps
Gallery
Manage
Dashboard
Documentation
0:00
0:00 / 0:11
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
Stream is live
Start typing a prompt
Tools
Run
[redacted]
PRO
Browser tabs: Gemini Voice Library; Story Canvas — Gemini Live; Live Translate; [Public] Live Jukebox | Goog[illegible]; Google AI Studio
Ask Gemini
Address: aistudio.google.com/live?model=[illegible]

### German translation response

AI Studio displays a Korean utterance and a German response about seeing Grandma. The speaker then says his Mandarin didn’t work out well.

[18:43](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1123s) [Google AI Studio displays Korean input, a German translation response, and an audio player.](https://assets.ai.engineer/sites/aiecode2025/sha256/fa52f5c0f2cf90736d8a40d2a5a6871da4bdf87ed538486e3b3f7e2ba80d2f7f.webp)

아, 보이지. 오, 보이죠. 어, 니 니가 좋죠. 엄마.

Model
Das bedeutet "Ich kann dich sehen, Oma. Oh, ich sehe dich. Ähm, du siehst gut aus, Oma." Möchtest du noch etwas übersetzen lassen?

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
0:00
0:00 / 0:09
0:00 / 0:07
Google AI models may make mistakes, so double-check outputs.
Stream is live
Start typing a prompt
Tools
Run
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
[redacted] PRO

### Requesting Mandarin

After saying his Mandarin attempt did not go well, Schaeff asks Gemini to speak Mandarin. The model replies in English and Chinese, asking what he would like to say.

[18:52](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1132s) [Google AI Studio responds to a request to speak Mandarin.](https://assets.ai.engineer/sites/aiecode2025/sha256/2ccd8e3b454e5556235cc2fc230d5c01e148d1d7309336ddc1a09abed78e9d8a.webp)

Ah! Okay. I think my uh my Mandarin didn't work out so well there. Uh can you also speak Mandarin, please?

Model
Yes, I can speak Mandarin too! What would you like to say now? 你想说什么?

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
0:00
0:00 / 0:17
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
Stream is live
Start typing a prompt
Tools
Run
PRO

[18:57](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1137s) [Google AI Studio shows a request to speak Mandarin and the model’s bilingual response.](https://assets.ai.engineer/sites/aiecode2025/sha256/01bfc4496d2725bfbba816a5dc8d8fa8d7b3e9aba91dc6c9b9c55e8aaab50cf3.webp)

Ah! Okay. I think my uh my Mandarin didn't work out so well there. Uh can you also speak Mandarin, please?

Model
Yes, I can speak Mandarin too! What would you like to say now? 你想说什么?

Google AI models may make mistakes, so double-check outputs.

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
0:00
0:00 / 0:17
0:00 / 0:04
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
Stream is live
Start typing a prompt
Tools
Run
PRO

### Gemini Live discusses eating

Schaeff asks Gemini what it has eaten. The model replies in Mandarin that it does not need to eat and asks whether he has eaten.

[19:05](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1145s) [Google AI Studio shows a Chinese response saying the model does not need to eat and asking whether the user has eaten.](https://assets.ai.engineer/sites/aiecode2025/sha256/4ab72a9be5a8fa89f28adb8fcec45d5debb0161f237889a84109897a00f87237.webp)

Ah! Nice to follow me.

Model
我不用吃饭, 因为我是一个人工智能。你呢? 你吃了吗?

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
0:00
0:00 / 0:07
0:00 / 0:04
Google AI models may make mistakes, so double-check outputs.
Stream is live
Start typing a prompt
Tools
Run
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
[redacted]
PRO

### Later multilingual exchange

The screen pairs a Portuguese transcription with a Chinese response asking what the user ate. The stream has ended, and AI Studio offers to start a new one.

[19:12](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1152s) [Google AI Studio shows a Portuguese transcription, a Chinese response, and a control to start a new stream.](https://assets.ai.engineer/sites/aiecode2025/sha256/b9fd18f21dd4ba29b823e0c08beaed6f201e699c3eeea8934147500726c53531.webp)

O que é o tabu da incerteza?

Model
你吃了什么好吃的? 可以跟我分享一下吗?

Interface text:
Google AI Studio
Playground
EXPLORE
Playground
History
BUILD
New app
My apps
Gallery
MANAGE
Dashboard
Documentation
0:00
0:00 / 0:07
0:00 / 0:03
Google AI models may make mistakes, so double-check outputs.
Start new stream
Clear the chat to start a new stream
Tools
Run
Run settings
Get code
System instructions
Optional tone and style instructions for the model
Voice
Puck
Media resolution
258 tokens / image
Thinking level
No Thinking
Session Context
Tools
Function calling
Edit
Automatic Function Response
Grounding with Google Search
[redacted]
PRO

## Full transcript

[0:12](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=12s) Can you hear me? Fantastic. Hello, everyone. Well, hello couple people that wandered in. Um, I'm sorry the schedule has been changing around. I'm sorry you didn't find the, a quiet working area you were looking  for. All right. Um, we are looking at some agents with Gemini API. Now, before we kick off, let's have a little listen to this.

[0:41](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=41s) Welcome one and all to the grandest stage on Earth. Ha. You're standing right at the heart of the largest AI engineer world's fair in history. Let's build the future together, shall we?

[0:57](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=57s) Does anyone know what model that is?

[1:01](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=61s) Yeah, you're, you, you, you're not allowed. Anyone else? Anyone know which model? No? Any guesses? It starts with G.

[1:17](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=77s) Gemini. Yes, it is a Gemini model, but which one?

[1:24](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=84s) No, it's not Live. That's incorrect.

[1:28](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=88s) Yes, is it Gemini TTS? Is that what you were about to say? No, okay. Um, so Gemini Text to Speech, uh, actually a really incredible model for expressive speech. You have inline audio text you can steer it with. Um, you can stream the speech. It's, uh, a lot lower latency now than it used to be. So, uh, if you are in the market for some speech, uh, text to speech that is, Gemini TTS is a good

[1:58](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=118s) model. Now, let's have a look at, uh, some of the things we're doing today. Uh, first of all, my name is Thor. I work on the developer experience at Google DeepMind, so that mostly includes Gemini API and Google AI Studio. So if you're using the Gemini models through the API, uh, you know, let me know if you have any feedback. Uh, I have my colleague Philip here as well, uh, who's working on managed agents on the API. So

[2:28](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=148s) if you're using that, uh, let us know as well. So this is a bit of a, a, a pre-look at the workshop we're doing tomorrow. Um, so tomorrow we'll do kind of Gemini API, especially focused on, um, live agents, uh, conversational agents, real-time, uh, conversational agents, multimodal real-time conversational agents. We're just packing everything in there. Um, and so what we can look at, actually, there's,

[2:58](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=178s) uh, been a lot of things that shipped. And so specifically what we want to look at is the multimodality of Gemini and the Gen Media models. Uh, so actually, this slide is already out of date because this morning we shipped a new model, uh, that is called... Anyone know which model did we ship? We shipped OmniFlash, which is, uh, a video generation and editing model. And we shipped NanoBanana Lite. So NanoBanana Lite is a really

[3:28](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=208s) exciting model because it cuts down the latency of image generation and image editing quite significantly. And so what I've done, we'll, we'll do a couple demos here. Uh, and one of the demos is, um, this example here, which I call the, uh, Gemini NanoBanana story canvas. Uh, and I just vibe-coded this up, uh, just recently, and I'm having some trouble with my mouse here. Um, but so what we can do is we can

[3:58](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=238s) come up with a story with, um, Gemini Live, and then have it use NanoBanana Lite to, to illustrate that story for us. So anyone, you know, this is a little bit like improv theater now. Anyone has a storyline they want to start with?

[4:15](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=255s) Where are we? Somewhere... Okay, let's say, um,

[4:23](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=263s) we're in Taipei in the 1980s.

[4:27](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=267s) The neon lights of Taipei's night market pulsed with an almost magical energy, reflecting off the wet pavement and the chrome of passing scooters. Rain had just passed, leaving a humid mist that hung in the air. Amidst the throng of shoppers and vendors, a young woman clutched a small, strangely glowing object under her jacket.

[4:48](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=288s) Something very exciting happened in 1980s Taipei. A new kind of drink emerged.

[4:54](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=294s) She paused at a stall where a man was vigorously shaking a metal cup, creating a frothy, milky concoction. He handed a cup to a customer, who took a long sip with wide eyes, savoring the strange tapioca pearls at the bottom. This was the birth of bubble tea, and its sweet, chewy magic was spreading like wildfire.

[5:18](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=318s) So what you can see here is, uh, from the tool call. So Gemini Live is getting a tool call.  Um, and then it's using the Gemini Interactions API to generate an image on the fly. Now, it still takes kind of, you know, two to three seconds for the actual image generation, but that is, um, pretty impressively fast. And, um, what you can look at is if you're looking at the code here, uh, we're using the Interactions API. And so the Interactions API

[5:48](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=348s) is exciting because it is a stateful API, which means we can chain our interactions very seamlessly. So you could see here the first image generation that we were doing, uh, it didn't have a previous interaction ID- But then the next generation that we're doing, we're passing this previous interaction ID to the interactions API, which means that we have the previous context of the previous interaction, and we can continue that in the

[6:17](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=377s) generation. So what that means specifically for our demo here is that you can see is our first, you know, interaction, uh, it generated an image with a very specific kind of visual feeling and also with characters, right? And so we can continue that interaction, and so we can keep the same consistency kinda across these generations. So if you haven't tried out the interactions API, uh, it is our unified API for interacting with all of the Google DeepMind models as well

[6:47](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=407s) as agents. Uh, and so do try that out. It is a, it is a really, really cool, um, way to interact with everything that Google DeepMind has. Okay. So this was, you know, a bunch of things that happened. Um, so this was kind of on the Gen Media side. We shipped, uh, NanoBanana Lite this morning. Now, we also recently shipped on the audio side our live translation model. Has anyone tried out Gemini 3.5 Live Translate?

[7:18](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=438s) Yes. Okay, a couple folks. Uh, that's fantastic. So what we can do is, uh, we can try that out as well. We can play, uh, a little demo. And so what I'm gonna do here is I'm gonna create this live translation session, and I want you to scan, uh, this QR code on your phone. And so what this is doing here is if I now enable, um, my microphone, so this is getting the, the

[7:47](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=467s) microphone of my speech, um, here into the session. And now if you go in and you select your preferred language, so we're seeing here we're getting some German, so someone is listening to me now speaking in German. And so if you are, uh, now listening to this, so you probably need some headphones. So if you have some noise-canceling headphones, you can now listen into this talk in your preferred language. And so Gemini Live Translate is a really

[8:17](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=497s) incredible model. It's near real-time translation, so as I'm speaking, as soon as the model is recognizing kind of the context of what I'm speaking about, uh, it will then start translating in real time to your desired language. So I see someone is listening in there. Do, do you get some audio out? Can you hear it? Yeah. Does it sound good? Fantastic. See. Um, so this is, this is really exciting because, you know, in a conference like this, obviously we

[8:47](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=527s) have a very, very international audience. And so I can speak in my preferred language, so I could switch to German now, for example.

[9:08](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=548s) Live.

[9:11](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=551s) Power from, uh, Gemini Live Translate. Um, it is a, a really incredible model. Um, it automatically detects the language the speaker is speaking and then translates it in real time, uh, into our preferred language. Okay, we have, uh, a very international crowd here today. Uh, who's listening in, in, uh, Chinese? Are you listening in Chinese?

[9:37](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=577s) Okay. Uh, I didn't understand that, but

[9:43](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=583s) Okay. Um, so that is Gemini Live Translate. Um, if you wanna try it out, you can add it to your, um, applications now. It is available on the Gemini API, so do try that out. Uh, it is a really incredible model, and, uh, would love to hear kind of what you're, what you're building with that. Now, obviously we're looking at, uh, agents, right? Can your agents, uh... What's the, the title of this talk? Gemini made up

[10:13](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=613s) this title, so I, uh, Can Your Agent Hear Me Now? I think that, that is the title. So our most recent real-time conversational model is called Gemini 3.1 Flash Live. Now, if you've talked to Gemini before in the Gemini app, that is exactly the model that you're speaking to. And so we can do a little demo here. So this demo was vibe-coded by Gemini in Google AI Studio. And if you remember the good old

[10:43](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=643s) times, you know, when you were listening to radio, you know, not too long ago, you could phone up the, the, the radio host, and you could ask for a song, right? You could wish for a song. So it's the same idea here. We're using Gemini Live to be our radio DJ, and then we're giving Gemini Live an API, a function call to the Gemini API to then use Lyria 3, which is our music generation model, to then generate some music

[11:13](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=673s) for us on demand. Okay. Let's, uh, try this out. We're, we're just going live demo to live demo here. Let's, let's see.

[11:22](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=682s) Well, hello there. You're live on the live jukebox right now. What absolute banger are we cooking up for you today, mate? To get this track properly sorted, I need a bit more of an idea. What genre are we thinking? Upbeat synth pop, smooth jazz, something with a bit more welly? And what's the vibe lyrics-wise? Give us some details.

[11:39](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=699s) Yeah, can we get, like, a really high energy reggaeton, please, about the AI Engineering World's Fair here in San Francisco?

[11:50](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=710s) Now, that sounds like a proper belter, mate. High energy reggaeton about the AI Engineering World's Fair? Brilliant. Right, to make this absolutely top of the pops, any specific bits of tech or big names you want thrown in, or are we just going for that huge festival vibe?

[12:05](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=725s) Yeah, just surprise me, please.

[12:09](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=729s) You got it, my friend. Prepare for a huge musical surprise. Generating that track right now. Mixing that reggaeton rhythm with the tech buzz for the world's fair. Watch this space, it's gonna blow your headphones off. Boom.

[12:20](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=740s) San Francisco, la bahía, con energía. AI Engineerin' fair, what a feeling, let's go. Siente el dembow, flow futurístico on the floor. Neural networks firing up, groundbreaking for sure. Clever algorithms dancing, la nueva sensación. Tech pioneers mingling, this is innovation. Bass line is pulsing heavy, can you feel the heat? Synth stabs electrifying, moving your feet. Celebrando la tecnología, no vamos a parar. En la ciudad con niebla, this is how we are. Feria de AI, la fiesta. ¡Dale! Feria de AI, la fiesta. ¡Wepa!

[12:50](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=770s) Yeah, that, that was pretty cool. You went, uh, multilingual there as well. Nice one. Cheers.

[12:57](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=777s) Cheers, mate. Glad you like the flavor. Keep those big ideas coming. This jukebox never sleeps. What we spinning next, then? Feeling-

[13:04](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=784s) All right, um, yes, you can try this out yourself. Uh, it is, is a, a demo that was built in Google AI Studio. Uh, so if you want, you can fork this in AI Studio and continue yourself. Um, I know this application might not necessarily be that useful for your business context, you know, if that's something you're looking to do. But I think the individual components are definitely useful. So if you're building real-time conversational agents,

[13:35](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=815s) uh, you know, you want them to be multilingual. You want them to have capabilities, uh, tool calls. So in this example, we're, we're just giving the agent a tool call to generate music with Lyria 3, uh, on demand. Now, obviously, this could also just call out to your CRM and get some, you know, customer information, or it could create, you know, a new ticket on the back end. Um, so, you know, really what I want you to take away is kind of this real-time, multimodal conversational, uh,

[14:05](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=845s) experience that you can build. And so the way that you can build with this model, so the, the Gemini 3.1 Flash Live real-time conversational model, is through the Live API. So the Live API is a low latency real-time voice and vision, uh, API, so you can build multimodal, uh, live real-time interactions. And so what this means is you have this stateful WebSocket API

[14:35](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=875s) where you create a session, and then over this WebSocket, you can send, in real time, text, audio, and video frames to the model, and the model will then respond in real time back in audio chunks. And these audio chunks you can then stream, uh, in your application. Uh, and in addition to the audio, you also get the transcript of what was said. So if you've worked with voice agents before, this is a little bit different from

[15:05](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=905s) kind of your cascading voice agent pipeline because this is a native audio model. So the model itself does not go through text, but rather the intelligence is baked into the audio model. So the audio model can understand the speech naturally, uh, and natively, and can then respond back in any of the 90 supported languages. So this is what allows us to, to switch very fluidly between different languages.

[15:36](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=936s) Um, so kind of the key features here, yeah, multilingual support, so 90, uh, different languages currently supported in the preview. Um, we can barge in. We can interrupt the model, right, with our DJ is talking a bit too lengthy. Uh, we, we can, we can barge in, interrupt it. We can give it, uh, the tools, and we have the audio transcription as well. Now, if you're building real-time conversational applications or real-time, um, multimodal applications, uh, you know, real-time

[16:06](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=966s) media can be quite challenging to scale. So we have a great lineup of partners. So, you know, LiveKit, Pipecat, they're all actually here in the expo as well. I recommend you go visit them. They do some really, really cool stuff to allow you to get up and running quickly with the Gemini Live API. Okay. So if you're now, um, you know, you're like, "Okay, this sounds great. I want this," um, the easiest way to try this out is through Google AI Studio.

[16:36](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=996s) So we spend a lot of money on this domain, ai.studio, so I do appreciate if you use that one, um, so we can get some ROI there. So ai.studio/live. You can try out the model. You can talk to it, and you can understand the capabilities that are available. So let's do that now. Uh, so we're just going to ai.dev. We also have ai.dev. Um, yeah, please do use that as well. That one was, was also not, not so cheap.

[17:07](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1027s) Uh, there we go, ai.dev/live. And so here, uh, we can also try out the Live Translate model if you want to, uh, but we can here select our Gemini 3.1 Flash, uh, Live model. We can select a different voice. Uh, for example, I like Puck here. And then also we can enable grounding with Google Search. So obviously the model is trained on a lot of information, but it does not have access to real-time information, right?

[17:37](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1057s) And so by giving it access to Google Search, it can actually go off, and it's basically like a built-in function call. And if we asked it about today's weather or, you know, why Germany played so, uh, well in the World Cup, uh, we can, we can do that, and it can use Google to, to look that up. Okay, let's give this a try.  Hey, can you hear me?

[18:01](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1081s) Yes, I can hear you perfectly. How can I help you today?

[18:04](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1084s) Can you see me?

[18:07](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1087s) Yes, I can see you clearly. You're wearing a black shirt and a black hat, sitting in front of a blue background. Anything specific you wanted to show me?

[18:15](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1095s) Yeah. What does it say on my sweatshirt?

[18:18](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1098s) I can see the word Gemini printed in yellow letters. Anything else you'd like me to identify?

[18:23](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1103s) Yeah. Was, uh, heißt das denn auf Deutsch?

[18:26](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1106s) Das heißt auf Deutsch Zwillinge. Möchtest du noch etwas übersetzen?

[18:36](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1116s) Das bedeutet, ich kann dich sehen, Oma. Oh, ich sehe dich. Ähm, du siehst gut aus, Oma. Möchtest du noch etwas übersetzen lassen?

[18:44](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1124s) Ah, okay. I think my, uh,  my Mandarin didn't work out so well there. Uh, can you also speak Mandarin, please?

[18:53](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1133s) Yes, I can speak Mandarin too. What would you like to say now?

[19:12](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1152s) Oh.   All right. If you don't read Mandarin, I do apologize. I was just asking the model what it has eaten, uh, and it just said it doesn't need to eat because it's, it's a model. So, uh, yeah, there we go. All right. So this is the easiest way for you to get started with, uh, Gemini Live, ai.studio/live. If you want to learn more about building with Gemini and the Gemini Live

[19:42](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1182s) API, join me tomorrow morning in, uh, room 2009, I think. Check the schedule. It, it does change every once in a while. Uh, but other than that,

[19:55](https://www.youtube.com/watch?v=xfPVBzDYjH4&t=1195s) . Thank you. Cheers.
