# Any-to-Any: Building Native Multimodal Agents

Patrick Löber

AI Engineer Europe 2026

[Talk page](https://ai.engineer/talks/GIRpQEfYf3U-any-any-building-native-multimodal-agents) · [Video](https://www.youtube.com/watch?v=GIRpQEfYf3U)

## Any-to-Any: Building Native Multimodal Agents

A NotebookLM-style study agent shows how to combine mixed-source understanding, tool-selected image and speech generation, and real-time conversation.

Basic JavaScript and familiarity with API requests and function calling will help you follow the implementation examples.

### What does an any-to-any agent actually need?

[0:31](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=31s)

How would you build a small NotebookLM-style application that understands source material and turns it into useful learning materials? Patrick Löber, who introduces himself as a member of the technical staff working on the Gemini API and AI Studio at Google DeepMind, approaches that problem through multimodal understanding, generation and real-time interaction.

**Gemini** accepts more than text: the API supports code, images, audio and video, alongside capabilities involving URLs and Google Search. Its output capabilities span text, images, speech, video, function calls and code. That breadth makes an any-to-any application possible, but the diagram describes an API ecosystem rather than a single model with every capability.

![Diagram with text, code, image, audio, video and other inputs flowing into a central multicolored symbol, with output branches on the right.](https://ai.engineer/api/talk-essay-media/d8e2629adcffaef047a44b19ce6fd2c13402c7210509ffb1cbea3aa5c2fc6d5b)

The inputs and outputs of “any-to-any” interaction.

In the model lineup described here, **Gemini 3** handles multimodal understanding and returns text. Specialized models handle native image and speech generation: **Nano Banana**, for example, supplies image generation based on Gemini. The distinction matters architecturally: an application must connect the reasoning model to the appropriate generator. Löber also points to **Gemma** as a local option, tentatively describing text, image and video inputs, with audio available on smaller models.

### Synthesize the sources, then choose the assets

[2:56](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=176s)

NotebookLM supplies the product reference: feed in several sources, then generate an audio overview or an infographic that helps explain them. Here, the goal is to let the agent decide what to create. **The distinction between an agent and a fixed workflow is who chooses the next asset.** A reasoning model makes that decision and invokes specialized models through function calls.

The architecture has two phases:

1. **Understand the sources.** Turn the mixed inputs into a synthesized study guide.
2. **Generate supporting materials.** Give Gemini the study guide and tools for producing images and speech. After a tool returns, the model considers whether the materials are sufficient or whether another asset would help.

The loop ends with a combination of text, visuals and audio, rather than a requirement that every source always produce every output type.

![Agentic architecture diagram showing inputs becoming a study guide, a function-calling loop for image and speech generation, and text, visuals and audio outputs.](https://ai.engineer/api/talk-essay-media/9afc5ea47a20bc200ace2e17fefcf9b90dba8e2f48e2f8b6a706ccf09aeee0fb)

Study-guide synthesis followed by an asset-generation loop.

The concrete study topic is [Attention Is All You Need](https://arxiv.org/abs/1706.03762). Inputs might include the paper as a PDF, images, a lecture or tutorial video, and voice memos. The useful capability is **cross-modal understanding**: the model should connect information across those sources, not merely summarize each file independently.

### Put the sources in one generation request

[4:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=290s)

The SDK path starts with an API key from AI Studio and an SDK installation. Upload the PDF, video and MP3 through the File API, or supply smaller files as inline data. Löber also recommends the [Gemini API development skills](https://github.com/google-gemini/gemini-skills), which give a coding agent guidance for working with the API.

The central call is `client.models.generateContent`: place the sources and the instruction together in `contents`, explain what the files represent, and ask for a synthesis. Löber names `gemini-3-flash` in the walkthrough. The JavaScript below retains that Generate Content API style while taking the callable model identifier from configuration. Current documentation has newer model names and, in some examples, the Interactions API; those examples are not the API sequence shown in this recording.

```javascript
import { GoogleGenAI, createPartFromUri } from "@google/genai";

const client = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
const model = process.env.GEMINI_MODEL;
if (!model) throw new Error("Set GEMINI_MODEL to an available model ID");

async function uploadReady(path) {
  let file = await client.files.upload({ file: path });
  while (file.state === "PROCESSING") {
    await new Promise(resolve => setTimeout(resolve, 2000));
    file = await client.files.get({ name: file.name });
  }
  if (file.state === "FAILED" || !file.uri || !file.mimeType) {
    throw new Error(`File unavailable: ${path}`);
  }
  return createPartFromUri(file.uri, file.mimeType);
}

const sources = await Promise.all([
  uploadReady("attention-is-all-you-need.pdf"),
  uploadReady("transformer-lecture.mp4"),
  uploadReady("study-notes.mp3"),
]);

const response = await client.models.generateContent({
  model,
  contents: [
    ...sources,
    "Analyze these resources together: the Transformer paper, a lecture, " +
      "and my voice notes. Create a study guide that connects their " +
      "explanations and identifies concepts needing further explanation.",
  ],
});

console.log(response.text);
```

The prompt supplies the relationship between the files. That relationship is what turns a collection of uploads into a request for a coherent study guide.

### Budget the media context and repeated queries

[6:09](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=369s)

Audio understanding can also serve a simpler task: transcription. Löber recommends asking **Flash** or **Flash Lite** directly to generate a transcript of an audio file. A separate transcription pipeline is not required for this use of the model.

Löber gives an audio rate of 1,920 tokens per minute and a one-million-token context budget, then describes that as more than nine hours. The arithmetic instead gives approximately 521 minutes, or 8.68 hours, before other input overhead. The [audio documentation](https://ai.google.dev/gemini-api/docs/audio) separately lists a 9.5-hour maximum per prompt; that stated limit does not make the two numbers in the talk reconcile.

For video, Löber gives a rough capacity of one hour and notes that configuration can extend it. The [video documentation](https://ai.google.dev/gemini-api/docs/video-understanding) makes the relevant condition explicit: for a one-million-token context window, it lists up to one hour at default media resolution or three hours at low resolution. Resolution therefore belongs in the input-budget decision. You can also narrow the task to a time interval—for example, analyze only minutes 5–15—use the File API for larger uploads, or supply a YouTube URL directly.

**Context caching** becomes useful when the same long sources support repeated questions. Löber reports 90% cost savings for repeated queries over long loaded files using context caching. Treat that as a caching claim, not a guaranteed reduction in the total application bill: the [caching documentation](https://ai.google.dev/gemini-api/docs/generate-content/caching) makes savings dependent on cache hits, while explicit caching also incurs storage costs tied to its lifetime. At this point, the application has the first essential capability: reading the source collection and producing a summary.

### Generate infographics and a spoken explanation

[7:56](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=476s)

The second phase uses Gemini as the reasoning model, with function calls connecting it to specialized generators. Once an asset has been generated, the loop can assess whether it needs more material. Image generation uses the same `client.models.generateContent` entry point, with a different model and a prompt describing the desired picture.

The image model is **Nano Banana 2**. Löber calls it Gemini 3.5 Flash image preview, whereas [Google’s launch announcement](https://blog.google/innovation-and-ai/technology/developers-tools/build-with-nano-banana-2/) identifies Nano Banana 2 as **Gemini 3.1 Flash Image**. Use the model’s documented identifier rather than turning the spoken name into an API string. The demonstrated task is straightforward: ask for an infographic and receive a slide-style graphic that explains the material.

Speech generation follows the same division of labor. Löber describes a text-to-speech model based on **Gemini 2.5**, with configuration for two speakers so the result can resemble a podcast conversation. He plays an excerpt that introduces the Transformer architecture and the 2017 paper, describing the complete output as a two-minute explanation of transformers. This is the study guide becoming another kind of learning material, not merely being read aloud in the interface.

### Let the model choose where each medium helps

[9:55](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=595s)

A function declaration tells the reasoning model what a tool does and what arguments it accepts. Give each tool a name, a useful description and a parameter schema. The image example needs one string: a detailed description of the desired image. Define an analogous speech tool and register both in the generation request.

Then pass in the synthesized study guide and assign the model the role of a research partner. Its job is to enhance the guide with materials where they help:

- **Visual explanation:** call `generateImage` for a concept complex enough to benefit from a diagram.
- **Audio explanation:** call `generateSpeech` for a section that would benefit from an audio summary.

The tool descriptions advertise capabilities; the prompt provides the criteria for choosing them. This JavaScript function constructs that decision request from the study guide produced in the first phase:

```javascript
function buildAssetRequest(model, studyGuide) {
  const functionDeclarations = [
    {
      name: "generateImage",
      description: "Generate a diagram or infographic for a study concept.",
      parameters: {
        type: "object",
        properties: {
          prompt: {
            type: "string",
            description: "Detailed description of the image to create.",
          },
        },
        required: ["prompt"],
      },
    },
    {
      name: "generateSpeech",
      description: "Generate a spoken summary of a study-guide section.",
      parameters: {
        type: "object",
        properties: {
          prompt: {
            type: "string",
            description: "Content and delivery instructions for the audio.",
          },
        },
        required: ["prompt"],
      },
    },
  ];

  return {
    model,
    contents: [
      "You are a research partner. Enhance this study guide with useful " +
        "materials. Call generateImage for concepts that need a visual " +
        "diagram. Call generateSpeech for sections that benefit from " +
        "an audio summary.\n\n" + studyGuide,
    ],
    config: { tools: [{ functionDeclarations }] },
  };
}
```

These declarations let the model request an asset; they are not the image or speech implementations. The application connects the selected function to its specialized generator.

### Why native understanding matters to generation

[11:18](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=678s)

Löber attributes native generation’s usefulness to the training it shares with the main Gemini models. The generator can use knowledge of the world and the meaning of the input, rather than treating a request only as a description of visual appearance.

Two examples make that distinction concrete:

- **A viewpoint from a map.** In a **Nano Banana 1** example Löber found on Twitter, arrows drawn on a map specify a viewing direction. The request asks for a picture of what would be visible there, and the output depicts the Golden Gate Bridge. The task requires interpreting the location and direction before generating the view.
- **Corrections on homework.** A Nano Banana 2 example returns an image of math homework with corrections. The desired output depends on understanding the mathematics as well as producing the annotated picture.

Löber adds that the image models can render code in images. For speech, the corresponding expressive capabilities are multilingual output, accents and tone. The British-accent sample uses regional wording about getting started on multimodal agents and then going to the pub; another accent sample follows. The room’s reaction favors the second example, while Löber explicitly notes that not every accent is supported. These examples show why voice instructions are part of generation, alongside the words to speak.

Understanding plus generation now supplies the core NotebookLM-style application: source material becomes a study guide, and the guide drives the creation of supporting media. Real-time interaction is an additional capability on top of that foundation.

### Add a conversation that can hear and see

[13:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=830s)

The **Live API** adds real-time interaction. Löber introduces **Gemini 3.1 Flash Live** as a native audio-to-audio model: audio enters and audio leaves a single architecture, rather than passing through a cascade of separate models. He connects that architecture to more natural-sounding conversation.

With limited time, he plays a prerecorded demonstration from his colleague Thor rather than attempting a live session. The interaction starts with speech, then adds a camera feed. When the demonstrator asks whether the assistant can see him, it describes a dark jacket over a blue shirt. The useful transition is from conversation about an unspecified scene to an answer grounded in the current camera view. Löber also tentatively mentions configuring the accent.

![Google AI Studio showing a camera preview, the question “Can you see me?” and a response describing the person, with video playback controls overlaid.](https://ai.engineer/api/talk-essay-media/d7c7d4afcf02b32893052967de45e9f205ade3be9749a1c9f7268c93087997cb)

A camera-enabled conversation in Google AI Studio.

AI Studio’s Live interface is the suggested place to try the interaction. Löber then shows a code slide and points again to development skills for implementation guidance. This completes the progression from understanding uploaded sources, through generating assets, to discussing material in real time.

### Extend the same architecture beyond study tools

[15:22](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=922s)

The study application is one use of a transferable pattern: understand heterogeneous inputs, reason over the result, and invoke the output capability the task needs. Löber closes with three extensions. A multimodal embedding model maps different modalities into a shared vector space for multimodal search. **[Gemma 4](https://ai.google.dev/gemma/docs/core)** offers a local multimodal-understanding option. **Veo** adds video generation with native audio. Each expands a different part of the application—finding source material, running understanding locally, or producing another output medium—without changing the central separation between understanding, deciding and generating.

## Resources

- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — The Transformer paper used as the study agent's source topic.
- [Gemini API development skills](https://github.com/google-gemini/gemini-skills) — Installable coding-agent guidance for Gemini API development and real-time Live API applications.
- [Build with Nano Banana 2](https://blog.google/innovation-and-ai/technology/developers-tools/build-with-nano-banana-2/) — Google's introduction to Nano Banana 2, also named Gemini 3.1 Flash Image.
- [Gemini audio understanding](https://ai.google.dev/gemini-api/docs/audio) — Current guidance for audio uploads, inline input, transcription and audio token accounting.
- [Gemini video understanding](https://ai.google.dev/gemini-api/docs/video-understanding) — Current video-input guidance, including how media resolution affects context capacity.
- [Context caching for Generate Content](https://ai.google.dev/gemini-api/docs/generate-content/caching) — Implicit and explicit caching, cache-hit accounting and storage lifetime for the API style used in the talk.
- [Gemini image generation](https://ai.google.dev/gemini-api/docs/image-generation) — Current image-generation model names and API guidance for the Nano Banana family.
- [Gemma 4 model overview](https://ai.google.dev/gemma/docs/core) — Open-weight multimodal models, supported inputs and local deployment memory considerations.

## Slides

### AI Engineer Europe

The recording opens with the AI Engineer Europe event logo.

[0:00](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=0s) [AI Engineer Europe logo in white on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/24e440bf5dfd4ecccd4fffdc9ef9243be07fff10b16e87c6b80fbc594828e3ea.webp)

AI Engineer
EUROPE

### Presenting sponsor

Google DeepMind is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=5s) [Google DeepMind presenting sponsor card.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a43b90b83b6eca77e82f00387f025461fb2cd84d99ed7f55e0f7cc2bbdddc7d.webp)

PRESENTING SPONSOR

Google DeepMind

### Platinum sponsors

Braintrust, WorkOS, and OpenAI are platinum sponsors.

[0:10](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=10s) [Platinum sponsors Braintrust, WorkOS, and OpenAI.](https://assets.ai.engineer/sites/aiecode2025/sha256/124a6d271f12dad563d0b3b5b9b9a89bea7230878f0d771617e634c110bc0e77.webp)

PLATINUM SPONSORS

Braintrust
WorkOS
OpenAI

### Overview

Patrick outlines multimodal understanding, generation, and real-time interaction before building a small NotebookLM clone with the audience.

[0:43](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=43s) [Overview lists multimodal understanding, multimodal generation, real-time interaction, and building an example app.](https://assets.ai.engineer/sites/aiecode2025/sha256/52941c8f5a9001b22ba11814ed4206b06b344ffa54e126cd5137bb42d5f89943.webp)

Overview

• Multimodal understanding
• Multimodal generation
• Real-Time interaction
• Build an example app

AIE
Google DeepMind

### What “any-to-any” means

The diagram maps Gemini API inputs such as code, images, audio and URLs to possible outputs including text, video and function calls.

[0:57](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=57s) [Diagram shows multiple inputs, including text, code, images, audio, video, speech, API information, and URLs, leading to text, image, audio, video, function call, and code outputs.](https://assets.ai.engineer/sites/aiecode2025/sha256/81e6ca83ddf6cace7e0b897b9d68dd68db7dcce1b7c3907128533399777c6340.webp)

What “any-to-any” means

Inputs:
system instructions
text
code
image
audio
video
speech
API information
URLs

Outputs:
text
image
audio
video
function call
code

Other visible text:
AIE
Engineering the future of AI

[1:12](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=72s) [Diagram showing text, code, images, audio, video and other inputs flowing into a model, with text, images, audio, video, function calls and code as outputs.](https://assets.ai.engineer/sites/aiecode2025/sha256/ee129549a93916f8c2fcd909b9de69a6a2c6172554ad6a7692dc9574f510c661.webp)

What “any-to-any” means

Inputs:
system instructions
text
code
image
audio
video
speech
API information
URLs

Outputs:
text
image
audio
video
function call
code

Other visible text:
AIE
Braintrust
WorkOS
OpenAI

### Native multimodal models

Gemini accepts multiple input types and outputs text. Separate models handle image generation, speech, live audio, video and embeddings; the speaker describes a single any-to-any model as a future vision.

[1:44](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=104s) [A list maps Gemini, Nano Banana, Gemini TTS, Gemini Flash Live, Gemma 4, Veo 3 and Gemini Embedding 2 to their input and output types.](https://assets.ai.engineer/sites/aiecode2025/sha256/81a20486c144f9eb8420f5355bbc2406e702f1eda8100919898e58087f6f7ff5.webp)

Native multimodal models

• text, audio, image, video, PDF → Gemini → text
• text, image → Nano Banana🍌 → text, image
• text → Gemini TTS → speech
• text, image, audio, video → Gemini Flash Live → text, audio

• text, audio, image, video → Gemma 4 → text
• text, image → Veo 3 → video+audio
• text, audio, image, video, PDF → Gemini Embedding 2 → embeddings

Interface text:
AIE
AI Engineer EUROPE

### Native multimodal models

The talk focuses on Gemini’s multimodal understanding, image generation and speech generation, with real-time audio through the Live API if time allows.

[2:40](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=160s) [Four bullets describe Gemini understanding, image generation, speech generation and real-time audio conversation.](https://assets.ai.engineer/sites/aiecode2025/sha256/1542b2c64fe7683b5489124783b3bc35b103b021f75756155165a4379321e33a.webp)

Native multimodal models

• Gemini: Understands text, images, audio, video, documents
• Native Image Generation (Nano Banana🍌): Generates & edits images with world understanding
• Native Speech / TTS: Generates natural speech with controllable style
• Gemini 3.1 Flash Live: Native audio-to-audio for real-time conversation

Interface text:
AIE
Engineering the future of AI

### What we’re building today

The proposed NotebookLM-style research agent takes multiple source formats and produces study guides, audio and visuals. The screenshot shows NotebookLM as the reference.

[2:54](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=174s) [A research agent takes text, images, audio, video and PDFs and produces a study guide, audio and visuals; a NotebookLM screenshot illustrates the idea.](https://assets.ai.engineer/sites/aiecode2025/sha256/6aee048cd76a5ea946762dfe8a5da70fe904122baec90e80d6f5df4411578ba1.webp)

What we’re building today

Research agent partner

text, images, audio, video, PDF
↓
agent
↓
study guide + audio + visuals

Interface text:
Introduction to NotebookLM
Sources
Chat
Studio
[illegible]

AIE
Engineering the future of AI

### Workflow vs agent

Instead of following a hardcoded pipeline, the agent decides what to create, calls specialized models through tools, evaluates the results, and repeats.

[3:21](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=201s) [Compares a hardcoded linear workflow with an agentic loop that reasons, decides, calls tools, evaluates, and repeats.](https://assets.ai.engineer/sites/aiecode2025/sha256/8b9bb4056caa62187df8c1349a83e89f8d1ac98124937562de4dd1d703a80e5a.webp)

Workflow vs agent

We want an agent that decides what to create.

❌ Linear workflow
Hardcoded pipeline

✅ Agentic loop
Reasons → decides → calls tools → evaluates → loops

Interface text:
AIE
Engineering the future of AI

### Agentic architecture

Gemini 3 Flash turns inputs into a text study guide, then calls image and speech tools and evaluates whether to loop again before producing text, visuals, and audio.

[3:47](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=227s) [Two-phase architecture creates a study guide, then uses function calls in an evaluation loop to produce text, visuals, and audio.](https://assets.ai.engineer/sites/aiecode2025/sha256/3d32b7a0aab8604620f88b107ac2e3d94594bb6683d92dbaa2efbfcb9b21d729.webp)

Agentic architecture

Phase 1: INPUTS → Gemini 3 Flash → Study Guide (text)

Phase 2: Study Guide → Gemini 3 Flash + Function Calling
↳ Calls: generate_image()
↳ Calls: generate_speech()
↺ Evaluates → Loops or finishes

OUTPUTS: Text + Visuals + Audio

AIE
AI Engineer
EUROPE

### Level 1: Multimodal understanding

The example aims to combine a paper, handwritten notes, a lecture or tutorial, and a voice memo so the model can connect information across sources.

[4:21](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=261s) [Research papers, handwritten notes, video, and audio are inputs for cross-modal understanding.](https://assets.ai.engineer/sites/aiecode2025/sha256/793723374674f641162908e713142ee969665e56d869710bd5b32861da5dce8f.webp)

Level 1: Multimodal understanding

Research paper (PDF)
Handwritten notes (Image)
Lecture/tutorial (Video)
Voice memo (Audio)

→ cross-modal understanding

Paper image: Attention Is All You Need
Video thumbnail: What are TRANSFORMERS?

AIE
AI Engineer
EUROPE

### Code: Multimodal understanding

The example uploads a PDF, video, and audio file, reads an image, and sends all four in one Gemini request. The Gemini skills repository is offered as a coding aid.

[4:51](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=291s) [Python example uploads document, video, and audio files, reads an image, and passes them to a Gemini model for analysis.](https://assets.ai.engineer/sites/aiecode2025/sha256/20102e8950859756c8bf490b4f2f71b2d77239b30ab8fcd121f01f32f6ad185f.webp)

Code: Multimodal understanding

from google import genai

client = genai.Client()

paper = client.files.upload(file="paper.pdf")
video = client.files.upload(file="lecture.mp4")
audio = client.files.upload(file="voice.mp3")
img = open("whiteboard.jpg", "rb").read()

response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents=[
                paper, video, audio, img,
                "Analyze..."
             ]
)

Tip:
github.com/google-gemini/gemini-skills

AIE
Engineering the future of AI

[5:40](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=340s) [Python example uploads PDF, video and audio files, reads an image, and sends them to Gemini for analysis.](https://assets.ai.engineer/sites/aiecode2025/sha256/a050d26dd87f93530c6dac5edbd2c2d2b2b193cdd33c758d190626d5110af78c.webp)

Code: Multimodal understanding

from google import genai

client = genai.Client()

paper = client.files.upload(file="paper.pdf")
video = client.files.upload(file="lecture.mp4")
audio = client.files.upload(file="voice.mp3")
img = open("whiteboard.jpg", "rb").read()

response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents=[
                paper, video, audio, img,
                "Analyze..."
             ]
)

Tip:
github.com/google-gemini/gemini-skills

AIE
AI Engineer
EUROPE

### Practical tips

Audio uses 1,920 tokens per minute. Video has an approximate one-hour maximum, or about three hours with low resolution and lower FPS. Timestamps select segments, and the File API handles larger uploads.

[6:08](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=368s) [Tips cover audio transcription, media limits, timestamps, file uploads, YouTube URLs and context caching.](https://assets.ai.engineer/sites/aiecode2025/sha256/6686c86a46f570bef2fe6f8ad4895bd221b4f8bdce13b34183d8a7a6f724e796.webp)

Practical tips

• Audio transcription — prompt “generate a transcript of...”
• Audio: 60s = 1,920 tokens → max ~9.5h
• Video: ~1h max, but ~3h with media_resolution=low and lower FPS
• Audio/Video: timestamps — prompt "analyze from 5:00 to 15:00"
• File API — `client.files.upload()` for large files vs inline for smaller files
• You can pass YouTube URLs
• Use context caching for longer files (90% cost savings)

AIE

### Level 1 checkpoint

Gemini 3 Flash turns PDFs, images, video and audio into text analysis. The checkpoint marks multimodal understanding complete; generation and real-time interaction remain unchecked.

[7:33](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=453s) [PDF, image, video and audio inputs pass through Gemini 3 Flash to text analysis; multimodal understanding is checked off.](https://assets.ai.engineer/sites/aiecode2025/sha256/f6a687c05e94e22df83054146abf9ce5300e295069979a847a2607d3ad42fcfa.webp)

Level 1 checkpoint

PDF + Image + Video + Audio → Gemini 3 Flash → Text analysis

☑ Multimodal Understanding
□ Multimodal Generation
□ Real-Time Interaction

AIE
Engineering the future of AI

### Level 2: Multimodal generation

Gemini 3 Flash calls specialized image and speech models, then reasons over the resulting visuals and audio to decide whether more assets are needed.

[7:47](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=467s) [Diagram of a reasoning model calling image and speech generation models and evaluating the resulting visuals and audio.](https://assets.ai.engineer/sites/aiecode2025/sha256/40540f9ce4e84917828379f822082d659a116c7b2399c6831d9503f81112f3da.webp)

Level 2: Multimodal generation

The agentic loop

Gemini 3 Flash
(reasoning core)

Reason & Evaluate

generate_image()
generate_speech()

Gemini 3.1 Flash Image
Gemini 2.5 Flash TTS

Visuals / Audio

Interface text:
AIE
Braintrust
WorkOS

### Code: Native image generation

The Python example prompts Gemini 3.1 Flash Image, then converts each returned inline image and saves it as a PNG.

[8:25](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=505s) [Python example calling an image generation model and saving the returned image.](https://assets.ai.engineer/sites/aiecode2025/sha256/9e49e13563891f6430e6aec7562abdf4b46853c2187ff88ddec11671cc0c329a.webp)

Code: Native image generation

from google import genai
from google.genai import types
from PIL import Image

client = genai.Client()

response = client.models.generate_content(
    model="gemini-3.1-flash-image-preview",
    contents="Create a picture of ...",
)

for part in response.parts:
    if part.inline_data is not None:
        image = part.as_image()
        image.save("generated_image.png")

AIE
AI Engineer EUROPE

### Text-to-speech example

The presenter plays a text-to-speech example explaining transformers; the audio player shows a 1:51 recording.

[9:22](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=562s) [An audio player displays a speaker icon and playback controls.](https://assets.ai.engineer/sites/aiecode2025/sha256/fd0d7df57b1dd8ca17bc0cf81572842759965a624cf5f63da5c0f6b5b669201f.webp)

Interface text:
AIE
0:01 / 1:51
AI Engineer
EUROPE

### Function calling

The image-generation function declaration gives Gemini a name, description and required prompt string. The model request then configures the available tools.

[9:59](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=599s) [Code declares an image-generation function with a prompt parameter and configures tools in a model request.](https://assets.ai.engineer/sites/aiecode2025/sha256/2daa6e847652c85202a82b9a650b8e7e61f49570535772d1ec6dc3fd4fb1f7dd.webp)

Function calling

# Define the function declarations
generate_image_decl = {
    "name": "generate_image",
    "description": (
        "Generate a visual diagram or illustration to explain a concept. "
        "Use this for complex concepts that benefit from visual explanation."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "prompt": {
                "type": "string",
                "description": (
                    "Detailed description of the image to generate. Include "
                    "style guidance (e.g., 'educational diagram', 'concept map')."
                ),
            },
        },
        "required": ["prompt"],
    },
}

# Send request with function declarations
response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents="...",
    config={"tools": ... }
)

Interface text:
AIE
Engineering the future of AI

### The agent prompt

The prompt asks a research agent to choose which parts of a synthesized study guide need diagrams or audio summaries, then explain what it created and why.

[10:38](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=638s) [A research-agent prompt asks when to create visual diagrams and audio summaries for a study guide.](https://assets.ai.engineer/sites/aiecode2025/sha256/76b76a4e94c104acd0611656dbe1e0d15149ec1398750b56d438cfee55c0343f.webp)

The agent prompt

AGENT_PROMPT = """
Here is a comprehensive study guide synthesized from multiple research sources (PDF, lecture video, audio notes, whiteboard photos):

{study_guide}

You are a research partner agent. Your job is to enhance this study guide with multimodal materials. Review the content and decide:

- Which concepts are complex enough to need a visual diagram?
  Call generate_image for those.
- Which sections would benefit from an audio summary that a student could listen to? Call generate_speech for those.

Be selective - not everything needs a diagram or audio. Focus on the concepts that are hardest to understand from text alone, or the sections that work best as spoken explanations.

After generating all materials, provide a final summary of what you created and why.
"""

Interface text:
AIE
AI Engineer EUROPE

### Why native matters: accents and tone

The speaker plays British and Bavarian accent samples to demonstrate audio generation, while noting that the model cannot produce every accent.

[12:39](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=759s) [Two audio buttons labeled British accent and Bavarian accent.](https://assets.ai.engineer/sites/aiecode2025/sha256/11f2e82fb90fc9bd40f853db89c7ed1cde16dd98bcc12ac36335fd6da872e934.webp)

Why native matters

Multilingual, understands accents and tone

British accent
Bavarian accent

Interface text:
AIE
AI Engineer EUROPE

### Level 2 checkpoint

Gemini 3 Flash turns inputs into a study guide, then uses function calling in an agent loop with speech and image generation. The checkpoint covers understanding and generation; real-time interaction remains unchecked.

[13:29](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=809s) [Two phases describe a study guide and an agent loop, with multimodal understanding and generation checked and real-time interaction unchecked.](https://assets.ai.engineer/sites/aiecode2025/sha256/87c7b1f3c45edeead0b5b57b2a127634f9a471aff3479bfc3343b22eb261360e.webp)

Level 2 checkpoint

Phase 1: Inputs → Gemini 3 Flash → Study guide

Phase 2: Study guide → Gemini 3 Flash + function calling → agent loop
↳ Gemini 2.5 Flash TTS ↔ Gemini 3.1 Flash Image

☑ Multimodal Understanding
☑ Multimodal Generation
☐ Real-Time Interaction

AIE

### Google AI Studio live demo

A prerecorded Google AI Studio demo pairs a live voice conversation with a camera feed. A system instruction asks Gemini to speak in a friendly Irish accent.

[14:43](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=883s) [Google AI Studio shows a live audio conversation alongside a camera feed.](https://assets.ai.engineer/sites/aiecode2025/sha256/6d935bf66a84c6655309f159f2cc83e0132a953a0d71a3111981e426989958ae.webp)

User
0:00 / 0:03
how you today

Model
Thoughts
Expand to view model thoughts

Ah, top o' the morning to ye! I'm doin' grand, thanks for [illegible]
how are things with you?

Stream is live

Run settings
Gemini 3.1 Flash Live
gemini-3.1-flash-live-preview

System instructions
Speak in a friendly Irish accent

No API Key
Switch to a paid API key to unlock higher quota and more features.

Voice
Puck

Media resolution
258 tokens / image

Thinking level
High

Session Context

Interface text:
Google AI Studio
Playground
Home
Enable saving
Build
Dashboard
Documentation
Get API key
Settings
Get code
Start typing a prompt
Tools
Run

## Full transcript

[0:00](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=0s) [upbeat music] Okay.

[0:18](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=18s) Yeah, go. Awesome, yeah. Thank you everyone for joining the session. Uh, if you joined the previous session, we're switching slightly the topic and talking more about Gemini now. But I have two slides about Gemma as well, so to make Gus happy.

[0:31](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=31s) Hey. Um, I'm Patrick. I'm a member of the technical staff at Google DeepMind. I work on the Gemini API and AI Studio. And today, I want to talk about any-to-any building native multimodal agents.

[0:43](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=43s) So I wanna talk about multimodal understanding, multimodal generation, real-time interactions, and then also build an example app together. So at the end of this session, you should be able to build this for yourself, a little NotebookLM clone.

[0:58](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=58s) Um, so what does any-to-any mean? Uh, these are all the capabilities you can do, uh, with the Gemini API. So there's a lot of use cases this enables, uh, because Gemini does not only understand text, right?

[1:12](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=72s) It's natively multimodal, so you can also feed in code, image, audio, video, and then some more like URLs and also Google Search. And then it can not only generate text, but now we're also able to generate images, uh, speech generations, video generations, uh, function calls, and of cour- of course, code generation.

[1:34](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=94s) So yeah, this enables a lot of really, really cool stuff. Um, but this slide is, is sl- slightly giving the wrong impression because actually there are still different models.

[1:46](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=106s) It's not one multimodal model yet. Uh, this is kind of a bigger vision that we have at Gemini, uh, at Google DeepMind to bring more of the generation capabilities also into Gemini.

[1:57](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=117s) But currently, it looks a little bit like this. This is an ugly slide, I know. Um, but yeah, we have the main Gemini model, a series right now, Gemini 3, and it's able to understand multiple modalities, but it only outputs text.

[2:12](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=132s) And then we have different specialized native generation models. For example, Nano Banana for native image generation and speech generation based on the main Gemini models. And I wanna talk about this in a moment a little bit.

[2:26](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=146s) And also, I mentioned Gemma here. Uh, correct me if I'm wrong, but it allows text, image, and video input, and the smaller models also audio inputs, so you can also build multimodal agents locally.

[2:37](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=157s) Um, yeah. So I wanna focus on, on four things, uh, four models. Uh, the multimodal understanding with Gemini, native image gener- generation, native speech generation, and then also if we still have time, a little bit about, uh, the Live API.

[2:56](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=176s) Um, and then build something together or at least give you the building blocks how you can build this, a little NotebookLM clone. Who has used NotebookLM before? Oh, [chuckles] almost everyone.

[3:07](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=187s) Okay, so I don't think I have to explain it. Um, but yeah, you can feed in multiple different sources. Um, and then the audio overview is pretty popular, where you can generate a podcast, uh, to explain topics for you.

[3:19](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=199s) And then also infographics are pretty cool. So we want to build the same thing, and we want to build this as an agent rather than a workflow. So this means that the agent should be able to decide what to create rather than where we hard code the pipeline.

[3:35](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=215s) Um, here we are having a reasoning model that can decide, um, what to create, and then it's hooked up via tool calls or function calls, and then calls the other specialized models.

[3:47](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=227s) Um, so this is roughly how the, the app or the agentic architecture looks like. We have the phase one for multimodal understanding, and then we have the phase two.

[3:59](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=239s) This is where we have the agentic loop, where we use Gemini as the reasoning model, and it can then call different tools, and these will then generate different modalities for us.

[4:10](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=250s) And then it, um, acts in a loop and reasons if we need more assets or if it's good enough. And then in the end, we get text, speech, um, and, and infographics as an output.

[4:23](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=263s) And yeah, I wanna do this as an example with, uh, some learning about Attention Is All You Need paper. So we want to be able to feed in PDFs, images, videos.

[4:34](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=274s) This can be a lecture, for example, or tutorial, and then voice memos. And ideally, we also want this cross-modal understanding, right? That we can, uh, draw information from all the different sources together and ma-- let the model make connections.

[4:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=290s) And it's actually extremely easy with Gemini to achieve this. This is basically the code you need. Uh, who has built with the Google Gemini SDK before?

[5:03](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=303s) Half. Half. Almost half of the room. Yeah. Basically, this is how you set it up. You get your API key, uh, for free at ai.studio, and then you install the SDK.

[5:13](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=313s) We have it available in different languages. And then you can simply upload different files. Uh, like here we are uploading a PDF, a video, and an MP3 file. Or you can also for smaller files directly use it as inline data.

[5:28](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=328s) And as a tip, uh, on the right side, I mentioned, uh, Gemini API skill. So you don't have to know this code now by heart. You can just, yeah, uh, hook up your agent with the Gemini skill and then tell it, um, to create this, and then it should know how to work with the Gemini models.

[5:46](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=346s) Um, yeah. The-- This is basically everything we need. And then we call client.models.generateContent, and here we're using gemini-3-flash. And then we can put everything together into the contents list and tell it, for example, "Analyze all these resources," give it a little bit more information what these resources contain, and then it should generate a summary.

[6:09](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=369s) And then a little bit of, of practical tips or nice to knows for, for understanding. Um, you can also use it to transcribe audio actually. Uh, Flash, and even the smallest ones, Flash Lite, is pretty good at transcribing audio if you just tell it in the prompt, generate a transcript of this file.

[6:29](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=389s) And then maybe nice to know is for audio, uh, one minute of audio translates to, uh, 1,920 tokens, and Gemini has a token, uh, limit of one million. So if you do the math, it translates to more than nine hours of content you can feed in, audio content.

[6:45](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=405s) For video, it's roughly one hour. Um, but there are, um, configurations you can tweak that give you more control, and you can even feed in longer videos. Then you can tell it to, um, look at only different timestamps.

[7:00](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=420s) So for example, only analyze from minute five to minute 15. Um, and then, yeah, you can use the File API that easily lets you upload larger files. You can even pass in URLs, uh, YouTube URLs directly.

[7:14](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=434s) And what's also nice to know is you can, um, combine it with context caching. This is built into the API. Uh, this is especially useful if you're, um, loading longer files into Gemini and doing repeated queries, because then it saves you 90% of the costs.

[7:33](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=453s) Um, so yeah, this is multimodal understanding in a nutshell. So here, doing a quick checkpoint. We're now able to use Gemini to understand all these different resources and generate a summary.

[7:47](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=467s) Um, and actually, the timer is not working, so I don't know how much longer I have. But I think that we are still a little bit good. Uh, so yeah.

[7:56](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=476s) Then the next phase is the, the multimodal generation part. Um, so for this, we're using the agentic loop, where we use Gemini, um, as the brain behind this, and we combine it with function calling.

[8:11](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=491s) And I will show you how to do this in a moment. And then these function call the specialized, uh, native generation models. And then it can reason if these assets are enough or if we need more.

[8:24](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=504s) And the way to do... to use these specialized models is also basically the same code. Once you have the SDKs, you call, again, client.models.generateContent. In this case, we're using Gemini 3.5 Flash image preview. [laughs]

[8:40](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=520s) It's not the nicest model name, but this is actually Nano Banana 2, the more famous model, uh, famous name for it. And then we tell it to, yeah, create a picture.

[8:49](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=529s) Or in this case, we can... It's pretty good at creating infographics, which is pretty cool. Just give it in your prompt, uh, create an infographic, and then it's creating these, these nice, uh, slide graphics for us.

[9:03](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=543s) And similar for text-to-speech. There we have a text-to-speech model, which currently is still based on Gemini 2.5. Um, and you can combine it with different configurations. You can also do, uh, two, uh, speaker, um, audio files.

[9:20](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=560s) So for example, this podcast style. And here is a nice example if the sound works.

[9:33](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=573s) Does it work?

[9:38](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=578s) Neural network architecture.

[9:40](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=580s) Yeah.

[9:40](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=580s) Introduced in a 2017 Google paper called Attention Is All You Need.

[9:45](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=585s) So this is an output of the text-to-speech model we have that then explains, uh, transformers in only two minutes for you.

[9:55](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=595s) And then the function calling, like I mentioned. So basically, to combine Gemini with function calling or tool calling, what you need to do is you create your function declarations.

[10:08](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=608s) So there you give it a name and a description. This helps the model to understand what this function is used for. And then also the different parameters. So in this case, we only want one string, which is, uh, then used for the prompt.

[10:23](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=623s) So the detailed description of how the image should look like. And then you do the same for, uh, the audio, um, generation function. And then if you set up, um, your model call, client.models.generateContent, you configure the tools.

[10:39](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=639s) And then you also need to add this to your prompt. So this is a small example prompt, an agent prompt, um, where you tell Gemini, "Hey, here's the study we synthesized before from the different modalities, and now you're a research agent partner.

[10:55](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=655s) Your job is to enhance the study guide with multiple materials." And then you do... Uh, you tell it the two functions. So, um, decide which concepts are complex enough to need a visual diagram, and for this, call generateImage.

[11:10](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=670s) And which sections would benefit from an audio s- audio summary, and for this, uh, call generateSpeech.

[11:18](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=678s) And yeah, this is basically everything you need to set up the agentic function calling for the multimodal generation part. And I also quickly wanted to touch on why native generation mo- uh, matters.

[11:32](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=692s) So we call this native image generation models, for example, because they are based on Gemini. So all the training or a lot of the training that goes into the main Gemini models are now also available in these models.

[11:46](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=706s) And this allows a, a lot of really, really cool use cases because these models understand the world. Uh, this on the left side is, for example, I found on Twitter, and, uh, example I really like from Nano Banana 1, where you can draw arrows on maps and just tell it, "Hey, um, create a picture of what you

[12:05](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=725s) see here." And since Gemini understands the world here, it's able to, to correctly, um, create a picture of the Golden, uh, Gate Bridge for you. And then on the right side, this is a nice example on the o- uh, from the educational space.

[12:18](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=738s) So you can use Nano Banana In this case, it was Nano Banana 2 to directly, um, correct your math homework, for example, and create pictures with the corrections because it understands math.

[12:31](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=751s) It can even generate code on images for you. So lots of nice use cases. And for the audio models, um, these are multilingual, and they understand accents and tone.

[12:43](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=763s) So you have

[12:44](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=764s) Ay-up, lads and lasses. We're getting stuck into building these multimodal agents today. No faffing about.

[12:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=770s) And one with a-

[12:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=770s) Let's just get 'em sorted so we can all nip to the pub for a proper pint. [laughs]

[12:55](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=775s) Was it a good British accent? Good. Yeah? It was a British accent. A one. Is anyone speaking German here? One, two, three. I have a [REDACTED:origin] accent because you can actually also tell it to create different accents.

[13:10](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=790s) Of course, not every accent in the world, but still. So here's one with a [REDACTED:origin] accent. [speaking [REDACTED:origin]] Was

[13:19](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=799s) it a good [REDACTED:origin] accent? [speaking [REDACTED:origin]] [laughs] So yeah, I would say this is pretty- That's better [REDACTED:origin] than British. [laughs] Okay. Uh, yeah.

[13:35](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=815s) And yeah, again, so quick checkpoint. You now know how to do un- the understanding part and the generation part. And this is basically already the, the NotebookLM clone. And now I quickly wanted to mention now, uh...

[13:50](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=830s) or we also have, um, a model for real-time interaction with it via our, what we call the Live API. And for this, we have a very new model which is also based on Gemini, uh, Gemini 3.1 Flash Live, and we call this an audio to audio model.

[14:06](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=846s) So native, um, audio generation. It's only one architecture. Uh, audio goes in and audio goes out, so you no longer have this cascaded pipeline with different models. And this allows a lot of really, really cool, um, natural sounding interactions.

[14:22](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=862s) I think I don't have time for a live demo, but you can try it at ai.studio/live. And here is a quick video from one of our colleagues, Thor. Um.

[14:36](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=876s) Hey, Gemini. You kind of, yeah- How are you today? ... just start it and start talking to it, and then you can also activate your camera. Ah, top of the morning to ya.

[14:42](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=882s) I'm doing grand. Thanks for asking. And it's also able to understand your video feed in real time. Just enjoying the chat, you know. This is, to me, one of the most impressive features.

[14:47](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=887s) And how are things with you? Um, can you see me?

[14:54](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=894s) Well, as plain as day. I see you there with your [REDACTED:physical_attribute]. I think you can also configure the accent or something. Wearing a grand dark jacket over a blue shirt.

[15:01](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=901s) But yeah, try it out for yourself, uh, at ai.studio/live. And I think that's almost it. Um,

[15:12](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=912s) uh, yeah, this is again how you can do it in the code. But there again, we have a skill for it that you configure, configure, and this... Yeah, now we are at all the three checkpoints.

[15:22](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=922s) And yeah, the pattern is transferable to every, every other field. Um, and maybe a few, uh, shout-outs as well to some other models. I'm not sure if you've seen the keynote this morning.

[15:34](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=934s) Um, but we now have a multimodal embedding model where you can combine all the different modalities into one unified vector space, which allows applications like multimodal search. And then again, you can go local with Gemma 4, and again, have this multimodal understanding, and Veo for, for image, uh, for video with native audio.

[15:54](https://www.youtube.com/watch?v=GIRpQEfYf3U&t=954s) And yeah, so that's it. Uh, thank you, and have fun building multimodal agents. [audience applauding] [upbeat music]
