Gemma, DeepMind's Family of Open Models

What changes when you can download a capable model, run it on your own devices, and fine-tune it for your application? Gemma, Google DeepMind’s family of open models, makes those choices available outside a hosted API. The speaker introduces Gemma 4 a week after its release, starting with the practical meaning of openness: control over where inference happens and how the model is adapted.

The preceding generation, Gemma 3, established the hardware-efficiency baseline. Its models ranged from 1B to 27B parameters, and the speaker describes them as the most capable open models that could fit on a single consumer GPU at their release about a year earlier. The comparison slide pairs LM Arena scores with dots representing how many H100 or A100 GPUs each model needed merely to load. That distinction matters: fitting the weights is a prerequisite for inference, not a measurement of generation speed. The talk does not specify the precision or memory assumptions behind those loading comparisons.

The speaker presents Gemma 4 as Google’s most capable open-model family yet, with distinct deployment roles rather than one model scaled uniformly. The launch lineup includes E2B, E4B, 26B A4B, and 31B. The official model card resolves the introductory size shorthand: the largest dense model is 31B. This is the four-model lineup discussed in the recording; the current card also includes a later 12B variant.

The smallest models target Android phones, iPhones, and even Raspberry Pi devices. The mixture-of-experts model emphasizes speed, while 31B is the choice for maximum capability within this family. The speaker says even 31B can run on a consumer GPU, although he does not identify the GPU or precision. The sizes slide explicitly labels its GPU-consumption column as 8-bit, a condition to retain when reading its memory figures.

The demonstrations turn those deployment roles into concrete behavior. On an Android phone, an agent can select from available skills; the example selects a piano-playing skill and then plays the piano. Another phone performs live coding in airplane mode, with no API calls. These examples put both model inference and the surrounding application behavior on the device.

On a laptop, ten Gemma instances run in parallel through llama.cpp, each generating a different SVG. The resulting images appear together as the agents finish. The speaker reports about 100 tokens per second during the ten-agent laptop SVG demonstration. He does not specify whether that is aggregate or per-instance throughput, nor the hardware, model variant, precision, or context length. The demonstration establishes concurrent local generation; its throughput figure cannot size a different deployment. Offline Android application development is another coding use case he identifies.

The next chart plots total model size in billions of parameters on the horizontal axis and LM Arena score on the vertical axis. The desirable region is the upper left: strong results from relatively small models. The speaker treats Arena as an imperfect proxy for community preferences in general conversation, including helpfulness and friendliness, rather than a complete measure of coding or agent reliability.

Across Gemma 2, Gemma 3, and Gemma 4, the progression he emphasizes is improving capability without increasing size. That makes the next generation interesting for devices already in people’s hands. He skips the detailed benchmark slides and returns to deployment: desktops, laptops, phones, and a community experiment using llama.cpp to try Gemma on a Nintendo Switch. The Switch example illustrates experimentation with new hardware targets, rather than a measured deployment recommendation.

Control also depends on licensing. Earlier Gemma releases drew requests for a conventional open-source license; Gemma 4 adopts Apache 2.0. That change complements the ability to download and modify the models with a familiar licensing framework for building on them.

The 26B model uses a mixture-of-experts architecture, but the E in E2B and E4B means something different: effective parameters. These smaller models use per-layer embeddings, an architecture the speaker says was introduced the previous summer. His spoken total-parameter estimates are approximate; the model card specifies 2.3B effective parameters and 5.1B including embeddings for E2B. The effective count describes the model’s computational footprint, not its entire weight inventory or a complete GPU-memory budget.

Per-layer embeddings supply information through lookup tables instead of requiring the usual dense matrix multiplications for those parameters. This changes where the weights need to reside. The lookup tables can be kept outside GPU memory, using CPU-accessible storage, while the computationally active portion stays on the accelerator. E2B and E4B use this separation to make mobile deployment more practical: a model can contain substantially more parameters than the portion that must remain on the GPU.

In llama.cpp, --override-tensor controls tensor placement. A shell invocation can make the model file and the tensor-name pattern explicit inputs:

: "${MODEL_GGUF:?Set MODEL_GGUF to your local GGUF file}"
: "${PLE_TENSOR_REGEX:?Set PLE_TENSOR_REGEX to the model's PLE tensor pattern}"

llama-cli \
  --model "$MODEL_GGUF" \
  --n-gpu-layers 99 \
  --override-tensor "${PLE_TENSOR_REGEX}=CPU" \
  --prompt "Write an SVG of a piano keyboard."

The override places matching tensors on CPU while requesting GPU offload for the other layers. The pattern must match the actual tensor names in the model file. The tensor-placement discussion supports this CPU-placement syntax, not a Gemma-specific direct-to-disk command. Disk-backed, memory-mapped per-layer embeddings are separately documented in the Gemma 4 edge launch article for LiteRT-LM.

The smallest models accept images, video, and audio. Audio understanding includes speech recognition and speech-to-translated-text: for example, Spanish speech producing French text. For the larger models, the speaker highlights fine-grained video understanding and spatial tasks such as pointing to a llama in a picture or detecting multiple objects. These are different interfaces to the model: transcription produces text from sound, while pointing and detection connect language to locations in an image.

The speaker says Gemma 4 was trained on over 140 languages. Its Gemini-based tokenizer draws on the multilingual research behind Gemini. Training coverage does not imply equal proficiency in every language, but tokenizer design also matters independently of the base model’s knowledge: it determines how a language is represented before fine-tuning begins.

That makes the tokenizer relevant to adaptation for languages with limited digital training data. The speaker gives Quechua in Peru and official Indian languages as examples where developers can bring their own data and train further. Multilingual and multimodal behavior can then combine in a single task, such as extracting or explaining Japanese text inside an image.

About a week after release, the speaker reports 10 million downloads of Gemma 4-based models and more than 1,000 derivatives, including quantizations and fine-tunes. He also reports more than 500 million downloads across the Gemma family. These are adoption figures at the time of the talk. The activity includes popularity on Hugging Face, Unsloth demonstrations of full-repository audits, and experiments placing Gemma on different devices.

The integration strategy is to support the ecosystem developers already inhabit. The speaker names Unsloth, MLX, Ollama, Hugging Face, vLLM, and SGLang as collaborators. A developer fine-tuning with Hugging Face Transformers should be able to stay there rather than switch to Keras. Launch-time support for Gemma and Gemini tooling depends on those maintainers as well as the model team.

Android Studio is the product-level example. Its agent mode helps write code and develop applications; the speaker describes an offline setup backed by Gemma served through llama.cpp, Ollama, or vLLM. He attributes part of Gemma’s Android development capability to Android-related datasets and benchmarks included during training. The local model therefore slots into an existing development workflow, instead of requiring a separate chat application.

Returning to adoption, the speaker corrects an outdated slide’s download total and reports more than 100,000 models derived from the Gemma family. The reasons to create a derivative extend beyond improving a chatbot: developers may want stronger task capability, a different conversational style, or a predictor for a specific context with no conversational interface at all.

Google’s official variants illustrate two forms of specialization:

• Policy screening: ShieldGemma is a family of guardrail models for identifying text or images that violate application policies, including toxic content.
• Medical understanding: MedGemma is described as a multimodal Gemma 3-based family for medical tasks, including radiology and chest X-ray understanding. Its open weights allow further fine-tuning for narrower use cases.

These models change the task the system performs: screening inputs or interpreting medical information requires different behavior from a general conversational assistant.

Community adaptation also serves regional language needs. AI Singapore researches and trains open models for Southeast Asian languages. The speaker then points to Sarvam in India, describing government investment in startups developing national models for official languages. Here, multilingual capability and sovereign AI meet: organizations can adapt a model around local language requirements and control its deployment.

The next example moves from language adaptation to scientific research. The speaker describes a paper he dates to the previous December in which researchers used Gemma 3 to propose cancer-therapy pathways and then tested proposals in a laboratory. He reports laboratory validation, not clinical treatment outcomes. The example expands the role of an open model from answering questions to generating hypotheses that another process can test.

Local deployment answers several different constraints. Finance or legal-review applications may need to keep data on private servers. Subway and airplane use may require inference without connectivity. A Chrome extension could use a local model to interpret what is on screen, while an on-device agent could act through available controls. The speaker’s closing observation is that increasingly complex agentic work can now happen entirely on a phone.

His invitation is concrete: spend an hour within the next two weeks trying current open models and discovering what they can do. For maximum raw intelligence, he still points to Gemini or another API-based model. When local execution matters, however, the available capabilities already support useful applications. His six-to-twelve-month outlook is for more capable models on personal devices, customized with their owners’ data and adapted to their particular needs. The next step is to try the models, build something, and share it.