# Running Gemma 4 On-Device: 40 Tokens/s on iPhone with MLX

Adrien Grondin

AI Engineer Europe 2026

[Talk page](https://ai.engineer/talks/a2muGkT4WD4-running-gemma-4-on-device-40-tokens) · [Video](https://www.youtube.com/watch?v=a2muGkT4WD4)

## Running Gemma 4 On-Device: 40 Tokens/s on iPhone with MLX

A native chatbot shows how MLX packages, compatible model weights, and quantization turn local iPhone inference into a practical app experience.

Basic familiarity with Swift app development and downloadable language-model weights will help with the integration example.

### A chatbot that runs on the phone

[0:15](https://www.youtube.com/watch?v=a2muGkT4WD4&t=15s)

How do you run **Gemma 4** on an iPhone? Adrien Grondin’s starting point is Locally AI, the native chatbot he develops. It runs models on the device through MLX and also offers **Apple Foundation** alongside compatible downloadable models. The opening app views show the three pieces users encounter: a conversation, Apple Foundation, and model management.

![Slide with three iPhone screens: a chat response, a Meet Apple Foundation screen, and a Manage models list.](https://ai.engineer/api/talk-essay-media/75e6fa26222c8eb129cdc7e096fcb0f5fcdb91e5887b5ff704915f817afe8cc1)

Three iPhone views show chat, Apple Foundation, and model management.

Smaller variants of Google DeepMind’s Gemma 4 family make this phone experience possible. Grondin has shown the app running them in social posts; underneath the interface is [MLX](https://github.com/ml-explore/mlx), Apple’s framework optimized for Apple silicon. The same ecosystem extends beyond the phone: Locally AI also runs on iPad and macOS.

### Start with the integration package

[1:49](https://www.youtube.com/watch?v=a2muGkT4WD4&t=109s)

Gemma is one option; **Qwen** and Hugging Face’s **SmolLM** are other model families Grondin points to. For a native iOS, iPadOS, or macOS application, the integration entry point is [MLX Swift LM](https://github.com/ml-explore/mlx-swift-lm), packaged as `mlx-swift-lm`. It supplies the machinery to download and run compatible models. Grondin leaves the detailed implementation to the developer and their coding agent, describing the API as straightforward. He estimates that an iOS app running a model can be assembled in less than ten minutes.

The surrounding ecosystem covers more than text chat:

- **Vision and omni models:** Prince’s [MLX-VLM](https://github.com/Blaizzy/mlx-vlm) provides another route for Python and Mac work.
- **Audio:** [MLX-Audio](https://github.com/Blaizzy/mlx-audio) covers speech workloads, including text-to-speech and speech-to-speech.
- **Image and video generation:** [MLX-Video](https://github.com/Blaizzy/mlx-video) extends the family of tools to generation workloads.

These projects explain the breadth of MLX’s ecosystem; the phone walkthrough remains focused on running a suitably sized language model.

### Choose compatible weights, then pass their ID

[3:12](https://www.youtube.com/watch?v=a2muGkT4WD4&t=192s)

**Installing the framework does not make every model runnable.** The next dependency is a compatible set of weights. Grondin directs developers to MLX Community on Hugging Face, where contributors publish both full-size and quantized versions. At the time of the talk, he estimates roughly 4,000–5,000 uploaded models and says quantizations can appear about 30 minutes after a lab releases a model. Those are his observations about community activity, rather than catalog counts or turnaround guarantees.

His iPhone example is **[Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it)**. The catalog offers multiple representations of its weights: he mentions BF16, MXFP4, and 5-bit and 6-bit variants. The model family identifies what you want to run; the particular repository identifies which representation you will download.

The integration sequence is short:

1. Add `mlx-swift-lm` to the application.
2. Select a compatible model repository from MLX Community.
3. Pass that repository’s ID to the framework, whose Hugging Face integration downloads the model.

For example, the available [4-bit Gemma 4 E2B checkpoint](https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit) gives a concrete identifier to put into a Swift model configuration:

```swift
import MLXLMCommon

let modelConfiguration = ModelConfiguration(
    id: "mlx-community/gemma-4-e2b-it-4bit"
)
```

This configuration selects the repository; the application’s loading and generation code then uses it. The significant handoff is the model ID, rather than a manually assembled download URL or a separate weight-transfer step.

![Hugging Face MLX Community page with model listings in two columns and member avatars at left.](https://ai.engineer/api/talk-essay-media/f5116e16441e63788a7188247432f260516a478624ef52324486c45589dcca41)

The MLX Community page on Hugging Face lists model variants.

### Fit the model to the workload

[4:41](https://www.youtube.com/watch?v=a2muGkT4WD4&t=281s)

For the proposed iPhone use, full-size weights are generally too large. **Quantization** reduces the precision used to represent weights, making a smaller representation available at a potential cost to output quality. Grondin initially mentions 3–8 bits, then settles on **4–8 bits** as his practical range.

| Weight precision | Grondin’s guidance |
| --- | --- |
| Below 4-bit | Output quality often suffers too much |
| 4-bit | His recommended lower bound |
| 8-bit | An option for very small models |

The choice therefore depends on model size as well as precision: the largest precision setting is most plausible when the model itself is small.

A chatbot is not the only useful destination. Grondin describes a **Liquid** model with 350 million parameters running through Shortcuts in his app. At that size, he uses the model for fast, efficient text processing and automation. He does not identify the exact Liquid checkpoint.

For the larger chat example, Grondin reports about 40 tokens per second for Gemma 4 E2B quantized to 4-bit on the latest iPhone. He does not name the iPhone model or provide a reproducible benchmark configuration, so this figure describes his reported setup rather than a hardware-wide expectation.

### What offline generation looks like

[6:01](https://www.youtube.com/watch?v=a2muGkT4WD4&t=361s)

The demonstration turns that throughput into a visible interaction. Grondin describes the phone as running live and offline, streaming its response as it generates. The displayed answer is relatively long; the selected view shows a multi-day Paris itinerary in the chat interface. Streaming lets the user begin reading while generation continues.

![A hand holds an iPhone displaying a dark chat interface with day-by-day travel recommendations.](https://ai.engineer/api/talk-essay-media/85295d9c9b908135c1ff9816d0b294b3145847d2ceaf2c996bd2811784480890)

An iPhone chat displays a multi-day Paris itinerary.

A different interface could wait and present the completed answer. Grondin suggests a four-second wait as an example of that design choice, not a measured completion time for every response. The appropriate presentation depends on how much output the task needs.

Older iPhones can also run these models, although Grondin says they should not be expected to reach the latest-phone figure. He offers 20 tokens per second as an illustrative rate that would still be useful for many applications, rather than reporting a measurement from a named older device. The practical question is whether generation is fast enough for the particular interaction.

### The download is still part of the product

[7:11](https://www.youtube.com/watch?v=a2muGkT4WD4&t=431s)

To try the experience, Grondin points attendees to an App Store QR code and describes Locally AI as free to use. Installing the app is only the first step: users also need the selected model’s weights. Grondin puts typical model downloads at roughly 1–3 GB, depending on the model.

He identifies model size as the biggest remaining barrier. Local generation removes the need to send each request to a hosted model, but it still asks users to download a substantial asset before using it. Smaller, more capable models and improving iPhone hardware make that tradeoff progressively easier.

### From phone inference to a local server

[7:38](https://www.youtube.com/watch?v=a2muGkT4WD4&t=458s)

Grondin expects successive iPhone generations to improve usability further. He then connects the phone app to the desktop ecosystem, announcing that LM Studio has acquired Locally AI. [LM Studio’s blog](https://lmstudio.ai/blog) dates the joining announcement to April 8, 2026.

LM Studio provides a desktop interface for downloading models from Hugging Face and running them locally. It supports both `llama.cpp` and MLX, allowing users to compare the engines. Its model browser shows the selection and download side of that workflow.

![Desktop model browser with a model list on the left and Gemma details and a download button on the right.](https://ai.engineer/api/talk-essay-media/fe7b8035b7ffc7ef29a9160a4b44d96cb1484b1e3e5103ad6121e9d75aa0817c)

LM Studio’s model browser shows a selected Gemma model and download options.

The next step is to expose the running model through a local server and connect another application to it. Grondin describes OpenAI-style and [Anthropic-style interfaces](https://lmstudio.ai/blog/claudecode), including streaming. This changes where inference lives: a phone app can embed the runtime directly, while an application using LM Studio sends requests to a separate local serving process.

### Tool calling and structured generation are separate capabilities

[9:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=542s)

An audience question asks whether the stack supports tool calling. Grondin says yes, while drawing a separate boundary around structured generation.

| Capability | Status described in the talk |
| --- | --- |
| Tool calling | Supported by `mlx-swift-lm` |
| Structured generation | Not yet supported directly |
| Structured-generation add-ons | Packages attempting to add support |

These describe the support boundary at the time of the talk, not a promise about every later package version. Tool calling lets a model request an operation in another system; it does not by itself establish that arbitrary output will be constrained to a required structure.

Grondin mentions Hugging Face’s work on additional packages without naming one. His concrete recommendation is that `mlx-swift-lm` already supports the tool-calling path for connecting to other systems, and he says models have become substantially better at selecting and using tools compared with a year earlier.

### Build an app or use the curated one

[9:43](https://www.youtube.com/watch?v=a2muGkT4WD4&t=583s)

The final questions separate two entry points that can otherwise sound like one setup process:

- **Developers:** Install `mlx-swift-lm` in their own app and obtain compatible model weights from Hugging Face.
- **Users:** Download Locally AI from the App Store and try the experience without setting up a development environment.

The ready-made app removes the integration work, while still requiring the model download discussed earlier.

That convenience also includes a selection boundary. Locally AI does not offer every open-source model: Grondin curates the catalog and checks that included models run correctly on the iPhone. **A model’s availability is not the same as its suitability for a phone.** The app’s final responsibility is to turn compatible weights and a capable runtime into choices that actually work on the user’s device.

## Resources

- [MLX Swift LM](https://github.com/ml-explore/mlx-swift-lm) — Swift package for integrating language and vision-language models into Apple-platform applications.
- [Gemma 4 E2B model card](https://huggingface.co/google/gemma-4-E2B-it) — Google's model card explains E2B's architecture, modalities, effective parameter count and usage.
- [Gemma 4 E2B 4-bit MLX weights](https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit) — MLX Community's 4-bit checkpoint with model-ID loading examples.
- [MLX-VLM](https://github.com/Blaizzy/mlx-vlm) — Run and fine-tune vision-language and omni models on Mac using MLX.
- [MLX-Audio](https://github.com/Blaizzy/mlx-audio) — Speech synthesis, recognition and speech-to-speech tools built on MLX.
- [MLX-Video](https://github.com/Blaizzy/mlx-video) — Image, video and audio generation model tooling for Mac.
- [LM Studio's Anthropic-compatible local server](https://lmstudio.ai/blog/claudecode) — Configure local models for Claude Code through LM Studio's Messages endpoint, including streaming and tool use.
- [Liquid AI LFM2-350M](https://huggingface.co/LiquidAI/LFM2-350M) — A compact 350M-parameter model designed for edge and on-device deployment.

## Slides

### AI Engineer Europe

The recording opens with the AI Engineer Europe logo.

[0:00](https://www.youtube.com/watch?v=a2muGkT4WD4&t=0s) [AI Engineer Europe logo on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/aab59c7da120f8504a96c9b8622254406bbff88706e2388a912d6dcc744c1118.webp)

AI Engineer
EUROPE

### Presenting sponsor

Google DeepMind is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=a2muGkT4WD4&t=5s) [Google DeepMind presenting sponsor card.](https://assets.ai.engineer/sites/aiecode2025/sha256/22e8cb940ef752894fb0f47ce4cfbb7a04b212fd252bdf1347f3f5879fa393f3.webp)

PRESENTING SPONSOR

Google DeepMind

### Platinum sponsors

[0:10](https://www.youtube.com/watch?v=a2muGkT4WD4&t=10s) [Braintrust, WorkOS, and OpenAI platinum sponsor logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/67467f5fe16f0400eb455787eacabd1245e81843e21f92324191574db695041f.webp)

PLATINUM SPONSORS

Braintrust
WorkOS
OpenAI

### Speaker introduction

Adrien introduces himself and points to his Twitter profile for more on on-device computing.

[0:21](https://www.youtube.com/watch?v=a2muGkT4WD4&t=21s) [A speaker portrait and profile QR code appear on the presentation screen.](https://assets.ai.engineer/sites/aiecode2025/sha256/f26779947e66e2e65f0054847aee8262e65f99c367cad853a6c9d7808d8f4409.webp)

@adrgrondin

AIE
Google DeepMind
AI Engineer

### Locally AI

Adrien introduces Locally AI, a native iPhone chatbot that runs on-device models using MLX.

[0:34](https://www.youtube.com/watch?v=a2muGkT4WD4&t=34s) [A blue app icon appears centered on a white slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/4f1824c648a70a007a4e4fc50a460a0dcb67d890b55cfc87a70c241ccc3eec6b.webp)

Interface text:
AIE
AI Engineer
EUROPE
Engineering the future of AI

### Locally AI app screens

Locally AI runs on-device models on iPhone with MLX. The screens show a Paris itinerary, Apple Foundation support, and a model picker with downloadable alternatives.

[0:47](https://www.youtube.com/watch?v=a2muGkT4WD4&t=47s) [Three iPhone screens show a chat, an Apple Foundation introduction, and model management.](https://assets.ai.engineer/sites/aiecode2025/sha256/a3c6ffffa5569606d984f87b8c1c6bfe9c0b001e5bfdc0c658f8f5e40630bef2.webp)

Apple Foundation

I want to plan a 7-day trip to Paris. Please suggest must-see attractions, recommend good neighborhoods to stay in, and create a daily itinerary including restaurants and activities.

Absolutely, Paris is a fantastic city with so much to offer! Here's a suggested itinerary, including must-see attractions, neighborhoods to stay in, and daily activities along with restaurant recommendations.

Must-See Attractions:
1. Eiffel Tower - Iconic landmark and one of the most visited monuments in the world.
2. Louvre Museum - Home to thousands of works of art, including the Mona Lisa.
3. Notre-Dame Cathedral - A masterpiece of French Gothic architecture.
4. Sacré-Cœur Basilica - Offers stunning views of Paris from Montmartre.
5. Champs-Élysées and Arc de Triomphe - Famous avenue and grand monument.
6. Montmartre and Sacré-Cœur - Artistic neighborhood with cobblestone streets.
7. Musée d'Orsay - [illegible] for its collection of Impressionist masterpieces.

Meet Apple Foundation

Locally AI now supports Apple’s on-device large language model, the same model that powers Apple Intelligence.

Plan: a trip to Paris
Explain: a complex topic simply
[illegible]

Manage models

System
Apple Foundation
On-device model by Apple. Excels at a diverse range of text generation tasks, like summarization, entity extraction, text understanding, refinement, generating creative content, and more.
New

Featured
SmolLM 3 (3B) — Download
A model made by Hugging Face. Great for complex reasoning, long conversations, and use in English, French, Spanish, German, Italian, and Portuguese. Recommended for iPhone 15 Pro or newer.
1,73 GB
Reasoning

Gemma 3n (E2B) — Download
A powerful model from Google, optimized for use in everyday devices. Great for content creation, text summarization, and conversational AI. Recommended for iPhone 15 Pro and newer.
2,51 GB
Recommended

Interface text:
09:41
Ask anything
AIE
AI Engineer Europe
Engineering the future of AI

[0:51](https://www.youtube.com/watch?v=a2muGkT4WD4&t=51s) [Three iPhone screens show a Paris itinerary, the Apple Foundation welcome screen, and model choices.](https://assets.ai.engineer/sites/aiecode2025/sha256/d8594167bd299dcb930bb98ddaa594a40219825c7a47e779e9234d67d8bd4279.webp)

Apple Foundation

I want to plan a 7-day trip to Paris. Please suggest must-see attractions, recommend good neighborhoods to stay in, and create a daily itinerary including restaurants and activities.

Absolutely, Paris is a fantastic city with so much to offer! Here's a suggested itinerary, including must-see attractions, neighborhoods to stay in, and daily activities along with restaurant recommendations.

Must-See Attractions:
1. Eiffel Tower - Iconic landmark and one of the most visited monuments in the world.
2. Louvre Museum - Home to thousands of works of art, including the Mona Lisa.
3. Notre-Dame Cathedral - A masterpiece of French Gothic architecture.
4. Sacré-Cœur Basilica - Offers stunning views of Paris from Montmartre.
5. Champs-Élysées and Arc de Triomphe - Famous avenue and grand monument.
6. Montmartre and Sacré-Cœur - Artistic neighborhood with cobblestone streets.
7. Musée d'Orsay - [illegible] for its collection of Impressionist masterpieces.

Meet Apple Foundation

Locally AI now supports Apple’s on-device large language model, the same model that powers Apple Intelligence.

Plan: a trip to Paris
Explain: a complex topic simply
[illegible]

Manage models

System
Apple Foundation
On-device model by Apple. Excels at a diverse range of text generation tasks, like summarization, entity extraction, text understanding, refinement, generating creative content, and more.
New

Featured
SmolLM 3 (3B) — Download
A model made by Hugging Face. Great for complex reasoning, long conversations, and use in English, French, Spanish, German, Italian, and Portuguese. Recommended for iPhone 15 Pro or newer.
1,73 GB
Reasoning

Gemma 3n (E2B) — Download
A powerful model from Google, optimized for use in everyday devices. Great for content creation, text summarization, and conversational AI. Recommended for iPhone 15 Pro and newer.
2,51 GB
Recommended

Interface text:
09:41
Ask anything
AIE
AI Engineer Europe
Braintrust
WorkOS
OpenAI

### Gemma 4

Gemma 4 includes smaller models that can run on an iPhone. The speaker introduces it as one of the models compatible with MLX.

[0:59](https://www.youtube.com/watch?v=a2muGkT4WD4&t=59s) [Gemma 4 wordmark and blue emblem on a white background.](https://assets.ai.engineer/sites/aiecode2025/sha256/93e1fd262115762c9e4c849436e7b6632a5527ae0c80e3b861d0e4b4356ad8c3.webp)

Gemma 4

Interface text:
AIE
Braintrust
WorkOS
OpenAI
AI Engineer
EUROPE

### Gemma 4 on iPhone

Adrien Grondin’s post shows a video preview of Gemma 4 E2B on an iPhone 17 Pro and reports ~40 tokens/s with MLX optimized for Apple Silicon.

[1:11](https://www.youtube.com/watch?v=a2muGkT4WD4&t=71s) [A social media post describes Gemma 4 E2B running on an iPhone and shows a video preview of the app.](https://assets.ai.engineer/sites/aiecode2025/sha256/7033443e10c9ac1df4593ebe1bf7deba0c99a26bfc2085f085ca98f24cace869.webp)

Adrien Grondin @adrgrondin · Apr 4

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro

Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed

Running at ~40tk/s with MLX optimized for Apple Silicon

Phone preview:
Gemma 4 E2B
Meet Gemma 4
[illegible]
Ask anything

Interface text:
0:12
249  504  5.9K  1M
AIE
AI Engineer EUROPE

### MLX

The app uses MLX, an Apple framework optimized for Apple silicon in iPhones and Macs.

[1:23](https://www.youtube.com/watch?v=a2muGkT4WD4&t=83s) [MLX lettering centered on a white slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/89d27c06926291ff3defd46bf0d357d39929e9ce1bf4edc48192a45b3ccdcfb2.webp)

MLX

Interface text:
AIE
AI Engineer
EUROPE

### Apple A19 Pro

The app uses Apple’s MLX framework, which is optimized for Apple silicon, including iPhone chips. The slide shows the A19 Pro.

[1:28](https://www.youtube.com/watch?v=a2muGkT4WD4&t=88s) [Apple A19 Pro chip on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/112f8ec543375630b8bd694bcfc2a362f0ace7cf41fac5cf679f3b18207653cd.webp)

A19
PRO

AIE

AI Engineer
EUROPE

### Apple M5

The M5 represents Apple silicon as the speaker describes MLX’s optimization for iPhone and Mac chips. The Locally app is also available on iPad and macOS.

[1:34](https://www.youtube.com/watch?v=a2muGkT4WD4&t=94s) [Apple M5 chip on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/c68459db761ce0748ff66f65e1755b0c11b287761e85c50a8add85e2e326c2fb.webp)

M5

AIE

AI Engineer
EUROPE

### MLX Swift LM repository

The speaker points iOS, iPadOS and macOS developers to mlx-swift-lm for downloading and running models on device.

[1:47](https://www.youtube.com/watch?v=a2muGkT4WD4&t=107s) [GitHub page for the mlx-swift-lm repository, described as LLMs and VLMs with MLX Swift.](https://assets.ai.engineer/sites/aiecode2025/sha256/c0f5cea6dd49056b2e6e6191c66f7755a4aca6a02d0cc273f0fd6ed0659d7894.webp)

ml-explore / mlx-swift-lm
Public

About
LLMs and VLMs with MLX Swift
Readme
MIT license
Code of conduct
Contributing
Activity
Custom properties
366 stars
14 watching
139 forks

main
9 Branches
17 Tags

aleroot: Fix tool calling for Llama 3 (#145) · 6 hours ago
248 Commits
.github — Fix doc comments and verify in CI (#176) — 3 days ago
Libraries — Fix tool calling for Llama 3 (#145) — 6 hours ago
Tests — Fix tool calling for Llama 3 (#145) — 6 hours ago
scripts — Fix doc comments and verify in CI (#176) — 3 days ago
skills — Decouple from tokenizer and downloader packages (#118) — last week
.gitignore — feat: implement gemma3n text model in MLXLLM (#346) — 9 months ago
.pre-commit-config.yaml — improve swift-format checks -- same as mlx-swift (#399) — 7 months ago
.spi.yml — prepare mlx-swift-lm — 5 months ago
.swift-format — initial commit — 2 years ago
ACKNOWLEDGMENTS.md — Add gemma 3 embedding model (#136) — 3 weeks ago
CODE_OF_CONDUCT.md — add admin files — 2 years ago
CONTRIBUTING.md — Add MNIST Digit Prediction/Inference (#22) — 2 years ago
LICENSE — add admin files — 2 years ago
Package.swift — Decouple from tokenizer and downloader packages (#118) — last week

Releases 6
2.31.3 Latest
last week
+ 5 releases
Packages
No packages published
Contributors 30

Interface text:
github.com
Platform · Solutions · Resources · Open Source · Enterprise · Pricing
Search or jump to... · Sign in · Sign up
Notifications · Fork 139 · Star 366
Code · Issues 33 · Pull requests 25 · Discussions · Actions · Security and quality · Insights
Go to file · Code · Report repository
AIE
AI Engineer EUROPE

### MLX Swift LM repository return

The speaker recommends mlx-swift-lm for downloading and running models in Apple-platform apps, and says an iOS app can run a model on-device in less than ten minutes.

[2:18](https://www.youtube.com/watch?v=a2muGkT4WD4&t=138s) [GitHub page for the mlx-swift-lm repository.](https://assets.ai.engineer/sites/aiecode2025/sha256/c8a7ff69caac44067c3d26d7b92047b98d2ecb47de27063eef68f56fd625902a.webp)

ml-explore / mlx-swift-lm

About
LLMs and VLMs with MLX Swift

aleroot  Fix tool calling for Llama 3 (#145)
[illegible] · 6 hours ago · 248 Commits

.github  Fix doc comments and verify in CI (#176)  3 days ago
Libraries  Fix tool calling for Llama 3 (#145)  6 hours ago
Tests  Fix tool calling for Llama 3 (#145)  6 hours ago
scripts  Fix doc comments and verify in CI (#176)  3 days ago
skills  Decouple from tokenizer and downloader packages (#118)  last week
.gitignore  feat: implement gemma3n text model in MLXLLM (#346)  9 months ago
.pre-commit-config.yaml  improve swift-format checks -- same as mlx-swift (#399)  7 months ago
.spi.yml  prepare mlx-swift-lm  5 months ago
.swift-format  initial commit  2 years ago
ACKNOWLEDGMENTS.md  Add gemma 3 embedding model (#136)  3 weeks ago
CODE_OF_CONDUCT.md  add admin files  2 years ago
CONTRIBUTING.md  Add MNIST Digit Prediction/Inference (#22)  2 years ago
LICENSE  add admin files  2 years ago
Package.swift  Decouple from tokenizer and downloader packages (#118)  last week

Readme
MIT license
Code of conduct
Contributing
Activity
Custom properties
366 stars
14 watching
139 forks
Report repository
Releases 6
2.31.3 Latest
last week
+ 5 releases
Packages
No packages published
Contributors 30

Engineering the future of AI
AIE
AI Engineer EUROPE

Interface text:
github.com
Private
Platform  Solutions  Resources  Open Source  Enterprise  Pricing
Search or jump to...
Sign in  Sign up
Public
Notifications  Fork 139  Star 366
Code  Issues 33  Pull requests 25  Discussions  Actions  Security and quality  Insights
main  9 Branches  17 Tags
Go to file  Code

### Hugging Face

Hugging Face is a place to get models for use with MLX.

[3:22](https://www.youtube.com/watch?v=a2muGkT4WD4&t=202s) [Hugging Face emoji centered on a white background.](https://assets.ai.engineer/sites/aiecode2025/sha256/88e1cb4b028c51656c66a505aa37c20c8e0c4f9d1a1347d6cd048470810be475.webp)

Interface text:
AIE
AI Engineer
EUROPE

### MLX Community model variants

MLX Community lists Gemma 4 E2B variants from BF16 to quantized formats. The speaker says to pass a chosen model ID to mlx-swift-lm to download it.

[3:30](https://www.youtube.com/watch?v=a2muGkT4WD4&t=210s) [Hugging Face MLX Community page showing Gemma 4 E2B model variants.](https://assets.ai.engineer/sites/aiecode2025/sha256/8c7854df7685f5da838060826825d5073a065e1b7af779614a21e787aa3a1601.webp)

MLX Community

AI & ML interests
None defined yet.

Recent Activity
acul3 updated a model · 2 days ago
mlx-community/VoxCPM2-4bit
acul3 updated a model · 2 days ago
mlx-community/VoxCPM2-8bit
acul3 published a model · 2 days ago
mlx-community/VoxCPM2-4bit
View all activity

Team members 4,305

mlx-community's models 20
Search: gemma-4-e2b
Sort: Recently updated

mlx-community/gemma-4-e2b-nvfp4
Any-to-Any · 2B · Updated 6 days ago · 1.62k downloads · 1 like
mlx-community/gemma-4-e2b-mxfp8
Any-to-Any · 2B · Updated 6 days ago · 969 downloads
mlx-community/gemma-4-e2b-mxfp4
Any-to-Any · 2B · Updated 6 days ago · 542 downloads
mlx-community/gemma-4-e2b-it-mxfp8
Any-to-Any · 2B · Updated 6 days ago · 690 downloads
mlx-community/gemma-4-e2b-it-mxfp4
Any-to-Any · 2B · Updated 6 days ago · 1.32k downloads
mlx-community/gemma-4-e2b-it-bf16
Any-to-Any · 5B · Updated 6 days ago · 872 downloads
mlx-community/gemma-4-e2b-it-8bit
Any-to-Any · 2B · Updated 6 days ago · 1.17k downloads · 1 like
mlx-community/gemma-4-e2b-it-6bit
Any-to-Any · 2B · Updated 6 days ago · 509 downloads
mlx-community/gemma-4-e2b-it-5bit
Any-to-Any · 1B · Updated 6 days ago · 553 downloads
mlx-community/gemma-4-e2b-it-4bit
Any-to-Any · 1B · Updated 6 days ago · 53.9k downloads · 4 likes
mlx-community/gemma-4-e2b-bf16
Any-to-Any · 5B · Updated 6 days ago · 660 downloads
mlx-community/gemma-4-e2b-8bit
Any-to-Any · 2B · Updated 6 days ago · 1.41k downloads · 1 like
mlx-community/gemma-4-e2b-6bit
Any-to-Any · 2B · Updated 6 days ago · 389 downloads
mlx-community/gemma-4-e2b-5bit
Any-to-Any · 1B · Updated 6 days ago · 366 downloads

Interface text:
Private
huggingface.co
Hugging Face
Search models, datasets, users...
Models  Datasets  Spaces  Buckets NEW  Docs  Pricing  Log In  Sign Up
Activity Feed  Request to join this org  Follow  10,893
AIE
AI Engineer

[4:18](https://www.youtube.com/watch?v=a2muGkT4WD4&t=258s) [Hugging Face MLX Community page listing Gemma 4 E2B model variants.](https://assets.ai.engineer/sites/aiecode2025/sha256/e549d86eae69952a70e7337270d1bb70cbf18bbec17dd455505ebcffd8f31848.webp)

MLX Community

AI & ML interests
None defined yet.

Recent Activity
acul3 updated a model · 2 days ago
mlx-community/VoxCPM2-4bit
acul3 updated a model · 2 days ago
mlx-community/VoxCPM2-8bit
acul3 published a model · 2 days ago
mlx-community/VoxCPM2-4bit
View all activity

Team members 4,305

mlx-community's models 20
Search: gemma-4-e2b
Sort: Recently updated

mlx-community/gemma-4-e2b-nvfp4
Any-to-Any · 2B · Updated 6 days ago · 1.62k downloads · 1 like
mlx-community/gemma-4-e2b-mxfp8
Any-to-Any · 2B · Updated 6 days ago · 969 downloads
mlx-community/gemma-4-e2b-mxfp4
Any-to-Any · 2B · Updated 6 days ago · 542 downloads
mlx-community/gemma-4-e2b-it-mxfp8
Any-to-Any · 2B · Updated 6 days ago · 690 downloads
mlx-community/gemma-4-e2b-it-mxfp4
Any-to-Any · 2B · Updated 6 days ago · 1.32k downloads
mlx-community/gemma-4-e2b-it-bf16
Any-to-Any · 5B · Updated 6 days ago · 872 downloads
mlx-community/gemma-4-e2b-it-8bit
Any-to-Any · 2B · Updated 6 days ago · 1.17k downloads · 1 like
mlx-community/gemma-4-e2b-it-6bit
Any-to-Any · 2B · Updated 6 days ago · 509 downloads
mlx-community/gemma-4-e2b-it-5bit
Any-to-Any · 1B · Updated 6 days ago · 553 downloads
mlx-community/gemma-4-e2b-it-4bit
Any-to-Any · 1B · Updated 6 days ago · 53.9k downloads · 4 likes
mlx-community/gemma-4-e2b-bf16
Any-to-Any · 5B · Updated 6 days ago · 660 downloads
mlx-community/gemma-4-e2b-8bit
Any-to-Any · 2B · Updated 6 days ago · 1.41k downloads · 1 like
mlx-community/gemma-4-e2b-6bit
Any-to-Any · 2B · Updated 6 days ago · 389 downloads
mlx-community/gemma-4-e2b-5bit
Any-to-Any · 1B · Updated 6 days ago · 366 downloads

Interface text:
Private
huggingface.co
Hugging Face
Search models, datasets, users...
Models  Datasets  Spaces  Buckets NEW  Docs  Pricing  Log In  Sign Up
Activity Feed  Request to join this org  Follow  10,893
AIE
AI Engineer
Engineering the future of AI

### Quantization options

For iPhone inference, the speaker recommends 4–8-bit quantization: full-size models are too large, while going below 4-bit can substantially affect output quality.

[4:40](https://www.youtube.com/watch?v=a2muGkT4WD4&t=280s) [Slide reading “4-bit / 6-bit / 8-bit.”](https://assets.ai.engineer/sites/aiecode2025/sha256/b5f116ebaaad16481e212e80373bdeae7a1b6c85d7ac368e9982aecc7e79342b.webp)

4-bit / 6-bit / 8-bit

AIE
Engineering the future of AI
AI Engineer
EUROPE

### iPhones

The speaker says Gemma 4 E2B quantized to 4-bit can run at 40 tokens per second on the latest iPhone.

[5:38](https://www.youtube.com/watch?v=a2muGkT4WD4&t=338s) [Four iPhones shown against a white background.](https://assets.ai.engineer/sites/aiecode2025/sha256/487608eeef757f585a77f5da4755684fb79edcc48f1ba82cd471f3e546531215.webp)

AIE

AI Engineer
EUROPE

AI Engineer

### Gemma 4 phone demo

The presenter shows Gemma 4 on a phone to illustrate 40 tokens per second, describing the demo as live, offline and streaming.

[6:04](https://www.youtube.com/watch?v=a2muGkT4WD4&t=364s) [A phone displays the Gemma 4 welcome screen and keyboard.](https://assets.ai.engineer/sites/aiecode2025/sha256/18d68b5559a1c52fc1d166ccec15b6395bfc10605d557f2a4eae652b516a23bf.webp)

Meet Gemma 4

Google's most intelligent open models to date — built for reasoning, image understanding, and multi-turn chat.

Plan a trip to Paris
Identify this landmark
Boost my produc[illegible]

Interface text:
21:10
Gemma 4 E2B
Ask anything
Tu  C'est  Oui
A Z E R T Y U I O P
Q S D F G H J K L M
W X C V B N
123
FR EN

AIE
AI Engineer EUROPE

### Gemma 4 Paris travel-planning demo

A phone streams a long Paris travel plan, including neighborhood recommendations. The speaker says the demo runs offline at 40 tokens per second.

[6:11](https://www.youtube.com/watch?v=a2muGkT4WD4&t=371s) [A phone displays a travel-planning prompt and the beginning of a generated response.](https://assets.ai.engineer/sites/aiecode2025/sha256/271459a655823e6ae6ca70441a65860529f89836d6541220e337327ec03e5527.webp)

I want to plan a 7-day trip to Paris. Please suggest must-see attractions, recommend good neighborhoods to stay in, and create a daily itinerary including restaurants and activities.

That sounds like a wonderful trip! Paris is a city of art, romance, and incredible history. To give you the best plan, I’ll create a balanced itinerary that mixes iconic sights with local charm, while ensuring you have time to savor the food and atmosphere.

Here is a comprehensive 7-day plan for your trip to Paris.

Your 7-Day Paris Adventure

Neighborhood Recommendations (Where to Stay)
Choosing the right neighborhood depends on your travel style. Here are three excellent options:
1. Saint-Germain-des-Prés (6th [illegible]

Interface text:
21:11
Gemma 4 E2B
Ask anything
AIE
AI Engineer EUROPE

[6:19](https://www.youtube.com/watch?v=a2muGkT4WD4&t=379s) [A phone displays neighborhood recommendations in a generated Paris travel plan.](https://assets.ai.engineer/sites/aiecode2025/sha256/c3d02051b7c97835a39b01ddca6a7bb05957189f496d83b0680954b0f22a5b31.webp)

• Best For: Foodies, shoppers, people who love independent shops, and those who want a vibrant nightlife.
• Pros: Incredible food markets, amazing architecture, great atmosphere.
• Cons: Can be very crowded during peak hours.

3. Latin Quarter (5th Arrondissement):
• Vibe: Student-focused, lively, bohemian, traditional Parisian heartbeat. Bustling cafés and vibrant student life.
• Best For: Budget-conscious travelers who want a lively, authentic Parisian experience, and those interested in books/history.
• Pros: Energetic, affordable dining options, great atmosphere around universities.
• Cons: Can feel chaotic.

Must-See Attractions (The Icons)
These are the non-negotiables:
• **The Eiffel

Interface text:
21:11
Gemma 4 E2B
Ask anything
AIE
Engineering the future of AI

### 40 tokens per second

The speaker says older iPhones run below 40 tokens per second, but even 20 tokens per second can be useful for many apps.

[6:47](https://www.youtube.com/watch?v=a2muGkT4WD4&t=407s) [A slide reads “40tk/s.”](https://assets.ai.engineer/sites/aiecode2025/sha256/871e1f248230eec88e336aa2a86fae5e741d0f574015dfc5aec85a788e6511cb.webp)

40tk/s

AIE
Engineering the future of AI

### Try the app

Scan the QR code to try the free iPhone app. The speaker says its model download is usually around 1 GB or 3 GB, depending on the model.

[7:06](https://www.youtube.com/watch?v=a2muGkT4WD4&t=426s) [A QR code with a blue star logo in its center.](https://assets.ai.engineer/sites/aiecode2025/sha256/96a23865261ae9aa929bbcd65ea40430cafbd5f19d5af3836334e7a2b8f032c7.webp)

Interface text:
AIE
Engineering the future of AI

### Locally AI and LM Studio

LM Studio acquired Locally AI. The speaker describes LM Studio as a way to download local models from Hugging Face and run them with llama.cpp or MLX.

[7:49](https://www.youtube.com/watch?v=a2muGkT4WD4&t=469s) [A purple app icon and a blue star app icon are separated by a plus sign.](https://assets.ai.engineer/sites/aiecode2025/sha256/ea589d7848e9e28f0addeb74a010f2d2ec7dfdfbc8c07211b954dc69d3d88ec3.webp)

+

Interface text:
AIE
AI Engineer
EUROPE

### LM Studio model browser

LM Studio’s browser shows a GGUF download for Gemma 4 26B A4B. The speaker says LM Studio can run local models with llama.cpp or MLX and expose a local server to applications.

[8:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=482s) [LM Studio model browser showing Gemma 4 26B A4B and its download options.](https://assets.ai.engineer/sites/aiecode2025/sha256/bd2c416a0d0f3939ead0848ee5fa31331d4e2cc6a871e2662a27509d068788c1.webp)

google/gemma-4-26b-a4b

Gemma 4 is the latest on-device model family from Google. This is the 26B (active 4B) MoE version. Supports vision and reasoning.

Params: 26B
Arch: gemma4
Domain: llm
Format: GGUF
Capabilities: Vision, Tool Use, Reasoning

Download Options
GGUF Gemma 4 26B A4B Instruct Q4_K_M — 17.99 GB
Full GPU Offload Possible
Download 17.99 GB

README
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on small models) and generating text output.

Gemma 4 introduces key capability and architectural advancements:
• Reasoning – All models in the family are designed as highly capable reasoners, with configurable thinking modes [cut off]

Interface text:
Unnamed Chat
Chats
Search local models by name or author...
Staff picks
GGUF, MLX
Best Match
649,378
95
Last updated: 7 days ago
Staff Pick
Gemma 4 26B A4B
Nemotron 3 Nano 4B
Qwen3.5 9B
Qwen3.5 35B A3B
Lfm2 24B A2B
Qwen3 Coder Next
Glm 4.7 Flash
Lfm2.5 1.2B

AIE
Google DeepMind

### Thank you

[8:43](https://www.youtube.com/watch?v=a2muGkT4WD4&t=523s) [A white slide with the words “Thank you!”](https://assets.ai.engineer/sites/aiecode2025/sha256/38dab5204b4f81320ca561ece8fbadfaee8a7dc1da4113e5aabd10c85c24b776.webp)

Thank you!

Interface text:
AIE
Braintrust
WorkOS
OpenAI

## Full transcript

[0:00](https://www.youtube.com/watch?v=a2muGkT4WD4&t=0s) [upbeat music] Okay.

[0:15](https://www.youtube.com/watch?v=a2muGkT4WD4&t=15s) Hello everybody. [clears throat] I'm going to show you today how to run Gemma 4 on iPhone with, uh, MLX. So first, let's introduce myself. I'm Adrien. You can find, uh, my Twitter if you want to learn more about, uh, all on-device things.

[0:29](https://www.youtube.com/watch?v=a2muGkT4WD4&t=29s) I'm the developer of, um, Locally AI, so maybe you have already seen, uh, the, the app. So Locally AI is a chatbot that allow you to run on-device models on your iPhone with, uh, MLX.

[0:42](https://www.youtube.com/watch?v=a2muGkT4WD4&t=42s) So I will just, uh, go through ML- what is MLX in a, in a few, few seconds. Basically, uh, lo- as I said, it's a, it's a chatbot. It's fully native.

[0:51](https://www.youtube.com/watch?v=a2muGkT4WD4&t=51s) You can also chat with, uh, Apple Foundation now with it. And the, and many models like, um, that are compatible with MLX and one of this model is, uh, Gemma 4.

[1:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=62s) So [clears throat] basically, Gemma 4, uh, by Google DeepMind have a lot of model and some of them like can run on iPhone, like the smaller ones and they are pretty great.

[1:13](https://www.youtube.com/watch?v=a2muGkT4WD4&t=73s) Maybe you have seen on, uh, on Twitter one of the post I've made that, uh, I should... I demo it running, uh, in the app, uh, on iPhone. It's really fast.

[1:20](https://www.youtube.com/watch?v=a2muGkT4WD4&t=80s) It runs really well on MLX and behind the app, so [clears throat] it's using MLX. MLX is a framework made by Apple that is optimized for Apple silicon, so mainly, mainly the, [clears throat] the chip in iPhone but also the chips on the, on the Mac.

[1:36](https://www.youtube.com/watch?v=a2muGkT4WD4&t=96s) The, the app Locally is available on iPad. It works also very well on this device and macOS and, uh, everything is, uh, is built to be as optimized as possible on the, on these, uh, devices.

[1:49](https://www.youtube.com/watch?v=a2muGkT4WD4&t=109s) So if you want, if you want to run on iPhone, um, a s- large model, so Gemma works well but you have a lot of, uh, model that you can run also.

[1:58](https://www.youtube.com/watch?v=a2muGkT4WD4&t=118s) The Qwen model and the, um, and the SmolLM model from the Hugging Face. You-- the place you, you will want to go is GitHub and go to the repo mlx-swift-lm.

[2:07](https://www.youtube.com/watch?v=a2muGkT4WD4&t=127s) I won't go into detail how you implement the repo. It's... I think I will let your agent in, uh, implement that, uh, for you but it's one repo that you need to install if you're developing, um, iOS or macOS or iPadOS app.

[2:19](https://www.youtube.com/watch?v=a2muGkT4WD4&t=139s) And you can use, um, you can use that to simply download the model and then run it. The API is very straightforward, very simple to, to implement. In less than ten minutes, you can have an iOS app with a model that is running on your, on your device.

[2:32](https://www.youtube.com/watch?v=a2muGkT4WD4&t=152s) That's very simple, uh, to do as, uh, as mentioned. MLX, uh, so this is mlx-swift-lm but if you're more into Python apps or macOS app you can also run mlx-vlm from, uh, from Prince.

[2:44](https://www.youtube.com/watch?v=a2muGkT4WD4&t=164s) Maybe you have, uh, seen him like he's doing, um, on device for audio with MLX Audio and visual model with, uh, MLX VLM and also MLX Video to, to run, um, to run, uh, image, uh, image generation model or image, uh, or video generation model.

[3:00](https://www.youtube.com/watch?v=a2muGkT4WD4&t=180s) MLX it's, uh, there's... The ecosystem is getting, uh, bigger or it's getting bigger. It's g- it's really great right now. You can do pretty much everything like, uh, omni models or as I said, text-to-speech, speech to, speech, speech-to-speech.

[3:12](https://www.youtube.com/watch?v=a2muGkT4WD4&t=192s) There's a lot of thing that you can do with, um, with the model. And basically, let's say you integrate the, you integrate, um, this model. You can't just run any model, uh, on it.

[3:23](https://www.youtube.com/watch?v=a2muGkT4WD4&t=203s) Like you need to, you need to get some model and there's a good place to get the model is Hugging Face. I'm pretty sure everybody heard, uh, of Hugging Face.

[3:31](https://www.youtube.com/watch?v=a2muGkT4WD4&t=211s) And on Hugging Face you will want to look for MLX Community. This is where like all the, the weight of a model, the quantized weight, the full size will be uploaded.

[3:41](https://www.youtube.com/watch?v=a2muGkT4WD4&t=221s) So you will just be able to go to this community, look for the models. I think right now there's almost 4,000 or 5,000 model uploaded so the community is really active on it.

[3:52](https://www.youtube.com/watch?v=a2muGkT4WD4&t=232s) When a model, uh, is, uh, is released by your, by your lab you will directly have it almost, uh, 30 minutes after release quantized in 4-bit, 6-bit and, and everything that you can, uh, you can imagine.

[4:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=242s) Here you have a, an example for Gemma 4 E2B which is the one I, I run on, um, on iPhone. There's like a lot of variant of it for, from BF16, MX- [clears throat] MXFP4 like 5-bit, 6-bit. [clears throat]

[4:16](https://www.youtube.com/watch?v=a2muGkT4WD4&t=256s) There's everything. So, so you download mlx-swift-lm, install it with your agent or anything. Then you go to MLX Community and you just choose the model you want to run and then with ID you can just pass it to the framework and it will directly, uh, it, it will be integrated with Hugging Face to download the model with,

[4:35](https://www.youtube.com/watch?v=a2muGkT4WD4&t=275s) uh, mlx-swift-lm directly. So you just need to grab the ID and the, and pass it to, to the framework.

[4:41](https://www.youtube.com/watch?v=a2muGkT4WD4&t=281s) Usually when you're running the, the model on, um, on iPhone what you want to, what you want to do is quantize... is select, is selecting some quanti- quantized version of the model because the full size will be way too large.

[4:54](https://www.youtube.com/watch?v=a2muGkT4WD4&t=294s) What I go recommend is going, it's trying depending on the size between 3-bit and, uh, and 8-bit usually. Uh, between 4-bit and 8-bit. A bit usually under like 4-bit it's getting, it's starting to get a lot of, uh, impact on the output and, um, and the model are not that great usually.

[5:12](https://www.youtube.com/watch?v=a2muGkT4WD4&t=312s) 4-bit is the lower I would go and 8-bit is the higher I would go if you're using really small models. And in my app for example I have like some bigger model like Gemma 4 but I also have some Liquid model where that is, uh, 350 million parameters and this it can run in Shortcuts.

[5:28](https://www.youtube.com/watch?v=a2muGkT4WD4&t=328s) So you can do a lot of kind of au- automation because the model are really fast and really efficient, uh, at that, uh, at that size to do some text processing and, uh, things like this.

[5:39](https://www.youtube.com/watch?v=a2muGkT4WD4&t=339s) And on the la- on the latest iPhone like if you take Gemma 4 E2B quantized in 4-bit it's extremely fast. Like it can run easily at, uh, 40 token, uh, 40 token for, uh, uh, per second.

[5:51](https://www.youtube.com/watch?v=a2muGkT4WD4&t=351s) I will just do an update in the slide because I remove, uh, I remove a slide where there's the video but maybe I can add it back because I have little bit more time.

[6:01](https://www.youtube.com/watch?v=a2muGkT4WD4&t=361s) Just go like this. So just to show you, uh, in demo what, what the 40 token per second means.

[6:10](https://www.youtube.com/watch?v=a2muGkT4WD4&t=370s) And that's running live offline. And as you can see, it's like really fast, like 40 token per second is more than acceptable for a lot of use case. This is, of course, streaming.

[6:22](https://www.youtube.com/watch?v=a2muGkT4WD4&t=382s) You can also like not do streaming and do a UI that just will wait for, for four second. And here the output is quite long, so it's generating a lot of tokens.

[6:31](https://www.youtube.com/watch?v=a2muGkT4WD4&t=391s) So on device with MLX, it's working. It's here. Like, um, it's really easy and really not hard to, to integrate. As I said, if you go to the, to the repo mlx-swift-lm, very...

[6:44](https://www.youtube.com/watch?v=a2muGkT4WD4&t=404s) It's a breeze to, to install. On top of that, as I said, like again, latest iPhone is really great, but it works also with older iPhone. Like you will not get 40 token per second, which is quite fast.

[6:55](https://www.youtube.com/watch?v=a2muGkT4WD4&t=415s) But even if you get 20 tokens per second, that's already great and useful for a lot of, um, for a lot of application, a lot of use case that you would want to do with, um, with your, um, with your, uh, with your app.

[7:11](https://www.youtube.com/watch?v=a2muGkT4WD4&t=431s) You can go to and scan this QR code if you want to try it by yourself. If you have a, if you have an iPhone, the app is on the App Store.

[7:17](https://www.youtube.com/watch?v=a2muGkT4WD4&t=437s) It's free to use. Only thing is that you will have to download the model that's u-usually around one gig-gigabytes or, or three gigabytes. Really depend on, uh, on which model.

[7:27](https://www.youtube.com/watch?v=a2muGkT4WD4&t=447s) But, uh, that's the biggest barrier right now. It's the size of a model, but this also is getting better. Model are getting smaller or getting smarter and, um, they also, uh, the iPhone is getting better.

[7:38](https://www.youtube.com/watch?v=a2muGkT4WD4&t=458s) So next iPhone and second and the next, next iPhone, everything will just, um, is reaching really great, uh, usability from, um, what I can see. And also on top of that, like, uh, maybe you have heard the news yesterday, uh, Locally AI have been acquired by, uh, LM Studio.

[7:55](https://www.youtube.com/watch?v=a2muGkT4WD4&t=475s) If you don't know, uh, LN- LM Studio, [clears throat] it's basically, um, uh, kind of a s- AI studio for all your local models. So you can download the, the model with L- any model with LM Studio directly from Hugging Face.

[8:10](https://www.youtube.com/watch?v=a2muGkT4WD4&t=490s) You can run them and you can open the server. You can run them with llama.cpp, but also MLX. So you can really compare the different engine, how they, uh, how they work.

[8:19](https://www.youtube.com/watch?v=a2muGkT4WD4&t=499s) You can, um, as I said, you can open a, a server locally and connect your application to this, uh, to this hosted server, uh, with va-various, um, various response, response type.

[8:31](https://www.youtube.com/watch?v=a2muGkT4WD4&t=511s) For example, open I- OpenAI, uh, response type or, um, [clears throat] or, um, or Anthropic response type, uh, for streaming anything and you can just get any model runnings, uh, really easily with that.

[8:45](https://www.youtube.com/watch?v=a2muGkT4WD4&t=525s) So, and I want to thank you. That was a very short introduction how you can do the same and run any model like in Gemma 4 if you want on your, on your iPhone.

[8:54](https://www.youtube.com/watch?v=a2muGkT4WD4&t=534s) If you have any question also. [audience applauding]

[9:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=542s) Uh, does, does this support tool calling?

[9:04](https://www.youtube.com/watch?v=a2muGkT4WD4&t=544s) Sorry?

[9:04](https://www.youtube.com/watch?v=a2muGkT4WD4&t=544s) Does this support tool calling?

[9:05](https://www.youtube.com/watch?v=a2muGkT4WD4&t=545s) Yes. Like, so yeah, I forgot about that. So it support tool calling. Um, not yet, um, custom, um, not yet structured, structured generation. There are some package on top of mlx-swift-lm that, uh, are trying to, to make this, uh, make this working.

[9:20](https://www.youtube.com/watch?v=a2muGkT4WD4&t=560s) Um, I will let you do... Uh, Hugging Face is doing it, but you, you can easily find them, uh, online. But, uh, mlx-swift-lm, yes, it support tool calling, so that's really useful if you want to do tool calling and call other s- other system.

[9:33](https://www.youtube.com/watch?v=a2muGkT4WD4&t=573s) And the model mo- the model are getting also better at tool calling. They were not so great like a year ago. Now it's getting much better.

[9:43](https://www.youtube.com/watch?v=a2muGkT4WD4&t=583s) I think you mentioned two things. First was the GitHub repo.

[9:46](https://www.youtube.com/watch?v=a2muGkT4WD4&t=586s) Yes.

[9:46](https://www.youtube.com/watch?v=a2muGkT4WD4&t=586s) And then the second thing was the app.

[9:49](https://www.youtube.com/watch?v=a2muGkT4WD4&t=589s) Um, so you have the GitHub repo for, um, for mlx-swift-lm. That's the, the package that will install in your app. And then you need to go to Hugging Face and, uh, and getting, uh, for, to, to get the weight of a model.

[10:02](https://www.youtube.com/watch?v=a2muGkT4WD4&t=602s) But for a, for a normal user, they can just download the app from the App Store?

[10:06](https://www.youtube.com/watch?v=a2muGkT4WD4&t=606s) Oh, and yes, if you want to try my app, uh, you can. If you want to try it like right now without having anything to install, you can do that.

[10:13](https://www.youtube.com/watch?v=a2muGkT4WD4&t=613s) And you can choose any open source model inside the app?

[10:16](https://www.youtube.com/watch?v=a2muGkT4WD4&t=616s) Uh, there's a selection, but, uh, that they are not... It's not any. Like I'm ensuring that all the model runs correctly, um, on the, on the iPhone because not all of them work, uh, work well.

[10:29](https://www.youtube.com/watch?v=a2muGkT4WD4&t=629s) Thank you very much. [audience applauding] [outro jingle]
