Running Gemma 4 On-Device: 40 Tokens/s on iPhone with MLX

How do you run Gemma 4 on an iPhone? Adrien Grondin’s starting point is Locally AI, the native chatbot he develops. It runs models on the device through MLX and also offers Apple Foundation alongside compatible downloadable models. The opening app views show the three pieces users encounter: a conversation, Apple Foundation, and model management.

Smaller variants of Google DeepMind’s Gemma 4 family make this phone experience possible. Grondin has shown the app running them in social posts; underneath the interface is MLX, Apple’s framework optimized for Apple silicon. The same ecosystem extends beyond the phone: Locally AI also runs on iPad and macOS.

Gemma is one option; Qwen and Hugging Face’s SmolLM are other model families Grondin points to. For a native iOS, iPadOS, or macOS application, the integration entry point is MLX Swift LM, packaged as mlx-swift-lm. It supplies the machinery to download and run compatible models. Grondin leaves the detailed implementation to the developer and their coding agent, describing the API as straightforward. He estimates that an iOS app running a model can be assembled in less than ten minutes.

The surrounding ecosystem covers more than text chat:

• Vision and omni models: Prince’s MLX-VLM provides another route for Python and Mac work.
• Audio: MLX-Audio covers speech workloads, including text-to-speech and speech-to-speech.
• Image and video generation: MLX-Video extends the family of tools to generation workloads.

These projects explain the breadth of MLX’s ecosystem; the phone walkthrough remains focused on running a suitably sized language model.

Installing the framework does not make every model runnable. The next dependency is a compatible set of weights. Grondin directs developers to MLX Community on Hugging Face, where contributors publish both full-size and quantized versions. At the time of the talk, he estimates roughly 4,000–5,000 uploaded models and says quantizations can appear about 30 minutes after a lab releases a model. Those are his observations about community activity, rather than catalog counts or turnaround guarantees.

His iPhone example is Gemma 4 E2B. The catalog offers multiple representations of its weights: he mentions BF16, MXFP4, and 5-bit and 6-bit variants. The model family identifies what you want to run; the particular repository identifies which representation you will download.

The integration sequence is short:

1. Add mlx-swift-lm to the application.
2. Select a compatible model repository from MLX Community.
3. Pass that repository’s ID to the framework, whose Hugging Face integration downloads the model.

For example, the available 4-bit Gemma 4 E2B checkpoint gives a concrete identifier to put into a Swift model configuration:

import MLXLMCommon

let modelConfiguration = ModelConfiguration(
    id: "mlx-community/gemma-4-e2b-it-4bit"
)

This configuration selects the repository; the application’s loading and generation code then uses it. The significant handoff is the model ID, rather than a manually assembled download URL or a separate weight-transfer step.

For the proposed iPhone use, full-size weights are generally too large. Quantization reduces the precision used to represent weights, making a smaller representation available at a potential cost to output quality. Grondin initially mentions 3–8 bits, then settles on 4–8 bits as his practical range.

The choice therefore depends on model size as well as precision: the largest precision setting is most plausible when the model itself is small.

A chatbot is not the only useful destination. Grondin describes a Liquid model with 350 million parameters running through Shortcuts in his app. At that size, he uses the model for fast, efficient text processing and automation. He does not identify the exact Liquid checkpoint.

For the larger chat example, Grondin reports about 40 tokens per second for Gemma 4 E2B quantized to 4-bit on the latest iPhone. He does not name the iPhone model or provide a reproducible benchmark configuration, so this figure describes his reported setup rather than a hardware-wide expectation.

The demonstration turns that throughput into a visible interaction. Grondin describes the phone as running live and offline, streaming its response as it generates. The displayed answer is relatively long; the selected view shows a multi-day Paris itinerary in the chat interface. Streaming lets the user begin reading while generation continues.

A different interface could wait and present the completed answer. Grondin suggests a four-second wait as an example of that design choice, not a measured completion time for every response. The appropriate presentation depends on how much output the task needs.

Older iPhones can also run these models, although Grondin says they should not be expected to reach the latest-phone figure. He offers 20 tokens per second as an illustrative rate that would still be useful for many applications, rather than reporting a measurement from a named older device. The practical question is whether generation is fast enough for the particular interaction.

To try the experience, Grondin points attendees to an App Store QR code and describes Locally AI as free to use. Installing the app is only the first step: users also need the selected model’s weights. Grondin puts typical model downloads at roughly 1–3 GB, depending on the model.

He identifies model size as the biggest remaining barrier. Local generation removes the need to send each request to a hosted model, but it still asks users to download a substantial asset before using it. Smaller, more capable models and improving iPhone hardware make that tradeoff progressively easier.

Grondin expects successive iPhone generations to improve usability further. He then connects the phone app to the desktop ecosystem, announcing that LM Studio has acquired Locally AI. LM Studio’s blog dates the joining announcement to April 8, 2026.

LM Studio provides a desktop interface for downloading models from Hugging Face and running them locally. It supports both llama.cpp and MLX, allowing users to compare the engines. Its model browser shows the selection and download side of that workflow.

The next step is to expose the running model through a local server and connect another application to it. Grondin describes OpenAI-style and Anthropic-style interfaces, including streaming. This changes where inference lives: a phone app can embed the runtime directly, while an application using LM Studio sends requests to a separate local serving process.

An audience question asks whether the stack supports tool calling. Grondin says yes, while drawing a separate boundary around structured generation.

These describe the support boundary at the time of the talk, not a promise about every later package version. Tool calling lets a model request an operation in another system; it does not by itself establish that arbitrary output will be constrained to a required structure.

Grondin mentions Hugging Face’s work on additional packages without naming one. His concrete recommendation is that mlx-swift-lm already supports the tool-calling path for connecting to other systems, and he says models have become substantially better at selecting and using tools compared with a year earlier.

The final questions separate two entry points that can otherwise sound like one setup process:

• Developers: Install mlx-swift-lm in their own app and obtain compatible model weights from Hugging Face.
• Users: Download Locally AI from the App Store and try the experience without setting up a development environment.

The ready-made app removes the integration work, while still requiring the model download discussed earlier.

That convenience also includes a selection boundary. Locally AI does not offer every open-source model: Grondin curates the catalog and checks that included models run correctly on the iPhone. A model’s availability is not the same as its suitability for a phone. The app’s final responsibility is to turn compatible weights and a capable runtime into choices that actually work on the user’s device.