Text Diffusion — Brendan O'Donoghue, Google DeepMind

Take a clean sentence, replace some of its tokens with random vocabulary entries, and ask a neural network to repair it. Repeat that training exercise at different noise levels. At inference time, start with an entirely random sequence and repeatedly remove the corruption until readable text emerges. This is text diffusion: the same broad denoising principle used for images and video, applied to a sequence of tokens.

Corruption can happen in a continuous space or directly over discrete tokens. Brendan O’Donoghue’s explanation uses the discrete case. A small Python example makes the training-side operation concrete: noise_probability controls how much of a clean token sequence is replaced, while the original sequence remains the target to recover.

import random


def corrupt_tokens(
    clean: list[int],
    vocabulary_size: int,
    noise_probability: float,
    rng: random.Random,
) -> list[int]:
    if vocabulary_size <= 0:
        raise ValueError("vocabulary_size must be positive")
    if not 0.0 <= noise_probability <= 1.0:
        raise ValueError("noise_probability must be between 0 and 1")

    return [
        rng.randrange(vocabulary_size)
        if rng.random() < noise_probability
        else token
        for token in clean
    ]


clean = [12, 41, 7, 93]
rng = random.Random(0)
training_input = corrupt_tokens(clean, 100, 0.5, rng)
inference_start = corrupt_tokens(clean, 100, 1.0, rng)

The token IDs here are teaching values. The learned part is the reverse operation: successive predictions turn the noisy canvas into a coherent sequence, filling in information in the order the model finds useful.

That process produces a recognizable visual effect: scattered text gradually settles into a clean response. Gemini Diffusion brought it to a research preview that O’Donoghue describes as reaching about 100,000 people. The model branched from the architecture of Gemini 2.0 Flash-Lite, which also served as its historical comparator. O’Donoghue describes broadly similar overall quality, with some advantages in code and disadvantages elsewhere, at substantially lower latency. These comparisons concern the preview he describes as roughly a year old, rather than a current model ranking.

An autoregressive model generates a response one token at a time. Each new token depends on the context and the tokens already produced. Diffusion instead initializes a canvas—potentially hundreds or thousands of tokens—and processes the whole block repeatedly. It still needs multiple forward passes, but each pass can update many positions rather than append only one token.

The immediate attraction is faster generation through better use of GPUs and TPUs. But the change in generation order also creates other capabilities:

• Bidirectional attention: a position can attend to later positions within the canvas, allowing later reasoning to inform an earlier answer.
• Adaptive computation: a model can learn to spend more refinement steps on a difficult response and fewer on an easy one.
• In-place editing: some tokens can remain fixed while other positions change—for example, holding a suffix constant and generating a compatible prefix.

These capabilities follow from treating the response as something to revise, rather than a stream whose emitted tokens are already committed.

Lower latency for one request does not imply higher throughput across many requests. Autoregressive serving can combine a large batch of queries, using the accelerator efficiently even while each individual user waits for sequential decoding. Diffusion repeatedly processes the same output positions and reaches the accelerator’s compute limit sooner.

O’Donoghue identifies this throughput penalty, and its corresponding serving cost, as the main obstacle to diffusion in large hosted models. His aside about Claude capacity concerns underscores why providers care about aggregate throughput even when users would prefer faster individual responses.

The hardware explanation starts with the gap between arithmetic capacity and memory bandwidth. Tensor cores perform matrix multiplications efficiently, but the weights, activations, and key-value cache reside in high-bandwidth memory, or HBM. Moving that data into the compute units takes time. GPUs and TPUs have abundant arithmetic capacity relative to the bandwidth available to feed it, so doing more useful computation per transfer can improve utilization.

At batch size one, autoregressive decoding is often memory-bound: the model streams weights and relevant cache data to produce a token, then repeats for the next token. A diffusion pass also moves model data, but it works on a block of output positions. O’Donoghue’s illustrative calculation uses 24 passes to generate 256 tokens: roughly ten times fewer memory transfers, with roughly tenfold speed possible only if execution remains truly memory-bound.

The condition matters: once arithmetic becomes the bottleneck, reducing transfer count no longer translates directly into the same latency reduction.

O’Donoghue reports approximately 2,000 tokens per second delivered in the Gemini Diffusion browser demo, including prefill, with speed depending on query and output length. Longer outputs amortize the initial processing of the prompt; a one-token answer can be dominated almost entirely by that prefill cost. The figure describes the browser experience he reports, not a hardware-independent decoding rate.

The arithmetic demonstration from Google I/O exposes what bidirectional generation makes possible. The prompt involves the square root of 81 and two-thirds squared, followed by additional arithmetic; the target answer is 39. After its first forward pass, Gemini Diffusion has already written an opening answer of 60 and begun a solution underneath it. That answer is still part of a mutable canvas.

On the second pass, the opening answer becomes 49 and the reasoning advances. On the third, the solution reaches 36 + 3 = 39, and the model changes the earlier answer to match.

Additional passes still clean up remaining text. The demonstration therefore shows the core correction by the third pass, not a complete response produced in exactly three passes. Later tokens provide information that can repair an earlier position before the answer is finalized.

O’Donoghue contrasts this with historical runs of the same prompt on two models he describes as much larger. GPT-4o initially answered 40, worked through the reasoning, and then appended a correction to 39. Gemini 2.5 Flash answered 42 and retained that mistake, eventually treating 36 + 3 as 42. These examples illustrate two consequences of committing an answer before completing its justification: a correction must be appended, or subsequent reasoning may accommodate the initial error. O’Donoghue acknowledges that modern thinking models can address this problem; diffusion’s distinctive capability is to revise the existing answer directly within the canvas.

Dynamic computation means increasing the available inference budget. More denoising passes give the model additional opportunities to inspect and repair an answer, even after it appears nearly clean. Across six internal coding evaluations, O’Donoghue reports generally improving quality as the denoising-step budget increases, though not strictly monotonically.

Adaptive computation is a separate choice: train the model to determine when it has finished, so different requests consume different amounts of the available budget. The preview examples make the difference visible:

O’Donoghue attributes the short pi run to memorization: the model can recover a familiar sequence quickly. He contrasts its four refinement passes with only four tokens from autoregressive generation in the same time. The code and explanatory paragraph use more passes because the model chooses to keep refining them.

The same pattern appears across tasks in the preview’s evaluations. GPQA Diamond was difficult for the targeted model size and took longer; Mostly Basic Python Problems, or MBPP, generally required less time. The stopping decision came from the model rather than a manually assigned step count for each benchmark. These observations concern the older preview and its evaluation setup.

Image inpainting provides a useful analogy for editing text. Remove part of an image and a diffusion model can fill the gap using the surrounding pixels and an instruction. The slide illustrates localized changes to a dog’s appearance. Unlike raster-order generation, which proceeds left to right and top to bottom, diffusion can condition a missing region on context throughout the image.

The text demonstrations apply the same principle at different scales. First, a request to fix a code bug produces a local indexing correction. Next, a request for documentation inserts explanatory text into the code. Finally, a story receives a new middle paragraph that can depend on both the first and third paragraphs. The important operation is context-preserving insertion or revision: the model uses material on both sides of the edit instead of simply regenerating every token from the beginning.

Low latency becomes more interesting when generation moves into an interaction loop. The first application demonstration resembles Wikipedia, but both the article text and its HTML are generated on demand. Clicking a link produces the next page quickly enough to resemble ordinary navigation. The model is generating the page itself, not merely answering a question inside a fixed interface.

A Reddit-style demonstration extends that idea to invented communities, posts, and comments, including a sharks-themed community. Gemini Diffusion generates text and HTML, while a separate image model supplies the pictures. O’Donoghue tentatively recalls that image model as Juno, from before Nano Banana; the text model itself does not generate images. The slower pictures arrive after the text, which he describes as about a minute later, while the site remains an interactive, generated simulation.

The next demonstration presents an operating-system simulation. Each click prompts generation of the next screen: opening a README produces its contents, other interactions produce web pages, and navigation can return to the desktop. The application opportunity is an interface whose content and screens are synthesized as the user explores them, provided generation stays within the latency budget of that interaction.

An external user’s demonstration moves the interaction loop into software creation. O’Donoghue corrects his introduction from an API to the Gemini Diffusion webpage: that is the interface used in this historical demo. The spoken requests build a to-do application incrementally:

1. Create the app and add 10 random to-dos.
2. Add a completed state and mark four random items complete.
3. Enable sorting by name and by state.
4. Test sorting, add an item, and delete several items.
5. Request dark mode.

The demonstrator reports that the tested interactions work and calls the sequence “15 seconds of work.” That is the demonstrator’s description, rather than an independently measured end-to-end duration.

The prepared talk ends with this product question: what becomes possible when a model responds quickly enough to keep up with a user’s next instruction? O’Donoghue anticipates another generation of models and invites engineers to explore those experiences, without supplying a release date.

The audience questions first separate data compatibility from algorithm compatibility. Asked about pretraining corpora and reinforcement learning from human feedback, O’Donoghue says the team uses the same data, but the algorithms need to change. He also confirms that diffusion models can be distilled, without describing a technique or identifying a published implementation. Asked about general availability, he anticipates a release soon but offers neither a date nor a definite general-availability commitment.

Increasing model size introduces another compute trade-off. O’Donoghue reports that larger models tend to need fewer denoising passes for the same output. Each pass costs more FLOPs, but fewer passes can partially offset that increase; he provides no quantitative scaling curve. Commercial pricing remains a separate, unanswered question: when asked whether tokens are the right billing unit, he says pricing has not yet been determined.

A text response has no predetermined dimensions like an image, so how large should the diffusion canvas be? The simplest approach uses a fixed-length window and generates successive windows autoregressively. Another option is a prediction head that estimates response length. A fixed window does not impose a fixed total answer length: the model can continue through as many windows as needed.

Adaptive stopping also operates inside a limit. O’Donoghue clarifies that the earlier examples all had a maximum denoising-step budget; the model could finish before exhausting it. This gives the system a cap on refinement work while preserving the ability to return easy responses early.

The next question establishes the boundary of self-correction. In the team’s implementation, completed windows are frozen before generation continues. Revisiting them is conceivable, but it is not the behavior he describes. Tokens are revisable within the active window; earlier completed windows are committed.

That makes the architecture a hybrid already: diffusion within blocks and autoregression across blocks. Context prefill remains the same; generation then proceeds in fixed-size blocks. O’Donoghue offers 512, 1,000, and 32 tokens as illustrative sizes, not a single disclosed configuration. Calling the process autoregressive across blocks does not mean that each token inside the block is generated sequentially.

Nor does the process require diffusion in a latent embedding space. A conventional prediction head can produce logits over token IDs. In the discrete process described here, every denoising step takes tokens in and returns tokens out. Latent-space approaches also exist, but O’Donoghue characterizes most contemporary text-diffusion work as discrete.

For the near term, O’Donoghue expects diffusion and autoregression to serve different use cases. On a phone or robot, a model may serve one local interaction rather than share an accelerator with thousands of unrelated requests. That removes much of the large-batch throughput advantage of autoregressive serving. He reports that text diffusion is already used in some on-device applications within Alphabet, mentioning robotics.

Asked whether diffusion can reach frontier-model quality, he identifies throughput rather than quality as the hosted-serving obstacle and affirms that his team’s research is roughly at that quality level. No supporting frontier benchmark is presented in this exchange. The deployment decision he describes is therefore about whether the application values a fast individual response enough to accept the cost of repeated refinement.

The remaining training questions receive brief confirmations. Reinforcement learning for reasoning is possible with algorithm changes. Diffusion and autoregressive models can also be combined, and O’Donoghue confirms that his team has done so, without explaining the implementation.

Initializing the canvas with a smaller model’s draft sounds like a way to avoid starting from noise, but it changes the input distribution. A denoiser trained on noise expects noise. To refine a smaller model’s output reliably, it would need training on that kind of output instead. O’Donoghue says the team does not usually take this route because it adds complexity. Asked for an implementation paper, he offers none from his team and points generally to external literature.

The final question turns the objective around: what happens if the user asks the model to generate noise? O’Donoghue says he does not know and does not think the team has tried it. His guess that it would produce something remains an open experiment—a fitting boundary between a demonstrated denoising mechanism and behavior that still needs to be tested.