What Is a Humanoid Foundation Model? An Introduction to GR00T N1

What happens when there is more physical work to do than people available to do it? Annika Brundyn and Aastha Jhunjhunwala, members of NVIDIA’s GR00T N1 team, open with that question. The motivating McKinsey chart covers 30 advanced economies: vacancies per unemployed person increased 4.2× between 2010 and 2023. That is a measure of labor-market tightness, not total job growth; more vacancies than unemployed workers occurred in seven of those economies, rather than across all 30.

The highlighted sectors make the engineering problem concrete: leisure and hospitality, healthcare, construction, transportation, and manufacturing. These jobs involve instruments, objects, and physical environments. A language model can help interpret an instruction, but text generation alone cannot carry it out. Physical AI must turn understanding into action.

Why give that intelligence a human-shaped body? Doors, workspaces, tools, and appliances were built around people. Brundyn’s rationale is that a compatible physical form makes a generalist robot easier to fit into those environments—not that every useful robot must be humanoid. The barista robot downstairs provides the contrast: it makes good espresso, but it cannot even cook rice. Excellence at one physical task does not automatically transfer to another.

Building such a robot involves a repeating lifecycle: collect or generate data, train a model on real and synthetic examples, and deploy it on the robot. NVIDIA calls this the three-computer problem because each stage puts different demands on the hardware.

Training can consume substantial centralized compute. Deployment imposes a different constraint: the model must be small and efficient enough to operate on the available edge device. These are workload examples, not a deployment recipe.

Project GR00T encompasses the compute infrastructure, software, and research needed for humanoid and other robots. GR00T N1 is the foundation model within that larger effort, announced at GTC in March 2025. The speakers describe it as open source and customizable. They round its size to two billion parameters; the paper specifies 2.2 billion. That is modest beside large language models but substantial for a robot. Its central promise is adaptation: start with shared foundation knowledge, then fine-tune for a particular robot body—its embodiment—and use case.

The data pyramid starts with a shortage. Language models can learn from enormous collections of text, but robotics has no comparable internet-scale collection of robots successfully performing tasks with corresponding action records. At the pyramid’s narrow top are real demonstrations: a human teleoperates a robot, perhaps using an Apple Vision Pro and gloves, and records the robot completing the task. These examples directly connect observations and actions to successful physical behavior.

That direct relevance comes at a cost. A robot and its operator occupy real time and equipment, and both need downtime. The slide’s 24 hours per robot per day is an illustrative upper limit, not achieved collection throughput. At the pyramid’s broad base, internet video offers abundance: a cooking tutorial may show a person solving exactly the kind of task a robot should learn. But it does not automatically supply the robot’s joint states or action labels. Human video is useful evidence about tasks, not a ready-made robot demonstration.

Synthetic data occupies the middle. A GPU can keep generating examples, but building a high-quality simulation environment requires skilled work. Another route starts with the teleoperation trajectories already collected and uses fine-tuned video or world foundation models to multiply them. Both the quality of that multiplication and the effective mixture of simulated and real examples remain research problems.

Brundyn introduces DreamGen as a recent Computex announcement in this data-generation effort. Its documented pipeline makes an essential bridge explicit: generated robot videos receive pseudo-action labels before they are used for policy learning. Generating a plausible-looking video alone does not produce commands a robot can execute. The larger data strategy combines task-grounded demonstrations, abundant human video, and synthetic examples rather than expecting any one source to solve the shortage.

Jhunjhunwala makes the model’s interface concrete with an industrial pick-and-place task. The input consists of an image observation, the robot’s current state, and a language instruction to pick up the industrial object and place it in the yellow bin. The output is a robot action trajectory. In the demonstration, the arm picks up the object and places it in the bin.

To a person watching, the output looks like a hand moving through space. At the control interface, it is a sequence of floating-point vectors that control the robot’s joints. Those numerical commands produce continuous motion. The distinction between state and action is temporal: state describes the robot’s situation now; action specifies what it should do next. Images provide visual context alongside that numerical state. A successful instruction-following model must connect all three inputs to usable motion, not merely describe what a successful motion would look like.

The architecture borrows its organizing analogy from Daniel Kahneman’s Thinking, Fast and Slow. System 2 interprets the task and supplies the context needed to act; the talk describes it as a slower planner that breaks complex work into simpler work. System 1 produces fast actions conditioned on that understanding. Jhunjhunwala describes System 1 as operating at almost 120 Hz; the paper identifies 120 Hz as the motor-action rate, not the frequency of a complete model inference. The planner analogy explains the division of labor without requiring a separate textual plan between the two systems.

Inside the action system, a state encoder converts the robot state into tokens, while an action encoder processes a noised action sequence. The talk relates that noise to imperfect sensors, but the published mechanism is deliberate flow-matching noise: training mixes demonstrated actions with sampled Gaussian noise. The model learns how to move from noise toward an action trajectory conditioned on the observations and instruction. Sensor error is a separate issue.

For the yellow-bin task, one training example therefore contains the observed scene, the current robot state, the instruction, and the demonstrated sequence of action vectors. The following Python expresses the noise-mixing operation on that action sequence. Here, t = 0 selects noise and t = 1 selects the demonstrated actions; target_velocity describes the direction between them.

import torch

def flow_matching_batch(expert_actions: torch.Tensor):
    # expert_actions: [batch, trajectory_length, action_dimensions]
    noise = torch.randn_like(expert_actions)
    t = torch.rand(
        expert_actions.shape[0], 1, 1,
        device=expert_actions.device,
        dtype=expert_actions.dtype,
    )
    mixed_actions = (1 - t) * noise + t * expert_actions
    target_velocity = expert_actions - noise
    return mixed_actions, t, target_velocity

This operation creates a training input and target; it does not execute the pick-and-place task. The encoded state and action tokens then enter a diffusion transformer with layers of self-attention and cross-attention.

The other branch supplies visual and language context. A vision encoder processes the image, and a text tokenizer processes the instruction. Their representations pass through Eagle 2, the vision-language model used in N1. Its output tokens condition the diffusion transformer through cross-attention. In the yellow-bin example, this connects the action-generation process to the scene and the requested destination. The action system does not generate motion independently of what the robot sees or what the user asked.

The transformer’s output tokens still are not commands for physical hardware. An embodiment-specific action decoder converts them into action vectors appropriate to the target robot, whether that is a humanoid hand or an industrial arm. The paper’s adaptation process also includes embodiment-specific state and action encoders and post-training; replacing the decoder alone is not the entire adaptation procedure. Shared representations make it possible to learn across embodiments, while the interfaces and adaptation connect that knowledge to one robot’s actual motion space.

Two broad approaches help explain how robot behavior is learned. Imitation learning uses expert demonstrations as the reference: the policy learns to match expert behavior. Reinforcement learning uses trial and error to optimize a reward, rather than requiring the policy to match a particular demonstrated trajectory.

Jhunjhunwala compares imitation to being told to behave like an older sibling: there is a specific exemplar to match. Reinforcement learning instead gives an objective to pursue. Each approach shifts the practical bottleneck.

The simulation-to-reality gap matters when reinforcement learning happens in simulation: behavior that earns reward in a modeled environment must still work with real physics, sensing, and hardware. The talk says N1 used both approaches in some ways, without identifying a reinforcement-learning component or stage. The published N1 training account establishes flow matching, so that broad remark should not be read as a specified RL training recipe.

The demonstration montage moves from kitchen pick-and-place tasks to a flowers-and-champagne scene, then to two robots performing industrial manipulation beside yellow bins. The polished social scene comes with a useful qualification: Jhunjhunwala acknowledges the fallen glasses and flowers behind the successful capture. A successful clip shows a capability, but not its reliability across attempts. The two-robot scene likewise shows an industrial task without establishing a particular coordination algorithm.

These examples illustrate why a reusable base model is appealing: related manipulation skills can support different settings and downstream tasks. Extending the model to other tasks and environments is the foundation-model ambition. The montage does not establish that the same checkpoint can already perform any task in any environment without further adaptation.

The closing discussion returns to the missing internet of action data. Simulation, teleoperation, and expert demonstrations are ways to build the experience from which a robot can learn. But collecting that experience is only part of the problem: the system that understands the instruction must also work coherently with the system that produces motion.

Co-training connects interpretation to action. Jhunjhunwala contrasts separately trained components, which can disagree, with N1’s joint optimization of System 1 and System 2. The goal is to improve the stack together rather than optimize each piece in isolation. Joint optimization does not mean every parameter changes: the paper freezes the language component during pre-training and post-training.

The resulting foundation knowledge is a starting point for adaptation, much as a base Llama model can be fine-tuned for a domain. In robotics, adaptation must account for both the downstream task and the body that performs it. GR00T N1’s generalist ambition is therefore a reusable learned foundation with interfaces and training that connect it to different embodiments—not a finished collection of specialist routines, and not a guarantee of universal physical competence.