Armen Aghajanyan

Armen Aghajanyan is the co-founder and CEO of Perceptron AI, where he is building models that connect visual understanding with physical action. His research spans language-model fine-tuning, joint text-and-image generation, and embodied foundation models: systems intended to understand a scene, reason about a task, and produce actions within one model.

From language models to multimodal intelligence

Aghajanyan began learning programming online while still in school. An early publication brought him into contact with Microsoft researcher Misha Bilenko and led to a Microsoft internship. He subsequently worked on language models at Microsoft before joining Meta’s FAIR research organization, where he spent nearly six years developing multimodal systems. His career account connects those early opportunities with his later work in AI research.

One strand of that research asked why large pretrained language models can adapt successfully to tasks with relatively little labeled data. With Sonal Gupta and Luke Zettlemoyer, he co-authored research on intrinsic dimensionality: the number of independent directions in which a model’s parameters need to change to learn a task. Their experiments showed that effective adaptation could occur within a much smaller parameter space than the model’s full size suggested. In one RoBERTa experiment, optimizing 200 parameters projected into the larger network reached approximately 90 percent of full fine-tuning performance on a paraphrase task. The result helped explain how extensive pretraining can make subsequent learning economical.

His work then expanded from adapting language representations to learning across text and images. He was the first author of CM3, a 2022 collaboration that trained a generative model on structured web documents containing text, markup, hyperlinks, and image tokens. Its training method moved masked passages to the end of a sequence, allowing the model to generate missing material with context from both sides. A single model could generate images, caption them, and fill gaps in multimodal documents.

Aghajanyan also co-authored CM3Leon, which extended this approach with retrieval-augmented pretraining and instruction tuning. It could generate both text and images, with applications including image editing and visual question answering. These projects developed a recurring theme in his work: bringing different kinds of information into a shared generative system, then finding training recipes that make the combination useful.

Taking multimodal research into robotics

In November 2024, Aghajanyan and fellow FAIR researcher Akshat Shrivastava founded Perceptron to develop physical-world intelligence. The transition extended their multimodal research into environments where recognizing an object is only part of the problem. A robot also needs to understand spatial relationships, follow a task through time, and produce actions appropriate to its hardware.

Aghajanyan connects this work to his research at FAIR on early fusion, which brings different modalities together early in a model’s processing. At Perceptron, the team applies that approach to perception, reasoning, and control. A book-sorting demonstration makes the connection concrete: one model reads a book’s title, identifies its type, and emits control tokens to place it in the appropriate bin. The task requires visual recognition and knowledge about the book to inform physical movement.

Perceptron’s model family includes commercial vision-language models and the open Isaac series. In August 2026, the team released Isaac 0.5, a sparse model with 36 billion parameters that combines video understanding, spatial grounding, task-progress estimation, and robot control. Images, language, robot state, and previous actions enter a shared sequence; the model can produce descriptions, coordinates, task-state outputs, or actions. Its release includes weights and companion training and inference code, giving other teams a starting point they can adapt to their own robots.

Learning what matters—and where to spend computation

Two related problems shape Aghajanyan’s approach to physical AI: deciding what a model should learn from visual experience and deciding which parts of that experience deserve computation.

• Semantic world modeling: Aghajanyan argues that useful physical prediction need not reconstruct every pixel of a future scene. His preferred direction is to predict changes that matter for the task, such as whether an object moved, a grasp held, contact occurred, or a procedure advanced. Sparse transcripts miss much of the information in video, while treating every pixel equally gives background details the same importance as a robot’s gripper or a contact point. Perceptron’s Isaac training approach uses descriptions derived from future observations to train future-percept prediction alongside perception and control. The detailed objective remains proprietary, although the model weights and substantial implementation code are public.

• Data sparsity and null experts: Continuous visual input contains repetitive or low-information material. Aghajanyan co-authored research combining weight and data sparsity that lets a model vary its computation across tokens. Ordinary mixture-of-experts routing chooses which learned experts process a token; zero-computation null routes also let it choose how much expert processing that token receives. The experiments found task-dependent allocation, preserving computation for useful visual details while skipping more redundant material. In his explanation of the approach, a request to segment fruit causes the model to allocate more computation to the fruit in an image.

Reducing the cost of robot learning

Aghajanyan’s team investigates how broad video experience can supplement expensive robot demonstrations. Perceptron’s Isaac 0.5 experiments measured combinations of video and robot data against held-out action-prediction loss. Increasing general-video training from 1,000 to one million hours reduced the teleoperation required to reach one specified loss threshold from roughly 5,900 hours to 28. This result applies to the tested training mixtures and prediction target; it does not establish an equivalent reduction for every deployed robot task.

That work depends on coordinating the training process. Aghajanyan’s positions on foundation-model development favor coherent ownership across data, pretraining, and later training stages. He emphasizes the risks of fragmented training efforts and dependence on distillation from another model. He also distinguishes the uses of public robot datasets: they can support intermediate training while remaining insufficient for the final adaptation of a precise robot policy.

He is candid about the gap between demonstrations and reliable operation. In his discussion of deployment limitations, temporal understanding remained insufficiently reliable for deployment. He reported improved tolerance of background and lighting changes from joint perception-and-control training, while acknowledging that stronger disturbances could still cause failure.

Research opportunity in Armenia

Aghajanyan’s interest in access to AI extends beyond model architecture. He has collaborated with YerevaNN and Armenian researchers and advocated greater investment in GPU infrastructure, arguing that strong engineers need computing resources and research opportunities to build ambitious technology.