The Best Models Still Reason Like Toddlers — Andrew Dai, Elorian

Show a model part of a chessboard and ask how many white squares are visible. In Andrew Dai’s example, the answer is 32: the number on a complete board. The model recognizes the object, supplies a familiar fact about it, and answers a different question from the one asked. Dai, Elorian’s co-founder and CEO, opens with this gap between recognizing a scene and inspecting its contents. These are examples he reports from frontier models, rather than a measured failure rate across all models or prompts.

The shortcut has a clear causal structure: identify a chessboard, recall that a complete chessboard has 32 white squares, then treat the partial image as though it contained the whole board. Pattern matching makes models useful at identifying plants, animals and flowers. Here, that same strength overrides the evidence needed for counting. The category is correct; the answer about the visible scene is wrong.

Board games make the distinction unusually easy to test. Dai, an enthusiastic collector, challenges the audience to put chess pieces in random positions and ask a model to locate all of them. Recognizing chess does little to solve that task: every piece needs a position grounded in this particular arrangement.

Catan exposes another way to avoid looking closely. Asked how many roads the blue player has on the board, one model response counted 10 blue roads off to the side and inferred that five must be on the board. Dai reports seven when the placed roads are counted directly. The inferred inventory supplies a plausible answer, but the question concerns the pieces actually placed. Extensive thinking does not help if the reasoning starts from a substitute for the visual evidence.

The next example adds time. A robot arm manipulates a cup and a cooker, lifts a lid, and later turns on the stove. Dai reports that models miss both the lid lift and the stove activation. Identifying the objects is insufficient: an account of the sequence must retain what changed earlier and notice what changes now. He calls the failure “context amnesia”—losing track of events and consistency across a long video.

Dai’s practical test for separating visual understanding from visual reasoning is to ask how long a person would need to answer. Naming the game, identifying a flower, or counting only three chess pieces can happen in less than a second. Inspecting a crowded board and checking its detailed arrangement takes longer. The one-second test is a design heuristic: it distinguishes a quick recognition task from a task that requires deliberate inspection, rather than establishing a universal timing threshold for model capability.

That distinction matters when designing an application. A prompt that sounds simple can still demand several visual operations: find the relevant objects, distinguish them from nearby objects, inspect their locations, and preserve those distinctions while answering. Dai’s immediate advice is to keep visual tasks simple when relying on existing frontier models. A request for more detailed inspection can move the task into precisely the territory where his examples fail.

High benchmark scores can obscure this distinction. In the benchmark discussion, Dai names ARC-AGI and objects to treating reported scores of 85% or 90% as a percentage of progress toward AGI. His resolution criticism concerns tasks he describes as using 32 × 32 or 64 × 64 pixel images: success on a tiny visual problem does not establish an ability to handle the detail in a complex real-world scene. Those dimensions are Dai’s characterization of the tasks, not a verified specification of the benchmark. This is a criticism of transfer from the benchmark to deployment, not a demonstration that small visual puzzles have no reasoning value.

A multimodal science exam can have a different weakness. Dai introduces an evaluation rendered as “MMU” in the captions and describes it as a multimodal extension of MMLU; its exact name remains uncertain. Adding images to text questions does not necessarily make detailed image inspection necessary. Dai describes questions that can be answered without the image, or with only a rough recognition of what it depicts. Such a test can reward scientific knowledge and category recognition while leaving counting, geometry and spatial relationships largely untested.

The proposed change is to evaluate abilities that visual applications actually need, including geometric alignment and spatial intelligence. A useful test must make the relevant relationship in the image necessary to the answer. This is a call for better industry benchmarks; the presentation does not define a consensus process or a completed shared evaluation framework. Dai connects this measurement gap to an opportunity: AI adoption has concentrated on coding, mathematics and documents, while many industries organized around visual work still lack systems that can handle their detailed requirements.

Dai calls the missing capability visual thinking. Two established kinds of visual AI explain what he means:

• Generation: High-fidelity images and videos can look convincing while lacking physical grounding and causal logic. Dai uses explosions and collapsing buildings as examples: Hollywood-style results may reproduce the disaster imagery found in films and game engines. His explanation is that the training distribution rewards that appearance; it is not an analysis of a particular generator’s training corpus.
• Understanding: Tools such as Google Lens and object detectors map pixels to labels, objects or segmented regions. Dai uses Lens to identify plants and flowers, and names YOLO as an object-detection example. These tools perform useful recognition, but their outputs alone do not answer a complex question about what should happen next.

Visual thinking adds active reasoning about space and time. The intended output is logic useful for planning, agent workflows and physical execution. This follows directly from the earlier failures: a system needs to inspect which objects are present, track their relationships and changes, and turn that information into something an action system can use.

Elorian’s approach combines four parts:

• Task-specific data: Collect and generate multimodal data aimed at visual reasoning. Dai says the needed data cannot simply be obtained online.
• A synthetic data flywheel: Use evaluations, agents, supervised fine-tuning (SFT) and reinforcement learning (RL) to improve the model.
• Architecture changes: Make improvements on top of a transformer-based architecture.
• Visual chain of thought: Let intermediate reasoning happen in visual space, rather than expressing all intermediate steps as text.

The presentation gives a concrete example of the last part, but leaves the architecture changes and the detailed operation of the training flywheel unspecified. These ingredients describe the approach without establishing their comparative contribution to performance.

The concrete question is: how many red hotels are built in this photo? The first step identifies hotels, drawing boxes around hotels and other objects. The next step narrows that visual selection to the red hotels. The observable change is from a photo containing many objects, to boxed candidates, to the subset relevant to the question. Unlike the chessboard shortcut, the intermediate steps remain attached to objects in this image.

What changes between the original photo and the red-hotel selection? The diagram follows the visual intermediate steps, with the bounding-box annotations carried into the narrowing step. Object identification comes before color filtering, so the system can narrow a set of located candidates rather than jump directly from the scene’s general appearance to an answer. The example explains the mechanism; it does not supply a final hotel count or an accuracy result.

The team background connects this proposal to experience building language and multimodal models. Dai describes 12 years at Google Brain and DeepMind, first authorship of work introducing pretraining and fine-tuning, and leadership roles on GLaM, PaLM 2 pretraining and Gemini data. Elorian has roughly 20 people at the time of the talk, including chief reasoning architect Dustin Tran, previously a post-training lead at xAI. This is the experience behind the research direction, rather than evidence that the proposed visual capabilities are already solved.

Robotics brings the temporal problem back into an action loop. Dai identifies complex, real-time physical actions in dynamic environments as a bottleneck. Static images and directed videos do not provide the same experience as active physical interaction, and he describes existing methods as overengineered and brittle. Elorian’s proposed API would deliver action-relevant scene understanding into existing robot planning and control systems. It is an integration point for better perception and reasoning, rather than a replacement for the entire control stack. At the June 2026 presentation, release by the end of the year was a plan.

Construction adds written rules to the visual scene. Safety requirements can vary by zone and change over time. Detecting a worker or a helmet is only part of the job: a system must connect policy language to the situation at the site and understand the spatial relationships that make the rule relevant. Dai’s examples include counting workers wearing helmets and checking whether construction follows previously defined plans.

The maintenance burden is important. Dai describes training separate models for different use cases and repeatedly retraining them as requirements change. The proposed alternative is a model that handles both language and vision: ground a video stream in written safety regulations, interpret the site policy, and use the camera infrastructure already installed. The practical promise is that policy text can participate directly in the reasoning task; the presentation describes this as a capability being built.

Architecture and mechanical design make counting errors consequential. A house with four bedrooms instead of the requested three is wrong even if it looks like a perfectly plausible house. The board-game problem returns at a larger scale: the system must preserve quantities and spatial constraints while working through a design.

Dai recounts a conversation with a mechanical engineering company: designing one small part of a robot testing platform took 100–200 hours, and he estimated 2,000–3,000 human hours for the whole platform. Those are figures from a reported company conversation, not an industry-wide productivity study. The obstacle he describes is detailed visual context across blueprints, 3D CAD and CAM files. Companies trying frontier models encountered errors in that context, leaving much of the constraint reasoning manual.

The proposed workflow pairs multimodal reasoning with validation. First, extract the geometric logic that matters to the design. Then use programmatic checks or simulation to test whether the mechanical device would work. Dai compares this to running code against unit tests: a design needs a checking mechanism beyond its apparent plausibility. He also suggests that the approach could extend to other design work, including the web and fashion, while keeping mechanical design as the concrete validation example.

Where does checking enter this proposed design workflow? The diagram separates extracting geometric logic from validating it. A simulator provides a test of the mechanical proposal, much as a unit test checks a program; generating or interpreting the design and checking its behavior are distinct jobs.

CAD and CAM quality control is another proposed application. Dai closes with the ambition of using visual reasoning to help produce faster cars, more efficient rockets and better batteries. Software remains useful for testing, but the objects being designed have geometry and physical constraints that must be understood. That is the opportunity the talk leaves open: bring the discipline of reasoning and validation into work whose essential content is visual.