# The Best Models Still Reason Like Toddlers — Andrew Dai, Elorian Andrew Dai, Elorian AI Engineer World's Fair 2026 [Talk page](https://ai.engineer/talks/A_I8mw8yfns-best-models-still-reason-like-toddlers) · [Video](https://www.youtube.com/watch?v=A_I8mw8yfns) ## The Best Models Still Reason Like Toddlers Andrew Dai traces counting and video-tracking failures to the gap between recognizing a scene and reasoning through its details, then explains Elorian’s approach to visual thinking and its proposed uses in robotics, construction and mechanical design. ### A familiar chessboard becomes the wrong answer [0:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=13s) Show a model part of a chessboard and ask how many white squares are visible. In [Andrew Dai](https://www.linkedin.com/in/andrewdai/)’s [example](https://www.youtube.com/watch?v=A_I8mw8yfns&t=72), the answer is 32: the number on a complete board. The model recognizes the object, supplies a familiar fact about it, and answers a different question from the one asked. Dai, Elorian’s co-founder and CEO, opens with this gap between recognizing a scene and inspecting its contents. These are examples he reports from frontier models, rather than a measured failure rate across all models or prompts. ![Source frame: A familiar chessboard becomes the wrong answer](https://ai.engineer/api/talk-essay-media/cf5675bf480070e8c7eb254759a694ff6380512e1a0381be2b33cb9315c6016d) Source frame: A familiar chessboard becomes the wrong answer The shortcut has a clear causal structure: identify a chessboard, recall that a complete chessboard has 32 white squares, then treat the partial image as though it contained the whole board. Pattern matching makes models useful at identifying plants, animals and flowers. Here, that same strength overrides the evidence needed for counting. The category is correct; the answer about the visible scene is wrong. Board games make the distinction unusually easy to test. Dai, an enthusiastic collector, challenges the audience to put chess pieces in random positions and ask a model to locate all of them. Recognizing chess does little to solve that task: every piece needs a position grounded in this particular arrangement. Catan exposes another way to avoid looking closely. Asked how many roads the blue player has on the board, one model response counted 10 blue roads off to the side and inferred that five must be on the board. Dai reports seven when the placed roads are counted directly. The inferred inventory supplies a plausible answer, but the question concerns the pieces actually placed. Extensive thinking does not help if the reasoning starts from a substitute for the visual evidence. ### Video needs remembered changes, not just recognized objects [3:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=204s) The [next example](https://www.youtube.com/watch?v=A_I8mw8yfns&t=204) adds time. A robot arm manipulates a cup and a cooker, lifts a lid, and later turns on the stove. Dai reports that models miss both the lid lift and the stove activation. Identifying the objects is insufficient: an account of the sequence must retain what changed earlier and notice what changes now. He calls the failure “context amnesia”—losing track of events and consistency across a long video. ![Source frame: Video needs remembered changes, not just recognized objects](https://ai.engineer/api/talk-essay-media/1d61f96ed1db696b7b8e178f7640a774b9d9c069c22cd87152e4391cc79eee17) Source frame: Video needs remembered changes, not just recognized objects Dai’s practical test for separating visual understanding from visual reasoning is to ask how long a person would need to answer. Naming the game, identifying a flower, or counting only three chess pieces can happen in less than a second. Inspecting a crowded board and checking its detailed arrangement takes longer. The one-second test is a design heuristic: it distinguishes a quick recognition task from a task that requires deliberate inspection, rather than establishing a universal timing threshold for model capability. That distinction matters when designing an application. A prompt that sounds simple can still demand several visual operations: find the relevant objects, distinguish them from nearby objects, inspect their locations, and preserve those distinctions while answering. Dai’s immediate advice is to keep visual tasks simple when relying on existing frontier models. A request for more detailed inspection can move the task into precisely the territory where his examples fail. ### What a visual benchmark actually makes the model do [5:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=344s) High benchmark scores can obscure this distinction. In the [benchmark discussion](https://www.youtube.com/watch?v=A_I8mw8yfns&t=344), Dai names **ARC-AGI** and objects to treating reported scores of 85% or 90% as a percentage of progress toward AGI. His resolution criticism concerns tasks he describes as using 32 × 32 or 64 × 64 pixel images: success on a tiny visual problem does not establish an ability to handle the detail in a complex real-world scene. Those dimensions are Dai’s characterization of the tasks, not a verified specification of the benchmark. This is a criticism of transfer from the benchmark to deployment, not a demonstration that small visual puzzles have no reasoning value. ![Source frame: What a visual benchmark actually makes the model do](https://ai.engineer/api/talk-essay-media/05e0ec03e0bb186277f1d03a058a6abaf0579107b5335b83100c7cec7ec06aee) Source frame: What a visual benchmark actually makes the model do A multimodal science exam can have a different weakness. Dai introduces an evaluation rendered as **“MMU”** in the captions and describes it as a multimodal extension of MMLU; its exact name remains uncertain. Adding images to text questions does not necessarily make detailed image inspection necessary. Dai describes questions that can be answered without the image, or with only a rough recognition of what it depicts. Such a test can reward scientific knowledge and category recognition while leaving counting, geometry and spatial relationships largely untested. The proposed change is to evaluate abilities that visual applications actually need, including geometric alignment and spatial intelligence. A useful test must make the relevant relationship in the image necessary to the answer. This is a call for better industry benchmarks; the presentation does not define a consensus process or a completed shared evaluation framework. Dai connects this measurement gap to an opportunity: AI adoption has concentrated on coding, mathematics and documents, while many industries organized around visual work still lack systems that can handle their detailed requirements. ### Generation and detection leave a reasoning gap [8:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=483s) Dai calls the missing capability visual thinking. Two established kinds of visual AI explain what he means: - **Generation:** High-fidelity images and videos can look convincing while lacking physical grounding and causal logic. Dai uses explosions and collapsing buildings as examples: Hollywood-style results may reproduce the disaster imagery found in films and game engines. His explanation is that the training distribution rewards that appearance; it is not an analysis of a particular generator’s training corpus. - **Understanding:** Tools such as Google Lens and object detectors map pixels to labels, objects or segmented regions. Dai uses Lens to identify plants and flowers, and names YOLO as an object-detection example. These tools perform useful recognition, but their outputs alone do not answer a complex question about what should happen next. ![Source frame: Generation and detection leave a reasoning gap](https://ai.engineer/api/talk-essay-media/e7e4f9ecb145028f30806397bb5286ada6c7df23511d1f33b9283951eb96cf15) Source frame: Generation and detection leave a reasoning gap Visual thinking adds active reasoning about space and time. The intended output is logic useful for planning, agent workflows and physical execution. This follows directly from the earlier failures: a system needs to inspect which objects are present, track their relationships and changes, and turn that information into something an action system can use. ### Make the intermediate steps visual [9:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=588s) Elorian’s approach combines four parts: - **Task-specific data:** Collect and generate multimodal data aimed at visual reasoning. Dai says the needed data cannot simply be obtained online. - **A synthetic data flywheel:** Use evaluations, agents, supervised fine-tuning (SFT) and reinforcement learning (RL) to improve the model. - **Architecture changes:** Make improvements on top of a transformer-based architecture. - **Visual chain of thought:** Let intermediate reasoning happen in visual space, rather than expressing all intermediate steps as text. The presentation gives a concrete example of the last part, but leaves the architecture changes and the detailed operation of the training flywheel unspecified. These ingredients describe the approach without establishing their comparative contribution to performance. ![Source frame: Make the intermediate steps visual](https://ai.engineer/api/talk-essay-media/e2211b11bc97fc4fb5189a52a72d94c6eb42583a01c847ead21f4e75fbfb90dd) Source frame: Make the intermediate steps visual The [concrete question](https://www.youtube.com/watch?v=A_I8mw8yfns&t=633) is: how many red hotels are built in this photo? The first step identifies hotels, drawing boxes around hotels and other objects. The next step narrows that visual selection to the red hotels. The observable change is from a photo containing many objects, to boxed candidates, to the subset relevant to the question. Unlike the chessboard shortcut, the intermediate steps remain attached to objects in this image. What changes between the original photo and the red-hotel selection? The diagram follows the visual intermediate steps, with the bounding-box annotations carried into the narrowing step. Object identification comes before color filtering, so the system can narrow a set of located candidates rather than jump directly from the scene’s general appearance to an answer. The example explains the mechanism; it does not supply a final hotel count or an accuracy result. The team background connects this proposal to experience building language and multimodal models. Dai describes 12 years at Google Brain and DeepMind, first authorship of work introducing pretraining and fine-tuning, and leadership roles on **GLaM**, **PaLM 2** pretraining and **Gemini** data. Elorian has roughly 20 people at the time of the talk, including chief reasoning architect Dustin Tran, previously a post-training lead at xAI. This is the experience behind the research direction, rather than evidence that the proposed visual capabilities are already solved. From a photo to the relevant hotel subset The visual chain-of-thought example locates candidates before narrowing them by color. The boxes are visual annotations connecting the located candidates to the red-hotel selection; the diagram does not specify an internal annotation format. - Photo and question — How many red hotels are built? - Box the candidates — Identify hotels; draw boxes around hotels and other objects. These annotations locate the candidates in the photo. - Keep the red hotels — Narrow the visual selection to the objects asked about. - Photo and question → Box the candidates (Identify objects and add box annotations) - Box the candidates → Keep the red hotels (Narrow the boxed candidates by color) ### Connect visual evidence to actions and written rules [12:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=744s) Robotics brings the temporal problem back into an action loop. Dai identifies complex, real-time physical actions in dynamic environments as a bottleneck. Static images and directed videos do not provide the same experience as active physical interaction, and he describes existing methods as overengineered and brittle. Elorian’s proposed API would deliver action-relevant scene understanding into existing robot planning and control systems. It is an integration point for better perception and reasoning, rather than a replacement for the entire control stack. At the June 2026 presentation, release by the end of the year was a plan. ![Source frame: Connect visual evidence to actions and written rules](https://ai.engineer/api/talk-essay-media/bbe774d6fc7b95d740121d7bb4ad1f2ae7da66d138e3cef001456ebf4cd0ceec) Source frame: Connect visual evidence to actions and written rules Construction adds written rules to the visual scene. Safety requirements can vary by zone and change over time. Detecting a worker or a helmet is only part of the job: a system must connect policy language to the situation at the site and understand the spatial relationships that make the rule relevant. Dai’s examples include counting workers wearing helmets and checking whether construction follows previously defined plans. The maintenance burden is important. Dai describes training separate models for different use cases and repeatedly retraining them as requirements change. The proposed alternative is a model that handles both language and vision: ground a video stream in written safety regulations, interpret the site policy, and use the camera infrastructure already installed. The practical promise is that policy text can participate directly in the reasoning task; the presentation describes this as a capability being built. ### Mechanical design needs geometric reasoning and a way to check it [14:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=886s) Architecture and mechanical design make counting errors consequential. A house with four bedrooms instead of the requested three is wrong even if it looks like a perfectly plausible house. The board-game problem returns at a larger scale: the system must preserve quantities and spatial constraints while working through a design. ![Source frame: Mechanical design needs geometric reasoning and a way to check it](https://ai.engineer/api/talk-essay-media/1c8636bf426e5da50f7f40d3422e37349278022543970a4df5dd614088a8e83d) Source frame: Mechanical design needs geometric reasoning and a way to check it Dai recounts a conversation with a mechanical engineering company: designing one small part of a robot testing platform took 100–200 hours, and he estimated 2,000–3,000 human hours for the whole platform. Those are figures from a reported company conversation, not an industry-wide productivity study. The obstacle he describes is detailed visual context across blueprints, 3D CAD and CAM files. Companies trying frontier models encountered errors in that context, leaving much of the constraint reasoning manual. The [proposed workflow](https://www.youtube.com/watch?v=A_I8mw8yfns&t=983) pairs multimodal reasoning with validation. First, extract the geometric logic that matters to the design. Then use programmatic checks or simulation to test whether the mechanical device would work. Dai compares this to running code against unit tests: a design needs a checking mechanism beyond its apparent plausibility. He also suggests that the approach could extend to other design work, including the web and fashion, while keeping mechanical design as the concrete validation example. Where does checking enter this proposed design workflow? The diagram separates extracting geometric logic from validating it. A simulator provides a test of the mechanical proposal, much as a unit test checks a program; generating or interpreting the design and checking its behavior are distinct jobs. CAD and CAM quality control is another proposed application. Dai closes with the ambition of using visual reasoning to help produce faster cars, more efficient rockets and better batteries. Software remains useful for testing, but the objects being designed have geometry and physical constraints that must be understood. That is the opportunity the talk leaves open: bring the discipline of reasoning and validation into work whose essential content is visual. Geometric reasoning followed by validation The proposed workflow connects visual design context to a separate programmatic or simulation check. - Visual design context — Blueprints, 3D CAD and CAM files. - Extract geometric logic — Use multimodal reasoning to interpret the design’s geometry. - Validate the design — Use programmatic checks or mechanical simulation to assess whether it will work. - Visual design context → Extract geometric logic (Multimodal reasoning) - Extract geometric logic → Validate the design (Programmatic or simulation checks) ## Key takeaways - Recognizing an object can supply the wrong answer when a task requires inspecting its visible parts. The partial chessboard and Catan examples show familiar patterns replacing direct counts. - Visual reasoning includes tracking changes over time. A robot video requires retaining earlier actions and noticing later ones, beyond identifying the objects in view. - Evaluate whether the image’s detailed relationships are necessary to answer the question. Tiny visual tasks and questions answerable from text or rough recognition do not establish readiness for complex visual work. - Visual intermediate steps can keep reasoning attached to the scene: locate hotel candidates with boxes, then narrow the selection to red hotels. - The proposed applications connect visual reasoning to an existing checking or action system: robot planners and controllers, written construction policies, or programmatic and simulation validation for mechanical designs. ## Slides ### The Best Models Still Reason Visually Like Toddlers Andrew Dai argues that frontier models still lag humans at visual reasoning. [0:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=27s) [Title slide for Andrew Dai’s talk on visual reasoning.](https://assets.ai.engineer/sites/aiecode2025/sha256/a2064aa67b373435679f02a6e1856eebaa33426ba31540a2709ac5ce56768818.webp) ELORIAN The Best Models Still Reason Visually Like Toddlers Andrew Dai June 2026 ### Static Visual Reasoning Failures Dai says models fill in a partially visible chessboard and answer 32 white squares. In the Catan image, he says a model guessed 5 blue roads where he counted 7. [0:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=57s) [Chessboard and Catan examples illustrate claimed counting and spatial reasoning failures.](https://assets.ai.engineer/sites/aiecode2025/sha256/3c12d88f0df72ff34b11a539a142af57101c31a589bdccc6183a8bd773d168f0.webp) Where Frontier Models Break Down: Static Visual Reasoning Failures Enumeration: Chessboard Hallucination Prompt: "How many white squares are in the picture?" Typical VLM Failure: "There are 32 white squares." Why it breaks: Models rely on pattern matching, leading to hallucination in cluttered or occluded environments. Lacks the mechanisms required for precise, sequential instance tracking. Spatial Understanding: Catan Board Prompt: "How many roads does the blue player have?" Typical VLM Failure: "The blue player currently has 5 roads." Why it breaks: Models excel at semantic guessing but completely fail at spatial grounding and detailed questions. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Temporal and Embodied Reasoning Failures A robot arm manipulates a cup at a stove. The presenter says the model’s caption misses both lifting the lid and turning on the stove, illustrating how it loses track of actions across a video. [3:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=211s) [A robot arm holds a cup near a stovetop beside text discussing missed actions in video captioning.](https://assets.ai.engineer/sites/aiecode2025/sha256/02773fc3425f7f55e0a4c7319049a1c5cf5cc4ad689a5de97f178ee40bdd9c40.webp) Where Frontier Models Break Down: Temporal & Embodied Reasoning Failures Video Captioning: Robotic Manipulation Prompt: "Describe the exact sequence of actions the robot arm took to complete the task." SOTA Model Output: “[Misses lifting the lid] ...The robotic arm approaches the small measuring cup filled with oats on the stovetop. The gripper grasps the sides of the measuring cup to secure a firm hold. The arm lifts the cup and positions it directly over the center of the small frying pan. The arm rotates to pour the oats into the pan. The arm tilts the empty cup back upright and places it back down in its original position on the stove. [Misses turning on the stove] [Context Amnesia]” Why it breaks: • Fails to maintain consistency across multi-step visual understanding. • Models suffer from "context amnesia," losing track of state changes over extended video sequences. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Robot Demonstration: Arm Over the Pan A model describes the robot pouring oats but misses lifting the lid and turning on the stove, illustrating how it can lose track of state changes across a multi-step video. [3:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=220s) [The robot arm is positioned over a pan on the stovetop, alongside the same video-captioning analysis.](https://assets.ai.engineer/sites/aiecode2025/sha256/9db2ec43ff704b425cee438a511998061f90a53877b0bb8d1705da868225b2cf.webp) Where Frontier Models Break Down: Temporal & Embodied Reasoning Failures Video Captioning: Robotic Manipulation Prompt: "Describe the exact sequence of actions the robot arm took to complete the task." SOTA Model Output: “[Misses lifting the lid] ...The robotic arm approaches the small measuring cup filled with oats on the stovetop. The gripper grasps the sides of the measuring cup to secure a firm hold. The arm lifts the cup and positions it directly over the center of the small frying pan. The arm rotates to pour the oats into the pan. The arm tilts the empty cup back upright and places it back down in its original position on the stove. [Misses turning on the stove] [Context Amnesia]” Why it breaks: • Fails to maintain consistency across multi-step visual understanding. • Models suffer from "context amnesia," losing track of state changes over extended video sequences. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Video Captioning: Robotic Manipulation A robot handles a cup and stove, but the model’s caption misses the earlier lid lift and the stove being turned on. Dai calls this context amnesia: losing track of actions across a video. [3:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=224s) [The robot arm is raised above the stove, with a cup beside a pan containing oats.](https://assets.ai.engineer/sites/aiecode2025/sha256/e88f05c0202bcf4e3f5df3db8bfb76af4cc1d093609510432e3c757466c60f9d.webp) Where Frontier Models Break Down: Temporal & Embodied Reasoning Failures Video Captioning: Robotic Manipulation Prompt: "Describe the exact sequence of actions the robot arm took to complete the task." SOTA Model Output: “[Misses lifting the lid] ...The robotic arm approaches the small measuring cup filled with oats on the stovetop. The gripper grasps the sides of the measuring cup to secure a firm hold. The arm lifts the cup and positions it directly over the center of the small frying pan. The arm rotates to pour the oats into the pan. The arm tilts the empty cup back upright and places it back down in its original position on the stove. [Misses turning on the stove] [Context Amnesia]” Why it breaks: • Fails to maintain consistency across multi-step visual understanding. • Models suffer from "context amnesia," losing track of state changes over extended video sequences. ELORIAN PROPRIETARY AND CONFIDENTIAL [3:53](https://www.youtube.com/watch?v=A_I8mw8yfns&t=233s) [A robotic arm beside a stove illustrates a video-captioning example about missed actions and tracking state changes.](https://assets.ai.engineer/sites/aiecode2025/sha256/af799c90627fefb53de44ffac09fd44759e85960d4905c9749666f19af4ea0f8.webp) Where Frontier Models Break Down: Temporal & Embodied Reasoning Failures Video Captioning: Robotic Manipulation Prompt: "Describe the exact sequence of actions the robot arm took to complete the task." SOTA Model Output: “[Misses lifting the lid] ...The robotic arm approaches the small measuring cup filled with oats on the stovetop. The gripper grasps the sides of the measuring cup to secure a firm hold. The arm lifts the cup and positions it directly over the center of the small frying pan. The arm rotates to pour the oats into the pan. The arm tilts the empty cup back upright and places it back down in its original position on the stove. [Misses turning on the stove] [Context Amnesia]” Why it breaks: • Fails to maintain consistency across multi-step visual understanding. • Models suffer from "context amnesia," losing track of state changes over extended video sequences. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Static Visual Reasoning Failures Dai contrasts recognizing a game with counting white squares or blue Catan roads. The slide presents model answers to the counting questions as failures. [4:34](https://www.youtube.com/watch?v=A_I8mw8yfns&t=274s) [Chessboard and Catan board examples illustrate counting and spatial grounding failures.](https://assets.ai.engineer/sites/aiecode2025/sha256/f48448198d2c0b6251ebca217f297d0cd4d0fc4bd4a5d13cf459e100cfc9becc.webp) Where Frontier Models Break Down: Static Visual Reasoning Failures Enumeration: Chessboard Hallucination Prompt: “How many white squares are in the picture?” Typical VLM Failure: “There are 32 white squares.” Why it breaks: Models rely on pattern matching, leading to hallucination in cluttered or occluded environments. Lacks the mechanisms required for precise, sequential instance tracking. Spatial Understanding: Catan Board Prompt: “How many roads does the blue player have?” Typical VLM Failure: “The blue player currently has 5 roads.” Why it breaks: Models excel at semantic guessing but completely fail at spatial grounding and detailed questions. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Visual Reasoning Needs Better Evaluation Dai argues that ARC-AGI’s low-resolution grids miss real-world complexity and many MMMU questions can be answered without looking at the image. He calls for benchmarks that isolate spatial reasoning and object permanence. [5:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=342s) [Three panels discuss ARC-AGI, MMMU and the need for benchmarks of real visual reasoning.](https://assets.ai.engineer/sites/aiecode2025/sha256/f4a2e3e12a319af49a861ce23158a527942b4077c5f369da20c8d2db7bf1b258.webp) Visual Reasoning Needs Better Evaluation ARC-AGI Trapped in Virtual Environments Real world problems cannot be reduced to 32x32 or 64x64 pixels. Fails to capture the high-resolution, continuous spatial dynamics of the real-world. MMMU Text-Biased Pseudo-Vision In benchmarks like Massive Multi-discipline Multimodal Understanding, models often bypass visual understanding entirely. They rely heavily on LLM priors and text hints embedded in the prompts, creating an illusion of multimodal capability. ? Benchmark Pioneering Real Visual Reasoning The industry needs rigorous benchmarks that isolate geometric alignment, object permanence, and spatial intelligence. ELORIAN [6:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=387s) [Three columns discuss ARC-AGI, MMMU, and a proposed visual reasoning benchmark.](https://assets.ai.engineer/sites/aiecode2025/sha256/11aafa94ef42ce002eb387af09c5b063d1aa51804e3066b4dc3efaeb26566567.webp) Visual Reasoning Needs Better Evaluation ARC-AGI Trapped in Virtual Environments Real world problems cannot be reduced to 32x32 or 64x64 pixels. Fails to capture the high-resolution, continuous spatial dynamics of the real-world. MMMU Text-Biased Pseudo-Vision In benchmarks like Massive Multi-discipline Multimodal Understanding, models often bypass visual understanding entirely. They rely heavily on LLM priors and text hints embedded in the prompts, creating an illusion of multimodal capability. ? Benchmark Pioneering Real Visual Reasoning The industry needs rigorous benchmarks that isolate geometric alignment, object permanence, and spatial intelligence. ELORIAN ### The Missing Paradigm in Visual AI Dai describes visual thinking as a frontier beyond image generation and passive perception: spatial and temporal intelligence that can support planning and physical execution. [8:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=481s) [Three columns compare generation, understanding, and thinking in visual AI.](https://assets.ai.engineer/sites/aiecode2025/sha256/e0d65f2d259063c88c71f7f2474d9eb7059b7dd15c52bfcb0f9e57cde5ab3b60.webp) The Missing Paradigm in Visual AI Generation • High-fidelity pixel generation (e.g., Diffusion). • Great visual appeal but lacks physical grounding and causal logic (hollywood-style/game style physics in place of real physics). Understanding • Mapping pixels to semantic labels (e.g., Google Lens, SAM3, YOLO, Mask R-CNN). • Highly robust, but fundamentally passive perception. THE FRONTIER Thinking • Active spatial and temporal intelligence. • Extracting actionable logic for multi-step planning, agentic workflows, and real-world/physical execution. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Elorian’s Approach Elorian combines human-labeled multimodal data, a synthetic data flywheel and architecture changes to enable visual chain-of-thought reasoning. The flywheel uses evals, agents, SFT and RL to iteratively improve the model. [9:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=586s) [Four columns describe multimodal data, a synthetic data flywheel, architecture advances, and native visual thinking.](https://assets.ai.engineer/sites/aiecode2025/sha256/9824010165bb69a8272677104d71776730dfdb3880dd31ce7073f5074deb5fcf.webp) Elorian’s Approach 01 Proprietary Multimodal Data Cross-domain human-labeled training data leads to fundamental visual reasoning capabilities. 02 Synthetic Data Flywheel Continuous improvement loop leveraging evals, agents, SFT, and RL to iteratively improve the model. 03 Architecture Advances Improved architecture designed for high-fidelity temporal and spatial understanding. 04 Native Visual Thinking Enabling complex visual chain-of-thought reasoning for refined thinking traces and nuanced content analysis. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Native Visual CoT: Beyond Text-Only Reasoning In a Monopoly example, the model boxes objects on the board, narrows them to red hotels, and counts three. [10:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=631s) [A five-step Monopoly board example identifies and counts three red hotels.](https://assets.ai.engineer/sites/aiecode2025/sha256/c594ab0514d503d182d2bbf83c52ec485a5dbb3ac3f7cb3fc38e20df88c2615a.webp) Native Visual CoT: Beyond Text-Only Reasoning 01 Prompt: “How many red hotels are built in this photo?” 02 Understanding “The user wants me to count the number of red hotel pieces on the Monopoly board. I need to look for the red building blocks placed on the game space” 03 Scanning board for properties Looking at the board, breaking down visually to locate all objects on the board [bounding box] 04 Distinguishing Making sure to only target red hotels, not to include other game components or cards, nor green hotels 05 Final count & response Counting the distinct red hotel pieces found, there are 3 of them. Based on the image, there are 3 red hotels on the board. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Elorian’s Team: Co-Founders Dai’s work spans LM pretraining, GLaM, PaLM 2 and Gemini data. Yang led research on Apple’s MM1 and worked on phrase grounding and UI understanding. [10:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=659s) [Profiles and photographs of co-founders Andrew Dai and Yinfei Yang, with their roles and experience.](https://assets.ai.engineer/sites/aiecode2025/sha256/1dea3a23514aaea91ad943214dde875202bd3d01a60f509646a4730f03e5aa45.webp) Elorian’s Team: Co-Founders Andrew Dai Co-Founder & CEO 12 years at Google Brain & DeepMind • Developed foundational techniques for modern LLMs. • First author of 2015 work introducing LM pretraining + SFT. • Co-led early MoE models including GLaM. • Lead for Gemini data; co-led PaLM 2 pretraining. Yinfei Yang Co-Founder & Chief Multimodal Architect Formerly Google Research & Apple • Led research for Apple's first public multimodal model, MM1. • Pioneered phrase grounding and UI understanding. • Shaped AI reasoning across language and vision. ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: 8 ### Our Team Dai says Elorian has roughly 20 people, including chief reasoning architect Dustin Tran, formerly xAI’s post-training lead. The roster lists experience at Gemini, Apple AI, Amazon Bedrock and other organizations. [11:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=718s) [Team roster with leadership roles and members’ prior affiliations.](https://assets.ai.engineer/sites/aiecode2025/sha256/c9f40d683ff56cbad7143933b8df52f954ce4113f1b6c5fa97d1623b2f56fc33.webp) Our Team Andrew Dai Co-Founder & CEO Yinfei Yang Co-Founder & CMMA Dustin Tran Chief Reasoning Architect Seth Neel Co-founder and Gemini Seojin Bang Gemini Marcella Valentine Gemini Richard Zhang Gemini Takeru Miyato Google Brain Forrest Huang Apple AI Zhen Xu Apple AI Le Xue BLIP paper lead Jared Lichtarge Gemini Yaning Hu Meta Gautam Kumar Amazon Bedrock Jing Gu xAI Arnav Das Gemini Jihua Huang SRI amazon Meta Gemini xAI DeepMind salesforce ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: [illegible] ### Robotics Use Case Elorian plans an API to pass action-relevant spatial scene understanding into robot planning and control systems. The slide illustrates an affordance map, a 3D point cloud and an action-primitive pipeline. [12:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=742s) [Robotics challenges and a proposed API connecting visual perception to planning and control, alongside an illustrated interface.](https://assets.ai.engineer/sites/aiecode2025/sha256/9b6f494f0c5922e579badb2345627e5a89f77449d4f04c23de3f464482eacec9.webp) Use Case Exploration / Robotics Robotics Use Case The Challenge Robots face critical bottlenecks performing complex, real-time physical actions in dynamic, unstructured environments. Existing vision models are trained from static images and pre-directed videos, not active physical interaction. Current paradigms are overly engineered, brittle, and expensive. Diagram: AFFORDANCE MAP (2D) SEMANTIC POINT CLOUD (3D) ELORIAN PRIMITIVE REASONING / API STREAM Visual Input Elorian Model (VLM) Affordance Calculator Action Primitive Generator Structured API Output [Other small diagram text illegible] Elorian’s Approach Connecting perception to action at inference time Building an API to deliver action-relevant spatial scene understanding directly into existing planning and control systems. ELORIAN PROPRIETARY AND CONFIDENTIAL ### Construction Use Case Elorian proposes matching construction camera streams to written, zone-specific safety rules. The illustration shows workers tracked for helmets and vests, while the speaker describes video understanding as a capability under development. [13:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=796s) [Construction safety challenges and a proposed automated auditing approach, illustrated with a monitored construction site.](https://assets.ai.engineer/sites/aiecode2025/sha256/747d61cb9250b3978f72eac1790137d6256254c75db522fde1af8ab0c9e7f1f5.webp) Use Case Exploration / Construction Construction Use Case The Challenge Rigid Vision: Construction sites run complex, zone-specific safety rules that outpace the rigidity of classical Computer Vision. Unstructured Policies: Traditional models cannot interpret unstructured policy language (like OSHA) or map spatial relationships. Brittle Retraining: Enforcing rules requires training separate models for every use case, making rapid adaptation challenging. Construction-site illustration: SAFETY ALERT: 0 0 HELMET USE: 98% Y-axis ticks: 100, 90, 80 X-axis ticks: 10, 20, 30, 40, 50 VEST USE: 100% Y-axis ticks: 100%, 50%, 0% X-axis labels: 0%, 100% EVENT LOG LIVE AI TRACKING ON HELMET (ID: H-12) VEST (ID: V-09) WORKER (ID: 104): HELMET: YES, VEST: YES [illegible tracking labels] Elorian’s Approach Automated safety auditing Directly ground video streams into complex, spatially-dependent safety regulations. Unlock natural language understanding to interpret site policies on existing camera infrastructure. ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: 11 ### Architecture & Design Use Case Elorian proposes extracting geometric logic from blueprints and CAD files for programmatic validation. The slide pairs a floor-plan compliance check with a mechanical-part tolerance check. [14:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=884s) [Architecture and mechanical design validation challenges, with proposed multimodal analysis and examples of a floor plan and mechanical part.](https://assets.ai.engineer/sites/aiecode2025/sha256/ec94055af694c1a0a92f9e5be0a8ef9ee1970f1a6d3c5b048523bb72f35cf412.webp) Use Case Exploration / Architecture & Design Architecture & Design Use Case The Challenge Validating complex spatial constraints and strict mechanical tolerances is heavily manual. Existing models struggle to parse unstructured visual context across architectural blueprints and complex 3D CAD/CAM files, leading to costly coordination errors. Elorian’s Approach From automated validation to generative design Leveraging multimodal reasoning to extract strict geometric logic, enabling programmatic validation (e.g., cross-disciplinary clash detection, CAD/CAM QC) and automating the future of mechanical design. ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: AI COMPLIANCE CHECKER BUILDING CODE PASS ACCESSIBILITY PASS FIRE SAFETY PASS [illegible] ELORIAN GEOMETRIC VALIDATION [illegible] [17:28](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1048s) [Slide describing challenges in architectural and mechanical design validation and an approach using multimodal reasoning, with blueprint and CAD examples.](https://assets.ai.engineer/sites/aiecode2025/sha256/89dd0b16a74023fbdaddd24d023761b3113d796b03d41ae3546c5d7dbaa779c7.webp) Use Case Exploration / Architecture & Design Architecture & Design Use Case The Challenge Validating complex spatial constraints and strict mechanical tolerances is heavily manual. Existing models struggle to parse unstructured visual context across architectural blueprints and complex 3D CAD/CAM files, leading to costly coordination errors. Elorian’s Approach From automated validation to generative design Leveraging multimodal reasoning to extract strict geometric logic, enabling programmatic validation (e.g., cross-disciplinary clash detection, CAD/CAM QC) and automating the future of mechanical design. ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: AI COMPLIANCE CHECKER BUILDING CODE PASS ACCESSIBILITY PASS FIRE SAFETY PASS [illegible] ELORIAN GEOMETRIC VALIDATION [illegible] ### Elorian Contact Information Elorian lists its website, X account, and LinkedIn page. Dai closes by inviting questions. [17:34](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1054s) [Closing slide with cards for the company's website, X account, and LinkedIn page.](https://assets.ai.engineer/sites/aiecode2025/sha256/67d0b70f4483cef4f659c5539733584a08738fb6ca260cad40d2022b18e96988.webp) Website elorian.ai Twitter / X x.com/ElorianAI LinkedIn Elorian AI ELORIAN PROPRIETARY AND CONFIDENTIAL Interface text: 14 / [illegible] ### AI Engineer World’s Fair [18:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1090s) [AI Engineer World’s Fair title in white on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/64426626b04e82c3bf8d0eedb56bd40ee2a082ec2f9ca366b1cb923c38d2e9d3.webp) AI Engineer World’s Fair HTTPS://AI.ENGINEER ## Full transcript [0:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1s) [music] [0:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=13s) Hi, thanks for coming everyone. Um, I'm [0:17](https://www.youtube.com/watch?v=A_I8mw8yfns&t=17s) the co-founder and CEO of Elorium and [0:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=20s) I'm here to talk about some of the [0:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=23s) issues with current models, current [0:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=26s) frontier models. This includes um Claude [0:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=30s) Chat GBD and Gemini um and how they [0:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=33s) handle visual problems and um uh this [0:37](https://www.youtube.com/watch?v=A_I8mw8yfns&t=37s) might be new to some of you who don't [0:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=40s) work in the visual space but actually [0:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=42s) there's quite a big gap between how [0:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=44s) these models handle visual reasoning and [0:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=47s) how humans deal with it. And you will [0:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=50s) see that we're actually quite far away [0:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=52s) from any definition of AGI for visual [0:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=56s) reasoning. [0:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=58s) So here are some examples of where uh [1:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=62s) how easy it is to find where models [1:04](https://www.youtube.com/watch?v=A_I8mw8yfns&t=64s) break down. And um you can find these [1:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=67s) examples yourself. Just takes a few [1:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=69s) minutes. Uh in this first example uh we [1:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=72s) have a chessboard hallucination and we [1:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=76s) give the models um this picture and ask [1:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=80s) how many white squares are in the image [1:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=83s) and any ordinary person uh who doesn't [1:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=86s) hallucinate would probably not say it's [1:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=89s) 32. So 32 of course the models say this [1:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=93s) because they see part of the chessboard [1:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=95s) and they hallucinate the complete board [1:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=98s) and as a result they give the wrong [1:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=100s) number and you see this quite a lot that [1:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=103s) models rely a lot current models rely a [1:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=105s) lot on pattern matching. That's what [1:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=107s) makes them so good at identifying plants [1:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=110s) and animals and flowers uh in the real [1:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=112s) world. But when it comes to like complex [1:55](https://www.youtube.com/watch?v=A_I8mw8yfns&t=115s) questions that part hurts them. So the [1:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=118s) pattern matching is actively hurting [1:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=119s) them in this case. Um so in their in [2:04](https://www.youtube.com/watch?v=A_I8mw8yfns&t=124s) what's going on in their reasoning is [2:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=125s) that oh this is a chess board. Chess [2:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=127s) boards all have 32 squares. Therefore [2:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=129s) this one must have 32 Y squares too. On [2:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=133s) the right example um I'm a big board [2:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=136s) game player. Have quite a collection. Um [2:19](https://www.youtube.com/watch?v=A_I8mw8yfns&t=139s) so you can see there's a a board game [2:21](https://www.youtube.com/watch?v=A_I8mw8yfns&t=141s) theme going on here. And actually you [2:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=142s) can uh reproduce this outside if you [2:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=145s) just go to you know outside the talks. [2:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=147s) It's a ball game area there. There are [2:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=149s) chess boards. Um I will bet if any of [2:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=152s) you uh place the pieces in some kind of [2:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=155s) random position, no frontier model will [2:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=158s) be able to tell you where those pieces [2:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=160s) are located, where all those pieces are [2:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=161s) located. Um and then another example [2:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=164s) here is katan. Um here another very [2:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=168s) simple question. How many rows does the [2:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=169s) blue player have? Um, these frontier [2:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=172s) models think extensively about this [2:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=174s) problem. Uh, one response I've seen is [2:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=176s) that, oh, the the guy has uh 10 blue [3:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=181s) rows off to the side of the board. [3:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=182s) Therefore, there must be five uh blue [3:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=185s) rows on the can board. Uh, but obviously [3:08](https://www.youtube.com/watch?v=A_I8mw8yfns&t=188s) that's not true. There's seven if you [3:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=190s) actually count. So these models um again [3:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=193s) are great at guessing uh great at [3:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=196s) pattern matching but they are not very [3:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=198s) spatially grounded and they just can't [3:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=200s) handle any kind of detailed questions. [3:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=204s) Um and then finally um we have this [3:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=206s) example where it actually um affects [3:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=210s) robots where here you have a robot arm [3:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=213s) uh manipulating this uh cup and um uh [3:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=218s) cooker basically and the [3:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=221s) state-of-the-art models today they miss [3:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=223s) the fact that the uh robot arm lifted [3:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=227s) the lid um and at the end they also [3:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=229s) missed the fact that the robot is [3:51](https://www.youtube.com/watch?v=A_I8mw8yfns&t=231s) turning on the stove like right now. So [3:53](https://www.youtube.com/watch?v=A_I8mw8yfns&t=233s) there's uh essentially context amnesia [3:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=236s) happening. Um and this is because these [4:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=240s) models can't maintain consistency um [4:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=243s) across long videos and they very easily [4:06](https://www.youtube.com/watch?v=A_I8mw8yfns&t=246s) lose track of what's happening. And a [4:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=249s) very common question I get is how do you [4:11](https://www.youtube.com/watch?v=A_I8mw8yfns&t=251s) define a visual reasoning problem versus [4:14](https://www.youtube.com/watch?v=A_I8mw8yfns&t=254s) a visual understanding problem? Um or [4:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=258s) you could say like visual thinking um [4:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=260s) compared to visual understanding. I [4:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=262s) think a very simple way to do it is just [4:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=265s) um ask yourself the same question. If [4:28](https://www.youtube.com/watch?v=A_I8mw8yfns&t=268s) you looked at an image or a video, how [4:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=271s) long would it take you to answer the [4:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=273s) question? So uh for example, in both of [4:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=276s) these cases, I doubt anyone in this room [4:39](https://www.youtube.com/watch?v=A_I8mw8yfns&t=279s) would be able to give an answer uh if [4:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=281s) they were only allowed one second to [4:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=282s) look at the image. So 1 second isn't [4:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=284s) enough to do these kind of like complex [4:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=287s) questions also called like system two um [4:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=290s) kind of thinking in Daniel Canon's book. [4:53](https://www.youtube.com/watch?v=A_I8mw8yfns&t=293s) But if I asked you what game is this uh [4:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=297s) or similarly what flower is this or if [5:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=300s) there are only three pieces on the [5:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=301s) chessboard if I asked you how many [5:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=302s) pieces are there those questions uh I'm [5:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=305s) sure all of you would be able to answer [5:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=307s) in less than a second and similarly all [5:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=309s) the frontier models would get that kind [5:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=310s) of question right. So that is the [5:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=312s) distinction um that we make between what [5:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=315s) is understanding uh what is like pattern [5:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=318s) recognition versus what is reasoning [5:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=320s) where you actually have to look in [5:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=322s) detail at the picture and um look at [5:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=325s) various things and this is exactly where [5:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=327s) frontier models uh fall apart today. So [5:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=331s) as you are designing your own systems uh [5:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=333s) that's something to keep in mind keep [5:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=335s) these visual tasks very simple otherwise [5:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=338s) you will have hallucinations a lot and a [5:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=340s) lot of hallucinations [5:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=344s) um so uh this leads into evals of course [5:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=348s) um frontier models there are already a [5:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=352s) bunch of multimodal reasoning evals or [5:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=354s) visual reasoning evals some that you [5:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=356s) might have heard of is arc agi this is [5:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=359s) uh quite often [6:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=360s) brought up to um people saying oh we the [6:04](https://www.youtube.com/watch?v=A_I8mw8yfns&t=364s) frontier models are 85% or 90% on RKGI [6:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=367s) therefore we are 90% of the way to a uh [6:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=370s) to AGI itself uh but I think these [6:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=373s) people they haven't really looked at any [6:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=375s) of the benchmark data because if you [6:17](https://www.youtube.com/watch?v=A_I8mw8yfns&t=377s) actually look at the data you will [6:19](https://www.youtube.com/watch?v=A_I8mw8yfns&t=379s) notice that the images are only 32x 32 [6:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=382s) or 64x 64 pixels and I would challenge [6:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=386s) anyone uh to give me like a real world [6:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=390s) complex task that can be reduced to a [6:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=392s) 32x 32 pixel problem. Um I think you'll [6:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=396s) very quickly realize almost no tasks [6:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=398s) almost no interesting tasks can be [6:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=400s) reduced to that kind of resolution. [6:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=402s) Another eval that people commonly uh [6:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=405s) bring up is MMU. Uh this is the massive [6:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=408s) multiddiscipline multimodal [6:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=410s) understanding. This is a step up from [6:51](https://www.youtube.com/watch?v=A_I8mw8yfns&t=411s) MMLU because it has um images rather [6:55](https://www.youtube.com/watch?v=A_I8mw8yfns&t=415s) than just pure text science questions. [6:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=417s) This is science questions based on [6:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=418s) images. But still images are a minor [7:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=422s) part of a lot of these questions. A lot [7:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=423s) of the questions you can just answer [7:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=425s) without looking at the image or just [7:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=427s) doing some pattern recognition just [7:08](https://www.youtube.com/watch?v=A_I8mw8yfns&t=428s) knowing roughly what the image is about. [7:11](https://www.youtube.com/watch?v=A_I8mw8yfns&t=431s) So what we really need is new visual [7:14](https://www.youtube.com/watch?v=A_I8mw8yfns&t=434s) reasoning benchmarks in the industry [7:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=436s) that really target the things that [7:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=438s) people care about like geometric align [7:21](https://www.youtube.com/watch?v=A_I8mw8yfns&t=441s) alignment, spatial intelligence, um [7:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=444s) object terminus and these are really [7:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=447s) critical for AI to be deployed in these [7:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=450s) visual use cases. And you might have [7:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=452s) noticed that still in a lot of [7:34](https://www.youtube.com/watch?v=A_I8mw8yfns&t=454s) industries that uh are primarily visual [7:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=458s) um which I will go into there isn't much [7:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=461s) uptake of AI right a lot of the AI [7:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=463s) uptake has been in the software [7:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=464s) engineering world and in the [7:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=466s) mathematician world um and in like [7:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=468s) documents um document handling etc. But [7:51](https://www.youtube.com/watch?v=A_I8mw8yfns&t=471s) this uh there is actually a huge gap [7:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=474s) huge opportunity that is just being [7:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=477s) looked over right now um based on the [8:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=480s) interest in coding. [8:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=483s) [clears throat] And so the missing [8:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=485s) paradigm in visual AI is thinking. So we [8:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=489s) have generation models very high quality [8:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=492s) generation models like bite dances [8:14](https://www.youtube.com/watch?v=A_I8mw8yfns&t=494s) seance model. Um, so we have these very [8:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=498s) high fidelity models and they look [8:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=500s) great, but they lack actual physical [8:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=502s) grounding um and causal logic. So you [8:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=504s) will you probably notice that if you ask [8:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=506s) these models to produce a a picture um a [8:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=509s) video of a some like uh something [8:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=512s) blowing up like um or a building falling [8:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=515s) down or these things they look very [8:37](https://www.youtube.com/watch?v=A_I8mw8yfns&t=517s) cartoonish they look Hollywood style [8:39](https://www.youtube.com/watch?v=A_I8mw8yfns&t=519s) kind of things and that's because they [8:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=521s) are just outputting what was in the [8:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=523s) training data and a lot of disaster [8:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=525s) videos um a lot of like action kind of [8:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=528s) videos on the internet are just going to [8:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=530s) be from Hollywood or game engines. So [8:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=532s) they're working to reproduce that and [8:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=534s) that's fundamentally a problem because [8:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=536s) it means they can be no better than [8:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=537s) those kind of uh videos. Um on [9:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=540s) understanding the what we are where we [9:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=543s) currently are is we have lot of tools [9:06](https://www.youtube.com/watch?v=A_I8mw8yfns&t=546s) that can map pixels to semantic labels [9:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=547s) like Google lens is obviously great to [9:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=550s) identify plants and flowers and I use [9:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=553s) that all the time. The SAM 3 for [9:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=555s) segmentation, YOLO for uh object [9:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=558s) recognition detection, mascaras CNN. [9:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=560s) These are of course highly robust and [9:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=562s) they're used everywhere in the industry, [9:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=564s) but they're fundamentally passive. So [9:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=565s) there's no reasoning capability to them. [9:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=567s) So they can't answer more complex [9:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=569s) questions. Um and really where the [9:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=571s) frontier is is uh with thinking visual [9:34](https://www.youtube.com/watch?v=A_I8mw8yfns&t=574s) thinking models. These models will have [9:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=576s) active spatial and temporal [9:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=578s) intelligence. They can extract actional [9:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=580s) logic for planning uh agentic workflows [9:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=584s) and physical execution. [9:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=588s) And so our approach uh is uh four stage. [9:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=592s) So we are collecting and generating our [9:55](https://www.youtube.com/watch?v=A_I8mw8yfns&t=595s) own uh multimodal data uh visual [9:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=598s) reasoning specific data. This this kind [10:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=600s) of data we found you just can't uh get [10:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=602s) online. Uh we have a synthetic data [10:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=605s) flywheel using evals agents SFT and RL [10:08](https://www.youtube.com/watch?v=A_I8mw8yfns&t=608s) to improve the model. We're making some [10:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=610s) uh we made some advances to the [10:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=612s) architecture um in terms of uh various [10:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=615s) different time um advance various [10:17](https://www.youtube.com/watch?v=A_I8mw8yfns&t=617s) different improvements on top of the [10:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=618s) transformer-based architecture and we're [10:21](https://www.youtube.com/watch?v=A_I8mw8yfns&t=621s) also enabling visual chain of thought [10:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=623s) reasoning and this is one of the key [10:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=625s) things that humans have that no frontier [10:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=627s) model has today since the frontier [10:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=629s) models are only textual uh chain of [10:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=631s) thought based [10:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=633s) um and this is one example of a visual [10:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=635s) chain of thought. So the question is [10:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=636s) like how many red hotels are built in [10:39](https://www.youtube.com/watch?v=A_I8mw8yfns&t=639s) this photo? Then the model realizes oh [10:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=643s) uh first we need to identify all the [10:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=645s) hotels. So it draws boxes around hotels [10:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=649s) um and other objects and then uh it [10:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=652s) reduces that to the red hotel. So it's [10:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=654s) this multi-step uh process happening in [10:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=657s) the visual space natively. [11:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=661s) So um about our company um I'm the [11:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=665s) co-founder and CEO. I spent the last 12 [11:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=667s) years at Google Brain and Deep Mind. Um [11:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=670s) I developed a lot of the foundational [11:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=672s) techniques for the model for modern [11:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=673s) LLMs. 11 years ago I was the first [11:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=676s) author of the work that introduced [11:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=678s) pre-training and fine-tuning. That's the [11:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=680s) work when combined with the transformer [11:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=682s) paper in 2017 led to the GBT series of [11:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=685s) models. So all the GBT uh papers site [11:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=687s) our paper. Um I co-led the earlye models [11:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=691s) uh the first model that was [11:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=692s) state-of-the-art called glam and then um [11:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=695s) more recently I co-led the palm to 2 [11:37](https://www.youtube.com/watch?v=A_I8mw8yfns&t=697s) pre-training architecture and I was [11:39](https://www.youtube.com/watch?v=A_I8mw8yfns&t=699s) co-lead for the gemini data area and my [11:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=702s) co-founder info [11:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=705s) and Google research he led research for [11:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=707s) Apple's first public multimodal model [11:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=709s) MM1 and he's has a a lot of experience [11:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=714s) in visual reasoning um across uh [11:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=717s) language as [11:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=719s) And this is our team. So we're roughly [12:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=722s) 20 people now. Um we've also have a [12:06](https://www.youtube.com/watch?v=A_I8mw8yfns&t=726s) chief reasoning architect Dustin Tran. [12:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=727s) Previously he was lead of post training [12:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=729s) at XAI. Um and we've hired a world-class [12:14](https://www.youtube.com/watch?v=A_I8mw8yfns&t=734s) team um across uh many other uh [12:17](https://www.youtube.com/watch?v=A_I8mw8yfns&t=737s) companies like Apple uh XAI um deep mind [12:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=740s) Amazon and so on. [12:24](https://www.youtube.com/watch?v=A_I8mw8yfns&t=744s) Um and in terms of the uh use cases that [12:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=747s) I mentioned, robotics is one primary use [12:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=750s) case. So robots have uh really critical [12:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=753s) bottlenecks performing complex real-time [12:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=756s) physical actions uh in these kind of [12:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=758s) like dynamic environments. Uh but [12:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=760s) existing vision models uh you probably [12:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=763s) realize are trained from static images [12:45](https://www.youtube.com/watch?v=A_I8mw8yfns&t=765s) and very directed videos. They are not [12:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=768s) like act they don't have active physical [12:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=770s) interaction. So existing methods are [12:53](https://www.youtube.com/watch?v=A_I8mw8yfns&t=773s) overengineered and brittle. Um and uh we [12:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=777s) are planning to release a model API [12:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=779s) available uh by the end of this year. Um [13:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=782s) and at that point the API can be used to [13:06](https://www.youtube.com/watch?v=A_I8mw8yfns&t=786s) deliver action relevant uh scene [13:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=789s) understanding into existing um planning [13:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=793s) and control systems for these um robots. [13:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=796s) Another important use case for uh visual [13:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=800s) reasoning um is construction. So [13:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=803s) construction sites they have these very [13:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=806s) complex zone specific safety rules. Um [13:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=810s) and uh computer vision can't adapt fast [13:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=812s) enough to changing safety rules. They [13:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=815s) also can't interpret things like OSHA [13:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=818s) policy language and match the that [13:42](https://www.youtube.com/watch?v=A_I8mw8yfns&t=822s) language to what's actually going on at [13:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=824s) the site or understand the spatial [13:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=826s) relationships that are important there. [13:48](https://www.youtube.com/watch?v=A_I8mw8yfns&t=828s) For example, like how many of these [13:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=829s) workers are wearing helmets or like is [13:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=832s) the construction um happening according [13:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=836s) to the plans that uh were defined [13:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=838s) earlier. [14:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=840s) And um currently enforcing these rules [14:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=843s) require training separate models for [14:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=845s) different use cases uh because they're [14:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=847s) very these models as I said before are [14:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=849s) very brittle. So you constantly have to [14:11](https://www.youtube.com/watch?v=A_I8mw8yfns&t=851s) do uh retraining. Um and our approach uh [14:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=855s) with the video um understanding [14:17](https://www.youtube.com/watch?v=A_I8mw8yfns&t=857s) capabilities that we are building into [14:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=858s) our models is um allows you to um ground [14:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=863s) these video streams in the safety [14:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=866s) regulations. And of course the safety [14:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=867s) regulations are in text, they're in [14:29](https://www.youtube.com/watch?v=A_I8mw8yfns&t=869s) language. So you have to be um the model [14:32](https://www.youtube.com/watch?v=A_I8mw8yfns&t=872s) has to manipulate both language and [14:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=873s) vision very well. [14:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=875s) And um yeah uh this will allow these [14:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=878s) models to interpret uh site policies [14:41](https://www.youtube.com/watch?v=A_I8mw8yfns&t=881s) using the current camera infrastructure [14:44](https://www.youtube.com/watch?v=A_I8mw8yfns&t=884s) that they have. [14:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=886s) Um and then finally architecture and [14:49](https://www.youtube.com/watch?v=A_I8mw8yfns&t=889s) design we think is also a very uh [14:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=892s) promising use case here. Uh this is [14:55](https://www.youtube.com/watch?v=A_I8mw8yfns&t=895s) exactly the use case where you need to [14:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=897s) be very detail oriented. So back to the [14:59](https://www.youtube.com/watch?v=A_I8mw8yfns&t=899s) board game example around counting [15:02](https://www.youtube.com/watch?v=A_I8mw8yfns&t=902s) spatial relationships. This shows up a [15:04](https://www.youtube.com/watch?v=A_I8mw8yfns&t=904s) lot in architecture and design. Uh like [15:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=907s) if you design a if you design a house [15:10](https://www.youtube.com/watch?v=A_I8mw8yfns&t=910s) with with four bedrooms instead of [15:11](https://www.youtube.com/watch?v=A_I8mw8yfns&t=911s) three, that homeowner is going to be [15:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=913s) very angry, right? Um so obviously [15:16](https://www.youtube.com/watch?v=A_I8mw8yfns&t=916s) counting is actually important. Um and [15:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=920s) also just understanding these spatial [15:22](https://www.youtube.com/watch?v=A_I8mw8yfns&t=922s) constraints, real world constraints is [15:25](https://www.youtube.com/watch?v=A_I8mw8yfns&t=925s) uh is a very manual process uh these [15:28](https://www.youtube.com/watch?v=A_I8mw8yfns&t=928s) days. We spoke to a mechanical [15:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=931s) engineering company just a few weeks ago [15:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=933s) and they said to design one small part [15:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=936s) of a robot testing platform takes 100 to [15:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=940s) 200 hours uh of the time to design the [15:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=943s) entire testing platform. Um, I believe [15:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=946s) it takes 2,000 to 3,000 hours of human [15:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=950s) uh time there. And they've uh a lot of [15:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=954s) these places they've tried frontier [15:56](https://www.youtube.com/watch?v=A_I8mw8yfns&t=956s) models, but they just don't work for [15:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=958s) these use cases. They really struggle to [16:00](https://www.youtube.com/watch?v=A_I8mw8yfns&t=960s) understand uh visual context across [16:03](https://www.youtube.com/watch?v=A_I8mw8yfns&t=963s) these like architecture blueprints, 3D [16:05](https://www.youtube.com/watch?v=A_I8mw8yfns&t=965s) CAD, CAM files. Um, and so there are [16:08](https://www.youtube.com/watch?v=A_I8mw8yfns&t=968s) lots of errors there. Um and similarly [16:12](https://www.youtube.com/watch?v=A_I8mw8yfns&t=972s) we believe that this can be useful [16:13](https://www.youtube.com/watch?v=A_I8mw8yfns&t=973s) useful for other kinds of design as well [16:15](https://www.youtube.com/watch?v=A_I8mw8yfns&t=975s) not just um architecture and engineering [16:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=978s) but maybe like designing [16:21](https://www.youtube.com/watch?v=A_I8mw8yfns&t=981s) um yeah for the web or fashion or other [16:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=983s) things and our approach is uh we're [16:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=986s) using multimodal reasoning to uh to [16:30](https://www.youtube.com/watch?v=A_I8mw8yfns&t=990s) extract um this uh geometric logic [16:33](https://www.youtube.com/watch?v=A_I8mw8yfns&t=993s) that's important. We're allowing [16:35](https://www.youtube.com/watch?v=A_I8mw8yfns&t=995s) programmatic validation or simulation [16:38](https://www.youtube.com/watch?v=A_I8mw8yfns&t=998s) validation. Just like in code, you can [16:40](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1000s) run code against unit tests. You can [16:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1003s) also run u mechanical devices through [16:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1006s) simulators that have been developed [16:47](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1007s) through seammens um and uh v various [16:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1010s) other companies to see if something will [16:52](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1012s) work in the real world. So there's a lot [16:54](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1014s) of parallels actually between uh this [16:57](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1017s) kind of like mechanical design and [16:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1018s) coding itself. But mechanical design is [17:01](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1021s) still relatively untouched by AI. [17:04](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1024s) Um and yeah, CAD CAM quality control is [17:07](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1027s) another potential use case and [17:09](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1029s) ultimately we believe that this is going [17:11](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1031s) to be a critical step to the future of [17:14](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1034s) mechanical design where the where AI can [17:18](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1038s) make faster cars, more efficient [17:20](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1040s) rockets, better batteries and all these [17:23](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1043s) things cannot be done just with code. Uh [17:26](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1046s) people are not coding up the next iPhone [17:27](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1047s) or coding up the next uh SpaceX rocket. [17:31](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1051s) It's all fundamentally very visual. [17:36](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1056s) So, um you can find out more about us [17:39](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1059s) through our website um lauren.ai, our [17:43](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1063s) Twitter page xx.comai [17:46](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1066s) or our LinkedIn uh page. And yeah, happy [17:50](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1070s) to take any questions. I'll be standing [17:53](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1073s) around here for for a little bit. [17:55](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1075s) Thanks. [17:58](https://www.youtube.com/watch?v=A_I8mw8yfns&t=1078s) [applause]