[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 13840,
    "text": "Hi, thanks for coming everyone. Um, I'm"
  },
  {
    "startMs": 17039,
    "text": "the co-founder and CEO of Elorium and"
  },
  {
    "startMs": 20720,
    "text": "I'm here to talk about some of the"
  },
  {
    "startMs": 23680,
    "text": "issues with current models, current"
  },
  {
    "startMs": 26640,
    "text": "frontier models. This includes um Claude"
  },
  {
    "startMs": 30240,
    "text": "Chat GBD and Gemini um and how they"
  },
  {
    "startMs": 33600,
    "text": "handle visual problems and um uh this"
  },
  {
    "startMs": 37920,
    "text": "might be new to some of you who don't"
  },
  {
    "startMs": 40320,
    "text": "work in the visual space but actually"
  },
  {
    "startMs": 42239,
    "text": "there's quite a big gap between how"
  },
  {
    "startMs": 44719,
    "text": "these models handle visual reasoning and"
  },
  {
    "startMs": 47840,
    "text": "how humans deal with it. And you will"
  },
  {
    "startMs": 50399,
    "text": "see that we're actually quite far away"
  },
  {
    "startMs": 52640,
    "text": "from any definition of AGI for visual"
  },
  {
    "startMs": 56559,
    "text": "reasoning."
  },
  {
    "startMs": 58879,
    "text": "So here are some examples of where uh"
  },
  {
    "startMs": 62160,
    "text": "how easy it is to find where models"
  },
  {
    "startMs": 64479,
    "text": "break down. And um you can find these"
  },
  {
    "startMs": 67600,
    "text": "examples yourself. Just takes a few"
  },
  {
    "startMs": 69200,
    "text": "minutes. Uh in this first example uh we"
  },
  {
    "startMs": 72400,
    "text": "have a chessboard hallucination and we"
  },
  {
    "startMs": 76000,
    "text": "give the models um this picture and ask"
  },
  {
    "startMs": 80799,
    "text": "how many white squares are in the image"
  },
  {
    "startMs": 83439,
    "text": "and any ordinary person uh who doesn't"
  },
  {
    "startMs": 86799,
    "text": "hallucinate would probably not say it's"
  },
  {
    "startMs": 89840,
    "text": "32. So 32 of course the models say this"
  },
  {
    "startMs": 93840,
    "text": "because they see part of the chessboard"
  },
  {
    "startMs": 95920,
    "text": "and they hallucinate the complete board"
  },
  {
    "startMs": 98320,
    "text": "and as a result they give the wrong"
  },
  {
    "startMs": 100400,
    "text": "number and you see this quite a lot that"
  },
  {
    "startMs": 103119,
    "text": "models rely a lot current models rely a"
  },
  {
    "startMs": 105680,
    "text": "lot on pattern matching. That's what"
  },
  {
    "startMs": 107439,
    "text": "makes them so good at identifying plants"
  },
  {
    "startMs": 110000,
    "text": "and animals and flowers uh in the real"
  },
  {
    "startMs": 112960,
    "text": "world. But when it comes to like complex"
  },
  {
    "startMs": 115200,
    "text": "questions that part hurts them. So the"
  },
  {
    "startMs": 118000,
    "text": "pattern matching is actively hurting"
  },
  {
    "startMs": 119840,
    "text": "them in this case. Um so in their in"
  },
  {
    "startMs": 124000,
    "text": "what's going on in their reasoning is"
  },
  {
    "startMs": 125439,
    "text": "that oh this is a chess board. Chess"
  },
  {
    "startMs": 127280,
    "text": "boards all have 32 squares. Therefore"
  },
  {
    "startMs": 129599,
    "text": "this one must have 32 Y squares too. On"
  },
  {
    "startMs": 133360,
    "text": "the right example um I'm a big board"
  },
  {
    "startMs": 136720,
    "text": "game player. Have quite a collection. Um"
  },
  {
    "startMs": 139360,
    "text": "so you can see there's a a board game"
  },
  {
    "startMs": 141120,
    "text": "theme going on here. And actually you"
  },
  {
    "startMs": 142959,
    "text": "can uh reproduce this outside if you"
  },
  {
    "startMs": 145280,
    "text": "just go to you know outside the talks."
  },
  {
    "startMs": 147520,
    "text": "It's a ball game area there. There are"
  },
  {
    "startMs": 149599,
    "text": "chess boards. Um I will bet if any of"
  },
  {
    "startMs": 152720,
    "text": "you uh place the pieces in some kind of"
  },
  {
    "startMs": 155840,
    "text": "random position, no frontier model will"
  },
  {
    "startMs": 158239,
    "text": "be able to tell you where those pieces"
  },
  {
    "startMs": 160160,
    "text": "are located, where all those pieces are"
  },
  {
    "startMs": 161680,
    "text": "located. Um and then another example"
  },
  {
    "startMs": 164239,
    "text": "here is katan. Um here another very"
  },
  {
    "startMs": 168000,
    "text": "simple question. How many rows does the"
  },
  {
    "startMs": 169920,
    "text": "blue player have? Um, these frontier"
  },
  {
    "startMs": 172239,
    "text": "models think extensively about this"
  },
  {
    "startMs": 174080,
    "text": "problem. Uh, one response I've seen is"
  },
  {
    "startMs": 176400,
    "text": "that, oh, the the guy has uh 10 blue"
  },
  {
    "startMs": 181120,
    "text": "rows off to the side of the board."
  },
  {
    "startMs": 182800,
    "text": "Therefore, there must be five uh blue"
  },
  {
    "startMs": 185360,
    "text": "rows on the can board. Uh, but obviously"
  },
  {
    "startMs": 188480,
    "text": "that's not true. There's seven if you"
  },
  {
    "startMs": 190560,
    "text": "actually count. So these models um again"
  },
  {
    "startMs": 193920,
    "text": "are great at guessing uh great at"
  },
  {
    "startMs": 196080,
    "text": "pattern matching but they are not very"
  },
  {
    "startMs": 198879,
    "text": "spatially grounded and they just can't"
  },
  {
    "startMs": 200879,
    "text": "handle any kind of detailed questions."
  },
  {
    "startMs": 204720,
    "text": "Um and then finally um we have this"
  },
  {
    "startMs": 206800,
    "text": "example where it actually um affects"
  },
  {
    "startMs": 210159,
    "text": "robots where here you have a robot arm"
  },
  {
    "startMs": 213920,
    "text": "uh manipulating this uh cup and um uh"
  },
  {
    "startMs": 218959,
    "text": "cooker basically and the"
  },
  {
    "startMs": 221440,
    "text": "state-of-the-art models today they miss"
  },
  {
    "startMs": 223760,
    "text": "the fact that the uh robot arm lifted"
  },
  {
    "startMs": 227440,
    "text": "the lid um and at the end they also"
  },
  {
    "startMs": 229840,
    "text": "missed the fact that the robot is"
  },
  {
    "startMs": 231040,
    "text": "turning on the stove like right now. So"
  },
  {
    "startMs": 233680,
    "text": "there's uh essentially context amnesia"
  },
  {
    "startMs": 236480,
    "text": "happening. Um and this is because these"
  },
  {
    "startMs": 240319,
    "text": "models can't maintain consistency um"
  },
  {
    "startMs": 243360,
    "text": "across long videos and they very easily"
  },
  {
    "startMs": 246239,
    "text": "lose track of what's happening. And a"
  },
  {
    "startMs": 249519,
    "text": "very common question I get is how do you"
  },
  {
    "startMs": 251920,
    "text": "define a visual reasoning problem versus"
  },
  {
    "startMs": 254799,
    "text": "a visual understanding problem? Um or"
  },
  {
    "startMs": 258160,
    "text": "you could say like visual thinking um"
  },
  {
    "startMs": 260239,
    "text": "compared to visual understanding. I"
  },
  {
    "startMs": 262079,
    "text": "think a very simple way to do it is just"
  },
  {
    "startMs": 265840,
    "text": "um ask yourself the same question. If"
  },
  {
    "startMs": 268560,
    "text": "you looked at an image or a video, how"
  },
  {
    "startMs": 271360,
    "text": "long would it take you to answer the"
  },
  {
    "startMs": 273600,
    "text": "question? So uh for example, in both of"
  },
  {
    "startMs": 276720,
    "text": "these cases, I doubt anyone in this room"
  },
  {
    "startMs": 279199,
    "text": "would be able to give an answer uh if"
  },
  {
    "startMs": 281280,
    "text": "they were only allowed one second to"
  },
  {
    "startMs": 282800,
    "text": "look at the image. So 1 second isn't"
  },
  {
    "startMs": 284960,
    "text": "enough to do these kind of like complex"
  },
  {
    "startMs": 287040,
    "text": "questions also called like system two um"
  },
  {
    "startMs": 290400,
    "text": "kind of thinking in Daniel Canon's book."
  },
  {
    "startMs": 293680,
    "text": "But if I asked you what game is this uh"
  },
  {
    "startMs": 297280,
    "text": "or similarly what flower is this or if"
  },
  {
    "startMs": 300000,
    "text": "there are only three pieces on the"
  },
  {
    "startMs": 301360,
    "text": "chessboard if I asked you how many"
  },
  {
    "startMs": 302639,
    "text": "pieces are there those questions uh I'm"
  },
  {
    "startMs": 305680,
    "text": "sure all of you would be able to answer"
  },
  {
    "startMs": 307039,
    "text": "in less than a second and similarly all"
  },
  {
    "startMs": 309039,
    "text": "the frontier models would get that kind"
  },
  {
    "startMs": 310560,
    "text": "of question right. So that is the"
  },
  {
    "startMs": 312479,
    "text": "distinction um that we make between what"
  },
  {
    "startMs": 315840,
    "text": "is understanding uh what is like pattern"
  },
  {
    "startMs": 318479,
    "text": "recognition versus what is reasoning"
  },
  {
    "startMs": 320800,
    "text": "where you actually have to look in"
  },
  {
    "startMs": 322560,
    "text": "detail at the picture and um look at"
  },
  {
    "startMs": 325520,
    "text": "various things and this is exactly where"
  },
  {
    "startMs": 327919,
    "text": "frontier models uh fall apart today. So"
  },
  {
    "startMs": 331360,
    "text": "as you are designing your own systems uh"
  },
  {
    "startMs": 333680,
    "text": "that's something to keep in mind keep"
  },
  {
    "startMs": 335600,
    "text": "these visual tasks very simple otherwise"
  },
  {
    "startMs": 338240,
    "text": "you will have hallucinations a lot and a"
  },
  {
    "startMs": 340560,
    "text": "lot of hallucinations"
  },
  {
    "startMs": 344000,
    "text": "um so uh this leads into evals of course"
  },
  {
    "startMs": 348800,
    "text": "um frontier models there are already a"
  },
  {
    "startMs": 352080,
    "text": "bunch of multimodal reasoning evals or"
  },
  {
    "startMs": 354080,
    "text": "visual reasoning evals some that you"
  },
  {
    "startMs": 356240,
    "text": "might have heard of is arc agi this is"
  },
  {
    "startMs": 359600,
    "text": "uh quite often"
  },
  {
    "startMs": 360960,
    "text": "brought up to um people saying oh we the"
  },
  {
    "startMs": 364400,
    "text": "frontier models are 85% or 90% on RKGI"
  },
  {
    "startMs": 367360,
    "text": "therefore we are 90% of the way to a uh"
  },
  {
    "startMs": 370560,
    "text": "to AGI itself uh but I think these"
  },
  {
    "startMs": 373440,
    "text": "people they haven't really looked at any"
  },
  {
    "startMs": 375440,
    "text": "of the benchmark data because if you"
  },
  {
    "startMs": 377360,
    "text": "actually look at the data you will"
  },
  {
    "startMs": 379199,
    "text": "notice that the images are only 32x 32"
  },
  {
    "startMs": 382800,
    "text": "or 64x 64 pixels and I would challenge"
  },
  {
    "startMs": 386400,
    "text": "anyone uh to give me like a real world"
  },
  {
    "startMs": 390000,
    "text": "complex task that can be reduced to a"
  },
  {
    "startMs": 392560,
    "text": "32x 32 pixel problem. Um I think you'll"
  },
  {
    "startMs": 396319,
    "text": "very quickly realize almost no tasks"
  },
  {
    "startMs": 398720,
    "text": "almost no interesting tasks can be"
  },
  {
    "startMs": 400319,
    "text": "reduced to that kind of resolution."
  },
  {
    "startMs": 402880,
    "text": "Another eval that people commonly uh"
  },
  {
    "startMs": 405280,
    "text": "bring up is MMU. Uh this is the massive"
  },
  {
    "startMs": 408479,
    "text": "multiddiscipline multimodal"
  },
  {
    "startMs": 410000,
    "text": "understanding. This is a step up from"
  },
  {
    "startMs": 411840,
    "text": "MMLU because it has um images rather"
  },
  {
    "startMs": 415199,
    "text": "than just pure text science questions."
  },
  {
    "startMs": 417520,
    "text": "This is science questions based on"
  },
  {
    "startMs": 418960,
    "text": "images. But still images are a minor"
  },
  {
    "startMs": 422000,
    "text": "part of a lot of these questions. A lot"
  },
  {
    "startMs": 423919,
    "text": "of the questions you can just answer"
  },
  {
    "startMs": 425120,
    "text": "without looking at the image or just"
  },
  {
    "startMs": 427039,
    "text": "doing some pattern recognition just"
  },
  {
    "startMs": 428479,
    "text": "knowing roughly what the image is about."
  },
  {
    "startMs": 431199,
    "text": "So what we really need is new visual"
  },
  {
    "startMs": 434160,
    "text": "reasoning benchmarks in the industry"
  },
  {
    "startMs": 436560,
    "text": "that really target the things that"
  },
  {
    "startMs": 438720,
    "text": "people care about like geometric align"
  },
  {
    "startMs": 441440,
    "text": "alignment, spatial intelligence, um"
  },
  {
    "startMs": 444479,
    "text": "object terminus and these are really"
  },
  {
    "startMs": 447280,
    "text": "critical for AI to be deployed in these"
  },
  {
    "startMs": 450400,
    "text": "visual use cases. And you might have"
  },
  {
    "startMs": 452800,
    "text": "noticed that still in a lot of"
  },
  {
    "startMs": 454880,
    "text": "industries that uh are primarily visual"
  },
  {
    "startMs": 458720,
    "text": "um which I will go into there isn't much"
  },
  {
    "startMs": 461039,
    "text": "uptake of AI right a lot of the AI"
  },
  {
    "startMs": 463360,
    "text": "uptake has been in the software"
  },
  {
    "startMs": 464880,
    "text": "engineering world and in the"
  },
  {
    "startMs": 466000,
    "text": "mathematician world um and in like"
  },
  {
    "startMs": 468240,
    "text": "documents um document handling etc. But"
  },
  {
    "startMs": 471599,
    "text": "this uh there is actually a huge gap"
  },
  {
    "startMs": 474400,
    "text": "huge opportunity that is just being"
  },
  {
    "startMs": 477039,
    "text": "looked over right now um based on the"
  },
  {
    "startMs": 480720,
    "text": "interest in coding."
  },
  {
    "startMs": 483395,
    "text": "[clears throat] And so the missing"
  },
  {
    "startMs": 485360,
    "text": "paradigm in visual AI is thinking. So we"
  },
  {
    "startMs": 489440,
    "text": "have generation models very high quality"
  },
  {
    "startMs": 492160,
    "text": "generation models like bite dances"
  },
  {
    "startMs": 494560,
    "text": "seance model. Um, so we have these very"
  },
  {
    "startMs": 498000,
    "text": "high fidelity models and they look"
  },
  {
    "startMs": 500080,
    "text": "great, but they lack actual physical"
  },
  {
    "startMs": 502400,
    "text": "grounding um and causal logic. So you"
  },
  {
    "startMs": 504960,
    "text": "will you probably notice that if you ask"
  },
  {
    "startMs": 506960,
    "text": "these models to produce a a picture um a"
  },
  {
    "startMs": 509919,
    "text": "video of a some like uh something"
  },
  {
    "startMs": 512640,
    "text": "blowing up like um or a building falling"
  },
  {
    "startMs": 515039,
    "text": "down or these things they look very"
  },
  {
    "startMs": 517120,
    "text": "cartoonish they look Hollywood style"
  },
  {
    "startMs": 519200,
    "text": "kind of things and that's because they"
  },
  {
    "startMs": 521440,
    "text": "are just outputting what was in the"
  },
  {
    "startMs": 523120,
    "text": "training data and a lot of disaster"
  },
  {
    "startMs": 525279,
    "text": "videos um a lot of like action kind of"
  },
  {
    "startMs": 528320,
    "text": "videos on the internet are just going to"
  },
  {
    "startMs": 530480,
    "text": "be from Hollywood or game engines. So"
  },
  {
    "startMs": 532880,
    "text": "they're working to reproduce that and"
  },
  {
    "startMs": 534959,
    "text": "that's fundamentally a problem because"
  },
  {
    "startMs": 536399,
    "text": "it means they can be no better than"
  },
  {
    "startMs": 537839,
    "text": "those kind of uh videos. Um on"
  },
  {
    "startMs": 540800,
    "text": "understanding the what we are where we"
  },
  {
    "startMs": 543600,
    "text": "currently are is we have lot of tools"
  },
  {
    "startMs": 546000,
    "text": "that can map pixels to semantic labels"
  },
  {
    "startMs": 547839,
    "text": "like Google lens is obviously great to"
  },
  {
    "startMs": 550399,
    "text": "identify plants and flowers and I use"
  },
  {
    "startMs": 553360,
    "text": "that all the time. The SAM 3 for"
  },
  {
    "startMs": 555600,
    "text": "segmentation, YOLO for uh object"
  },
  {
    "startMs": 558720,
    "text": "recognition detection, mascaras CNN."
  },
  {
    "startMs": 560959,
    "text": "These are of course highly robust and"
  },
  {
    "startMs": 562800,
    "text": "they're used everywhere in the industry,"
  },
  {
    "startMs": 564320,
    "text": "but they're fundamentally passive. So"
  },
  {
    "startMs": 565920,
    "text": "there's no reasoning capability to them."
  },
  {
    "startMs": 567839,
    "text": "So they can't answer more complex"
  },
  {
    "startMs": 569279,
    "text": "questions. Um and really where the"
  },
  {
    "startMs": 571440,
    "text": "frontier is is uh with thinking visual"
  },
  {
    "startMs": 574000,
    "text": "thinking models. These models will have"
  },
  {
    "startMs": 576959,
    "text": "active spatial and temporal"
  },
  {
    "startMs": 578240,
    "text": "intelligence. They can extract actional"
  },
  {
    "startMs": 580959,
    "text": "logic for planning uh agentic workflows"
  },
  {
    "startMs": 584560,
    "text": "and physical execution."
  },
  {
    "startMs": 588240,
    "text": "And so our approach uh is uh four stage."
  },
  {
    "startMs": 592000,
    "text": "So we are collecting and generating our"
  },
  {
    "startMs": 595200,
    "text": "own uh multimodal data uh visual"
  },
  {
    "startMs": 598000,
    "text": "reasoning specific data. This this kind"
  },
  {
    "startMs": 600240,
    "text": "of data we found you just can't uh get"
  },
  {
    "startMs": 602560,
    "text": "online. Uh we have a synthetic data"
  },
  {
    "startMs": 605519,
    "text": "flywheel using evals agents SFT and RL"
  },
  {
    "startMs": 608399,
    "text": "to improve the model. We're making some"
  },
  {
    "startMs": 610720,
    "text": "uh we made some advances to the"
  },
  {
    "startMs": 612240,
    "text": "architecture um in terms of uh various"
  },
  {
    "startMs": 615600,
    "text": "different time um advance various"
  },
  {
    "startMs": 617600,
    "text": "different improvements on top of the"
  },
  {
    "startMs": 618880,
    "text": "transformer-based architecture and we're"
  },
  {
    "startMs": 621040,
    "text": "also enabling visual chain of thought"
  },
  {
    "startMs": 623279,
    "text": "reasoning and this is one of the key"
  },
  {
    "startMs": 625519,
    "text": "things that humans have that no frontier"
  },
  {
    "startMs": 627600,
    "text": "model has today since the frontier"
  },
  {
    "startMs": 629680,
    "text": "models are only textual uh chain of"
  },
  {
    "startMs": 631680,
    "text": "thought based"
  },
  {
    "startMs": 633600,
    "text": "um and this is one example of a visual"
  },
  {
    "startMs": 635360,
    "text": "chain of thought. So the question is"
  },
  {
    "startMs": 636880,
    "text": "like how many red hotels are built in"
  },
  {
    "startMs": 639839,
    "text": "this photo? Then the model realizes oh"
  },
  {
    "startMs": 643360,
    "text": "uh first we need to identify all the"
  },
  {
    "startMs": 645120,
    "text": "hotels. So it draws boxes around hotels"
  },
  {
    "startMs": 649440,
    "text": "um and other objects and then uh it"
  },
  {
    "startMs": 652480,
    "text": "reduces that to the red hotel. So it's"
  },
  {
    "startMs": 654959,
    "text": "this multi-step uh process happening in"
  },
  {
    "startMs": 657680,
    "text": "the visual space natively."
  },
  {
    "startMs": 661519,
    "text": "So um about our company um I'm the"
  },
  {
    "startMs": 665279,
    "text": "co-founder and CEO. I spent the last 12"
  },
  {
    "startMs": 667920,
    "text": "years at Google Brain and Deep Mind. Um"
  },
  {
    "startMs": 670480,
    "text": "I developed a lot of the foundational"
  },
  {
    "startMs": 672079,
    "text": "techniques for the model for modern"
  },
  {
    "startMs": 673600,
    "text": "LLMs. 11 years ago I was the first"
  },
  {
    "startMs": 676480,
    "text": "author of the work that introduced"
  },
  {
    "startMs": 678240,
    "text": "pre-training and fine-tuning. That's the"
  },
  {
    "startMs": 680640,
    "text": "work when combined with the transformer"
  },
  {
    "startMs": 682399,
    "text": "paper in 2017 led to the GBT series of"
  },
  {
    "startMs": 685360,
    "text": "models. So all the GBT uh papers site"
  },
  {
    "startMs": 687760,
    "text": "our paper. Um I co-led the earlye models"
  },
  {
    "startMs": 691440,
    "text": "uh the first model that was"
  },
  {
    "startMs": 692480,
    "text": "state-of-the-art called glam and then um"
  },
  {
    "startMs": 695200,
    "text": "more recently I co-led the palm to 2"
  },
  {
    "startMs": 697360,
    "text": "pre-training architecture and I was"
  },
  {
    "startMs": 699360,
    "text": "co-lead for the gemini data area and my"
  },
  {
    "startMs": 702000,
    "text": "co-founder info"
  },
  {
    "startMs": 705120,
    "text": "and Google research he led research for"
  },
  {
    "startMs": 707200,
    "text": "Apple's first public multimodal model"
  },
  {
    "startMs": 709680,
    "text": "MM1 and he's has a a lot of experience"
  },
  {
    "startMs": 714079,
    "text": "in visual reasoning um across uh"
  },
  {
    "startMs": 717279,
    "text": "language as"
  },
  {
    "startMs": 719680,
    "text": "And this is our team. So we're roughly"
  },
  {
    "startMs": 722320,
    "text": "20 people now. Um we've also have a"
  },
  {
    "startMs": 726079,
    "text": "chief reasoning architect Dustin Tran."
  },
  {
    "startMs": 727920,
    "text": "Previously he was lead of post training"
  },
  {
    "startMs": 729839,
    "text": "at XAI. Um and we've hired a world-class"
  },
  {
    "startMs": 734079,
    "text": "team um across uh many other uh"
  },
  {
    "startMs": 737120,
    "text": "companies like Apple uh XAI um deep mind"
  },
  {
    "startMs": 740959,
    "text": "Amazon and so on."
  },
  {
    "startMs": 744160,
    "text": "Um and in terms of the uh use cases that"
  },
  {
    "startMs": 747279,
    "text": "I mentioned, robotics is one primary use"
  },
  {
    "startMs": 750079,
    "text": "case. So robots have uh really critical"
  },
  {
    "startMs": 753600,
    "text": "bottlenecks performing complex real-time"
  },
  {
    "startMs": 756079,
    "text": "physical actions uh in these kind of"
  },
  {
    "startMs": 758399,
    "text": "like dynamic environments. Uh but"
  },
  {
    "startMs": 760800,
    "text": "existing vision models uh you probably"
  },
  {
    "startMs": 763360,
    "text": "realize are trained from static images"
  },
  {
    "startMs": 765279,
    "text": "and very directed videos. They are not"
  },
  {
    "startMs": 768560,
    "text": "like act they don't have active physical"
  },
  {
    "startMs": 770320,
    "text": "interaction. So existing methods are"
  },
  {
    "startMs": 773680,
    "text": "overengineered and brittle. Um and uh we"
  },
  {
    "startMs": 777600,
    "text": "are planning to release a model API"
  },
  {
    "startMs": 779760,
    "text": "available uh by the end of this year. Um"
  },
  {
    "startMs": 782880,
    "text": "and at that point the API can be used to"
  },
  {
    "startMs": 786240,
    "text": "deliver action relevant uh scene"
  },
  {
    "startMs": 789040,
    "text": "understanding into existing um planning"
  },
  {
    "startMs": 793040,
    "text": "and control systems for these um robots."
  },
  {
    "startMs": 796880,
    "text": "Another important use case for uh visual"
  },
  {
    "startMs": 800639,
    "text": "reasoning um is construction. So"
  },
  {
    "startMs": 803440,
    "text": "construction sites they have these very"
  },
  {
    "startMs": 806079,
    "text": "complex zone specific safety rules. Um"
  },
  {
    "startMs": 810000,
    "text": "and uh computer vision can't adapt fast"
  },
  {
    "startMs": 812480,
    "text": "enough to changing safety rules. They"
  },
  {
    "startMs": 815440,
    "text": "also can't interpret things like OSHA"
  },
  {
    "startMs": 818240,
    "text": "policy language and match the that"
  },
  {
    "startMs": 822000,
    "text": "language to what's actually going on at"
  },
  {
    "startMs": 824399,
    "text": "the site or understand the spatial"
  },
  {
    "startMs": 826320,
    "text": "relationships that are important there."
  },
  {
    "startMs": 828079,
    "text": "For example, like how many of these"
  },
  {
    "startMs": 829440,
    "text": "workers are wearing helmets or like is"
  },
  {
    "startMs": 832560,
    "text": "the construction um happening according"
  },
  {
    "startMs": 836000,
    "text": "to the plans that uh were defined"
  },
  {
    "startMs": 838880,
    "text": "earlier."
  },
  {
    "startMs": 840480,
    "text": "And um currently enforcing these rules"
  },
  {
    "startMs": 843440,
    "text": "require training separate models for"
  },
  {
    "startMs": 845440,
    "text": "different use cases uh because they're"
  },
  {
    "startMs": 847600,
    "text": "very these models as I said before are"
  },
  {
    "startMs": 849519,
    "text": "very brittle. So you constantly have to"
  },
  {
    "startMs": 851360,
    "text": "do uh retraining. Um and our approach uh"
  },
  {
    "startMs": 855040,
    "text": "with the video um understanding"
  },
  {
    "startMs": 857199,
    "text": "capabilities that we are building into"
  },
  {
    "startMs": 858880,
    "text": "our models is um allows you to um ground"
  },
  {
    "startMs": 863519,
    "text": "these video streams in the safety"
  },
  {
    "startMs": 866000,
    "text": "regulations. And of course the safety"
  },
  {
    "startMs": 867360,
    "text": "regulations are in text, they're in"
  },
  {
    "startMs": 869279,
    "text": "language. So you have to be um the model"
  },
  {
    "startMs": 872000,
    "text": "has to manipulate both language and"
  },
  {
    "startMs": 873600,
    "text": "vision very well."
  },
  {
    "startMs": 875760,
    "text": "And um yeah uh this will allow these"
  },
  {
    "startMs": 878880,
    "text": "models to interpret uh site policies"
  },
  {
    "startMs": 881680,
    "text": "using the current camera infrastructure"
  },
  {
    "startMs": 884000,
    "text": "that they have."
  },
  {
    "startMs": 886800,
    "text": "Um and then finally architecture and"
  },
  {
    "startMs": 889279,
    "text": "design we think is also a very uh"
  },
  {
    "startMs": 892480,
    "text": "promising use case here. Uh this is"
  },
  {
    "startMs": 895199,
    "text": "exactly the use case where you need to"
  },
  {
    "startMs": 897199,
    "text": "be very detail oriented. So back to the"
  },
  {
    "startMs": 899680,
    "text": "board game example around counting"
  },
  {
    "startMs": 902480,
    "text": "spatial relationships. This shows up a"
  },
  {
    "startMs": 904800,
    "text": "lot in architecture and design. Uh like"
  },
  {
    "startMs": 907360,
    "text": "if you design a if you design a house"
  },
  {
    "startMs": 910000,
    "text": "with with four bedrooms instead of"
  },
  {
    "startMs": 911440,
    "text": "three, that homeowner is going to be"
  },
  {
    "startMs": 913600,
    "text": "very angry, right? Um so obviously"
  },
  {
    "startMs": 916320,
    "text": "counting is actually important. Um and"
  },
  {
    "startMs": 920240,
    "text": "also just understanding these spatial"
  },
  {
    "startMs": 922160,
    "text": "constraints, real world constraints is"
  },
  {
    "startMs": 925519,
    "text": "uh is a very manual process uh these"
  },
  {
    "startMs": 928079,
    "text": "days. We spoke to a mechanical"
  },
  {
    "startMs": 931279,
    "text": "engineering company just a few weeks ago"
  },
  {
    "startMs": 933120,
    "text": "and they said to design one small part"
  },
  {
    "startMs": 936320,
    "text": "of a robot testing platform takes 100 to"
  },
  {
    "startMs": 940240,
    "text": "200 hours uh of the time to design the"
  },
  {
    "startMs": 943600,
    "text": "entire testing platform. Um, I believe"
  },
  {
    "startMs": 946639,
    "text": "it takes 2,000 to 3,000 hours of human"
  },
  {
    "startMs": 950480,
    "text": "uh time there. And they've uh a lot of"
  },
  {
    "startMs": 954639,
    "text": "these places they've tried frontier"
  },
  {
    "startMs": 956480,
    "text": "models, but they just don't work for"
  },
  {
    "startMs": 958000,
    "text": "these use cases. They really struggle to"
  },
  {
    "startMs": 960320,
    "text": "understand uh visual context across"
  },
  {
    "startMs": 963040,
    "text": "these like architecture blueprints, 3D"
  },
  {
    "startMs": 965680,
    "text": "CAD, CAM files. Um, and so there are"
  },
  {
    "startMs": 968560,
    "text": "lots of errors there. Um and similarly"
  },
  {
    "startMs": 972160,
    "text": "we believe that this can be useful"
  },
  {
    "startMs": 973920,
    "text": "useful for other kinds of design as well"
  },
  {
    "startMs": 975759,
    "text": "not just um architecture and engineering"
  },
  {
    "startMs": 978959,
    "text": "but maybe like designing"
  },
  {
    "startMs": 981680,
    "text": "um yeah for the web or fashion or other"
  },
  {
    "startMs": 983920,
    "text": "things and our approach is uh we're"
  },
  {
    "startMs": 986959,
    "text": "using multimodal reasoning to uh to"
  },
  {
    "startMs": 990160,
    "text": "extract um this uh geometric logic"
  },
  {
    "startMs": 993600,
    "text": "that's important. We're allowing"
  },
  {
    "startMs": 995040,
    "text": "programmatic validation or simulation"
  },
  {
    "startMs": 998560,
    "text": "validation. Just like in code, you can"
  },
  {
    "startMs": 1000800,
    "text": "run code against unit tests. You can"
  },
  {
    "startMs": 1003360,
    "text": "also run u mechanical devices through"
  },
  {
    "startMs": 1006000,
    "text": "simulators that have been developed"
  },
  {
    "startMs": 1007600,
    "text": "through seammens um and uh v various"
  },
  {
    "startMs": 1010800,
    "text": "other companies to see if something will"
  },
  {
    "startMs": 1012880,
    "text": "work in the real world. So there's a lot"
  },
  {
    "startMs": 1014639,
    "text": "of parallels actually between uh this"
  },
  {
    "startMs": 1017120,
    "text": "kind of like mechanical design and"
  },
  {
    "startMs": 1018720,
    "text": "coding itself. But mechanical design is"
  },
  {
    "startMs": 1021040,
    "text": "still relatively untouched by AI."
  },
  {
    "startMs": 1024640,
    "text": "Um and yeah, CAD CAM quality control is"
  },
  {
    "startMs": 1027679,
    "text": "another potential use case and"
  },
  {
    "startMs": 1029520,
    "text": "ultimately we believe that this is going"
  },
  {
    "startMs": 1031360,
    "text": "to be a critical step to the future of"
  },
  {
    "startMs": 1034720,
    "text": "mechanical design where the where AI can"
  },
  {
    "startMs": 1038079,
    "text": "make faster cars, more efficient"
  },
  {
    "startMs": 1040959,
    "text": "rockets, better batteries and all these"
  },
  {
    "startMs": 1043760,
    "text": "things cannot be done just with code. Uh"
  },
  {
    "startMs": 1046000,
    "text": "people are not coding up the next iPhone"
  },
  {
    "startMs": 1047919,
    "text": "or coding up the next uh SpaceX rocket."
  },
  {
    "startMs": 1051039,
    "text": "It's all fundamentally very visual."
  },
  {
    "startMs": 1056240,
    "text": "So, um you can find out more about us"
  },
  {
    "startMs": 1059919,
    "text": "through our website um lauren.ai, our"
  },
  {
    "startMs": 1063600,
    "text": "Twitter page xx.comai"
  },
  {
    "startMs": 1066880,
    "text": "or our LinkedIn uh page. And yeah, happy"
  },
  {
    "startMs": 1070400,
    "text": "to take any questions. I'll be standing"
  },
  {
    "startMs": 1073200,
    "text": "around here for for a little bit."
  },
  {
    "startMs": 1075760,
    "text": "Thanks."
  },
  {
    "startMs": 1078142,
    "text": "[applause]"
  }
]