[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 13040,
    "text": "Hi. Can everyone hear me? Sweet. We have"
  },
  {
    "startMs": 16720,
    "text": "a short 20 minutes here and so I wanted"
  },
  {
    "startMs": 19279,
    "text": "to jump in and get right to it. Uh my"
  },
  {
    "startMs": 21600,
    "text": "name is De. I'm the co-founder and CEO"
  },
  {
    "startMs": 22960,
    "text": "of Redducto. Uh and today we wanted to"
  },
  {
    "startMs": 25680,
    "text": "talk about one of the I think really"
  },
  {
    "startMs": 28080,
    "text": "practical but maybe less sexy parts of"
  },
  {
    "startMs": 31359,
    "text": "building agents that actually work in"
  },
  {
    "startMs": 33200,
    "text": "the real world uh which is data. Um I'm"
  },
  {
    "startMs": 35680,
    "text": "sure you've seen plenty of talks about"
  },
  {
    "startMs": 37280,
    "text": "data today. We primarily have focused on"
  },
  {
    "startMs": 39920,
    "text": "building infrastructure for anybody"
  },
  {
    "startMs": 41520,
    "text": "working with some of the hardest sources"
  },
  {
    "startMs": 43040,
    "text": "of data which is unstructured images,"
  },
  {
    "startMs": 45440,
    "text": "PDFs, spreadsheets, everything that"
  },
  {
    "startMs": 47680,
    "text": "humans are used to using day-to-day. Uh,"
  },
  {
    "startMs": 50239,
    "text": "have any of you used Reduct already or"
  },
  {
    "startMs": 53280,
    "text": "trial it? Cool. Um, I guess helpful"
  },
  {
    "startMs": 57360,
    "text": "context maybe to start is we're an"
  },
  {
    "startMs": 59120,
    "text": "agentic document processing platform."
  },
  {
    "startMs": 60960,
    "text": "Um, so we help a lot of the world's"
  },
  {
    "startMs": 63120,
    "text": "leading AI teams build uh both AI"
  },
  {
    "startMs": 66000,
    "text": "applications and also workflows"
  },
  {
    "startMs": 67439,
    "text": "depending on what they're trying to do."
  },
  {
    "startMs": 69119,
    "text": "Uh, that includes a lot of the AI"
  },
  {
    "startMs": 70640,
    "text": "natives that you've probably seen today."
  },
  {
    "startMs": 72159,
    "text": "It includes the Harveys, the Lores, the"
  },
  {
    "startMs": 73840,
    "text": "Rogos of the world. Uh, but also some of"
  },
  {
    "startMs": 76080,
    "text": "the largest enterprises in the world."
  },
  {
    "startMs": 77520,
    "text": "And I think that's important context for"
  },
  {
    "startMs": 79200,
    "text": "what we're going to talk about. Um, we"
  },
  {
    "startMs": 81200,
    "text": "work with the largest tech companies,"
  },
  {
    "startMs": 83200,
    "text": "global financial institutions, insurance"
  },
  {
    "startMs": 85680,
    "text": "orgs, people that have decades worth of"
  },
  {
    "startMs": 88560,
    "text": "historical data that historically has"
  },
  {
    "startMs": 90799,
    "text": "been really, really hard to actually use"
  },
  {
    "startMs": 92479,
    "text": "outside of a demo context. And across"
  },
  {
    "startMs": 95119,
    "text": "all these companies at this point, we've"
  },
  {
    "startMs": 96799,
    "text": "processed many billions of documents for"
  },
  {
    "startMs": 98880,
    "text": "our customers. I've gotten kind of lazy"
  },
  {
    "startMs": 100960,
    "text": "with that plus symbol at the end, uh,"
  },
  {
    "startMs": 102960,
    "text": "but the number keeps changing and so"
  },
  {
    "startMs": 104720,
    "text": "we'll let it sit there. Uh but the main"
  },
  {
    "startMs": 107600,
    "text": "thing that we've learned across those"
  },
  {
    "startMs": 108799,
    "text": "and what I wanted to focus on today is"
  },
  {
    "startMs": 110479,
    "text": "not actually a reduct product itself."
  },
  {
    "startMs": 112720,
    "text": "It's the intricacies of what we've"
  },
  {
    "startMs": 114240,
    "text": "learned that hopefully you can actually"
  },
  {
    "startMs": 116320,
    "text": "take home and implement in the work that"
  },
  {
    "startMs": 118719,
    "text": "you're doing as well. Um and I think a"
  },
  {
    "startMs": 120799,
    "text": "lot of the work that we've done and a"
  },
  {
    "startMs": 122079,
    "text": "lot of the learnings we've had are in"
  },
  {
    "startMs": 124000,
    "text": "this sort of broader thematic context of"
  },
  {
    "startMs": 126079,
    "text": "the scope of AI applications has changed"
  },
  {
    "startMs": 128080,
    "text": "a lot recently. Uh from just like"
  },
  {
    "startMs": 130239,
    "text": "information synthesis products to actual"
  },
  {
    "startMs": 132560,
    "text": "agents doing work. And with that we"
  },
  {
    "startMs": 135360,
    "text": "found that there are a few things that"
  },
  {
    "startMs": 136720,
    "text": "are worth talking through. One is"
  },
  {
    "startMs": 137920,
    "text": "framing the problem itself. Uh the"
  },
  {
    "startMs": 139440,
    "text": "bottleneck that people face. Uh why PDFs"
  },
  {
    "startMs": 142160,
    "text": "in particular are hard even though"
  },
  {
    "startMs": 143440,
    "text": "you've probably seen two dozen different"
  },
  {
    "startMs": 146400,
    "text": "PDF processing launches on Twitter. Uh"
  },
  {
    "startMs": 149200,
    "text": "where we find strengths and weaknesses"
  },
  {
    "startMs": 150800,
    "text": "with different tools. Uh we think"
  },
  {
    "startMs": 152239,
    "text": "there's a right place in time for"
  },
  {
    "startMs": 153599,
    "text": "traditional CV versus BLMs. Um and then"
  },
  {
    "startMs": 156480,
    "text": "more interestingly I think the latter"
  },
  {
    "startMs": 157760,
    "text": "half of this talk will actually be"
  },
  {
    "startMs": 158959,
    "text": "focused on the next frontier. uh what"
  },
  {
    "startMs": 161599,
    "text": "we've seen be possible as a result of"
  },
  {
    "startMs": 163280,
    "text": "having agents in the loop uh things that"
  },
  {
    "startMs": 165440,
    "text": "we found as a result of being able to"
  },
  {
    "startMs": 166800,
    "text": "use harnesses for different types of"
  },
  {
    "startMs": 168239,
    "text": "tasks uh and our learnings around things"
  },
  {
    "startMs": 170480,
    "text": "like evaluations uh as you go from rag"
  },
  {
    "startMs": 172959,
    "text": "to building agent products but I'll"
  },
  {
    "startMs": 175920,
    "text": "start with the first thing which is"
  },
  {
    "startMs": 177040,
    "text": "something that I assume has been harped"
  },
  {
    "startMs": 179120,
    "text": "on a lot if you went to AI engineer a"
  },
  {
    "startMs": 181040,
    "text": "few years ago the word that you would"
  },
  {
    "startMs": 183280,
    "text": "have heard in every single talk would"
  },
  {
    "startMs": 184800,
    "text": "have been rag um and everybody was"
  },
  {
    "startMs": 186879,
    "text": "building some form of rag application"
  },
  {
    "startMs": 188879,
    "text": "for a while all applications ations were"
  },
  {
    "startMs": 191360,
    "text": "really some form of information"
  },
  {
    "startMs": 192640,
    "text": "synthesis, right? You would pull in"
  },
  {
    "startMs": 195200,
    "text": "information from some context, whether"
  },
  {
    "startMs": 197120,
    "text": "that's a perfect prompt or a file that a"
  },
  {
    "startMs": 199120,
    "text": "user uploaded and you would have"
  },
  {
    "startMs": 201360,
    "text": "something like a search product. Um,"
  },
  {
    "startMs": 202879,
    "text": "you'd have enterprise search, you'd have"
  },
  {
    "startMs": 204640,
    "text": "a chatbot that would do simple question"
  },
  {
    "startMs": 206480,
    "text": "answer on top of the content and that"
  },
  {
    "startMs": 208720,
    "text": "was it. But today, the buzzword that"
  },
  {
    "startMs": 211920,
    "text": "you've probably heard a million times is"
  },
  {
    "startMs": 214159,
    "text": "agents. Uh, and for those of you that"
  },
  {
    "startMs": 216720,
    "text": "are engineers, you've probably had cloud"
  },
  {
    "startMs": 218799,
    "text": "code or similar tool do a lot of your"
  },
  {
    "startMs": 220640,
    "text": "endto-end work for many, many tasks. And"
  },
  {
    "startMs": 223040,
    "text": "that same sort of shift is starting to"
  },
  {
    "startMs": 224480,
    "text": "happen for all sorts of white collar"
  },
  {
    "startMs": 226400,
    "text": "work. Um, whether you're in finance or"
  },
  {
    "startMs": 228400,
    "text": "insurance or healthcare, people are"
  },
  {
    "startMs": 230319,
    "text": "starting to make autonomous decisions."
  },
  {
    "startMs": 232239,
    "text": "They're trying to create end-to-end work"
  },
  {
    "startMs": 234480,
    "text": "products, not just answer questions from"
  },
  {
    "startMs": 236080,
    "text": "a PDF, but generate and modify PDFs as"
  },
  {
    "startMs": 238480,
    "text": "well. And that's a very different"
  },
  {
    "startMs": 240159,
    "text": "framing. the tools that you need, the"
  },
  {
    "startMs": 241760,
    "text": "problems that you face change a lot"
  },
  {
    "startMs": 243599,
    "text": "versus just trying to build a retrieval"
  },
  {
    "startMs": 246319,
    "text": "uh platform."
  },
  {
    "startMs": 248319,
    "text": "And the common core for all of this is"
  },
  {
    "startMs": 252239,
    "text": "it actually ends up being an even more"
  },
  {
    "startMs": 254959,
    "text": "importance problem to solve for when you"
  },
  {
    "startMs": 257359,
    "text": "start having these multi-step pipelines."
  },
  {
    "startMs": 259120,
    "text": "If you're just doing question answer,"
  },
  {
    "startMs": 260720,
    "text": "there is obviously a risk that comes"
  },
  {
    "startMs": 263360,
    "text": "with just answering the question. But"
  },
  {
    "startMs": 265520,
    "text": "when agents are making multiple"
  },
  {
    "startMs": 267199,
    "text": "decisions that compound over the course"
  },
  {
    "startMs": 269040,
    "text": "of a source of files, when they're"
  },
  {
    "startMs": 270639,
    "text": "pulling in multiple sources of data, the"
  },
  {
    "startMs": 272720,
    "text": "risk of bad inputs becomes really,"
  },
  {
    "startMs": 274960,
    "text": "really pervasive in your pipeline. And"
  },
  {
    "startMs": 277759,
    "text": "that's what we focus on because at the"
  },
  {
    "startMs": 280000,
    "text": "end of the day, a lot of the value of"
  },
  {
    "startMs": 282800,
    "text": "language model tools in the real world"
  },
  {
    "startMs": 284960,
    "text": "only applies in the context to which you"
  },
  {
    "startMs": 287120,
    "text": "apply that intelligence. And for a lot"
  },
  {
    "startMs": 289040,
    "text": "of enterprises, data is unstructured, it"
  },
  {
    "startMs": 292160,
    "text": "is scattered, and it is multimodal. uh"
  },
  {
    "startMs": 294400,
    "text": "it's not this like cleanly organized"
  },
  {
    "startMs": 296479,
    "text": "repository. You have random teams and"
  },
  {
    "startMs": 298880,
    "text": "different organizations that go through"
  },
  {
    "startMs": 300960,
    "text": "and put things in Google Drive and Box"
  },
  {
    "startMs": 302720,
    "text": "and wherever else you might have things."
  },
  {
    "startMs": 304800,
    "text": "Um you're going to have data for formats"
  },
  {
    "startMs": 306960,
    "text": "that are unstructured by default. You"
  },
  {
    "startMs": 308720,
    "text": "don't necessarily know what is in your"
  },
  {
    "startMs": 310400,
    "text": "corpus of information. And you're going"
  },
  {
    "startMs": 312720,
    "text": "to have all sorts of downstream problems"
  },
  {
    "startMs": 314320,
    "text": "that come with that. Um some of the"
  },
  {
    "startMs": 316320,
    "text": "problems are going to be parsing"
  },
  {
    "startMs": 317520,
    "text": "extraction accuracy and that definitely"
  },
  {
    "startMs": 318960,
    "text": "matters. But it's also going to be about"
  },
  {
    "startMs": 320560,
    "text": "problems like are you retrieving the"
  },
  {
    "startMs": 322080,
    "text": "right context? It's going to be about"
  },
  {
    "startMs": 323759,
    "text": "how do you actually interact with that"
  },
  {
    "startMs": 325199,
    "text": "context and apply modifications to it."
  },
  {
    "startMs": 328720,
    "text": "And the thing that you've probably heard"
  },
  {
    "startMs": 331039,
    "text": "harp on and others in the space is PDFs"
  },
  {
    "startMs": 333759,
    "text": "are surprisingly still a very hard"
  },
  {
    "startMs": 336080,
    "text": "problem. Um I don't know if any of you"
  },
  {
    "startMs": 338479,
    "text": "follow Serge which is a data lab that"
  },
  {
    "startMs": 340400,
    "text": "works with a lot of the foundation model"
  },
  {
    "startMs": 342160,
    "text": "companies. uh they have this really"
  },
  {
    "startMs": 343840,
    "text": "great benchmark called GDP PDF where"
  },
  {
    "startMs": 346160,
    "text": "even Fable uh like current frontier of"
  },
  {
    "startMs": 349360,
    "text": "model intelligence is at about 30% on"
  },
  {
    "startMs": 351840,
    "text": "their benchmark um and it's entirely"
  },
  {
    "startMs": 353520,
    "text": "predicated on this idea of can we have"
  },
  {
    "startMs": 355360,
    "text": "models go through and actually make"
  },
  {
    "startMs": 357520,
    "text": "determinations off of contents that"
  },
  {
    "startMs": 359280,
    "text": "would be in documents like PDFs uh and"
  },
  {
    "startMs": 362000,
    "text": "the reason why they're hard I'll come"
  },
  {
    "startMs": 363440,
    "text": "back to that benchmark in a second is"
  },
  {
    "startMs": 365759,
    "text": "fundamentally PDFs as a file format are"
  },
  {
    "startMs": 368960,
    "text": "both very old but were designed in a"
  },
  {
    "startMs": 370960,
    "text": "very different context I've genuinely"
  },
  {
    "startMs": 373600,
    "text": "met people that have worked on PDF"
  },
  {
    "startMs": 375199,
    "text": "processing longer than I've been alive."
  },
  {
    "startMs": 376800,
    "text": "Uh I've met the people that worked on"
  },
  {
    "startMs": 378720,
    "text": "printer drivers for printers to print"
  },
  {
    "startMs": 380479,
    "text": "PDFs in the early 1990s. And that's what"
  },
  {
    "startMs": 382720,
    "text": "it was like. You wanted to be able to"
  },
  {
    "startMs": 384240,
    "text": "represent what was faithfully on the"
  },
  {
    "startMs": 386479,
    "text": "documents when you originally created it"
  },
  {
    "startMs": 388560,
    "text": "and have it be printable at the end of"
  },
  {
    "startMs": 390240,
    "text": "it. A lot of the considerations today"
  },
  {
    "startMs": 392319,
    "text": "are not around that. Um ultimately what"
  },
  {
    "startMs": 394800,
    "text": "we want is something like a markdown"
  },
  {
    "startMs": 397280,
    "text": "representation, something that agents"
  },
  {
    "startMs": 398560,
    "text": "will effectively reason on. And in the"
  },
  {
    "startMs": 401440,
    "text": "real world, humans encode so much"
  },
  {
    "startMs": 403680,
    "text": "context visually. Like the average"
  },
  {
    "startMs": 406880,
    "text": "financial analyst that is a new grad in"
  },
  {
    "startMs": 409600,
    "text": "their IB role uh is not going through"
  },
  {
    "startMs": 412560,
    "text": "and thinking in terms of will agents"
  },
  {
    "startMs": 414319,
    "text": "consume this deck. They're creating"
  },
  {
    "startMs": 415759,
    "text": "these really creative slides. I'm sure"
  },
  {
    "startMs": 418080,
    "text": "you've seen the softbank slides with uh"
  },
  {
    "startMs": 420479,
    "text": "goose laying eggs. Those details matter,"
  },
  {
    "startMs": 423199,
    "text": "right? Like a lot of the data that"
  },
  {
    "startMs": 424560,
    "text": "you're going to reason on is going to be"
  },
  {
    "startMs": 426000,
    "text": "tabular structures that maybe don't have"
  },
  {
    "startMs": 428000,
    "text": "clean grid lines and separate out the"
  },
  {
    "startMs": 430160,
    "text": "merge cells. You're going to have things"
  },
  {
    "startMs": 431440,
    "text": "like line charts and graphs. You're"
  },
  {
    "startMs": 433440,
    "text": "going to have messy handwriting that"
  },
  {
    "startMs": 435199,
    "text": "even I as a human would often struggle"
  },
  {
    "startMs": 437120,
    "text": "to read. And that's the sort of problem"
  },
  {
    "startMs": 439120,
    "text": "that you need to solve for if you're"
  },
  {
    "startMs": 440720,
    "text": "dealing with that long tail."
  },
  {
    "startMs": 443599,
    "text": "And we started the company in 2023"
  },
  {
    "startMs": 446479,
    "text": "because we felt that there was a sort of"
  },
  {
    "startMs": 448160,
    "text": "step change in what was possible here."
  },
  {
    "startMs": 450479,
    "text": "uh for a while as people would think"
  },
  {
    "startMs": 452800,
    "text": "about any sort of PDF processing"
  },
  {
    "startMs": 454639,
    "text": "problem, it used to be some modified"
  },
  {
    "startMs": 457120,
    "text": "version of an NLP pipeline. You do a"
  },
  {
    "startMs": 459440,
    "text": "simple OCR pass and then you try to"
  },
  {
    "startMs": 461199,
    "text": "post-process that text. And that worked"
  },
  {
    "startMs": 463680,
    "text": "when you would have really consistent"
  },
  {
    "startMs": 465520,
    "text": "layouts, right? Like if you knew that a"
  },
  {
    "startMs": 467120,
    "text": "W2 is always going to look like a W2,"
  },
  {
    "startMs": 469599,
    "text": "that's a constrained problem space that"
  },
  {
    "startMs": 471039,
    "text": "you can template your way around. But"
  },
  {
    "startMs": 473520,
    "text": "VLMs are interesting because they are"
  },
  {
    "startMs": 475280,
    "text": "fundamentally horizontal in nature. um"
  },
  {
    "startMs": 477599,
    "text": "for the first time you can have this"
  },
  {
    "startMs": 479599,
    "text": "premise of read the document the way"
  },
  {
    "startMs": 481840,
    "text": "that a human would have. You can have"
  },
  {
    "startMs": 482960,
    "text": "this premise of we want to address the"
  },
  {
    "startMs": 484479,
    "text": "longtail. Uh and so we found that they"
  },
  {
    "startMs": 487039,
    "text": "were incredible for all sorts of things"
  },
  {
    "startMs": 489039,
    "text": "like handwritten text in a way that"
  },
  {
    "startMs": 490800,
    "text": "traditional OCR just never was. But the"
  },
  {
    "startMs": 493919,
    "text": "flip side is that we don't think that"
  },
  {
    "startMs": 495520,
    "text": "they're a one-sizefits-all solution. And"
  },
  {
    "startMs": 497520,
    "text": "if you're solving this problem at scale,"
  },
  {
    "startMs": 499120,
    "text": "if you're a company dealing with"
  },
  {
    "startMs": 500319,
    "text": "hundreds of millions of documents, there"
  },
  {
    "startMs": 502400,
    "text": "are all sorts of secondary"
  },
  {
    "startMs": 503759,
    "text": "considerations like determinism. you"
  },
  {
    "startMs": 506000,
    "text": "care a lot about the efficiency of your"
  },
  {
    "startMs": 507599,
    "text": "processing. Uh, and so we find that"
  },
  {
    "startMs": 510000,
    "text": "there are some things where traditional"
  },
  {
    "startMs": 512000,
    "text": "CV is actually still really really"
  },
  {
    "startMs": 513919,
    "text": "strong. And this has been really"
  },
  {
    "startMs": 515279,
    "text": "underappreciated as autonomous vehicle"
  },
  {
    "startMs": 517599,
    "text": "research has gotten better. Techniques"
  },
  {
    "startMs": 518880,
    "text": "like object detection are more"
  },
  {
    "startMs": 520479,
    "text": "sophisticated than they were a decade"
  },
  {
    "startMs": 522560,
    "text": "ago. And so we find that subund million"
  },
  {
    "startMs": 524959,
    "text": "parameter models are really effective"
  },
  {
    "startMs": 526720,
    "text": "for things like detecting the layout of"
  },
  {
    "startMs": 528480,
    "text": "a document. Um, you can go really really"
  },
  {
    "startMs": 530800,
    "text": "far without even needing a large"
  },
  {
    "startMs": 532399,
    "text": "foundation model. And these are models"
  },
  {
    "startMs": 534080,
    "text": "that can actually run on CPU. You can"
  },
  {
    "startMs": 535839,
    "text": "run them at scale. You can make sure"
  },
  {
    "startMs": 537200,
    "text": "that you understand on a region level"
  },
  {
    "startMs": 539279,
    "text": "what are the hard things for you to"
  },
  {
    "startMs": 540720,
    "text": "process. And VLMs introduce this notion"
  },
  {
    "startMs": 543040,
    "text": "of semantics. You can go through and"
  },
  {
    "startMs": 544800,
    "text": "actually identify and make correct the"
  },
  {
    "startMs": 546560,
    "text": "sorts of mistakes that you were likely"
  },
  {
    "startMs": 547920,
    "text": "to have in your pipeline."
  },
  {
    "startMs": 550720,
    "text": "And off of that idea of semantics, uh,"
  },
  {
    "startMs": 554080,
    "text": "when you're deconstructing this problem"
  },
  {
    "startMs": 555519,
    "text": "and you have a clear sense of where the"
  },
  {
    "startMs": 557279,
    "text": "more nuanced things are, if you've"
  },
  {
    "startMs": 559120,
    "text": "segmented the text on your page and you"
  },
  {
    "startMs": 560880,
    "text": "understand where the handwriting is, you"
  },
  {
    "startMs": 562800,
    "text": "can introduce this notion of a sort of"
  },
  {
    "startMs": 564880,
    "text": "agent in the loop. Whereas historically,"
  },
  {
    "startMs": 566560,
    "text": "you would have had a human review team"
  },
  {
    "startMs": 568000,
    "text": "go through and annotate and correct"
  },
  {
    "startMs": 569440,
    "text": "mistakes. Uh, VLMs can now present this"
  },
  {
    "startMs": 572240,
    "text": "sort of idea of what we call agentic"
  },
  {
    "startMs": 573920,
    "text": "OCR. Really for us what that looks like"
  },
  {
    "startMs": 576800,
    "text": "is if you've ever used you know a tool"
  },
  {
    "startMs": 579040,
    "text": "like cursor that's applying fast edits"
  },
  {
    "startMs": 580880,
    "text": "in your IDE there's this notion of"
  },
  {
    "startMs": 582959,
    "text": "speculative decoding where you're"
  },
  {
    "startMs": 584080,
    "text": "applying token level edits to your"
  },
  {
    "startMs": 585360,
    "text": "output. You can apply a similar sort of"
  },
  {
    "startMs": 587440,
    "text": "principle here uh that is not just you"
  },
  {
    "startMs": 589680,
    "text": "know sending OCR to Gemini writing a"
  },
  {
    "startMs": 592720,
    "text": "really pretty prompt asking it nicely to"
  },
  {
    "startMs": 595200,
    "text": "not deviate too much from the original"
  },
  {
    "startMs": 596880,
    "text": "because we find that when you're doing"
  },
  {
    "startMs": 598000,
    "text": "that sort of next token prediction you"
  },
  {
    "startMs": 599839,
    "text": "introduce net new loss cases where"
  },
  {
    "startMs": 601680,
    "text": "models that are really intelligent will"
  },
  {
    "startMs": 603519,
    "text": "start actually correcting things not"
  },
  {
    "startMs": 605120,
    "text": "faithfully to what was in the document."
  },
  {
    "startMs": 606480,
    "text": "They'll see the word total and if the"
  },
  {
    "startMs": 608160,
    "text": "human made a mistake in that table"
  },
  {
    "startMs": 609600,
    "text": "models will actually sometimes go"
  },
  {
    "startMs": 610959,
    "text": "through and add up the values in the"
  },
  {
    "startMs": 613120,
    "text": "table themselves. What you want is to"
  },
  {
    "startMs": 615519,
    "text": "correct the token level edits that you"
  },
  {
    "startMs": 616959,
    "text": "want. Maybe you messed up a period"
  },
  {
    "startMs": 618240,
    "text": "versus a comma, a zero versus an O."
  },
  {
    "startMs": 620720,
    "text": "Those sorts of details really, really"
  },
  {
    "startMs": 622160,
    "text": "matter. And that's a question of how do"
  },
  {
    "startMs": 624000,
    "text": "we actually represent what a human would"
  },
  {
    "startMs": 625760,
    "text": "have seen if they had read that"
  },
  {
    "startMs": 626959,
    "text": "document."
  },
  {
    "startMs": 629120,
    "text": "So the way that we see this is agentic"
  },
  {
    "startMs": 631360,
    "text": "OCR is almost like that human loop"
  },
  {
    "startMs": 634000,
    "text": "analogy where you have the first inputs"
  },
  {
    "startMs": 637120,
    "text": "uh go through with a CD plus VLM parse"
  },
  {
    "startMs": 639680,
    "text": "but then you have a verification"
  },
  {
    "startMs": 641120,
    "text": "correction layer that ends up leading to"
  },
  {
    "startMs": 642959,
    "text": "a high confidence output."
  },
  {
    "startMs": 645680,
    "text": "But I mentioned earlier that we don't"
  },
  {
    "startMs": 647519,
    "text": "see the range of problems as purely just"
  },
  {
    "startMs": 649920,
    "text": "parsing and extraction. And a lot of"
  },
  {
    "startMs": 652720,
    "text": "what this looks like is I think it's"
  },
  {
    "startMs": 654720,
    "text": "important to think through the details"
  },
  {
    "startMs": 656560,
    "text": "of your pipeline. Even if you have a"
  },
  {
    "startMs": 658640,
    "text": "great documents to markdown pipeline and"
  },
  {
    "startMs": 661440,
    "text": "a great example of this is if you've"
  },
  {
    "startMs": 663519,
    "text": "built any sort of rag platform um you've"
  },
  {
    "startMs": 666240,
    "text": "probably had some consideration around"
  },
  {
    "startMs": 667760,
    "text": "things like tables."
  },
  {
    "startMs": 669760,
    "text": "There are a lot of things that you can"
  },
  {
    "startMs": 671040,
    "text": "encode well in something like markdown."
  },
  {
    "startMs": 674000,
    "text": "uh but things like this table where the"
  },
  {
    "startMs": 676160,
    "text": "merge cells actually encode a lot of"
  },
  {
    "startMs": 678160,
    "text": "meaning. It matters that you're"
  },
  {
    "startMs": 680079,
    "text": "preserving that sort of structure and"
  },
  {
    "startMs": 681519,
    "text": "it's not a model limitation. LMS are"
  },
  {
    "startMs": 683360,
    "text": "incredible at reasoning through like an"
  },
  {
    "startMs": 685120,
    "text": "HTML structure of the same table, but"
  },
  {
    "startMs": 687120,
    "text": "you're also wasting a lot of tokens and"
  },
  {
    "startMs": 688720,
    "text": "that gets expensive quickly. And"
  },
  {
    "startMs": 690720,
    "text": "obviously on the other end of the"
  },
  {
    "startMs": 691920,
    "text": "spectrum, you probably don't want to go"
  },
  {
    "startMs": 694079,
    "text": "through and encode simple tables in HTML"
  },
  {
    "startMs": 696399,
    "text": "because then you have a lot of HTML tags"
  },
  {
    "startMs": 698240,
    "text": "that are erroneous. And so what we ended"
  },
  {
    "startMs": 700320,
    "text": "up doing was looking at this as sort of"
  },
  {
    "startMs": 702000,
    "text": "like a dynamic problem of when you have"
  },
  {
    "startMs": 703680,
    "text": "a simple table, great, we can"
  },
  {
    "startMs": 705920,
    "text": "approximate that data in markdown. When"
  },
  {
    "startMs": 708399,
    "text": "you have a more complex table, you may"
  },
  {
    "startMs": 710000,
    "text": "want to use something like HTML. But"
  },
  {
    "startMs": 712640,
    "text": "it's not only language models that"
  },
  {
    "startMs": 714480,
    "text": "should be a consideration in your"
  },
  {
    "startMs": 715680,
    "text": "pipeline. If you're doing anything"
  },
  {
    "startMs": 717120,
    "text": "related to embedding, you're also going"
  },
  {
    "startMs": 719040,
    "text": "to have the secondary problem of"
  },
  {
    "startMs": 720640,
    "text": "retrieval of that context. Right? That"
  },
  {
    "startMs": 722959,
    "text": "same table that I showed you earlier, if"
  },
  {
    "startMs": 725040,
    "text": "you look at the HTML representation is"
  },
  {
    "startMs": 728240,
    "text": "really really messy. The vast majority"
  },
  {
    "startMs": 730399,
    "text": "of that snippets is just HTML tags. It's"
  },
  {
    "startMs": 732880,
    "text": "just classifying the structure of the"
  },
  {
    "startMs": 734320,
    "text": "document. And the unfortunate thing is"
  },
  {
    "startMs": 736800,
    "text": "whereas in some blanket evals, you might"
  },
  {
    "startMs": 738959,
    "text": "have contents that is really trying to"
  },
  {
    "startMs": 742000,
    "text": "do the work for the model and say"
  },
  {
    "startMs": 744000,
    "text": "exactly what you're looking for. A real"
  },
  {
    "startMs": 746720,
    "text": "world person does not enumerate the"
  },
  {
    "startMs": 748399,
    "text": "values in the table. They just say how"
  },
  {
    "startMs": 750160,
    "text": "did revenue change over time and they"
  },
  {
    "startMs": 752079,
    "text": "assume that you're going to retrieve the"
  },
  {
    "startMs": 753519,
    "text": "right table when it's relevant. And"
  },
  {
    "startMs": 755440,
    "text": "whereas language models can reason"
  },
  {
    "startMs": 757040,
    "text": "through that text effectively if you're"
  },
  {
    "startMs": 758560,
    "text": "pulling from a large corpus, we find"
  },
  {
    "startMs": 760480,
    "text": "that embedding models really struggle to"
  },
  {
    "startMs": 762639,
    "text": "correlate that natural language human"
  },
  {
    "startMs": 764880,
    "text": "prompt with this messy blob of HTML tags"
  },
  {
    "startMs": 767600,
    "text": "and numbers. And so a big thing that you"
  },
  {
    "startMs": 769920,
    "text": "can do that actually takes very little"
  },
  {
    "startMs": 771440,
    "text": "effort is creating a representation"
  },
  {
    "startMs": 773600,
    "text": "that's more so designed for the"
  },
  {
    "startMs": 774959,
    "text": "embedding model itself. taking the same"
  },
  {
    "startMs": 776639,
    "text": "table, we're creating a natural language"
  },
  {
    "startMs": 778560,
    "text": "representation of that table. So you"
  },
  {
    "startMs": 780160,
    "text": "have the best of both worlds. When"
  },
  {
    "startMs": 781360,
    "text": "you're actually passing this into the"
  },
  {
    "startMs": 782639,
    "text": "model for reasoning, you're using the"
  },
  {
    "startMs": 784480,
    "text": "HTML table representation. And when"
  },
  {
    "startMs": 786399,
    "text": "you're trying to make sure that you"
  },
  {
    "startMs": 787360,
    "text": "retrieve the right snippets, you're"
  },
  {
    "startMs": 788959,
    "text": "using that natural language block."
  },
  {
    "startMs": 792720,
    "text": "Off of that, uh there's also this idea"
  },
  {
    "startMs": 795120,
    "text": "of there's a lot to do that is not just"
  },
  {
    "startMs": 797600,
    "text": "parsing and extraction. Um, and I think"
  },
  {
    "startMs": 799440,
    "text": "a lot of the industry's focus has been"
  },
  {
    "startMs": 801360,
    "text": "on parsing and extraction historically"
  },
  {
    "startMs": 803360,
    "text": "because we do think that there's a"
  },
  {
    "startMs": 804880,
    "text": "massive uplift there. And I talked"
  },
  {
    "startMs": 807120,
    "text": "earlier about the GDF GDP PDF benchmark"
  },
  {
    "startMs": 811519,
    "text": "which I think is a great illustrative"
  },
  {
    "startMs": 813120,
    "text": "example of what you can see uh as a"
  },
  {
    "startMs": 815920,
    "text": "result of improving your data pipeline."
  },
  {
    "startMs": 818639,
    "text": "So what we found is that if you take the"
  },
  {
    "startMs": 820480,
    "text": "same exact benchmark that I mentioned"
  },
  {
    "startMs": 822079,
    "text": "earlier, uh, unfortunately we couldn't"
  },
  {
    "startMs": 823600,
    "text": "test it on Fable because our access was"
  },
  {
    "startMs": 826079,
    "text": "cut. uh but if you test on other models"
  },
  {
    "startMs": 828639,
    "text": "and you give it both the original PDF"
  },
  {
    "startMs": 830959,
    "text": "but also a structured representation of"
  },
  {
    "startMs": 832560,
    "text": "the PDF like the parse results here"
  },
  {
    "startMs": 835120,
    "text": "across models whether it's Gemini"
  },
  {
    "startMs": 837120,
    "text": "whether it's anthropic or openi um you"
  },
  {
    "startMs": 840560,
    "text": "find that you actually improve end LLM"
  },
  {
    "startMs": 843279,
    "text": "performance just from better inputs and"
  },
  {
    "startMs": 845760,
    "text": "it's to an extreme where models like GPT"
  },
  {
    "startMs": 848399,
    "text": "5.5 and opus actually outperform"
  },
  {
    "startMs": 851600,
    "text": "something like Fable out of the box not"
  },
  {
    "startMs": 853839,
    "text": "just on an accuracy basis but as a"
  },
  {
    "startMs": 856240,
    "text": "result of giving better inputs, the"
  },
  {
    "startMs": 858320,
    "text": "models end up needing to use fewer"
  },
  {
    "startMs": 860160,
    "text": "reasoning tokens as well. Um, they're"
  },
  {
    "startMs": 861920,
    "text": "focused less on representing the data"
  },
  {
    "startMs": 864000,
    "text": "and more on the actual outputs and as a"
  },
  {
    "startMs": 866160,
    "text": "result they end up driving down latency"
  },
  {
    "startMs": 868480,
    "text": "and also getting to the correct answer"
  },
  {
    "startMs": 870160,
    "text": "more quickly. But even once you have"
  },
  {
    "startMs": 873680,
    "text": "that sort of pipeline and you've gone"
  },
  {
    "startMs": 875440,
    "text": "through and you've actually inspected"
  },
  {
    "startMs": 878480,
    "text": "everything in your parsing layer, a lot"
  },
  {
    "startMs": 881040,
    "text": "of human work is going to require"
  },
  {
    "startMs": 882720,
    "text": "actually understanding the range of what"
  },
  {
    "startMs": 884480,
    "text": "you have in your corpus, routing it to"
  },
  {
    "startMs": 886560,
    "text": "the appropriate pipeline and sort of"
  },
  {
    "startMs": 888000,
    "text": "decomposing that problem or even at the"
  },
  {
    "startMs": 890320,
    "text": "end editing and modifying your document."
  },
  {
    "startMs": 892800,
    "text": "And so what we tried to do is look at"
  },
  {
    "startMs": 894560,
    "text": "this as this problem of how do you make"
  },
  {
    "startMs": 896000,
    "text": "sure that every interaction that a"
  },
  {
    "startMs": 897519,
    "text": "language model has with a document is as"
  },
  {
    "startMs": 899600,
    "text": "effective as if a human would have done"
  },
  {
    "startMs": 901120,
    "text": "it. Um if you're filling out a form, how"
  },
  {
    "startMs": 903040,
    "text": "do you make sure that you have precision"
  },
  {
    "startMs": 904240,
    "text": "in where you fill out fields? And a"
  },
  {
    "startMs": 906800,
    "text": "really good example of this uh on the"
  },
  {
    "startMs": 908639,
    "text": "orchestration side is I think"
  },
  {
    "startMs": 910399,
    "text": "classification splitting are a very"
  },
  {
    "startMs": 912639,
    "text": "underappreciated way to have an LM do"
  },
  {
    "startMs": 914720,
    "text": "its best work. Obviously, you can just"
  },
  {
    "startMs": 916639,
    "text": "go through and dump as much context as"
  },
  {
    "startMs": 918160,
    "text": "you want. And if you're doing a sort of"
  },
  {
    "startMs": 919680,
    "text": "needle in the haststack test, that might"
  },
  {
    "startMs": 921279,
    "text": "be fine. But in practice, there is"
  },
  {
    "startMs": 923519,
    "text": "erosion that you find in quality outside"
  },
  {
    "startMs": 925600,
    "text": "of just the token economics as a result"
  },
  {
    "startMs": 927920,
    "text": "of passing in too much. And instead,"
  },
  {
    "startMs": 930959,
    "text": "what we find is you can get a lot of"
  },
  {
    "startMs": 933120,
    "text": "headroom by thinking through things like"
  },
  {
    "startMs": 935680,
    "text": "how do you classify the right documents"
  },
  {
    "startMs": 937519,
    "text": "to the right sort of pipeline? And even"
  },
  {
    "startMs": 939839,
    "text": "for large documents, how do you make"
  },
  {
    "startMs": 941440,
    "text": "sure that you're passing in the snippets"
  },
  {
    "startMs": 943279,
    "text": "uh that are actually relevant? We see"
  },
  {
    "startMs": 945199,
    "text": "use cases where people will have things"
  },
  {
    "startMs": 946959,
    "text": "like paper mail. Uh and these paper mail"
  },
  {
    "startMs": 949600,
    "text": "packets can be hundreds of pages long."
  },
  {
    "startMs": 951680,
    "text": "You don't necessarily know what is going"
  },
  {
    "startMs": 953199,
    "text": "to be contained within it. Um you might"
  },
  {
    "startMs": 955120,
    "text": "have issues like a person interle the"
  },
  {
    "startMs": 957519,
    "text": "content and having the model do that"
  },
  {
    "startMs": 959839,
    "text": "sort of work is almost like a"
  },
  {
    "startMs": 961920,
    "text": "distraction from the work that you're"
  },
  {
    "startMs": 963279,
    "text": "actually trying to achieve which might"
  },
  {
    "startMs": 964480,
    "text": "be extracting the data from the paper"
  },
  {
    "startMs": 966079,
    "text": "mail reasoning on it or making a"
  },
  {
    "startMs": 967680,
    "text": "decision."
  },
  {
    "startMs": 970639,
    "text": "Again, uh I mentioned earlier that the"
  },
  {
    "startMs": 973360,
    "text": "second half I think is the more"
  },
  {
    "startMs": 974959,
    "text": "interesting piece. Um which is once you"
  },
  {
    "startMs": 977519,
    "text": "have the sort of initial classification"
  },
  {
    "startMs": 980079,
    "text": "and splitting layer, you've figured out"
  },
  {
    "startMs": 982000,
    "text": "sorting. Uh I think the thing that we"
  },
  {
    "startMs": 985360,
    "text": "are really excited about as a team is"
  },
  {
    "startMs": 987519,
    "text": "agent harnesses have been this really"
  },
  {
    "startMs": 989360,
    "text": "really interesting frontier to push past"
  },
  {
    "startMs": 991920,
    "text": "what canonically used to be hard"
  },
  {
    "startMs": 994160,
    "text": "unsolved problems. Um one good example"
  },
  {
    "startMs": 997120,
    "text": "of this that I'll talk about in a second"
  },
  {
    "startMs": 998720,
    "text": "is things like line charts. uh we work"
  },
  {
    "startMs": 1000800,
    "text": "with many of the largest hedge funds in"
  },
  {
    "startMs": 1002240,
    "text": "the world and things like line charts"
  },
  {
    "startMs": 1004079,
    "text": "historically have been really really"
  },
  {
    "startMs": 1005279,
    "text": "difficult because one they're an imaged"
  },
  {
    "startMs": 1006800,
    "text": "format but two there's a lot of pixel"
  },
  {
    "startMs": 1008959,
    "text": "level granularity that if you're doing"
  },
  {
    "startMs": 1010639,
    "text": "anything with a traditional vision"
  },
  {
    "startMs": 1012399,
    "text": "encoder you're probably going to lose"
  },
  {
    "startMs": 1014480,
    "text": "you're going to get a rough plot of how"
  },
  {
    "startMs": 1016000,
    "text": "revenue trended but you're not going to"
  },
  {
    "startMs": 1017360,
    "text": "get the individual data points and so"
  },
  {
    "startMs": 1019360,
    "text": "we've been thinking through how do we"
  },
  {
    "startMs": 1020639,
    "text": "give agents the ability to have the"
  },
  {
    "startMs": 1021920,
    "text": "right tools to solve for the specific"
  },
  {
    "startMs": 1024480,
    "text": "type of problem that you're looking at"
  },
  {
    "startMs": 1026480,
    "text": "in the case of chart extraction the"
  },
  {
    "startMs": 1028559,
    "text": "chart on the left encodes codes a"
  },
  {
    "startMs": 1030559,
    "text": "massive table of data. If you actually"
  },
  {
    "startMs": 1032400,
    "text": "went through and tried to plot every"
  },
  {
    "startMs": 1033839,
    "text": "single pixel, it would be really really"
  },
  {
    "startMs": 1035760,
    "text": "difficult. U but it would also be hard"
  },
  {
    "startMs": 1038079,
    "text": "for a model to even approximate the"
  },
  {
    "startMs": 1040559,
    "text": "intricacies of the lines in between. And"
  },
  {
    "startMs": 1042880,
    "text": "there's no model that out of the box can"
  },
  {
    "startMs": 1044640,
    "text": "do this as a singleshot problem. What"
  },
  {
    "startMs": 1046319,
    "text": "you're seeing on the right is a"
  },
  {
    "startMs": 1048079,
    "text": "reconstruction of the markdown table"
  },
  {
    "startMs": 1050000,
    "text": "that we're able to generate off of the"
  },
  {
    "startMs": 1051840,
    "text": "initial line chart. And the only way"
  },
  {
    "startMs": 1053679,
    "text": "that we were able to get there was to"
  },
  {
    "startMs": 1055520,
    "text": "have an agent with all sorts of tools."
  },
  {
    "startMs": 1057200,
    "text": "It has its own code interpreter. It has"
  },
  {
    "startMs": 1058799,
    "text": "the ability to visualize the chart that"
  },
  {
    "startMs": 1060400,
    "text": "it's generating. And it's iteratively"
  },
  {
    "startMs": 1062000,
    "text": "going through. It's finding mistakes in"
  },
  {
    "startMs": 1063600,
    "text": "the line chart again and again and again"
  },
  {
    "startMs": 1066000,
    "text": "until it's able to get to the final"
  },
  {
    "startMs": 1067520,
    "text": "output. That applies for problems like"
  },
  {
    "startMs": 1070320,
    "text": "structured extraction as well where for"
  },
  {
    "startMs": 1072400,
    "text": "a while we've had this documents to"
  },
  {
    "startMs": 1074480,
    "text": "structured output feature. Uh but you"
  },
  {
    "startMs": 1076799,
    "text": "can really take it a step further by"
  },
  {
    "startMs": 1078480,
    "text": "having an agent harness around that same"
  },
  {
    "startMs": 1080559,
    "text": "sort of task. You can have a parent"
  },
  {
    "startMs": 1081919,
    "text": "agent go through set validation criteria"
  },
  {
    "startMs": 1084480,
    "text": "for sub agents to follow. Uh and this"
  },
  {
    "startMs": 1087200,
    "text": "means that if you have something like a"
  },
  {
    "startMs": 1089280,
    "text": "CBP form with tens of thousands of"
  },
  {
    "startMs": 1091520,
    "text": "fields, that's the sort of problem where"
  },
  {
    "startMs": 1093840,
    "text": "you end up finding a lot of issues that"
  },
  {
    "startMs": 1095440,
    "text": "are silent in nature like you drop"
  },
  {
    "startMs": 1097039,
    "text": "content, you drop rows and micro one"
  },
  {
    "startMs": 1099280,
    "text": "actually released a really good"
  },
  {
    "startMs": 1100640,
    "text": "benchmark in the space this morning"
  },
  {
    "startMs": 1102799,
    "text": "where there's this like bifurcation in"
  },
  {
    "startMs": 1105039,
    "text": "the market. Uh Frontier models with Max"
  },
  {
    "startMs": 1107200,
    "text": "Reasoning are really really precise."
  },
  {
    "startMs": 1109200,
    "text": "Like provided that they extracted a row,"
  },
  {
    "startMs": 1110880,
    "text": "odds are it's not a hallucinated row"
  },
  {
    "startMs": 1112480,
    "text": "like they actually got it correct. but"
  },
  {
    "startMs": 1114799,
    "text": "they silently drop a lot of the contents"
  },
  {
    "startMs": 1116640,
    "text": "across the benchmark. Uh recall really"
  },
  {
    "startMs": 1118799,
    "text": "really struggles. On the flip side, a"
  },
  {
    "startMs": 1120960,
    "text": "lot of dedicated document processing"
  },
  {
    "startMs": 1122480,
    "text": "services are actually behind frontier"
  },
  {
    "startMs": 1124880,
    "text": "models from a precision perspective, but"
  },
  {
    "startMs": 1127200,
    "text": "close that gap on a recall perspective."
  },
  {
    "startMs": 1129600,
    "text": "And so there's always been this sort of"
  },
  {
    "startMs": 1131120,
    "text": "trade-off and it was only with an agent"
  },
  {
    "startMs": 1133360,
    "text": "harness that we were able to find that"
  },
  {
    "startMs": 1134799,
    "text": "sort of local maximum of both precision"
  },
  {
    "startMs": 1137440,
    "text": "and also recall for this sort of task."
  },
  {
    "startMs": 1141360,
    "text": "The last thing and maybe the most"
  },
  {
    "startMs": 1143360,
    "text": "important thing from this talk uh is"
  },
  {
    "startMs": 1146000,
    "text": "that I think at the end of the day eval"
  },
  {
    "startMs": 1149440,
    "text": "should underpin all of your decisions"
  },
  {
    "startMs": 1151200,
    "text": "and it's been a big part of how we think"
  },
  {
    "startMs": 1152799,
    "text": "about our product. Um that applies both"
  },
  {
    "startMs": 1154640,
    "text": "to off-the-shelf data sets that you eval"
  },
  {
    "startMs": 1157360,
    "text": "against but also to things like"
  },
  {
    "startMs": 1158720,
    "text": "real-time production monitoring because"
  },
  {
    "startMs": 1160559,
    "text": "your production data is going to differ"
  },
  {
    "startMs": 1162240,
    "text": "from whatever else you have in your"
  },
  {
    "startMs": 1164640,
    "text": "contrived set. And I really think it's"
  },
  {
    "startMs": 1167919,
    "text": "important to think of eval not as just"
  },
  {
    "startMs": 1169919,
    "text": "this like macrolevel view, but also the"
  },
  {
    "startMs": 1173120,
    "text": "best teams that we work with look at"
  },
  {
    "startMs": 1174400,
    "text": "eval on a granular level for each step"
  },
  {
    "startMs": 1176320,
    "text": "of their pipeline. Uh the first thing"
  },
  {
    "startMs": 1178480,
    "text": "might be that you want to make sure that"
  },
  {
    "startMs": 1179840,
    "text": "the inputs to your pipeline are great."
  },
  {
    "startMs": 1181600,
    "text": "And of course, you should eval things"
  },
  {
    "startMs": 1183280,
    "text": "like your parsing pipeline. But even"
  },
  {
    "startMs": 1185679,
    "text": "perfect parsing with a horrible"
  },
  {
    "startMs": 1187360,
    "text": "retrieval pipeline is not going to help"
  },
  {
    "startMs": 1189120,
    "text": "if you're not passing the right context."
  },
  {
    "startMs": 1190799,
    "text": "And so it's important that you're"
  },
  {
    "startMs": 1191919,
    "text": "thinking through details like your"
  },
  {
    "startMs": 1193760,
    "text": "retrieval pipeline, your formatting at"
  },
  {
    "startMs": 1196000,
    "text": "the end of the pipeline, and also"
  },
  {
    "startMs": 1197440,
    "text": "ultimately the most important thing is"
  },
  {
    "startMs": 1199440,
    "text": "are you able to improve end agent"
  },
  {
    "startMs": 1201520,
    "text": "performance."
  },
  {
    "startMs": 1203919,
    "text": "I'll close off just with a a sense of"
  },
  {
    "startMs": 1206080,
    "text": "where we are headed and where we've seen"
  },
  {
    "startMs": 1207760,
    "text": "the industry head. Uh the most important"
  },
  {
    "startMs": 1210480,
    "text": "thing I think is as agents get better"
  },
  {
    "startMs": 1212720,
    "text": "and better, you can deviate from the"
  },
  {
    "startMs": 1215120,
    "text": "sort of deterministic pipeline that you"
  },
  {
    "startMs": 1216720,
    "text": "would have had a few years ago. Um, a"
  },
  {
    "startMs": 1218480,
    "text": "lot of our customers will actually"
  },
  {
    "startMs": 1220080,
    "text": "create effectively a file system for"
  },
  {
    "startMs": 1221919,
    "text": "their agent to go through and navigate"
  },
  {
    "startMs": 1223679,
    "text": "and let the agent decide what sorts of"
  },
  {
    "startMs": 1225600,
    "text": "tools it wants to use. So, we create a"
  },
  {
    "startMs": 1227200,
    "text": "CLI where instead of people creating a"
  },
  {
    "startMs": 1229280,
    "text": "endto-end pipeline where documents"
  },
  {
    "startMs": 1231280,
    "text": "always follow one specific flow, um, the"
  },
  {
    "startMs": 1233919,
    "text": "agent will decide if it needs to read a"
  },
  {
    "startMs": 1235760,
    "text": "certain type of document and they'll"
  },
  {
    "startMs": 1237520,
    "text": "split that into two sets. Uh, one is a"
  },
  {
    "startMs": 1239760,
    "text": "content field which the agent can read"
  },
  {
    "startMs": 1241760,
    "text": "as it needs to. Uh, but the other is all"
  },
  {
    "startMs": 1244400,
    "text": "the metadata that you would have wanted."
  },
  {
    "startMs": 1245760,
    "text": "If you're doing things like citations,"
  },
  {
    "startMs": 1247039,
    "text": "you may want bounding boxes, so on and"
  },
  {
    "startMs": 1249360,
    "text": "so forth."
  },
  {
    "startMs": 1251360,
    "text": "Um, I'll skip this part on editing. I"
  },
  {
    "startMs": 1253600,
    "text": "think there's a lot of interesting work"
  },
  {
    "startMs": 1254799,
    "text": "being done here. We've already released"
  },
  {
    "startMs": 1256240,
    "text": "some of it, but in the next few months,"
  },
  {
    "startMs": 1258159,
    "text": "you will see us look more and more"
  },
  {
    "startMs": 1260159,
    "text": "towards things like document generation."
  },
  {
    "startMs": 1262559,
    "text": "But the recap for today, and I really"
  },
  {
    "startMs": 1264720,
    "text": "appreciate your time, is one, I highly"
  },
  {
    "startMs": 1267440,
    "text": "recommend that you decompose the parsing"
  },
  {
    "startMs": 1269280,
    "text": "problem. To the extent possible, you"
  },
  {
    "startMs": 1271039,
    "text": "should think of it as the right tool for"
  },
  {
    "startMs": 1272240,
    "text": "the right task so that you can hit that"
  },
  {
    "startMs": 1273840,
    "text": "perimeter frontier of accuracy, cost,"
  },
  {
    "startMs": 1275760,
    "text": "and latency. Two, I think agentic"
  },
  {
    "startMs": 1278480,
    "text": "verification is the biggest step change"
  },
  {
    "startMs": 1280240,
    "text": "that the industry has had for a while"
  },
  {
    "startMs": 1281840,
    "text": "and it's a really good opportunity for"
  },
  {
    "startMs": 1283520,
    "text": "you to make sure that you're building"
  },
  {
    "startMs": 1285280,
    "text": "pipelines that work in production."
  },
  {
    "startMs": 1287600,
    "text": "Three, I think it takes very little"
  },
  {
    "startMs": 1289760,
    "text": "effort, but there's a lot of headroom"
  },
  {
    "startMs": 1291600,
    "text": "from details like formatting the data"
  },
  {
    "startMs": 1293520,
    "text": "for its consumer. or similar vein, I"
  },
  {
    "startMs": 1296960,
    "text": "think it's really really important to"
  },
  {
    "startMs": 1298080,
    "text": "think about not just the data processing"
  },
  {
    "startMs": 1300240,
    "text": "but the the data orchestration and so"
  },
  {
    "startMs": 1302320,
    "text": "you should always think about tools like"
  },
  {
    "startMs": 1303919,
    "text": "classify and split as a way to augment"
  },
  {
    "startMs": 1306400,
    "text": "your pipeline. Five uh make sure that"
  },
  {
    "startMs": 1309280,
    "text": "you eval at every stage and six think"
  },
  {
    "startMs": 1311679,
    "text": "through what that next frontier looks"
  },
  {
    "startMs": 1313200,
    "text": "like for you because I think most"
  },
  {
    "startMs": 1314960,
    "text": "successful companies in today's era have"
  },
  {
    "startMs": 1317120,
    "text": "deviated a lot from what we used to do"
  },
  {
    "startMs": 1318880,
    "text": "two three years ago. But if you have any"
  },
  {
    "startMs": 1321679,
    "text": "questions uh please feel free to reach"
  },
  {
    "startMs": 1323360,
    "text": "out at any point. My email is just first"
  },
  {
    "startMs": 1325520,
    "text": "namered reductto.ai. Uh and you can also"
  },
  {
    "startMs": 1328240,
    "text": "reach out on our website if we can be"
  },
  {
    "startMs": 1329760,
    "text": "helpful for your use case. Thank you."
  }
]