[
  {
    "startMs": 1964,
    "text": "[music]"
  },
  {
    "startMs": 14000,
    "text": "Uh afternoon everybody. Uh I'm going to"
  },
  {
    "startMs": 16710,
    "text": "Uh afternoon everybody. Uh I'm going to"
  },
  {
    "startMs": 16720,
    "text": "Uh afternoon everybody. Uh I'm going to talk today about engineering agency out"
  },
  {
    "startMs": 19429,
    "text": "talk today about engineering agency out"
  },
  {
    "startMs": 19439,
    "text": "talk today about engineering agency out of the happy path. Um this is going to"
  },
  {
    "startMs": 21830,
    "text": "of the happy path. Um this is going to"
  },
  {
    "startMs": 21840,
    "text": "of the happy path. Um this is going to be focused on AI leadership, engineering"
  },
  {
    "startMs": 25189,
    "text": "be focused on AI leadership, engineering"
  },
  {
    "startMs": 25199,
    "text": "be focused on AI leadership, engineering leadership at growth stage companies."
  },
  {
    "startMs": 28630,
    "text": "leadership at growth stage companies."
  },
  {
    "startMs": 28640,
    "text": "leadership at growth stage companies. Um, and I'd like to start by asking"
  },
  {
    "startMs": 30550,
    "text": "Um, and I'd like to start by asking"
  },
  {
    "startMs": 30560,
    "text": "Um, and I'd like to start by asking everybody to think about the strongest"
  },
  {
    "startMs": 32790,
    "text": "everybody to think about the strongest"
  },
  {
    "startMs": 32800,
    "text": "everybody to think about the strongest domain experts on your team. So these"
  },
  {
    "startMs": 35110,
    "text": "domain experts on your team. So these"
  },
  {
    "startMs": 35120,
    "text": "domain experts on your team. So these are people who uh, you know, if sales"
  },
  {
    "startMs": 37350,
    "text": "are people who uh, you know, if sales"
  },
  {
    "startMs": 37360,
    "text": "are people who uh, you know, if sales wants to build a new feature or deliver"
  },
  {
    "startMs": 39190,
    "text": "wants to build a new feature or deliver"
  },
  {
    "startMs": 39200,
    "text": "wants to build a new feature or deliver something new in order to close an"
  },
  {
    "startMs": 40549,
    "text": "something new in order to close an"
  },
  {
    "startMs": 40559,
    "text": "something new in order to close an account or an urgent customer ticket"
  },
  {
    "startMs": 42790,
    "text": "account or an urgent customer ticket"
  },
  {
    "startMs": 42800,
    "text": "account or an urgent customer ticket appears, uh, these are the people who"
  },
  {
    "startMs": 44470,
    "text": "appears, uh, these are the people who"
  },
  {
    "startMs": 44480,
    "text": "appears, uh, these are the people who get at paint. Uh, they're the people"
  },
  {
    "startMs": 46790,
    "text": "get at paint. Uh, they're the people"
  },
  {
    "startMs": 46800,
    "text": "get at paint. Uh, they're the people maybe who are responsible for a small"
  },
  {
    "startMs": 48790,
    "text": "maybe who are responsible for a small"
  },
  {
    "startMs": 48800,
    "text": "maybe who are responsible for a small team or for a critical business process."
  },
  {
    "startMs": 51430,
    "text": "team or for a critical business process."
  },
  {
    "startMs": 51440,
    "text": "team or for a critical business process. And I want to think about how these"
  },
  {
    "startMs": 52709,
    "text": "And I want to think about how these"
  },
  {
    "startMs": 52719,
    "text": "And I want to think about how these people actually spend most of their work"
  },
  {
    "startMs": 54470,
    "text": "people actually spend most of their work"
  },
  {
    "startMs": 54480,
    "text": "people actually spend most of their work weeks."
  },
  {
    "startMs": 56229,
    "text": "weeks."
  },
  {
    "startMs": 56239,
    "text": "weeks. Uh my experience is that most experts"
  },
  {
    "startMs": 58389,
    "text": "Uh my experience is that most experts"
  },
  {
    "startMs": 58399,
    "text": "Uh my experience is that most experts I've worked with have their attention"
  },
  {
    "startMs": 60150,
    "text": "I've worked with have their attention"
  },
  {
    "startMs": 60160,
    "text": "I've worked with have their attention sort of smeared over multitasking,"
  },
  {
    "startMs": 62790,
    "text": "sort of smeared over multitasking,"
  },
  {
    "startMs": 62800,
    "text": "sort of smeared over multitasking, loading up on complicated situations,"
  },
  {
    "startMs": 65270,
    "text": "loading up on complicated situations,"
  },
  {
    "startMs": 65280,
    "text": "loading up on complicated situations, making decisions that could have been"
  },
  {
    "startMs": 66870,
    "text": "making decisions that could have been"
  },
  {
    "startMs": 66880,
    "text": "making decisions that could have been routine or obvious in retrospect and"
  },
  {
    "startMs": 69750,
    "text": "routine or obvious in retrospect and"
  },
  {
    "startMs": 69760,
    "text": "routine or obvious in retrospect and only a small fraction of their attention"
  },
  {
    "startMs": 71830,
    "text": "only a small fraction of their attention"
  },
  {
    "startMs": 71840,
    "text": "only a small fraction of their attention is left over to sort of move the"
  },
  {
    "startMs": 73270,
    "text": "is left over to sort of move the"
  },
  {
    "startMs": 73280,
    "text": "is left over to sort of move the business forward."
  },
  {
    "startMs": 75990,
    "text": "business forward."
  },
  {
    "startMs": 76000,
    "text": "business forward. Um now like the cost of like raw white"
  },
  {
    "startMs": 78630,
    "text": "Um now like the cost of like raw white"
  },
  {
    "startMs": 78640,
    "text": "Um now like the cost of like raw white collar output is dropping. It's"
  },
  {
    "startMs": 81350,
    "text": "collar output is dropping. It's"
  },
  {
    "startMs": 81360,
    "text": "collar output is dropping. It's cratering. I've talked to a bunch of"
  },
  {
    "startMs": 82710,
    "text": "cratering. I've talked to a bunch of"
  },
  {
    "startMs": 82720,
    "text": "cratering. I've talked to a bunch of people over the conference and"
  },
  {
    "startMs": 84149,
    "text": "people over the conference and"
  },
  {
    "startMs": 84159,
    "text": "people over the conference and regardless of what you think about"
  },
  {
    "startMs": 85350,
    "text": "regardless of what you think about"
  },
  {
    "startMs": 85360,
    "text": "regardless of what you think about frontier models and where the future's"
  },
  {
    "startMs": 86950,
    "text": "frontier models and where the future's"
  },
  {
    "startMs": 86960,
    "text": "frontier models and where the future's going to go in terms of new capacity."
  },
  {
    "startMs": 88710,
    "text": "going to go in terms of new capacity."
  },
  {
    "startMs": 88720,
    "text": "going to go in terms of new capacity. Everybody I've talked to has agreed that"
  },
  {
    "startMs": 90390,
    "text": "Everybody I've talked to has agreed that"
  },
  {
    "startMs": 90400,
    "text": "Everybody I've talked to has agreed that existing capacity is going to get very"
  },
  {
    "startMs": 92310,
    "text": "existing capacity is going to get very"
  },
  {
    "startMs": 92320,
    "text": "existing capacity is going to get very very cheap."
  },
  {
    "startMs": 94149,
    "text": "very cheap."
  },
  {
    "startMs": 94159,
    "text": "very cheap. Cheap execution increases the load and"
  },
  {
    "startMs": 96789,
    "text": "Cheap execution increases the load and"
  },
  {
    "startMs": 96799,
    "text": "Cheap execution increases the load and the on on judgment on deciding what to"
  },
  {
    "startMs": 99109,
    "text": "the on on judgment on deciding what to"
  },
  {
    "startMs": 99119,
    "text": "the on on judgment on deciding what to do. Uh organizational judgment was"
  },
  {
    "startMs": 102310,
    "text": "do. Uh organizational judgment was"
  },
  {
    "startMs": 102320,
    "text": "do. Uh organizational judgment was already in short supply. Uh and this is"
  },
  {
    "startMs": 104310,
    "text": "already in short supply. Uh and this is"
  },
  {
    "startMs": 104320,
    "text": "already in short supply. Uh and this is going to make it even worse. I think"
  },
  {
    "startMs": 105670,
    "text": "going to make it even worse. I think"
  },
  {
    "startMs": 105680,
    "text": "going to make it even worse. I think this is currently underpriced."
  },
  {
    "startMs": 108389,
    "text": "this is currently underpriced."
  },
  {
    "startMs": 108399,
    "text": "this is currently underpriced. So with that in mind, I'm going to talk"
  },
  {
    "startMs": 109510,
    "text": "So with that in mind, I'm going to talk"
  },
  {
    "startMs": 109520,
    "text": "So with that in mind, I'm going to talk today about using AI for full"
  },
  {
    "startMs": 110870,
    "text": "today about using AI for full"
  },
  {
    "startMs": 110880,
    "text": "today about using AI for full automations. By full automations, I mean"
  },
  {
    "startMs": 113510,
    "text": "automations. By full automations, I mean"
  },
  {
    "startMs": 113520,
    "text": "automations. By full automations, I mean taking humans out of the happy path"
  },
  {
    "startMs": 115429,
    "text": "taking humans out of the happy path"
  },
  {
    "startMs": 115439,
    "text": "taking humans out of the happy path completely. Using humans maybe for"
  },
  {
    "startMs": 117109,
    "text": "completely. Using humans maybe for"
  },
  {
    "startMs": 117119,
    "text": "completely. Using humans maybe for exceptions, but not not human in the"
  },
  {
    "startMs": 118870,
    "text": "exceptions, but not not human in the"
  },
  {
    "startMs": 118880,
    "text": "exceptions, but not not human in the loop um style automations. I think that"
  },
  {
    "startMs": 121830,
    "text": "loop um style automations. I think that"
  },
  {
    "startMs": 121840,
    "text": "loop um style automations. I think that these types of processes can improve"
  },
  {
    "startMs": 123910,
    "text": "these types of processes can improve"
  },
  {
    "startMs": 123920,
    "text": "these types of processes can improve margins kind of obvious. They can reduce"
  },
  {
    "startMs": 126310,
    "text": "margins kind of obvious. They can reduce"
  },
  {
    "startMs": 126320,
    "text": "margins kind of obvious. They can reduce complexity of a business by"
  },
  {
    "startMs": 127590,
    "text": "complexity of a business by"
  },
  {
    "startMs": 127600,
    "text": "complexity of a business by encapsulating whole chunks of work. Uh"
  },
  {
    "startMs": 129749,
    "text": "encapsulating whole chunks of work. Uh"
  },
  {
    "startMs": 129759,
    "text": "encapsulating whole chunks of work. Uh and then you can recover judgment from"
  },
  {
    "startMs": 131190,
    "text": "and then you can recover judgment from"
  },
  {
    "startMs": 131200,
    "text": "and then you can recover judgment from your experts and hopefully route it"
  },
  {
    "startMs": 132949,
    "text": "your experts and hopefully route it"
  },
  {
    "startMs": 132959,
    "text": "your experts and hopefully route it towards more important um tasks or"
  },
  {
    "startMs": 135110,
    "text": "towards more important um tasks or"
  },
  {
    "startMs": 135120,
    "text": "towards more important um tasks or questions. I'm going to talk about that"
  },
  {
    "startMs": 136470,
    "text": "questions. I'm going to talk about that"
  },
  {
    "startMs": 136480,
    "text": "questions. I'm going to talk about that specifically a bit later."
  },
  {
    "startMs": 138790,
    "text": "specifically a bit later."
  },
  {
    "startMs": 138800,
    "text": "specifically a bit later. Um, now the key to full automations and"
  },
  {
    "startMs": 141510,
    "text": "Um, now the key to full automations and"
  },
  {
    "startMs": 141520,
    "text": "Um, now the key to full automations and my experience has been engineering"
  },
  {
    "startMs": 142869,
    "text": "my experience has been engineering"
  },
  {
    "startMs": 142879,
    "text": "my experience has been engineering agency out of the happy path out of"
  },
  {
    "startMs": 145270,
    "text": "agency out of the happy path out of"
  },
  {
    "startMs": 145280,
    "text": "agency out of the happy path out of these automations. This is a bit of a"
  },
  {
    "startMs": 147110,
    "text": "these automations. This is a bit of a"
  },
  {
    "startMs": 147120,
    "text": "these automations. This is a bit of a reversal. You know, agents have"
  },
  {
    "startMs": 148790,
    "text": "reversal. You know, agents have"
  },
  {
    "startMs": 148800,
    "text": "reversal. You know, agents have permanently changed and expanded my idea"
  },
  {
    "startMs": 150869,
    "text": "permanently changed and expanded my idea"
  },
  {
    "startMs": 150879,
    "text": "permanently changed and expanded my idea of what I think is possible with"
  },
  {
    "startMs": 152229,
    "text": "of what I think is possible with"
  },
  {
    "startMs": 152239,
    "text": "of what I think is possible with software. But in enterprise, agents have"
  },
  {
    "startMs": 154550,
    "text": "software. But in enterprise, agents have"
  },
  {
    "startMs": 154560,
    "text": "software. But in enterprise, agents have problems. They're a pain to maintain."
  },
  {
    "startMs": 157110,
    "text": "problems. They're a pain to maintain."
  },
  {
    "startMs": 157120,
    "text": "problems. They're a pain to maintain. Uh, especially if skills and harnesses"
  },
  {
    "startMs": 158790,
    "text": "Uh, especially if skills and harnesses"
  },
  {
    "startMs": 158800,
    "text": "Uh, especially if skills and harnesses are changing. Uh, they fail in unusually"
  },
  {
    "startMs": 162070,
    "text": "are changing. Uh, they fail in unusually"
  },
  {
    "startMs": 162080,
    "text": "are changing. Uh, they fail in unusually shaped ways that enterprises find hard"
  },
  {
    "startMs": 163750,
    "text": "shaped ways that enterprises find hard"
  },
  {
    "startMs": 163760,
    "text": "shaped ways that enterprises find hard to work with. And then they increase the"
  },
  {
    "startMs": 166070,
    "text": "to work with. And then they increase the"
  },
  {
    "startMs": 166080,
    "text": "to work with. And then they increase the burden of decisions. When I see an"
  },
  {
    "startMs": 168070,
    "text": "burden of decisions. When I see an"
  },
  {
    "startMs": 168080,
    "text": "burden of decisions. When I see an expert or a leader responding to agent"
  },
  {
    "startMs": 170470,
    "text": "expert or a leader responding to agent"
  },
  {
    "startMs": 170480,
    "text": "expert or a leader responding to agent notifications, you know, maybe on their"
  },
  {
    "startMs": 171830,
    "text": "notifications, you know, maybe on their"
  },
  {
    "startMs": 171840,
    "text": "notifications, you know, maybe on their phone or or on their laptop, maybe they"
  },
  {
    "startMs": 173430,
    "text": "phone or or on their laptop, maybe they"
  },
  {
    "startMs": 173440,
    "text": "phone or or on their laptop, maybe they have an app for it. I'm often"
  },
  {
    "startMs": 174949,
    "text": "have an app for it. I'm often"
  },
  {
    "startMs": 174959,
    "text": "have an app for it. I'm often suspicious. Is that actually the best"
  },
  {
    "startMs": 176550,
    "text": "suspicious. Is that actually the best"
  },
  {
    "startMs": 176560,
    "text": "suspicious. Is that actually the best use of their, you know, valuable time"
  },
  {
    "startMs": 178390,
    "text": "use of their, you know, valuable time"
  },
  {
    "startMs": 178400,
    "text": "use of their, you know, valuable time and attention?"
  },
  {
    "startMs": 180309,
    "text": "and attention?"
  },
  {
    "startMs": 180319,
    "text": "and attention? Full automations, on the other hand,"
  },
  {
    "startMs": 181589,
    "text": "Full automations, on the other hand,"
  },
  {
    "startMs": 181599,
    "text": "Full automations, on the other hand, they free up attention. They're legible."
  },
  {
    "startMs": 184309,
    "text": "they free up attention. They're legible."
  },
  {
    "startMs": 184319,
    "text": "they free up attention. They're legible. So this means that, you know, they take"
  },
  {
    "startMs": 185830,
    "text": "So this means that, you know, they take"
  },
  {
    "startMs": 185840,
    "text": "So this means that, you know, they take an input, they produce an output, they"
  },
  {
    "startMs": 187430,
    "text": "an input, they produce an output, they"
  },
  {
    "startMs": 187440,
    "text": "an input, they produce an output, they encapsulate the whole thing, and they"
  },
  {
    "startMs": 189030,
    "text": "encapsulate the whole thing, and they"
  },
  {
    "startMs": 189040,
    "text": "encapsulate the whole thing, and they hide the complexity. And they're"
  },
  {
    "startMs": 191430,
    "text": "hide the complexity. And they're"
  },
  {
    "startMs": 191440,
    "text": "hide the complexity. And they're organizationally familiar. Like large"
  },
  {
    "startMs": 193270,
    "text": "organizationally familiar. Like large"
  },
  {
    "startMs": 193280,
    "text": "organizationally familiar. Like large human organizations have been building"
  },
  {
    "startMs": 194790,
    "text": "human organizations have been building"
  },
  {
    "startMs": 194800,
    "text": "human organizations have been building automations in software for decades."
  },
  {
    "startMs": 197030,
    "text": "automations in software for decades."
  },
  {
    "startMs": 197040,
    "text": "automations in software for decades. Think marketing, think DevOps. And"
  },
  {
    "startMs": 199190,
    "text": "Think marketing, think DevOps. And"
  },
  {
    "startMs": 199200,
    "text": "Think marketing, think DevOps. And they've been building uh automations in"
  },
  {
    "startMs": 201830,
    "text": "they've been building uh automations in"
  },
  {
    "startMs": 201840,
    "text": "they've been building uh automations in industry for centuries. Full automations"
  },
  {
    "startMs": 203830,
    "text": "industry for centuries. Full automations"
  },
  {
    "startMs": 203840,
    "text": "industry for centuries. Full automations are one of the key tools by which human"
  },
  {
    "startMs": 206149,
    "text": "are one of the key tools by which human"
  },
  {
    "startMs": 206159,
    "text": "are one of the key tools by which human organizations can do bigger and bigger"
  },
  {
    "startMs": 207830,
    "text": "organizations can do bigger and bigger"
  },
  {
    "startMs": 207840,
    "text": "organizations can do bigger and bigger things."
  },
  {
    "startMs": 214000,
    "text": "And that is giving the whole system an"
  },
  {
    "startMs": 215750,
    "text": "And that is giving the whole system an"
  },
  {
    "startMs": 215760,
    "text": "And that is giving the whole system an escape hatch. So, one of the problems"
  },
  {
    "startMs": 217670,
    "text": "escape hatch. So, one of the problems"
  },
  {
    "startMs": 217680,
    "text": "escape hatch. So, one of the problems with full automations is, and I'm sure"
  },
  {
    "startMs": 219509,
    "text": "with full automations is, and I'm sure"
  },
  {
    "startMs": 219519,
    "text": "with full automations is, and I'm sure people have seen this, when you get uh"
  },
  {
    "startMs": 221190,
    "text": "people have seen this, when you get uh"
  },
  {
    "startMs": 221200,
    "text": "people have seen this, when you get uh you sign up for something and maybe you"
  },
  {
    "startMs": 223110,
    "text": "you sign up for something and maybe you"
  },
  {
    "startMs": 223120,
    "text": "you sign up for something and maybe you accidentally get a whole marketing drip"
  },
  {
    "startMs": 224949,
    "text": "accidentally get a whole marketing drip"
  },
  {
    "startMs": 224959,
    "text": "accidentally get a whole marketing drip email when you were signing up as a"
  },
  {
    "startMs": 226229,
    "text": "email when you were signing up as a"
  },
  {
    "startMs": 226239,
    "text": "email when you were signing up as a vendor. Um, one of the interesting"
  },
  {
    "startMs": 228390,
    "text": "vendor. Um, one of the interesting"
  },
  {
    "startMs": 228400,
    "text": "vendor. Um, one of the interesting things you can do with AI and agency"
  },
  {
    "startMs": 230149,
    "text": "things you can do with AI and agency"
  },
  {
    "startMs": 230159,
    "text": "things you can do with AI and agency with automations is you can pre-filter"
  },
  {
    "startMs": 231750,
    "text": "with automations is you can pre-filter"
  },
  {
    "startMs": 231760,
    "text": "with automations is you can pre-filter inputs. You can check does this task"
  },
  {
    "startMs": 234949,
    "text": "inputs. You can check does this task"
  },
  {
    "startMs": 234959,
    "text": "inputs. You can check does this task actually match the happy path of the"
  },
  {
    "startMs": 236630,
    "text": "actually match the happy path of the"
  },
  {
    "startMs": 236640,
    "text": "actually match the happy path of the automation. You can reject adversarial"
  },
  {
    "startMs": 238949,
    "text": "automation. You can reject adversarial"
  },
  {
    "startMs": 238959,
    "text": "automation. You can reject adversarial situations. If something looks like it's"
  },
  {
    "startMs": 240869,
    "text": "situations. If something looks like it's"
  },
  {
    "startMs": 240879,
    "text": "situations. If something looks like it's business critical, sort of not a normal"
  },
  {
    "startMs": 242550,
    "text": "business critical, sort of not a normal"
  },
  {
    "startMs": 242560,
    "text": "business critical, sort of not a normal course of business operation, you can"
  },
  {
    "startMs": 244309,
    "text": "course of business operation, you can"
  },
  {
    "startMs": 244319,
    "text": "course of business operation, you can escalate that to a human. I think this"
  },
  {
    "startMs": 246470,
    "text": "escalate that to a human. I think this"
  },
  {
    "startMs": 246480,
    "text": "escalate that to a human. I think this is kind of neat. This is the use of AI,"
  },
  {
    "startMs": 248789,
    "text": "is kind of neat. This is the use of AI,"
  },
  {
    "startMs": 248799,
    "text": "is kind of neat. This is the use of AI, AI agency to reduce the variance of an"
  },
  {
    "startMs": 251670,
    "text": "AI agency to reduce the variance of an"
  },
  {
    "startMs": 251680,
    "text": "AI agency to reduce the variance of an automation to make things more smooth,"
  },
  {
    "startMs": 253910,
    "text": "automation to make things more smooth,"
  },
  {
    "startMs": 253920,
    "text": "automation to make things more smooth, which thereby lets us expand the scope"
  },
  {
    "startMs": 255589,
    "text": "which thereby lets us expand the scope"
  },
  {
    "startMs": 255599,
    "text": "which thereby lets us expand the scope of what we can automate. So I'm going to"
  },
  {
    "startMs": 257990,
    "text": "of what we can automate. So I'm going to"
  },
  {
    "startMs": 258000,
    "text": "of what we can automate. So I'm going to give a case study today. My background"
  },
  {
    "startMs": 259830,
    "text": "give a case study today. My background"
  },
  {
    "startMs": 259840,
    "text": "give a case study today. My background is working in bioarma data at scale. I"
  },
  {
    "startMs": 261990,
    "text": "is working in bioarma data at scale. I"
  },
  {
    "startMs": 262000,
    "text": "is working in bioarma data at scale. I had a company called deal enablement for"
  },
  {
    "startMs": 264710,
    "text": "had a company called deal enablement for"
  },
  {
    "startMs": 264720,
    "text": "had a company called deal enablement for bioharmaceutical mergers, licensing and"
  },
  {
    "startMs": 266790,
    "text": "bioharmaceutical mergers, licensing and"
  },
  {
    "startMs": 266800,
    "text": "bioharmaceutical mergers, licensing and acquisitions. In 23 we sold to clarvate."
  },
  {
    "startMs": 269590,
    "text": "acquisitions. In 23 we sold to clarvate."
  },
  {
    "startMs": 269600,
    "text": "acquisitions. In 23 we sold to clarvate. Clarvate is a really interesting"
  },
  {
    "startMs": 270629,
    "text": "Clarvate is a really interesting"
  },
  {
    "startMs": 270639,
    "text": "Clarvate is a really interesting company. They have an annotated history"
  },
  {
    "startMs": 272310,
    "text": "company. They have an annotated history"
  },
  {
    "startMs": 272320,
    "text": "company. They have an annotated history of the entire you know drug development"
  },
  {
    "startMs": 274310,
    "text": "of the entire you know drug development"
  },
  {
    "startMs": 274320,
    "text": "of the entire you know drug development industry. hundreds of millions of"
  },
  {
    "startMs": 275909,
    "text": "industry. hundreds of millions of"
  },
  {
    "startMs": 275919,
    "text": "industry. hundreds of millions of document or tens of millions of"
  },
  {
    "startMs": 277110,
    "text": "document or tens of millions of"
  },
  {
    "startMs": 277120,
    "text": "document or tens of millions of documents, hundreds of thousands of"
  },
  {
    "startMs": 278550,
    "text": "documents, hundreds of thousands of"
  },
  {
    "startMs": 278560,
    "text": "documents, hundreds of thousands of edits every year and a rich ontology for"
  },
  {
    "startMs": 280629,
    "text": "edits every year and a rich ontology for"
  },
  {
    "startMs": 280639,
    "text": "edits every year and a rich ontology for working of those sort of tens of"
  },
  {
    "startMs": 281990,
    "text": "working of those sort of tens of"
  },
  {
    "startMs": 282000,
    "text": "working of those sort of tens of thousands of ways of indexing it."
  },
  {
    "startMs": 284918,
    "text": "thousands of ways of indexing it."
  },
  {
    "startMs": 284928,
    "text": "thousands of ways of indexing it. [snorts] Um the automation we built was"
  },
  {
    "startMs": 286870,
    "text": "[snorts] Um the automation we built was"
  },
  {
    "startMs": 286880,
    "text": "[snorts] Um the automation we built was automating the indexing and the"
  },
  {
    "startMs": 288710,
    "text": "automating the indexing and the"
  },
  {
    "startMs": 288720,
    "text": "automating the indexing and the relationships between those documents."
  },
  {
    "startMs": 290629,
    "text": "relationships between those documents."
  },
  {
    "startMs": 290639,
    "text": "relationships between those documents. So this is processing tens of thousands"
  },
  {
    "startMs": 292390,
    "text": "So this is processing tens of thousands"
  },
  {
    "startMs": 292400,
    "text": "So this is processing tens of thousands of new tasks a month when a document's"
  },
  {
    "startMs": 294150,
    "text": "of new tasks a month when a document's"
  },
  {
    "startMs": 294160,
    "text": "of new tasks a month when a document's edited or inserted or something changes."
  },
  {
    "startMs": 296629,
    "text": "edited or inserted or something changes."
  },
  {
    "startMs": 296639,
    "text": "edited or inserted or something changes. We were able to achieve three and a half"
  },
  {
    "startMs": 298150,
    "text": "We were able to achieve three and a half"
  },
  {
    "startMs": 298160,
    "text": "We were able to achieve three and a half n of accuracy on the tasks that the"
  },
  {
    "startMs": 300550,
    "text": "n of accuracy on the tasks that the"
  },
  {
    "startMs": 300560,
    "text": "n of accuracy on the tasks that the business most valued. And I'll talk"
  },
  {
    "startMs": 301830,
    "text": "business most valued. And I'll talk"
  },
  {
    "startMs": 301840,
    "text": "business most valued. And I'll talk about what that means in a few minutes."
  },
  {
    "startMs": 303830,
    "text": "about what that means in a few minutes."
  },
  {
    "startMs": 303840,
    "text": "about what that means in a few minutes. Um, this was expensive to build. This"
  },
  {
    "startMs": 305749,
    "text": "Um, this was expensive to build. This"
  },
  {
    "startMs": 305759,
    "text": "Um, this was expensive to build. This was a capex cost. AI engineers aren't"
  },
  {
    "startMs": 307749,
    "text": "was a capex cost. AI engineers aren't"
  },
  {
    "startMs": 307759,
    "text": "was a capex cost. AI engineers aren't cheap. But once it was built, it was"
  },
  {
    "startMs": 309670,
    "text": "cheap. But once it was built, it was"
  },
  {
    "startMs": 309680,
    "text": "cheap. But once it was built, it was cheap to operate. It mostly stayed"
  },
  {
    "startMs": 311749,
    "text": "cheap to operate. It mostly stayed"
  },
  {
    "startMs": 311759,
    "text": "cheap to operate. It mostly stayed within its encapsulation. Um, and it was"
  },
  {
    "startMs": 315110,
    "text": "within its encapsulation. Um, and it was"
  },
  {
    "startMs": 315120,
    "text": "within its encapsulation. Um, and it was stable."
  },
  {
    "startMs": 317909,
    "text": "stable."
  },
  {
    "startMs": 317919,
    "text": "stable. So building this out, our initial"
  },
  {
    "startMs": 319830,
    "text": "So building this out, our initial"
  },
  {
    "startMs": 319840,
    "text": "So building this out, our initial approach, I think in 2024, we were"
  },
  {
    "startMs": 322230,
    "text": "approach, I think in 2024, we were"
  },
  {
    "startMs": 322240,
    "text": "approach, I think in 2024, we were excited by 40, we were excited by 03. We"
  },
  {
    "startMs": 325430,
    "text": "excited by 40, we were excited by 03. We"
  },
  {
    "startMs": 325440,
    "text": "excited by 40, we were excited by 03. We did something, maybe we were early for"
  },
  {
    "startMs": 326710,
    "text": "did something, maybe we were early for"
  },
  {
    "startMs": 326720,
    "text": "did something, maybe we were early for agents, but we built something that's"
  },
  {
    "startMs": 328310,
    "text": "agents, but we built something that's"
  },
  {
    "startMs": 328320,
    "text": "agents, but we built something that's sort of similar to what people would"
  },
  {
    "startMs": 329590,
    "text": "sort of similar to what people would"
  },
  {
    "startMs": 329600,
    "text": "sort of similar to what people would today think of as a research and"
  },
  {
    "startMs": 330950,
    "text": "today think of as a research and"
  },
  {
    "startMs": 330960,
    "text": "today think of as a research and assignment agent. After a few sprints,"
  },
  {
    "startMs": 333510,
    "text": "assignment agent. After a few sprints,"
  },
  {
    "startMs": 333520,
    "text": "assignment agent. After a few sprints, we hit, you know, 80 83% performance. We"
  },
  {
    "startMs": 335749,
    "text": "we hit, you know, 80 83% performance. We"
  },
  {
    "startMs": 335759,
    "text": "we hit, you know, 80 83% performance. We were really excited. We had been working"
  },
  {
    "startMs": 336950,
    "text": "were really excited. We had been working"
  },
  {
    "startMs": 336960,
    "text": "were really excited. We had been working in NLP for a decade. We were used to"
  },
  {
    "startMs": 338950,
    "text": "in NLP for a decade. We were used to"
  },
  {
    "startMs": 338960,
    "text": "in NLP for a decade. We were used to spending months fine-tuning a BERT model"
  },
  {
    "startMs": 340790,
    "text": "spending months fine-tuning a BERT model"
  },
  {
    "startMs": 340800,
    "text": "spending months fine-tuning a BERT model to just do one sort of component of one"
  },
  {
    "startMs": 342629,
    "text": "to just do one sort of component of one"
  },
  {
    "startMs": 342639,
    "text": "to just do one sort of component of one of these pipelines. We compiled a"
  },
  {
    "startMs": 345110,
    "text": "of these pipelines. We compiled a"
  },
  {
    "startMs": 345120,
    "text": "of these pipelines. We compiled a thousand devalves. We went we sat down"
  },
  {
    "startMs": 346950,
    "text": "thousand devalves. We went we sat down"
  },
  {
    "startMs": 346960,
    "text": "thousand devalves. We went we sat down with the domain experts."
  },
  {
    "startMs": 349189,
    "text": "with the domain experts."
  },
  {
    "startMs": 349199,
    "text": "with the domain experts. Um, I will say one good thing that we"
  },
  {
    "startMs": 351270,
    "text": "Um, I will say one good thing that we"
  },
  {
    "startMs": 351280,
    "text": "Um, I will say one good thing that we did is that we brought the domain"
  },
  {
    "startMs": 352710,
    "text": "did is that we brought the domain"
  },
  {
    "startMs": 352720,
    "text": "did is that we brought the domain experts a whole bunch of examples. And"
  },
  {
    "startMs": 355830,
    "text": "experts a whole bunch of examples. And"
  },
  {
    "startMs": 355840,
    "text": "experts a whole bunch of examples. And um I've worked on other projects where"
  },
  {
    "startMs": 358390,
    "text": "um I've worked on other projects where"
  },
  {
    "startMs": 358400,
    "text": "um I've worked on other projects where you go to a domain expert, you give them"
  },
  {
    "startMs": 359830,
    "text": "you go to a domain expert, you give them"
  },
  {
    "startMs": 359840,
    "text": "you go to a domain expert, you give them a spreadsheet and you say, \"Hey, fill"
  },
  {
    "startMs": 361430,
    "text": "a spreadsheet and you say, \"Hey, fill"
  },
  {
    "startMs": 361440,
    "text": "a spreadsheet and you say, \"Hey, fill out a 100 evals.\" That doesn't go well."
  },
  {
    "startMs": 364150,
    "text": "out a 100 evals.\" That doesn't go well."
  },
  {
    "startMs": 364160,
    "text": "out a 100 evals.\" That doesn't go well. Uh bringing a blank spreadsheet to a"
  },
  {
    "startMs": 366150,
    "text": "Uh bringing a blank spreadsheet to a"
  },
  {
    "startMs": 366160,
    "text": "Uh bringing a blank spreadsheet to a busy expert uh doesn't create momentum."
  },
  {
    "startMs": 369270,
    "text": "busy expert uh doesn't create momentum."
  },
  {
    "startMs": 369280,
    "text": "busy expert uh doesn't create momentum. Uh by bringing them all these these"
  },
  {
    "startMs": 371029,
    "text": "Uh by bringing them all these these"
  },
  {
    "startMs": 371039,
    "text": "Uh by bringing them all these these these evals, these places we'"
  },
  {
    "startMs": 372710,
    "text": "these evals, these places we'"
  },
  {
    "startMs": 372720,
    "text": "these evals, these places we' experienced, we very quickly were able"
  },
  {
    "startMs": 374070,
    "text": "experienced, we very quickly were able"
  },
  {
    "startMs": 374080,
    "text": "experienced, we very quickly were able to get a the domain experts on board"
  },
  {
    "startMs": 376230,
    "text": "to get a the domain experts on board"
  },
  {
    "startMs": 376240,
    "text": "to get a the domain experts on board with what's possible and b get them"
  },
  {
    "startMs": 378230,
    "text": "with what's possible and b get them"
  },
  {
    "startMs": 378240,
    "text": "with what's possible and b get them fixated and interested in the places in"
  },
  {
    "startMs": 380230,
    "text": "fixated and interested in the places in"
  },
  {
    "startMs": 380240,
    "text": "fixated and interested in the places in which it failed. Um I do think that most"
  },
  {
    "startMs": 383350,
    "text": "which it failed. Um I do think that most"
  },
  {
    "startMs": 383360,
    "text": "which it failed. Um I do think that most automations are going to require input"
  },
  {
    "startMs": 384950,
    "text": "automations are going to require input"
  },
  {
    "startMs": 384960,
    "text": "automations are going to require input with domain experts working on these"
  },
  {
    "startMs": 386550,
    "text": "with domain experts working on these"
  },
  {
    "startMs": 386560,
    "text": "with domain experts working on these cases. If you can help them get started"
  },
  {
    "startMs": 389510,
    "text": "cases. If you can help them get started"
  },
  {
    "startMs": 389520,
    "text": "cases. If you can help them get started with evaluation,"
  },
  {
    "startMs": 394720,
    "text": "but writing evals is an art and it can"
  },
  {
    "startMs": 396309,
    "text": "but writing evals is an art and it can"
  },
  {
    "startMs": 396319,
    "text": "but writing evals is an art and it can help get them started more quickly. But"
  },
  {
    "startMs": 399029,
    "text": "help get them started more quickly. But"
  },
  {
    "startMs": 399039,
    "text": "help get them started more quickly. But experts, we sat down with these eval."
  },
  {
    "startMs": 401029,
    "text": "experts, we sat down with these eval."
  },
  {
    "startMs": 401039,
    "text": "experts, we sat down with these eval. What did we find? Well, there was a"
  },
  {
    "startMs": 402870,
    "text": "What did we find? Well, there was a"
  },
  {
    "startMs": 402880,
    "text": "What did we find? Well, there was a large bucket of undecided or contentious"
  },
  {
    "startMs": 406629,
    "text": "large bucket of undecided or contentious"
  },
  {
    "startMs": 406639,
    "text": "large bucket of undecided or contentious cases. These were tasks where actually"
  },
  {
    "startMs": 409350,
    "text": "cases. These were tasks where actually"
  },
  {
    "startMs": 409360,
    "text": "cases. These were tasks where actually two experts might confidently disagree"
  },
  {
    "startMs": 411270,
    "text": "two experts might confidently disagree"
  },
  {
    "startMs": 411280,
    "text": "two experts might confidently disagree about what the answer is or where the"
  },
  {
    "startMs": 413830,
    "text": "about what the answer is or where the"
  },
  {
    "startMs": 413840,
    "text": "about what the answer is or where the organization itself hadn't quite"
  },
  {
    "startMs": 415350,
    "text": "organization itself hadn't quite"
  },
  {
    "startMs": 415360,
    "text": "organization itself hadn't quite decided. and everybody was not sure."
  },
  {
    "startMs": 417350,
    "text": "decided. and everybody was not sure."
  },
  {
    "startMs": 417360,
    "text": "decided. and everybody was not sure. Mixed in with that sort of"
  },
  {
    "startMs": 418550,
    "text": "Mixed in with that sort of"
  },
  {
    "startMs": 418560,
    "text": "Mixed in with that sort of heterogeneously was a small bucket of"
  },
  {
    "startMs": 420390,
    "text": "heterogeneously was a small bucket of"
  },
  {
    "startMs": 420400,
    "text": "heterogeneously was a small bucket of critical errors. These were errors where"
  },
  {
    "startMs": 422309,
    "text": "critical errors. These were errors where"
  },
  {
    "startMs": 422319,
    "text": "critical errors. These were errors where everybody agreed a misstep would, you"
  },
  {
    "startMs": 424150,
    "text": "everybody agreed a misstep would, you"
  },
  {
    "startMs": 424160,
    "text": "everybody agreed a misstep would, you know, threaten material accounts. We we"
  },
  {
    "startMs": 426390,
    "text": "know, threaten material accounts. We we"
  },
  {
    "startMs": 426400,
    "text": "know, threaten material accounts. We we just couldn't have them. And then mixed"
  },
  {
    "startMs": 428070,
    "text": "just couldn't have them. And then mixed"
  },
  {
    "startMs": 428080,
    "text": "just couldn't have them. And then mixed in with that was a whole sort of maybe 3"
  },
  {
    "startMs": 430710,
    "text": "in with that was a whole sort of maybe 3"
  },
  {
    "startMs": 430720,
    "text": "in with that was a whole sort of maybe 3 or 5% of exceptions or escalations,"
  },
  {
    "startMs": 433670,
    "text": "or 5% of exceptions or escalations,"
  },
  {
    "startMs": 433680,
    "text": "or 5% of exceptions or escalations, places that needed more attention."
  },
  {
    "startMs": 436950,
    "text": "places that needed more attention."
  },
  {
    "startMs": 436960,
    "text": "places that needed more attention. Uh do a quick example of these three. So"
  },
  {
    "startMs": 439189,
    "text": "Uh do a quick example of these three. So"
  },
  {
    "startMs": 439199,
    "text": "Uh do a quick example of these three. So the errors pretty straightforward. Even"
  },
  {
    "startMs": 441110,
    "text": "the errors pretty straightforward. Even"
  },
  {
    "startMs": 441120,
    "text": "the errors pretty straightforward. Even top models, if you have multi-step"
  },
  {
    "startMs": 442629,
    "text": "top models, if you have multi-step"
  },
  {
    "startMs": 442639,
    "text": "top models, if you have multi-step workflows fail 2 3% of the time. It's"
  },
  {
    "startMs": 445430,
    "text": "workflows fail 2 3% of the time. It's"
  },
  {
    "startMs": 445440,
    "text": "workflows fail 2 3% of the time. It's annoying when you see it at scale."
  },
  {
    "startMs": 446790,
    "text": "annoying when you see it at scale."
  },
  {
    "startMs": 446800,
    "text": "annoying when you see it at scale. Sometimes you can't even reproduce it,"
  },
  {
    "startMs": 448309,
    "text": "Sometimes you can't even reproduce it,"
  },
  {
    "startMs": 448319,
    "text": "Sometimes you can't even reproduce it, but it happens. For undecided or"
  },
  {
    "startMs": 450390,
    "text": "but it happens. For undecided or"
  },
  {
    "startMs": 450400,
    "text": "but it happens. For undecided or ambiguous cases, quick example, um, in"
  },
  {
    "startMs": 453189,
    "text": "ambiguous cases, quick example, um, in"
  },
  {
    "startMs": 453199,
    "text": "ambiguous cases, quick example, um, in drug development, uh, drug developers"
  },
  {
    "startMs": 455189,
    "text": "drug development, uh, drug developers"
  },
  {
    "startMs": 455199,
    "text": "drug development, uh, drug developers care a lot about targets. These are the,"
  },
  {
    "startMs": 457350,
    "text": "care a lot about targets. These are the,"
  },
  {
    "startMs": 457360,
    "text": "care a lot about targets. These are the, you know, key pathway or molecule that"
  },
  {
    "startMs": 458790,
    "text": "you know, key pathway or molecule that"
  },
  {
    "startMs": 458800,
    "text": "you know, key pathway or molecule that your drug actually impacts. Gout is a"
  },
  {
    "startMs": 461270,
    "text": "your drug actually impacts. Gout is a"
  },
  {
    "startMs": 461280,
    "text": "your drug actually impacts. Gout is a disease that's caused by overprouction"
  },
  {
    "startMs": 463029,
    "text": "disease that's caused by overprouction"
  },
  {
    "startMs": 463039,
    "text": "disease that's caused by overprouction of interlucan one beta. Culticine is a"
  },
  {
    "startMs": 465749,
    "text": "of interlucan one beta. Culticine is a"
  },
  {
    "startMs": 465759,
    "text": "of interlucan one beta. Culticine is a drug that reduces interlucan one beta by"
  },
  {
    "startMs": 468150,
    "text": "drug that reduces interlucan one beta by"
  },
  {
    "startMs": 468160,
    "text": "drug that reduces interlucan one beta by preventing a chain reaction basically."
  },
  {
    "startMs": 470710,
    "text": "preventing a chain reaction basically."
  },
  {
    "startMs": 470720,
    "text": "preventing a chain reaction basically. uh so colon itself binds to a protein"
  },
  {
    "startMs": 474230,
    "text": "uh so colon itself binds to a protein"
  },
  {
    "startMs": 474240,
    "text": "uh so colon itself binds to a protein called tubulin. Our system wanted to"
  },
  {
    "startMs": 476390,
    "text": "called tubulin. Our system wanted to"
  },
  {
    "startMs": 476400,
    "text": "called tubulin. Our system wanted to know does cultureine target interlucan"
  },
  {
    "startMs": 478469,
    "text": "know does cultureine target interlucan"
  },
  {
    "startMs": 478479,
    "text": "know does cultureine target interlucan one beta. And if you ask GPT55 sometimes"
  },
  {
    "startMs": 481589,
    "text": "one beta. And if you ask GPT55 sometimes"
  },
  {
    "startMs": 481599,
    "text": "one beta. And if you ask GPT55 sometimes it'll say no and if you ask you know"
  },
  {
    "startMs": 483430,
    "text": "it'll say no and if you ask you know"
  },
  {
    "startMs": 483440,
    "text": "it'll say no and if you ask you know opus 48 sometimes it'll say yes. They"
  },
  {
    "startMs": 485749,
    "text": "opus 48 sometimes it'll say yes. They"
  },
  {
    "startMs": 485759,
    "text": "opus 48 sometimes it'll say yes. They actually sort of both go back and forth"
  },
  {
    "startMs": 487110,
    "text": "actually sort of both go back and forth"
  },
  {
    "startMs": 487120,
    "text": "actually sort of both go back and forth on this. This is a toy case. It's"
  },
  {
    "startMs": 489029,
    "text": "on this. This is a toy case. It's"
  },
  {
    "startMs": 489039,
    "text": "on this. This is a toy case. It's actually the real situation is a little"
  },
  {
    "startMs": 490390,
    "text": "actually the real situation is a little"
  },
  {
    "startMs": 490400,
    "text": "actually the real situation is a little more complicated but it's"
  },
  {
    "startMs": 491350,
    "text": "more complicated but it's"
  },
  {
    "startMs": 491360,
    "text": "more complicated but it's representative. It gives a sort of"
  },
  {
    "startMs": 492550,
    "text": "representative. It gives a sort of"
  },
  {
    "startMs": 492560,
    "text": "representative. It gives a sort of flavor of um what undecided cases look"
  },
  {
    "startMs": 495110,
    "text": "flavor of um what undecided cases look"
  },
  {
    "startMs": 495120,
    "text": "flavor of um what undecided cases look like. Uh in terms of unusual or"
  },
  {
    "startMs": 497749,
    "text": "like. Uh in terms of unusual or"
  },
  {
    "startMs": 497759,
    "text": "like. Uh in terms of unusual or exception, I've got another bio example"
  },
  {
    "startMs": 499189,
    "text": "exception, I've got another bio example"
  },
  {
    "startMs": 499199,
    "text": "exception, I've got another bio example that I really like. Uh, this is a really"
  },
  {
    "startMs": 501029,
    "text": "that I really like. Uh, this is a really"
  },
  {
    "startMs": 501039,
    "text": "that I really like. Uh, this is a really exciting new potential drug called Verve"
  },
  {
    "startMs": 502950,
    "text": "exciting new potential drug called Verve"
  },
  {
    "startMs": 502960,
    "text": "exciting new potential drug called Verve 102, possibly cure heart disease, which"
  },
  {
    "startMs": 505350,
    "text": "102, possibly cure heart disease, which"
  },
  {
    "startMs": 505360,
    "text": "102, possibly cure heart disease, which is like it's pretty cool. I hope to get"
  },
  {
    "startMs": 507189,
    "text": "is like it's pretty cool. I hope to get"
  },
  {
    "startMs": 507199,
    "text": "is like it's pretty cool. I hope to get this or something like it for my little"
  },
  {
    "startMs": 508710,
    "text": "this or something like it for my little"
  },
  {
    "startMs": 508720,
    "text": "this or something like it for my little boy when he turns 18. Um, this does have"
  },
  {
    "startMs": 512550,
    "text": "boy when he turns 18. Um, this does have"
  },
  {
    "startMs": 512560,
    "text": "boy when he turns 18. Um, this does have a decidable output. We do want to index"
  },
  {
    "startMs": 514790,
    "text": "a decidable output. We do want to index"
  },
  {
    "startMs": 514800,
    "text": "a decidable output. We do want to index the sort of six mechanisms of this drug,"
  },
  {
    "startMs": 517029,
    "text": "the sort of six mechanisms of this drug,"
  },
  {
    "startMs": 517039,
    "text": "the sort of six mechanisms of this drug, but the four of them that I've circled"
  },
  {
    "startMs": 518469,
    "text": "but the four of them that I've circled"
  },
  {
    "startMs": 518479,
    "text": "but the four of them that I've circled here are all actually just new to"
  },
  {
    "startMs": 519750,
    "text": "here are all actually just new to"
  },
  {
    "startMs": 519760,
    "text": "here are all actually just new to science. So, like these didn't exist a"
  },
  {
    "startMs": 521670,
    "text": "science. So, like these didn't exist a"
  },
  {
    "startMs": 521680,
    "text": "science. So, like these didn't exist a few years ago. So, in order to properly"
  },
  {
    "startMs": 524149,
    "text": "few years ago. So, in order to properly"
  },
  {
    "startMs": 524159,
    "text": "few years ago. So, in order to properly solve this, we're going to require some"
  },
  {
    "startMs": 525509,
    "text": "solve this, we're going to require some"
  },
  {
    "startMs": 525519,
    "text": "solve this, we're going to require some human attention. The humans are going to"
  },
  {
    "startMs": 526790,
    "text": "human attention. The humans are going to"
  },
  {
    "startMs": 526800,
    "text": "human attention. The humans are going to have to think about how to index"
  },
  {
    "startMs": 528389,
    "text": "have to think about how to index"
  },
  {
    "startMs": 528399,
    "text": "have to think about how to index ontologies. This is an escalation"
  },
  {
    "startMs": 532470,
    "text": "ontologies. This is an escalation"
  },
  {
    "startMs": 532480,
    "text": "ontologies. This is an escalation path forward. How are we going to handle"
  },
  {
    "startMs": 533750,
    "text": "path forward. How are we going to handle"
  },
  {
    "startMs": 533760,
    "text": "path forward. How are we going to handle these three buckets? We need to make"
  },
  {
    "startMs": 535110,
    "text": "these three buckets? We need to make"
  },
  {
    "startMs": 535120,
    "text": "these three buckets? We need to make sure that errors both get routed into a"
  },
  {
    "startMs": 538310,
    "text": "sure that errors both get routed into a"
  },
  {
    "startMs": 538320,
    "text": "sure that errors both get routed into a place where we can grind out high"
  },
  {
    "startMs": 539670,
    "text": "place where we can grind out high"
  },
  {
    "startMs": 539680,
    "text": "place where we can grind out high performance and then we need to grind"
  },
  {
    "startMs": 541350,
    "text": "performance and then we need to grind"
  },
  {
    "startMs": 541360,
    "text": "performance and then we need to grind out the performance for exceptions and"
  },
  {
    "startMs": 543350,
    "text": "out the performance for exceptions and"
  },
  {
    "startMs": 543360,
    "text": "out the performance for exceptions and for undecided cases. We want to route"
  },
  {
    "startMs": 544949,
    "text": "for undecided cases. We want to route"
  },
  {
    "startMs": 544959,
    "text": "for undecided cases. We want to route those into places to keep them out of"
  },
  {
    "startMs": 546710,
    "text": "those into places to keep them out of"
  },
  {
    "startMs": 546720,
    "text": "those into places to keep them out of the normal system because they're going"
  },
  {
    "startMs": 548310,
    "text": "the normal system because they're going"
  },
  {
    "startMs": 548320,
    "text": "the normal system because they're going to require different types of handling."
  },
  {
    "startMs": 551030,
    "text": "to require different types of handling."
  },
  {
    "startMs": 551040,
    "text": "to require different types of handling. And I think probably in retrospect we"
  },
  {
    "startMs": 553030,
    "text": "And I think probably in retrospect we"
  },
  {
    "startMs": 553040,
    "text": "And I think probably in retrospect we spent at least as much time building"
  },
  {
    "startMs": 555110,
    "text": "spent at least as much time building"
  },
  {
    "startMs": 555120,
    "text": "spent at least as much time building case routing as we actually did on the"
  },
  {
    "startMs": 557110,
    "text": "case routing as we actually did on the"
  },
  {
    "startMs": 557120,
    "text": "case routing as we actually did on the final error climbing for the sort of"
  },
  {
    "startMs": 559030,
    "text": "final error climbing for the sort of"
  },
  {
    "startMs": 559040,
    "text": "final error climbing for the sort of critical set. Is the AI ever confused"
  },
  {
    "startMs": 561509,
    "text": "critical set. Is the AI ever confused"
  },
  {
    "startMs": 561519,
    "text": "critical set. Is the AI ever confused about what's going on? Is it unusual a"
  },
  {
    "startMs": 563350,
    "text": "about what's going on? Is it unusual a"
  },
  {
    "startMs": 563360,
    "text": "about what's going on? Is it unusual a case? Abort. Use the AI to describe"
  },
  {
    "startMs": 565430,
    "text": "case? Abort. Use the AI to describe"
  },
  {
    "startMs": 565440,
    "text": "case? Abort. Use the AI to describe what's going on. Categorize it away."
  },
  {
    "startMs": 567350,
    "text": "what's going on. Categorize it away."
  },
  {
    "startMs": 567360,
    "text": "what's going on. Categorize it away. Skip it from the pipeline. Is it"
  },
  {
    "startMs": 569430,
    "text": "Skip it from the pipeline. Is it"
  },
  {
    "startMs": 569440,
    "text": "Skip it from the pipeline. Is it plausibly business critical? Okay, then"
  },
  {
    "startMs": 571590,
    "text": "plausibly business critical? Okay, then"
  },
  {
    "startMs": 571600,
    "text": "plausibly business critical? Okay, then send that to the extremely robustly"
  },
  {
    "startMs": 573670,
    "text": "send that to the extremely robustly"
  },
  {
    "startMs": 573680,
    "text": "send that to the extremely robustly tested, you know, thousands of evals"
  },
  {
    "startMs": 575350,
    "text": "tested, you know, thousands of evals"
  },
  {
    "startMs": 575360,
    "text": "tested, you know, thousands of evals pipeline. Hopefully you can ship an"
  },
  {
    "startMs": 577190,
    "text": "pipeline. Hopefully you can ship an"
  },
  {
    "startMs": 577200,
    "text": "pipeline. Hopefully you can ship an answer. If through that pipeline it's"
  },
  {
    "startMs": 579350,
    "text": "answer. If through that pipeline it's"
  },
  {
    "startMs": 579360,
    "text": "answer. If through that pipeline it's still not sure, fine. and ship it back"
  },
  {
    "startMs": 580790,
    "text": "still not sure, fine. and ship it back"
  },
  {
    "startMs": 580800,
    "text": "still not sure, fine. and ship it back to a human as an exception. But vast"
  },
  {
    "startMs": 582630,
    "text": "to a human as an exception. But vast"
  },
  {
    "startMs": 582640,
    "text": "to a human as an exception. But vast majority of the time you can ship an"
  },
  {
    "startMs": 583910,
    "text": "majority of the time you can ship an"
  },
  {
    "startMs": 583920,
    "text": "majority of the time you can ship an answer"
  },
  {
    "startMs": 586070,
    "text": "answer"
  },
  {
    "startMs": 586080,
    "text": "answer is it undecided pipeline that out as"
  },
  {
    "startMs": 587910,
    "text": "is it undecided pipeline that out as"
  },
  {
    "startMs": 587920,
    "text": "is it undecided pipeline that out as well. Um for the majority of cases"
  },
  {
    "startMs": 589910,
    "text": "well. Um for the majority of cases"
  },
  {
    "startMs": 589920,
    "text": "well. Um for the majority of cases though they didn't fall into any of"
  },
  {
    "startMs": 591030,
    "text": "though they didn't fall into any of"
  },
  {
    "startMs": 591040,
    "text": "though they didn't fall into any of these cases and we could just ship as"
  },
  {
    "startMs": 592790,
    "text": "these cases and we could just ship as"
  },
  {
    "startMs": 592800,
    "text": "these cases and we could just ship as usual."
  },
  {
    "startMs": 598080,
    "text": "um in terms of the critical path. So you"
  },
  {
    "startMs": 600389,
    "text": "um in terms of the critical path. So you"
  },
  {
    "startMs": 600399,
    "text": "um in terms of the critical path. So you know we had this this small set of tasks"
  },
  {
    "startMs": 602150,
    "text": "know we had this this small set of tasks"
  },
  {
    "startMs": 602160,
    "text": "know we had this this small set of tasks that were absolutely business critical"
  },
  {
    "startMs": 604389,
    "text": "that were absolutely business critical"
  },
  {
    "startMs": 604399,
    "text": "that were absolutely business critical once we took out the ambiguous cases and"
  },
  {
    "startMs": 606389,
    "text": "once we took out the ambiguous cases and"
  },
  {
    "startMs": 606399,
    "text": "once we took out the ambiguous cases and once we took out the sort of um unusual"
  },
  {
    "startMs": 608949,
    "text": "once we took out the sort of um unusual"
  },
  {
    "startMs": 608959,
    "text": "once we took out the sort of um unusual escalation cases we're able to get to"
  },
  {
    "startMs": 610710,
    "text": "escalation cases we're able to get to"
  },
  {
    "startMs": 610720,
    "text": "escalation cases we're able to get to about 95% accuracy with just sort of"
  },
  {
    "startMs": 612710,
    "text": "about 95% accuracy with just sort of"
  },
  {
    "startMs": 612720,
    "text": "about 95% accuracy with just sort of standard approaches going from 95 to 3"
  },
  {
    "startMs": 615269,
    "text": "standard approaches going from 95 to 3"
  },
  {
    "startMs": 615279,
    "text": "standard approaches going from 95 to 3 and a half 9s. This is a case of I've"
  },
  {
    "startMs": 618069,
    "text": "and a half 9s. This is a case of I've"
  },
  {
    "startMs": 618079,
    "text": "and a half 9s. This is a case of I've got a whole bag of tricks on it. It's a"
  },
  {
    "startMs": 619509,
    "text": "got a whole bag of tricks on it. It's a"
  },
  {
    "startMs": 619519,
    "text": "got a whole bag of tricks on it. It's a case of hill climbing. It's a case of"
  },
  {
    "startMs": 620870,
    "text": "case of hill climbing. It's a case of"
  },
  {
    "startMs": 620880,
    "text": "case of hill climbing. It's a case of looking at a lot of evals using all of"
  },
  {
    "startMs": 622710,
    "text": "looking at a lot of evals using all of"
  },
  {
    "startMs": 622720,
    "text": "looking at a lot of evals using all of the normal tricks to juice performance"
  },
  {
    "startMs": 624150,
    "text": "the normal tricks to juice performance"
  },
  {
    "startMs": 624160,
    "text": "the normal tricks to juice performance out of LLMs. I basically do believe"
  },
  {
    "startMs": 626310,
    "text": "out of LLMs. I basically do believe"
  },
  {
    "startMs": 626320,
    "text": "out of LLMs. I basically do believe though that if your inputs are well"
  },
  {
    "startMs": 627990,
    "text": "though that if your inputs are well"
  },
  {
    "startMs": 628000,
    "text": "though that if your inputs are well structured and you're looking at a"
  },
  {
    "startMs": 629350,
    "text": "structured and you're looking at a"
  },
  {
    "startMs": 629360,
    "text": "structured and you're looking at a specific set of cases that you've"
  },
  {
    "startMs": 630710,
    "text": "specific set of cases that you've"
  },
  {
    "startMs": 630720,
    "text": "specific set of cases that you've articulated from your business, getting"
  },
  {
    "startMs": 632470,
    "text": "articulated from your business, getting"
  },
  {
    "startMs": 632480,
    "text": "articulated from your business, getting to three and a half nines is possible"
  },
  {
    "startMs": 634069,
    "text": "to three and a half nines is possible"
  },
  {
    "startMs": 634079,
    "text": "to three and a half nines is possible for most automations."
  },
  {
    "startMs": 636630,
    "text": "for most automations."
  },
  {
    "startMs": 636640,
    "text": "for most automations. Um, [snorts] we deployed this only once"
  },
  {
    "startMs": 638389,
    "text": "Um, [snorts] we deployed this only once"
  },
  {
    "startMs": 638399,
    "text": "Um, [snorts] we deployed this only once a random sample showed that this"
  },
  {
    "startMs": 640230,
    "text": "a random sample showed that this"
  },
  {
    "startMs": 640240,
    "text": "a random sample showed that this performance was better than the historic"
  },
  {
    "startMs": 641829,
    "text": "performance was better than the historic"
  },
  {
    "startMs": 641839,
    "text": "performance was better than the historic work. This is cool. So, um, I do think"
  },
  {
    "startMs": 645269,
    "text": "work. This is cool. So, um, I do think"
  },
  {
    "startMs": 645279,
    "text": "work. This is cool. So, um, I do think specifically um, I'm going to claim that"
  },
  {
    "startMs": 647190,
    "text": "specifically um, I'm going to claim that"
  },
  {
    "startMs": 647200,
    "text": "specifically um, I'm going to claim that this system performed better than PhD"
  },
  {
    "startMs": 649110,
    "text": "this system performed better than PhD"
  },
  {
    "startMs": 649120,
    "text": "this system performed better than PhD humans at scale. And I'll make that"
  },
  {
    "startMs": 651190,
    "text": "humans at scale. And I'll make that"
  },
  {
    "startMs": 651200,
    "text": "humans at scale. And I'll make that qualifier because I know that a PhD"
  },
  {
    "startMs": 653110,
    "text": "qualifier because I know that a PhD"
  },
  {
    "startMs": 653120,
    "text": "qualifier because I know that a PhD human if they're sitting down on a"
  },
  {
    "startMs": 654710,
    "text": "human if they're sitting down on a"
  },
  {
    "startMs": 654720,
    "text": "human if they're sitting down on a singular case and they're looking at all"
  },
  {
    "startMs": 656150,
    "text": "singular case and they're looking at all"
  },
  {
    "startMs": 656160,
    "text": "singular case and they're looking at all the details, they will outperform this"
  },
  {
    "startMs": 658069,
    "text": "the details, they will outperform this"
  },
  {
    "startMs": 658079,
    "text": "the details, they will outperform this system, but not at scale. Not when they"
  },
  {
    "startMs": 660230,
    "text": "system, but not at scale. Not when they"
  },
  {
    "startMs": 660240,
    "text": "system, but not at scale. Not when they have to look at hundreds of cases a"
  },
  {
    "startMs": 661509,
    "text": "have to look at hundreds of cases a"
  },
  {
    "startMs": 661519,
    "text": "have to look at hundreds of cases a week, not when they have fatigue, not"
  },
  {
    "startMs": 663269,
    "text": "week, not when they have fatigue, not"
  },
  {
    "startMs": 663279,
    "text": "week, not when they have fatigue, not when if you know one in a thousand times"
  },
  {
    "startMs": 664710,
    "text": "when if you know one in a thousand times"
  },
  {
    "startMs": 664720,
    "text": "when if you know one in a thousand times they misclick and not uh you know given"
  },
  {
    "startMs": 667350,
    "text": "they misclick and not uh you know given"
  },
  {
    "startMs": 667360,
    "text": "they misclick and not uh you know given time pressure of their other work."
  },
  {
    "startMs": 671440,
    "text": "Um so full automation is the happy path."
  },
  {
    "startMs": 673590,
    "text": "Um so full automation is the happy path."
  },
  {
    "startMs": 673600,
    "text": "Um so full automation is the happy path. Saves you time. That's great. You get"
  },
  {
    "startMs": 675509,
    "text": "Saves you time. That's great. You get"
  },
  {
    "startMs": 675519,
    "text": "Saves you time. That's great. You get better critical business critical task"
  },
  {
    "startMs": 677110,
    "text": "better critical business critical task"
  },
  {
    "startMs": 677120,
    "text": "better critical business critical task outcomes. That's also great. You know,"
  },
  {
    "startMs": 678630,
    "text": "outcomes. That's also great. You know,"
  },
  {
    "startMs": 678640,
    "text": "outcomes. That's also great. You know, this is a if you can take the places"
  },
  {
    "startMs": 680310,
    "text": "this is a if you can take the places"
  },
  {
    "startMs": 680320,
    "text": "this is a if you can take the places that are most valuable to the business"
  },
  {
    "startMs": 681990,
    "text": "that are most valuable to the business"
  },
  {
    "startMs": 682000,
    "text": "that are most valuable to the business and grind them out until you're"
  },
  {
    "startMs": 683110,
    "text": "and grind them out until you're"
  },
  {
    "startMs": 683120,
    "text": "and grind them out until you're performing better than humans at scale,"
  },
  {
    "startMs": 684550,
    "text": "performing better than humans at scale,"
  },
  {
    "startMs": 684560,
    "text": "performing better than humans at scale, that's great. There's a third benefit"
  },
  {
    "startMs": 686310,
    "text": "that's great. There's a third benefit"
  },
  {
    "startMs": 686320,
    "text": "that's great. There's a third benefit here, which is that it organizes the"
  },
  {
    "startMs": 687829,
    "text": "here, which is that it organizes the"
  },
  {
    "startMs": 687839,
    "text": "here, which is that it organizes the exhaust from the automation in a way"
  },
  {
    "startMs": 689350,
    "text": "exhaust from the automation in a way"
  },
  {
    "startMs": 689360,
    "text": "exhaust from the automation in a way that your experts can pay attention to."
  },
  {
    "startMs": 690790,
    "text": "that your experts can pay attention to."
  },
  {
    "startMs": 690800,
    "text": "that your experts can pay attention to. And I think this is pretty neat. So,"
  },
  {
    "startMs": 693350,
    "text": "And I think this is pretty neat. So,"
  },
  {
    "startMs": 693360,
    "text": "And I think this is pretty neat. So, back to that expert from the beginning."
  },
  {
    "startMs": 695190,
    "text": "back to that expert from the beginning."
  },
  {
    "startMs": 695200,
    "text": "back to that expert from the beginning. Hopefully, we've cleared their plate"
  },
  {
    "startMs": 696790,
    "text": "Hopefully, we've cleared their plate"
  },
  {
    "startMs": 696800,
    "text": "Hopefully, we've cleared their plate somewhat. We've taken some of the"
  },
  {
    "startMs": 697910,
    "text": "somewhat. We've taken some of the"
  },
  {
    "startMs": 697920,
    "text": "somewhat. We've taken some of the routine work out of their job. In"
  },
  {
    "startMs": 699990,
    "text": "routine work out of their job. In"
  },
  {
    "startMs": 700000,
    "text": "routine work out of their job. In addition, because we've invested so"
  },
  {
    "startMs": 701430,
    "text": "addition, because we've invested so"
  },
  {
    "startMs": 701440,
    "text": "addition, because we've invested so heavily in routing, we've now organized"
  },
  {
    "startMs": 703430,
    "text": "heavily in routing, we've now organized"
  },
  {
    "startMs": 703440,
    "text": "heavily in routing, we've now organized that we can organize their attention in"
  },
  {
    "startMs": 704870,
    "text": "that we can organize their attention in"
  },
  {
    "startMs": 704880,
    "text": "that we can organize their attention in a structured way among the around the"
  },
  {
    "startMs": 707269,
    "text": "a structured way among the around the"
  },
  {
    "startMs": 707279,
    "text": "a structured way among the around the remaining cases. So they're going to get"
  },
  {
    "startMs": 709829,
    "text": "remaining cases. So they're going to get"
  },
  {
    "startMs": 709839,
    "text": "remaining cases. So they're going to get a pile organized undecided and ambiguous"
  },
  {
    "startMs": 712710,
    "text": "a pile organized undecided and ambiguous"
  },
  {
    "startMs": 712720,
    "text": "a pile organized undecided and ambiguous cases, exceptions and unusual cases, and"
  },
  {
    "startMs": 715430,
    "text": "cases, exceptions and unusual cases, and"
  },
  {
    "startMs": 715440,
    "text": "cases, exceptions and unusual cases, and then a small number of critical cases"
  },
  {
    "startMs": 716949,
    "text": "then a small number of critical cases"
  },
  {
    "startMs": 716959,
    "text": "then a small number of critical cases the automation wasn't sure about. For"
  },
  {
    "startMs": 719430,
    "text": "the automation wasn't sure about. For"
  },
  {
    "startMs": 719440,
    "text": "the automation wasn't sure about. For those last ones, the sort of ones that"
  },
  {
    "startMs": 720790,
    "text": "those last ones, the sort of ones that"
  },
  {
    "startMs": 720800,
    "text": "those last ones, the sort of ones that are critical, they can quickly review"
  },
  {
    "startMs": 722389,
    "text": "are critical, they can quickly review"
  },
  {
    "startMs": 722399,
    "text": "are critical, they can quickly review them, very very few in number, and ship"
  },
  {
    "startMs": 724310,
    "text": "them, very very few in number, and ship"
  },
  {
    "startMs": 724320,
    "text": "them, very very few in number, and ship them. [snorts] For the ones that are"
  },
  {
    "startMs": 726710,
    "text": "them. [snorts] For the ones that are"
  },
  {
    "startMs": 726720,
    "text": "them. [snorts] For the ones that are organizationally ambiguous, this is"
  },
  {
    "startMs": 728470,
    "text": "organizationally ambiguous, this is"
  },
  {
    "startMs": 728480,
    "text": "organizationally ambiguous, this is actually kind of interesting. You know,"
  },
  {
    "startMs": 729990,
    "text": "actually kind of interesting. You know,"
  },
  {
    "startMs": 730000,
    "text": "actually kind of interesting. You know, the first thought is maybe this is a way"
  },
  {
    "startMs": 731990,
    "text": "the first thought is maybe this is a way"
  },
  {
    "startMs": 732000,
    "text": "the first thought is maybe this is a way to incentivize the organization to"
  },
  {
    "startMs": 733910,
    "text": "to incentivize the organization to"
  },
  {
    "startMs": 733920,
    "text": "to incentivize the organization to specify. You know, you've got 20% maybe"
  },
  {
    "startMs": 736150,
    "text": "specify. You know, you've got 20% maybe"
  },
  {
    "startMs": 736160,
    "text": "specify. You know, you've got 20% maybe of the total tasks that the organization"
  },
  {
    "startMs": 738389,
    "text": "of the total tasks that the organization"
  },
  {
    "startMs": 738399,
    "text": "of the total tasks that the organization wasn't sure with"
  },
  {
    "startMs": 742240,
    "text": "these are often one of the reasons that"
  },
  {
    "startMs": 744790,
    "text": "these are often one of the reasons that"
  },
  {
    "startMs": 744800,
    "text": "these are often one of the reasons that they're not yet fully specified is that"
  },
  {
    "startMs": 746629,
    "text": "they're not yet fully specified is that"
  },
  {
    "startMs": 746639,
    "text": "they're not yet fully specified is that they're only marginally valuable. So the"
  },
  {
    "startMs": 749030,
    "text": "they're only marginally valuable. So the"
  },
  {
    "startMs": 749040,
    "text": "they're only marginally valuable. So the alternative here is that you can"
  },
  {
    "startMs": 750389,
    "text": "alternative here is that you can"
  },
  {
    "startMs": 750399,
    "text": "alternative here is that you can actually if the expert has a big pile of"
  },
  {
    "startMs": 753269,
    "text": "actually if the expert has a big pile of"
  },
  {
    "startMs": 753279,
    "text": "actually if the expert has a big pile of them in front of them they can actually"
  },
  {
    "startMs": 754949,
    "text": "them in front of them they can actually"
  },
  {
    "startMs": 754959,
    "text": "them in front of them they can actually advocate that hey let's permanently"
  },
  {
    "startMs": 756470,
    "text": "advocate that hey let's permanently"
  },
  {
    "startMs": 756480,
    "text": "advocate that hey let's permanently freeze this work let's prioritize it"
  },
  {
    "startMs": 758710,
    "text": "freeze this work let's prioritize it"
  },
  {
    "startMs": 758720,
    "text": "freeze this work let's prioritize it let's not think about it again until"
  },
  {
    "startMs": 760069,
    "text": "let's not think about it again until"
  },
  {
    "startMs": 760079,
    "text": "let's not think about it again until there's a business case too and let's"
  },
  {
    "startMs": 761670,
    "text": "there's a business case too and let's"
  },
  {
    "startMs": 761680,
    "text": "there's a business case too and let's get it off our plate that is a form of"
  },
  {
    "startMs": 764069,
    "text": "get it off our plate that is a form of"
  },
  {
    "startMs": 764079,
    "text": "get it off our plate that is a form of again reducing complexity in an"
  },
  {
    "startMs": 765670,
    "text": "again reducing complexity in an"
  },
  {
    "startMs": 765680,
    "text": "again reducing complexity in an organization having fewer things to"
  },
  {
    "startMs": 767590,
    "text": "organization having fewer things to"
  },
  {
    "startMs": 767600,
    "text": "organization having fewer things to think about so the focus and attention"
  },
  {
    "startMs": 769030,
    "text": "think about so the focus and attention"
  },
  {
    "startMs": 769040,
    "text": "think about so the focus and attention can go in the right place"
  },
  {
    "startMs": 771588,
    "text": "can go in the right place"
  },
  {
    "startMs": 771598,
    "text": "can go in the right place [snorts] I think this is even more"
  },
  {
    "startMs": 772790,
    "text": "[snorts] I think this is even more"
  },
  {
    "startMs": 772800,
    "text": "[snorts] I think this is even more interesting so groundbreaking work like"
  },
  {
    "startMs": 774790,
    "text": "interesting so groundbreaking work like"
  },
  {
    "startMs": 774800,
    "text": "interesting so groundbreaking work like that verve 102 drug for for heart"
  },
  {
    "startMs": 776790,
    "text": "that verve 102 drug for for heart"
  },
  {
    "startMs": 776800,
    "text": "that verve 102 drug for for heart disease it's only like 3% % of new drug"
  },
  {
    "startMs": 779030,
    "text": "disease it's only like 3% % of new drug"
  },
  {
    "startMs": 779040,
    "text": "disease it's only like 3% % of new drug development work. The vast majority of"
  },
  {
    "startMs": 780470,
    "text": "development work. The vast majority of"
  },
  {
    "startMs": 780480,
    "text": "development work. The vast majority of drug development work sort of pattern"
  },
  {
    "startMs": 781670,
    "text": "drug development work sort of pattern"
  },
  {
    "startMs": 781680,
    "text": "drug development work sort of pattern matches. Everybody's familiar with the"
  },
  {
    "startMs": 783030,
    "text": "matches. Everybody's familiar with the"
  },
  {
    "startMs": 783040,
    "text": "matches. Everybody's familiar with the rough shape of what you're looking at."
  },
  {
    "startMs": 784710,
    "text": "rough shape of what you're looking at."
  },
  {
    "startMs": 784720,
    "text": "rough shape of what you're looking at. So an expert might see this once a week,"
  },
  {
    "startMs": 786870,
    "text": "So an expert might see this once a week,"
  },
  {
    "startMs": 786880,
    "text": "So an expert might see this once a week, once every other week, one of these"
  },
  {
    "startMs": 788389,
    "text": "once every other week, one of these"
  },
  {
    "startMs": 788399,
    "text": "once every other week, one of these cases, they're scattered throughout. Uh"
  },
  {
    "startMs": 790870,
    "text": "cases, they're scattered throughout. Uh"
  },
  {
    "startMs": 790880,
    "text": "cases, they're scattered throughout. Uh the the expert might think, hey, I could"
  },
  {
    "startMs": 792790,
    "text": "the the expert might think, hey, I could"
  },
  {
    "startMs": 792800,
    "text": "the the expert might think, hey, I could build something to handle these cases."
  },
  {
    "startMs": 794870,
    "text": "build something to handle these cases."
  },
  {
    "startMs": 794880,
    "text": "build something to handle these cases. But if you're only seeing it"
  },
  {
    "startMs": 795910,
    "text": "But if you're only seeing it"
  },
  {
    "startMs": 795920,
    "text": "But if you're only seeing it sporadically, it's hard to build an"
  },
  {
    "startMs": 797350,
    "text": "sporadically, it's hard to build an"
  },
  {
    "startMs": 797360,
    "text": "sporadically, it's hard to build an intuition for how important it is, how"
  },
  {
    "startMs": 799030,
    "text": "intuition for how important it is, how"
  },
  {
    "startMs": 799040,
    "text": "intuition for how important it is, how expensive it would be, who would pay for"
  },
  {
    "startMs": 800470,
    "text": "expensive it would be, who would pay for"
  },
  {
    "startMs": 800480,
    "text": "expensive it would be, who would pay for it. You start to bucket those. And now"
  },
  {
    "startMs": 803110,
    "text": "it. You start to bucket those. And now"
  },
  {
    "startMs": 803120,
    "text": "it. You start to bucket those. And now the expert can think maybe when they sit"
  },
  {
    "startMs": 804870,
    "text": "the expert can think maybe when they sit"
  },
  {
    "startMs": 804880,
    "text": "the expert can think maybe when they sit down with two or three hours and look at"
  },
  {
    "startMs": 806230,
    "text": "down with two or three hours and look at"
  },
  {
    "startMs": 806240,
    "text": "down with two or three hours and look at them in aggregate, hey there's patterns"
  },
  {
    "startMs": 808310,
    "text": "them in aggregate, hey there's patterns"
  },
  {
    "startMs": 808320,
    "text": "them in aggregate, hey there's patterns here or hey maybe this is the specific"
  },
  {
    "startMs": 810710,
    "text": "here or hey maybe this is the specific"
  },
  {
    "startMs": 810720,
    "text": "here or hey maybe this is the specific investment I propose to start solving"
  },
  {
    "startMs": 812710,
    "text": "investment I propose to start solving"
  },
  {
    "startMs": 812720,
    "text": "investment I propose to start solving these for the organization and then if"
  },
  {
    "startMs": 815269,
    "text": "these for the organization and then if"
  },
  {
    "startMs": 815279,
    "text": "these for the organization and then if an an account you know a real customer"
  },
  {
    "startMs": 817190,
    "text": "an an account you know a real customer"
  },
  {
    "startMs": 817200,
    "text": "an an account you know a real customer comes and says hey I have a problem this"
  },
  {
    "startMs": 819190,
    "text": "comes and says hey I have a problem this"
  },
  {
    "startMs": 819200,
    "text": "comes and says hey I have a problem this expert now is ready to figure out how to"
  },
  {
    "startMs": 821509,
    "text": "expert now is ready to figure out how to"
  },
  {
    "startMs": 821519,
    "text": "expert now is ready to figure out how to how to deal with that how to move the"
  },
  {
    "startMs": 822949,
    "text": "how to deal with that how to move the"
  },
  {
    "startMs": 822959,
    "text": "how to deal with that how to move the business forward on it. So we saw this"
  },
  {
    "startMs": 824710,
    "text": "business forward on it. So we saw this"
  },
  {
    "startMs": 824720,
    "text": "business forward on it. So we saw this happen organically on a 8 figure account"
  },
  {
    "startMs": 827670,
    "text": "happen organically on a 8 figure account"
  },
  {
    "startMs": 827680,
    "text": "happen organically on a 8 figure account um and the expert had basically by"
  },
  {
    "startMs": 830629,
    "text": "um and the expert had basically by"
  },
  {
    "startMs": 830639,
    "text": "um and the expert had basically by having the data structured had already"
  },
  {
    "startMs": 832069,
    "text": "having the data structured had already"
  },
  {
    "startMs": 832079,
    "text": "having the data structured had already pre-formed an opinion about where to"
  },
  {
    "startMs": 833670,
    "text": "pre-formed an opinion about where to"
  },
  {
    "startMs": 833680,
    "text": "pre-formed an opinion about where to invest and where to build forward. I"
  },
  {
    "startMs": 836150,
    "text": "invest and where to build forward. I"
  },
  {
    "startMs": 836160,
    "text": "invest and where to build forward. I think this is actually the case for most"
  },
  {
    "startMs": 838550,
    "text": "think this is actually the case for most"
  },
  {
    "startMs": 838560,
    "text": "think this is actually the case for most scaled business processes. If you're"
  },
  {
    "startMs": 841430,
    "text": "scaled business processes. If you're"
  },
  {
    "startMs": 841440,
    "text": "scaled business processes. If you're working with the with with the market"
  },
  {
    "startMs": 843110,
    "text": "working with the with with the market"
  },
  {
    "startMs": 843120,
    "text": "working with the with with the market with data in aggregate you're collecting"
  },
  {
    "startMs": 845350,
    "text": "with data in aggregate you're collecting"
  },
  {
    "startMs": 845360,
    "text": "with data in aggregate you're collecting leading indicators of market demand,"
  },
  {
    "startMs": 847189,
    "text": "leading indicators of market demand,"
  },
  {
    "startMs": 847199,
    "text": "leading indicators of market demand, right? you know, the places that are"
  },
  {
    "startMs": 848870,
    "text": "right? you know, the places that are"
  },
  {
    "startMs": 848880,
    "text": "right? you know, the places that are slightly unusual or the places that um"
  },
  {
    "startMs": 851269,
    "text": "slightly unusual or the places that um"
  },
  {
    "startMs": 851279,
    "text": "slightly unusual or the places that um that the market's moving now that they"
  },
  {
    "startMs": 852629,
    "text": "that the market's moving now that they"
  },
  {
    "startMs": 852639,
    "text": "that the market's moving now that they weren't a few years ago, but they look"
  },
  {
    "startMs": 854389,
    "text": "weren't a few years ago, but they look"
  },
  {
    "startMs": 854399,
    "text": "weren't a few years ago, but they look like exceptions and if they're"
  },
  {
    "startMs": 855509,
    "text": "like exceptions and if they're"
  },
  {
    "startMs": 855519,
    "text": "like exceptions and if they're encountered infrequently, it's not"
  },
  {
    "startMs": 856870,
    "text": "encountered infrequently, it's not"
  },
  {
    "startMs": 856880,
    "text": "encountered infrequently, it's not enough signal to have somebody think"
  },
  {
    "startMs": 858310,
    "text": "enough signal to have somebody think"
  },
  {
    "startMs": 858320,
    "text": "enough signal to have somebody think about what to do with them. They're"
  },
  {
    "startMs": 859829,
    "text": "about what to do with them. They're"
  },
  {
    "startMs": 859839,
    "text": "about what to do with them. They're difficult to prioritize if they occur"
  },
  {
    "startMs": 861509,
    "text": "difficult to prioritize if they occur"
  },
  {
    "startMs": 861519,
    "text": "difficult to prioritize if they occur randomly. If you can organize this"
  },
  {
    "startMs": 863670,
    "text": "randomly. If you can organize this"
  },
  {
    "startMs": 863680,
    "text": "randomly. If you can organize this signal, this exhaust these exceptional"
  },
  {
    "startMs": 865269,
    "text": "signal, this exhaust these exceptional"
  },
  {
    "startMs": 865279,
    "text": "signal, this exhaust these exceptional cases that don't land in the happy path,"
  },
  {
    "startMs": 867189,
    "text": "cases that don't land in the happy path,"
  },
  {
    "startMs": 867199,
    "text": "cases that don't land in the happy path, your best talent can actually start to"
  },
  {
    "startMs": 868949,
    "text": "your best talent can actually start to"
  },
  {
    "startMs": 868959,
    "text": "your best talent can actually start to think about what to do with it. So,"
  },
  {
    "startMs": 871430,
    "text": "think about what to do with it. So,"
  },
  {
    "startMs": 871440,
    "text": "think about what to do with it. So, quick, we built a full automation. We"
  },
  {
    "startMs": 874470,
    "text": "quick, we built a full automation. We"
  },
  {
    "startMs": 874480,
    "text": "quick, we built a full automation. We can save hours. We can improve task"
  },
  {
    "startMs": 876710,
    "text": "can save hours. We can improve task"
  },
  {
    "startMs": 876720,
    "text": "can save hours. We can improve task level outcomes. We can reduce"
  },
  {
    "startMs": 878629,
    "text": "level outcomes. We can reduce"
  },
  {
    "startMs": 878639,
    "text": "level outcomes. We can reduce organizational complexity by taking a"
  },
  {
    "startMs": 880389,
    "text": "organizational complexity by taking a"
  },
  {
    "startMs": 880399,
    "text": "organizational complexity by taking a big bucket of tasks, encapsulating them,"
  },
  {
    "startMs": 882310,
    "text": "big bucket of tasks, encapsulating them,"
  },
  {
    "startMs": 882320,
    "text": "big bucket of tasks, encapsulating them, and having nobody have to think about"
  },
  {
    "startMs": 883509,
    "text": "and having nobody have to think about"
  },
  {
    "startMs": 883519,
    "text": "and having nobody have to think about them anymore. And then we can route"
  },
  {
    "startMs": 885269,
    "text": "them anymore. And then we can route"
  },
  {
    "startMs": 885279,
    "text": "them anymore. And then we can route talent towards higher value problems."
  },
  {
    "startMs": 887430,
    "text": "talent towards higher value problems."
  },
  {
    "startMs": 887440,
    "text": "talent towards higher value problems. This is a form of organizational"
  },
  {
    "startMs": 889189,
    "text": "This is a form of organizational"
  },
  {
    "startMs": 889199,
    "text": "This is a form of organizational refactoring."
  },
  {
    "startMs": 891509,
    "text": "refactoring."
  },
  {
    "startMs": 891519,
    "text": "refactoring. I will say that full automations,"
  },
  {
    "startMs": 894069,
    "text": "I will say that full automations,"
  },
  {
    "startMs": 894079,
    "text": "I will say that full automations, especially if you're trying to very high"
  },
  {
    "startMs": 896069,
    "text": "especially if you're trying to very high"
  },
  {
    "startMs": 896079,
    "text": "especially if you're trying to very high quality output, something like doing a"
  },
  {
    "startMs": 897990,
    "text": "quality output, something like doing a"
  },
  {
    "startMs": 898000,
    "text": "quality output, something like doing a PhD level expert's job, are expensive."
  },
  {
    "startMs": 901030,
    "text": "PhD level expert's job, are expensive."
  },
  {
    "startMs": 901040,
    "text": "PhD level expert's job, are expensive. AI engineers are expensive. Having an AI"
  },
  {
    "startMs": 903670,
    "text": "AI engineers are expensive. Having an AI"
  },
  {
    "startMs": 903680,
    "text": "AI engineers are expensive. Having an AI engineer spend hours reading a"
  },
  {
    "startMs": 904949,
    "text": "engineer spend hours reading a"
  },
  {
    "startMs": 904959,
    "text": "engineer spend hours reading a spreadsheet of eval is expensive. Once"
  },
  {
    "startMs": 907829,
    "text": "spreadsheet of eval is expensive. Once"
  },
  {
    "startMs": 907839,
    "text": "spreadsheet of eval is expensive. Once you get to that 95% going from 95 to"
  },
  {
    "startMs": 910150,
    "text": "you get to that 95% going from 95 to"
  },
  {
    "startMs": 910160,
    "text": "you get to that 95% going from 95 to 99.95, that's an expensive process,"
  },
  {
    "startMs": 912550,
    "text": "99.95, that's an expensive process,"
  },
  {
    "startMs": 912560,
    "text": "99.95, that's an expensive process, etc., etc., etc. Beyond that, you"
  },
  {
    "startMs": 914870,
    "text": "etc., etc., etc. Beyond that, you"
  },
  {
    "startMs": 914880,
    "text": "etc., etc., etc. Beyond that, you actually have to think if you're going"
  },
  {
    "startMs": 915910,
    "text": "actually have to think if you're going"
  },
  {
    "startMs": 915920,
    "text": "actually have to think if you're going to be refactoring a um a non-engineering"
  },
  {
    "startMs": 919189,
    "text": "to be refactoring a um a non-engineering"
  },
  {
    "startMs": 919199,
    "text": "to be refactoring a um a non-engineering process, you're going to have"
  },
  {
    "startMs": 920310,
    "text": "process, you're going to have"
  },
  {
    "startMs": 920320,
    "text": "process, you're going to have organizational change, trust building,"
  },
  {
    "startMs": 922949,
    "text": "organizational change, trust building,"
  },
  {
    "startMs": 922959,
    "text": "organizational change, trust building, ownership change. If you're going to"
  },
  {
    "startMs": 924790,
    "text": "ownership change. If you're going to"
  },
  {
    "startMs": 924800,
    "text": "ownership change. If you're going to make an investment of this size,"
  },
  {
    "startMs": 925990,
    "text": "make an investment of this size,"
  },
  {
    "startMs": 926000,
    "text": "make an investment of this size, hopefully you're also working with your"
  },
  {
    "startMs": 927189,
    "text": "hopefully you're also working with your"
  },
  {
    "startMs": 927199,
    "text": "hopefully you're also working with your sales and marketing team to figure out"
  },
  {
    "startMs": 928870,
    "text": "sales and marketing team to figure out"
  },
  {
    "startMs": 928880,
    "text": "sales and marketing team to figure out how to juice that for revenue, how to"
  },
  {
    "startMs": 930470,
    "text": "how to juice that for revenue, how to"
  },
  {
    "startMs": 930480,
    "text": "how to juice that for revenue, how to increase customer satisfaction, how to"
  },
  {
    "startMs": 931910,
    "text": "increase customer satisfaction, how to"
  },
  {
    "startMs": 931920,
    "text": "increase customer satisfaction, how to expand into a neighboring market. So,"
  },
  {
    "startMs": 933509,
    "text": "expand into a neighboring market. So,"
  },
  {
    "startMs": 933519,
    "text": "expand into a neighboring market. So, this is a major project. Um, thinking"
  },
  {
    "startMs": 936870,
    "text": "this is a major project. Um, thinking"
  },
  {
    "startMs": 936880,
    "text": "this is a major project. Um, thinking about where to do them takes a little"
  },
  {
    "startMs": 938550,
    "text": "about where to do them takes a little"
  },
  {
    "startMs": 938560,
    "text": "about where to do them takes a little bit of of um of judgment. You want to"
  },
  {
    "startMs": 940949,
    "text": "bit of of um of judgment. You want to"
  },
  {
    "startMs": 940959,
    "text": "bit of of um of judgment. You want to pick a domain that is scaled and that"
  },
  {
    "startMs": 942949,
    "text": "pick a domain that is scaled and that"
  },
  {
    "startMs": 942959,
    "text": "pick a domain that is scaled and that already has material ROI. I would"
  },
  {
    "startMs": 944870,
    "text": "already has material ROI. I would"
  },
  {
    "startMs": 944880,
    "text": "already has material ROI. I would actually not propose building full"
  },
  {
    "startMs": 946550,
    "text": "actually not propose building full"
  },
  {
    "startMs": 946560,
    "text": "actually not propose building full automations if you're a startup, if"
  },
  {
    "startMs": 948230,
    "text": "automations if you're a startup, if"
  },
  {
    "startMs": 948240,
    "text": "automations if you're a startup, if you're in discovery, if you're looking"
  },
  {
    "startMs": 949430,
    "text": "you're in discovery, if you're looking"
  },
  {
    "startMs": 949440,
    "text": "you're in discovery, if you're looking at new and new businesses. Keep the"
  },
  {
    "startMs": 950949,
    "text": "at new and new businesses. Keep the"
  },
  {
    "startMs": 950959,
    "text": "at new and new businesses. Keep the flexibility of humans. But if you have a"
  },
  {
    "startMs": 953030,
    "text": "flexibility of humans. But if you have a"
  },
  {
    "startMs": 953040,
    "text": "flexibility of humans. But if you have a bucket of tasks that's associated with"
  },
  {
    "startMs": 954790,
    "text": "bucket of tasks that's associated with"
  },
  {
    "startMs": 954800,
    "text": "bucket of tasks that's associated with seven figures of revenue and that maybe"
  },
  {
    "startMs": 956790,
    "text": "seven figures of revenue and that maybe"
  },
  {
    "startMs": 956800,
    "text": "seven figures of revenue and that maybe there's a customer ticket, hopefully"
  },
  {
    "startMs": 958310,
    "text": "there's a customer ticket, hopefully"
  },
  {
    "startMs": 958320,
    "text": "there's a customer ticket, hopefully there's 10 customer tickets, the"
  },
  {
    "startMs": 959910,
    "text": "there's 10 customer tickets, the"
  },
  {
    "startMs": 959920,
    "text": "there's 10 customer tickets, the organization knows that there's pain"
  },
  {
    "startMs": 961269,
    "text": "organization knows that there's pain"
  },
  {
    "startMs": 961279,
    "text": "organization knows that there's pain here, that can justify an investment and"
  },
  {
    "startMs": 963910,
    "text": "here, that can justify an investment and"
  },
  {
    "startMs": 963920,
    "text": "here, that can justify an investment and that is usually something that a CFO can"
  },
  {
    "startMs": 965749,
    "text": "that is usually something that a CFO can"
  },
  {
    "startMs": 965759,
    "text": "that is usually something that a CFO can understand. You know, if we say, hey,"
  },
  {
    "startMs": 967509,
    "text": "understand. You know, if we say, hey,"
  },
  {
    "startMs": 967519,
    "text": "understand. You know, if we say, hey, this is associated with seven figures of"
  },
  {
    "startMs": 969590,
    "text": "this is associated with seven figures of"
  },
  {
    "startMs": 969600,
    "text": "this is associated with seven figures of revenue and there's five tickets."
  },
  {
    "startMs": 970949,
    "text": "revenue and there's five tickets."
  },
  {
    "startMs": 970959,
    "text": "revenue and there's five tickets. Customers don't like it or they want"
  },
  {
    "startMs": 972230,
    "text": "Customers don't like it or they want"
  },
  {
    "startMs": 972240,
    "text": "Customers don't like it or they want something else here. That's a pretty"
  },
  {
    "startMs": 973590,
    "text": "something else here. That's a pretty"
  },
  {
    "startMs": 973600,
    "text": "something else here. That's a pretty good leading signal."
  },
  {
    "startMs": 976389,
    "text": "good leading signal."
  },
  {
    "startMs": 976399,
    "text": "good leading signal. Sort of like in terms of the tasks"
  },
  {
    "startMs": 978069,
    "text": "Sort of like in terms of the tasks"
  },
  {
    "startMs": 978079,
    "text": "Sort of like in terms of the tasks themselves, look for business processes"
  },
  {
    "startMs": 980310,
    "text": "themselves, look for business processes"
  },
  {
    "startMs": 980320,
    "text": "themselves, look for business processes that are running. ideal if it has"
  },
  {
    "startMs": 982310,
    "text": "that are running. ideal if it has"
  },
  {
    "startMs": 982320,
    "text": "that are running. ideal if it has written specifications. I wouldn't worry"
  },
  {
    "startMs": 983829,
    "text": "written specifications. I wouldn't worry"
  },
  {
    "startMs": 983839,
    "text": "written specifications. I wouldn't worry too much about if the written"
  },
  {
    "startMs": 984870,
    "text": "too much about if the written"
  },
  {
    "startMs": 984880,
    "text": "too much about if the written specifications are any good."
  },
  {
    "startMs": 986710,
    "text": "specifications are any good."
  },
  {
    "startMs": 986720,
    "text": "specifications are any good. Specifications are really hard and I"
  },
  {
    "startMs": 988629,
    "text": "Specifications are really hard and I"
  },
  {
    "startMs": 988639,
    "text": "Specifications are really hard and I think that even if there has been"
  },
  {
    "startMs": 990389,
    "text": "think that even if there has been"
  },
  {
    "startMs": 990399,
    "text": "think that even if there has been momentum to build specifications that"
  },
  {
    "startMs": 993269,
    "text": "momentum to build specifications that"
  },
  {
    "startMs": 993279,
    "text": "momentum to build specifications that reflects that on the inside we the team"
  },
  {
    "startMs": 996069,
    "text": "reflects that on the inside we the team"
  },
  {
    "startMs": 996079,
    "text": "reflects that on the inside we the team believes that this is specifiable the"
  },
  {
    "startMs": 998310,
    "text": "believes that this is specifiable the"
  },
  {
    "startMs": 998320,
    "text": "believes that this is specifiable the team believes that this is not just raw"
  },
  {
    "startMs": 999990,
    "text": "team believes that this is not just raw"
  },
  {
    "startMs": 1000000,
    "text": "team believes that this is not just raw judgment complicated detail orient"
  },
  {
    "startMs": 1005920,
    "text": "work now which is actually"
  },
  {
    "startMs": 1008310,
    "text": "work now which is actually"
  },
  {
    "startMs": 1008320,
    "text": "work now which is actually shockingly shockingly new and very very"
  },
  {
    "startMs": 1010550,
    "text": "shockingly shockingly new and very very"
  },
  {
    "startMs": 1010560,
    "text": "shockingly shockingly new and very very interesting. It's ideal if this is"
  },
  {
    "startMs": 1012069,
    "text": "interesting. It's ideal if this is"
  },
  {
    "startMs": 1012079,
    "text": "interesting. It's ideal if this is detail- oriented work because you can"
  },
  {
    "startMs": 1014069,
    "text": "detail- oriented work because you can"
  },
  {
    "startMs": 1014079,
    "text": "detail- oriented work because you can take this sort of expensive thought,"
  },
  {
    "startMs": 1016870,
    "text": "take this sort of expensive thought,"
  },
  {
    "startMs": 1016880,
    "text": "take this sort of expensive thought, memory, um, wrote work off of of humans"
  },
  {
    "startMs": 1020470,
    "text": "memory, um, wrote work off of of humans"
  },
  {
    "startMs": 1020480,
    "text": "memory, um, wrote work off of of humans who tend to fail at it at scale."
  },
  {
    "startMs": 1022870,
    "text": "who tend to fail at it at scale."
  },
  {
    "startMs": 1022880,
    "text": "who tend to fail at it at scale. Subjective judgment work is not. So"
  },
  {
    "startMs": 1024710,
    "text": "Subjective judgment work is not. So"
  },
  {
    "startMs": 1024720,
    "text": "Subjective judgment work is not. So that's where you have to draw the line."
  },
  {
    "startMs": 1026069,
    "text": "that's where you have to draw the line."
  },
  {
    "startMs": 1026079,
    "text": "that's where you have to draw the line. Uh, AI for like if your business is"
  },
  {
    "startMs": 1028390,
    "text": "Uh, AI for like if your business is"
  },
  {
    "startMs": 1028400,
    "text": "Uh, AI for like if your business is built around a process, AI to automate"
  },
  {
    "startMs": 1030870,
    "text": "built around a process, AI to automate"
  },
  {
    "startMs": 1030880,
    "text": "built around a process, AI to automate that specific differentiated process is"
  },
  {
    "startMs": 1032870,
    "text": "that specific differentiated process is"
  },
  {
    "startMs": 1032880,
    "text": "that specific differentiated process is still in my opinion um, it's not where"
  },
  {
    "startMs": 1035110,
    "text": "still in my opinion um, it's not where"
  },
  {
    "startMs": 1035120,
    "text": "still in my opinion um, it's not where the action is."
  },
  {
    "startMs": 1037029,
    "text": "the action is."
  },
  {
    "startMs": 1037039,
    "text": "the action is. So a bag of tricks here. I'm not going"
  },
  {
    "startMs": 1038630,
    "text": "So a bag of tricks here. I'm not going"
  },
  {
    "startMs": 1038640,
    "text": "So a bag of tricks here. I'm not going to get into this, but if you are working"
  },
  {
    "startMs": 1040150,
    "text": "to get into this, but if you are working"
  },
  {
    "startMs": 1040160,
    "text": "to get into this, but if you are working on something where you're going from 95"
  },
  {
    "startMs": 1041829,
    "text": "on something where you're going from 95"
  },
  {
    "startMs": 1041839,
    "text": "on something where you're going from 95 to 99.95%,"
  },
  {
    "startMs": 1043590,
    "text": "to 99.95%,"
  },
  {
    "startMs": 1043600,
    "text": "to 99.95%, here's a bag of tricks. This talk will"
  },
  {
    "startMs": 1044949,
    "text": "here's a bag of tricks. This talk will"
  },
  {
    "startMs": 1044959,
    "text": "here's a bag of tricks. This talk will be online. You can take a photo of this,"
  },
  {
    "startMs": 1046390,
    "text": "be online. You can take a photo of this,"
  },
  {
    "startMs": 1046400,
    "text": "be online. You can take a photo of this, contact me. I'd love to chat. I'm very"
  },
  {
    "startMs": 1048069,
    "text": "contact me. I'd love to chat. I'm very"
  },
  {
    "startMs": 1048079,
    "text": "contact me. I'd love to chat. I'm very interested in the ways that models fail."
  },
  {
    "startMs": 1049990,
    "text": "interested in the ways that models fail."
  },
  {
    "startMs": 1050000,
    "text": "interested in the ways that models fail. I feel like knowing where models fail is"
  },
  {
    "startMs": 1051669,
    "text": "I feel like knowing where models fail is"
  },
  {
    "startMs": 1051679,
    "text": "I feel like knowing where models fail is really interesting for what to build"
  },
  {
    "startMs": 1053270,
    "text": "really interesting for what to build"
  },
  {
    "startMs": 1053280,
    "text": "really interesting for what to build next. Um, but I'd be I'd love to chat"
  },
  {
    "startMs": 1055270,
    "text": "next. Um, but I'd be I'd love to chat"
  },
  {
    "startMs": 1055280,
    "text": "next. Um, but I'd be I'd love to chat about any places where you have clean"
  },
  {
    "startMs": 1057029,
    "text": "about any places where you have clean"
  },
  {
    "startMs": 1057039,
    "text": "about any places where you have clean input, uh, a sort of specified process,"
  },
  {
    "startMs": 1059669,
    "text": "input, uh, a sort of specified process,"
  },
  {
    "startMs": 1059679,
    "text": "input, uh, a sort of specified process, and you can't get to three and a half"
  },
  {
    "startMs": 1061110,
    "text": "and you can't get to three and a half"
  },
  {
    "startMs": 1061120,
    "text": "and you can't get to three and a half nights of accuracy. I think the two that"
  },
  {
    "startMs": 1063190,
    "text": "nights of accuracy. I think the two that"
  },
  {
    "startMs": 1063200,
    "text": "nights of accuracy. I think the two that I'll flag out of this is the top one"
  },
  {
    "startMs": 1064710,
    "text": "I'll flag out of this is the top one"
  },
  {
    "startMs": 1064720,
    "text": "I'll flag out of this is the top one which is read and write evals and the"
  },
  {
    "startMs": 1066470,
    "text": "which is read and write evals and the"
  },
  {
    "startMs": 1066480,
    "text": "which is read and write evals and the bottom one which is build a thousand"
  },
  {
    "startMs": 1067830,
    "text": "bottom one which is build a thousand"
  },
  {
    "startMs": 1067840,
    "text": "bottom one which is build a thousand evals and expect to spend time reading"
  },
  {
    "startMs": 1069510,
    "text": "evals and expect to spend time reading"
  },
  {
    "startMs": 1069520,
    "text": "evals and expect to spend time reading them. Um that was really one of the big"
  },
  {
    "startMs": 1072230,
    "text": "them. Um that was really one of the big"
  },
  {
    "startMs": 1072240,
    "text": "them. Um that was really one of the big learnings from my team is is everybody"
  },
  {
    "startMs": 1074310,
    "text": "learnings from my team is is everybody"
  },
  {
    "startMs": 1074320,
    "text": "learnings from my team is is everybody had to invest in reading exactly why"
  },
  {
    "startMs": 1076630,
    "text": "had to invest in reading exactly why"
  },
  {
    "startMs": 1076640,
    "text": "had to invest in reading exactly why this time the AI did not do the thing we"
  },
  {
    "startMs": 1079190,
    "text": "this time the AI did not do the thing we"
  },
  {
    "startMs": 1079200,
    "text": "this time the AI did not do the thing we expected on the emails and it's"
  },
  {
    "startMs": 1080470,
    "text": "expected on the emails and it's"
  },
  {
    "startMs": 1080480,
    "text": "expected on the emails and it's exhausting work because you're reading"
  },
  {
    "startMs": 1081990,
    "text": "exhausting work because you're reading"
  },
  {
    "startMs": 1082000,
    "text": "exhausting work because you're reading psychopantic text and you're trying to"
  },
  {
    "startMs": 1083750,
    "text": "psychopantic text and you're trying to"
  },
  {
    "startMs": 1083760,
    "text": "psychopantic text and you're trying to figure out what specifically went wrong"
  },
  {
    "startMs": 1086470,
    "text": "figure out what specifically went wrong"
  },
  {
    "startMs": 1086480,
    "text": "figure out what specifically went wrong or against my expectations."
  },
  {
    "startMs": 1088710,
    "text": "or against my expectations."
  },
  {
    "startMs": 1088720,
    "text": "or against my expectations. I'm close with this quote. This is one"
  },
  {
    "startMs": 1090390,
    "text": "I'm close with this quote. This is one"
  },
  {
    "startMs": 1090400,
    "text": "I'm close with this quote. This is one of my favorite quotes is Rockefeller uh"
  },
  {
    "startMs": 1092789,
    "text": "of my favorite quotes is Rockefeller uh"
  },
  {
    "startMs": 1092799,
    "text": "of my favorite quotes is Rockefeller uh paraphrase as soon as you can hire"
  },
  {
    "startMs": 1094630,
    "text": "paraphrase as soon as you can hire"
  },
  {
    "startMs": 1094640,
    "text": "paraphrase as soon as you can hire someone to do your job. Sit down, kick"
  },
  {
    "startMs": 1096789,
    "text": "someone to do your job. Sit down, kick"
  },
  {
    "startMs": 1096799,
    "text": "someone to do your job. Sit down, kick up your feet, think about how to make"
  },
  {
    "startMs": 1098310,
    "text": "up your feet, think about how to make"
  },
  {
    "startMs": 1098320,
    "text": "up your feet, think about how to make the company more money. What is he"
  },
  {
    "startMs": 1100390,
    "text": "the company more money. What is he"
  },
  {
    "startMs": 1100400,
    "text": "the company more money. What is he saying here? He's saying if your job is"
  },
  {
    "startMs": 1102950,
    "text": "saying here? He's saying if your job is"
  },
  {
    "startMs": 1102960,
    "text": "saying here? He's saying if your job is specifiable, get someone else to do it."
  },
  {
    "startMs": 1105510,
    "text": "specifiable, get someone else to do it."
  },
  {
    "startMs": 1105520,
    "text": "specifiable, get someone else to do it. What we want you to be doing is"
  },
  {
    "startMs": 1106789,
    "text": "What we want you to be doing is"
  },
  {
    "startMs": 1106799,
    "text": "What we want you to be doing is thinking, applying your judgment,"
  },
  {
    "startMs": 1108470,
    "text": "thinking, applying your judgment,"
  },
  {
    "startMs": 1108480,
    "text": "thinking, applying your judgment, looking for new opportunity. I think"
  },
  {
    "startMs": 1110549,
    "text": "looking for new opportunity. I think"
  },
  {
    "startMs": 1110559,
    "text": "looking for new opportunity. I think that's perennial advice. It's great"
  },
  {
    "startMs": 1111909,
    "text": "that's perennial advice. It's great"
  },
  {
    "startMs": 1111919,
    "text": "that's perennial advice. It's great advice. My hope is that AI might be able"
  },
  {
    "startMs": 1114310,
    "text": "advice. My hope is that AI might be able"
  },
  {
    "startMs": 1114320,
    "text": "advice. My hope is that AI might be able to help us do this even better. If AI"
  },
  {
    "startMs": 1116710,
    "text": "to help us do this even better. If AI"
  },
  {
    "startMs": 1116720,
    "text": "to help us do this even better. If AI can help route our attention towards the"
  },
  {
    "startMs": 1119110,
    "text": "can help route our attention towards the"
  },
  {
    "startMs": 1119120,
    "text": "can help route our attention towards the most valuable places, the most valuable"
  },
  {
    "startMs": 1120870,
    "text": "most valuable places, the most valuable"
  },
  {
    "startMs": 1120880,
    "text": "most valuable places, the most valuable opportunities, then maybe we can sit"
  },
  {
    "startMs": 1122789,
    "text": "opportunities, then maybe we can sit"
  },
  {
    "startMs": 1122799,
    "text": "opportunities, then maybe we can sit down and actually think about the most"
  },
  {
    "startMs": 1124789,
    "text": "down and actually think about the most"
  },
  {
    "startMs": 1124799,
    "text": "down and actually think about the most valuable things first. Um, my bet is"
  },
  {
    "startMs": 1128070,
    "text": "valuable things first. Um, my bet is"
  },
  {
    "startMs": 1128080,
    "text": "valuable things first. Um, my bet is that orchestration layer and and um the"
  },
  {
    "startMs": 1131590,
    "text": "that orchestration layer and and um the"
  },
  {
    "startMs": 1131600,
    "text": "that orchestration layer and and um the right harnesses, the market's going to"
  },
  {
    "startMs": 1133350,
    "text": "right harnesses, the market's going to"
  },
  {
    "startMs": 1133360,
    "text": "right harnesses, the market's going to sort that out and there's not going to"
  },
  {
    "startMs": 1134870,
    "text": "sort that out and there's not going to"
  },
  {
    "startMs": 1134880,
    "text": "sort that out and there's not going to be a huge amount of differentiated"
  },
  {
    "startMs": 1136310,
    "text": "be a huge amount of differentiated"
  },
  {
    "startMs": 1136320,
    "text": "be a huge amount of differentiated advantage in terms of picking the right"
  },
  {
    "startMs": 1138470,
    "text": "advantage in terms of picking the right"
  },
  {
    "startMs": 1138480,
    "text": "advantage in terms of picking the right AI orchestration software in 2026."
  },
  {
    "startMs": 1141590,
    "text": "AI orchestration software in 2026."
  },
  {
    "startMs": 1141600,
    "text": "AI orchestration software in 2026. My bet is that the companies that"
  },
  {
    "startMs": 1143830,
    "text": "My bet is that the companies that"
  },
  {
    "startMs": 1143840,
    "text": "My bet is that the companies that succeed the most with AI are going to be"
  },
  {
    "startMs": 1145990,
    "text": "succeed the most with AI are going to be"
  },
  {
    "startMs": 1146000,
    "text": "succeed the most with AI are going to be the ones who figure out how to best"
  },
  {
    "startMs": 1147590,
    "text": "the ones who figure out how to best"
  },
  {
    "startMs": 1147600,
    "text": "the ones who figure out how to best manage their collective attention and"
  },
  {
    "startMs": 1149190,
    "text": "manage their collective attention and"
  },
  {
    "startMs": 1149200,
    "text": "manage their collective attention and judgment because the premium on that is"
  },
  {
    "startMs": 1150950,
    "text": "judgment because the premium on that is"
  },
  {
    "startMs": 1150960,
    "text": "judgment because the premium on that is going to only go up at least for so long"
  },
  {
    "startMs": 1153590,
    "text": "going to only go up at least for so long"
  },
  {
    "startMs": 1153600,
    "text": "going to only go up at least for so long as human judgment holds a premium. Um"
  },
  {
    "startMs": 1156470,
    "text": "as human judgment holds a premium. Um"
  },
  {
    "startMs": 1156480,
    "text": "as human judgment holds a premium. Um that's it for my talk. It's exciting"
  },
  {
    "startMs": 1158230,
    "text": "that's it for my talk. It's exciting"
  },
  {
    "startMs": 1158240,
    "text": "that's it for my talk. It's exciting times everyone. Uh I hope everybody"
  },
  {
    "startMs": 1160070,
    "text": "times everyone. Uh I hope everybody"
  },
  {
    "startMs": 1160080,
    "text": "times everyone. Uh I hope everybody enjoys the conference and uh really"
  },
  {
    "startMs": 1162150,
    "text": "enjoys the conference and uh really"
  },
  {
    "startMs": 1162160,
    "text": "enjoys the conference and uh really excited about all the stuff that we're"
  },
  {
    "startMs": 1163270,
    "text": "excited about all the stuff that we're"
  },
  {
    "startMs": 1163280,
    "text": "excited about all the stuff that we're going to be building."
  },
  {
    "startMs": 1177703,
    "text": "[music]"
  }
]