[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12559,
    "text": "Uh, I'm Jason. I'm from Modal and today"
  },
  {
    "startMs": 15669,
    "text": "Uh, I'm Jason. I'm from Modal and today"
  },
  {
    "startMs": 15679,
    "text": "Uh, I'm Jason. I'm from Modal and today I'm talking about scaling agent RL"
  },
  {
    "startMs": 18390,
    "text": "I'm talking about scaling agent RL"
  },
  {
    "startMs": 18400,
    "text": "I'm talking about scaling agent RL across the globe. Um, but first I think"
  },
  {
    "startMs": 23029,
    "text": "across the globe. Um, but first I think"
  },
  {
    "startMs": 23039,
    "text": "across the globe. Um, but first I think a lot of AI engineers view models as"
  },
  {
    "startMs": 26150,
    "text": "a lot of AI engineers view models as"
  },
  {
    "startMs": 26160,
    "text": "a lot of AI engineers view models as kind of like static backends that um"
  },
  {
    "startMs": 29429,
    "text": "kind of like static backends that um"
  },
  {
    "startMs": 29439,
    "text": "kind of like static backends that um that they"
  },
  {
    "startMs": 31910,
    "text": "that they"
  },
  {
    "startMs": 31920,
    "text": "that they kind of swap out like as the new models"
  },
  {
    "startMs": 33750,
    "text": "kind of swap out like as the new models"
  },
  {
    "startMs": 33760,
    "text": "kind of swap out like as the new models come out they'll maybe uh test the"
  },
  {
    "startMs": 36950,
    "text": "come out they'll maybe uh test the"
  },
  {
    "startMs": 36960,
    "text": "come out they'll maybe uh test the models, run evals, uh do AV AB tests and"
  },
  {
    "startMs": 40869,
    "text": "models, run evals, uh do AV AB tests and"
  },
  {
    "startMs": 40879,
    "text": "models, run evals, uh do AV AB tests and kind of like rebuild their harness"
  },
  {
    "startMs": 42310,
    "text": "kind of like rebuild their harness"
  },
  {
    "startMs": 42320,
    "text": "kind of like rebuild their harness around whatever model is cheapest but"
  },
  {
    "startMs": 45430,
    "text": "around whatever model is cheapest but"
  },
  {
    "startMs": 45440,
    "text": "around whatever model is cheapest but also highest quality for for their use"
  },
  {
    "startMs": 47510,
    "text": "also highest quality for for their use"
  },
  {
    "startMs": 47520,
    "text": "also highest quality for for their use cases. Um, and the claim I want to make"
  },
  {
    "startMs": 51430,
    "text": "cases. Um, and the claim I want to make"
  },
  {
    "startMs": 51440,
    "text": "cases. Um, and the claim I want to make for this talk is that this is actually a"
  },
  {
    "startMs": 54310,
    "text": "for this talk is that this is actually a"
  },
  {
    "startMs": 54320,
    "text": "for this talk is that this is actually a losing strategy. Uh, when you end up"
  },
  {
    "startMs": 56869,
    "text": "losing strategy. Uh, when you end up"
  },
  {
    "startMs": 56879,
    "text": "losing strategy. Uh, when you end up having to compete on like not only the"
  },
  {
    "startMs": 59750,
    "text": "having to compete on like not only the"
  },
  {
    "startMs": 59760,
    "text": "having to compete on like not only the agent economics but also the experience"
  },
  {
    "startMs": 61750,
    "text": "agent economics but also the experience"
  },
  {
    "startMs": 61760,
    "text": "agent economics but also the experience of the agent and like these two"
  },
  {
    "startMs": 63830,
    "text": "of the agent and like these two"
  },
  {
    "startMs": 63840,
    "text": "of the agent and like these two competing priorities of like offering"
  },
  {
    "startMs": 66149,
    "text": "competing priorities of like offering"
  },
  {
    "startMs": 66159,
    "text": "competing priorities of like offering differentiated value. Uh, and I think"
  },
  {
    "startMs": 69590,
    "text": "differentiated value. Uh, and I think"
  },
  {
    "startMs": 69600,
    "text": "differentiated value. Uh, and I think this is what we hear from our customers"
  },
  {
    "startMs": 71590,
    "text": "this is what we hear from our customers"
  },
  {
    "startMs": 71600,
    "text": "this is what we hear from our customers who do fine-tuning. Um it's like the"
  },
  {
    "startMs": 74630,
    "text": "who do fine-tuning. Um it's like the"
  },
  {
    "startMs": 74640,
    "text": "who do fine-tuning. Um it's like the first the first area where I think um"
  },
  {
    "startMs": 77830,
    "text": "first the first area where I think um"
  },
  {
    "startMs": 77840,
    "text": "first the first area where I think um fine-tuning took off is is people who"
  },
  {
    "startMs": 80070,
    "text": "fine-tuning took off is is people who"
  },
  {
    "startMs": 80080,
    "text": "fine-tuning took off is is people who had agent deployments who um"
  },
  {
    "startMs": 84630,
    "text": "had agent deployments who um"
  },
  {
    "startMs": 84640,
    "text": "had agent deployments who um need reliable SLOs's like like they they"
  },
  {
    "startMs": 87030,
    "text": "need reliable SLOs's like like they they"
  },
  {
    "startMs": 87040,
    "text": "need reliable SLOs's like like they they need reliability SLOs's if they if they"
  },
  {
    "startMs": 89350,
    "text": "need reliability SLOs's if they if they"
  },
  {
    "startMs": 89360,
    "text": "need reliability SLOs's if they if they need those then they need to own their"
  },
  {
    "startMs": 90469,
    "text": "need those then they need to own their"
  },
  {
    "startMs": 90479,
    "text": "need those then they need to own their own inference and uh the open source"
  },
  {
    "startMs": 92710,
    "text": "own inference and uh the open source"
  },
  {
    "startMs": 92720,
    "text": "own inference and uh the open source models out of the box were not always"
  },
  {
    "startMs": 94630,
    "text": "models out of the box were not always"
  },
  {
    "startMs": 94640,
    "text": "models out of the box were not always capable of supporting their deployments"
  },
  {
    "startMs": 97590,
    "text": "capable of supporting their deployments"
  },
  {
    "startMs": 97600,
    "text": "capable of supporting their deployments uh without you know uh degrading in in"
  },
  {
    "startMs": 100149,
    "text": "uh without you know uh degrading in in"
  },
  {
    "startMs": 100159,
    "text": "uh without you know uh degrading in in quality and I think fine-tuning"
  },
  {
    "startMs": 102710,
    "text": "quality and I think fine-tuning"
  },
  {
    "startMs": 102720,
    "text": "quality and I think fine-tuning especially"
  },
  {
    "startMs": 104390,
    "text": "especially"
  },
  {
    "startMs": 104400,
    "text": "especially you know after these deployments became"
  },
  {
    "startMs": 106149,
    "text": "you know after these deployments became"
  },
  {
    "startMs": 106159,
    "text": "you know after these deployments became more mature I think was the only path to"
  },
  {
    "startMs": 108789,
    "text": "more mature I think was the only path to"
  },
  {
    "startMs": 108799,
    "text": "more mature I think was the only path to owning your own inference uh but more"
  },
  {
    "startMs": 111030,
    "text": "owning your own inference uh but more"
  },
  {
    "startMs": 111040,
    "text": "owning your own inference uh but more importantly kind of like pushing"
  },
  {
    "startMs": 112630,
    "text": "importantly kind of like pushing"
  },
  {
    "startMs": 112640,
    "text": "importantly kind of like pushing yourself up this curve in terms of"
  },
  {
    "startMs": 115030,
    "text": "yourself up this curve in terms of"
  },
  {
    "startMs": 115040,
    "text": "yourself up this curve in terms of latency quality"
  },
  {
    "startMs": 116581,
    "text": "latency quality"
  },
  {
    "startMs": 116591,
    "text": "latency quality [clears throat]"
  },
  {
    "startMs": 116710,
    "text": "[clears throat]"
  },
  {
    "startMs": 116720,
    "text": "[clears throat] uh and cost al together like if you if"
  },
  {
    "startMs": 119030,
    "text": "uh and cost al together like if you if"
  },
  {
    "startMs": 119040,
    "text": "uh and cost al together like if you if you do fine tuning you like own all"
  },
  {
    "startMs": 120550,
    "text": "you do fine tuning you like own all"
  },
  {
    "startMs": 120560,
    "text": "you do fine tuning you like own all these yourself kind of um there were"
  },
  {
    "startMs": 123590,
    "text": "these yourself kind of um there were"
  },
  {
    "startMs": 123600,
    "text": "these yourself kind of um there were other use cases though I think one we"
  },
  {
    "startMs": 125429,
    "text": "other use cases though I think one we"
  },
  {
    "startMs": 125439,
    "text": "other use cases though I think one we heard which was interesting was uh"
  },
  {
    "startMs": 128070,
    "text": "heard which was interesting was uh"
  },
  {
    "startMs": 128080,
    "text": "heard which was interesting was uh proprietary data sets that their agents"
  },
  {
    "startMs": 130869,
    "text": "proprietary data sets that their agents"
  },
  {
    "startMs": 130879,
    "text": "proprietary data sets that their agents actually needed to reason about and"
  },
  {
    "startMs": 132309,
    "text": "actually needed to reason about and"
  },
  {
    "startMs": 132319,
    "text": "actually needed to reason about and understand and take action on. But uh"
  },
  {
    "startMs": 135030,
    "text": "understand and take action on. But uh"
  },
  {
    "startMs": 135040,
    "text": "understand and take action on. But uh these these data sets were like not"
  },
  {
    "startMs": 136790,
    "text": "these these data sets were like not"
  },
  {
    "startMs": 136800,
    "text": "these these data sets were like not present in any pre-training set. And so"
  },
  {
    "startMs": 139830,
    "text": "present in any pre-training set. And so"
  },
  {
    "startMs": 139840,
    "text": "present in any pre-training set. And so there like out of the box sometimes they"
  },
  {
    "startMs": 142070,
    "text": "there like out of the box sometimes they"
  },
  {
    "startMs": 142080,
    "text": "there like out of the box sometimes they they would build agents on that had to"
  },
  {
    "startMs": 144470,
    "text": "they would build agents on that had to"
  },
  {
    "startMs": 144480,
    "text": "they would build agents on that had to deal with this type of data. And Claude"
  },
  {
    "startMs": 147670,
    "text": "deal with this type of data. And Claude"
  },
  {
    "startMs": 147680,
    "text": "deal with this type of data. And Claude Gbt just like would not work at all uh"
  },
  {
    "startMs": 150150,
    "text": "Gbt just like would not work at all uh"
  },
  {
    "startMs": 150160,
    "text": "Gbt just like would not work at all uh for these tasks. And so they actually"
  },
  {
    "startMs": 152790,
    "text": "for these tasks. And so they actually"
  },
  {
    "startMs": 152800,
    "text": "for these tasks. And so they actually just had no other alternative but to"
  },
  {
    "startMs": 155270,
    "text": "just had no other alternative but to"
  },
  {
    "startMs": 155280,
    "text": "just had no other alternative but to fine-tune open source models."
  },
  {
    "startMs": 158309,
    "text": "fine-tune open source models."
  },
  {
    "startMs": 158319,
    "text": "fine-tune open source models. And lastly, I think like there's an"
  },
  {
    "startMs": 160309,
    "text": "And lastly, I think like there's an"
  },
  {
    "startMs": 160319,
    "text": "And lastly, I think like there's an emerging attitude around fine-tuning,"
  },
  {
    "startMs": 162390,
    "text": "emerging attitude around fine-tuning,"
  },
  {
    "startMs": 162400,
    "text": "emerging attitude around fine-tuning, which is um in order to build"
  },
  {
    "startMs": 165670,
    "text": "which is um in order to build"
  },
  {
    "startMs": 165680,
    "text": "which is um in order to build differentiated value with agents, uh"
  },
  {
    "startMs": 168390,
    "text": "differentiated value with agents, uh"
  },
  {
    "startMs": 168400,
    "text": "differentiated value with agents, uh like more and more people are deciding"
  },
  {
    "startMs": 170070,
    "text": "like more and more people are deciding"
  },
  {
    "startMs": 170080,
    "text": "like more and more people are deciding to vertically integrate across all the"
  },
  {
    "startMs": 172949,
    "text": "to vertically integrate across all the"
  },
  {
    "startMs": 172959,
    "text": "to vertically integrate across all the different levels of the stack. Um these"
  },
  {
    "startMs": 175270,
    "text": "different levels of the stack. Um these"
  },
  {
    "startMs": 175280,
    "text": "different levels of the stack. Um these are this is most common in industries"
  },
  {
    "startMs": 177350,
    "text": "are this is most common in industries"
  },
  {
    "startMs": 177360,
    "text": "are this is most common in industries where uh people are already competing on"
  },
  {
    "startMs": 180150,
    "text": "where uh people are already competing on"
  },
  {
    "startMs": 180160,
    "text": "where uh people are already competing on like some metric of task intelligence"
  },
  {
    "startMs": 183110,
    "text": "like some metric of task intelligence"
  },
  {
    "startMs": 183120,
    "text": "like some metric of task intelligence per dollar uh like coding. But I think"
  },
  {
    "startMs": 186630,
    "text": "per dollar uh like coding. But I think"
  },
  {
    "startMs": 186640,
    "text": "per dollar uh like coding. But I think um what we should see as the industry"
  },
  {
    "startMs": 189270,
    "text": "um what we should see as the industry"
  },
  {
    "startMs": 189280,
    "text": "um what we should see as the industry matures outside of just coding is like"
  },
  {
    "startMs": 192470,
    "text": "matures outside of just coding is like"
  },
  {
    "startMs": 192480,
    "text": "matures outside of just coding is like we should see this trend kind of"
  },
  {
    "startMs": 194070,
    "text": "we should see this trend kind of"
  },
  {
    "startMs": 194080,
    "text": "we should see this trend kind of continue elsewhere. Um so I think that's"
  },
  {
    "startMs": 197910,
    "text": "continue elsewhere. Um so I think that's"
  },
  {
    "startMs": 197920,
    "text": "continue elsewhere. Um so I think that's that's why I think a lot of engineers"
  },
  {
    "startMs": 199750,
    "text": "that's why I think a lot of engineers"
  },
  {
    "startMs": 199760,
    "text": "that's why I think a lot of engineers have not really been fully bought in"
  },
  {
    "startMs": 201350,
    "text": "have not really been fully bought in"
  },
  {
    "startMs": 201360,
    "text": "have not really been fully bought in because the the latest model comes out"
  },
  {
    "startMs": 203030,
    "text": "because the the latest model comes out"
  },
  {
    "startMs": 203040,
    "text": "because the the latest model comes out and it's better than any model they"
  },
  {
    "startMs": 204390,
    "text": "and it's better than any model they"
  },
  {
    "startMs": 204400,
    "text": "and it's better than any model they could fine-tune from the last"
  },
  {
    "startMs": 205670,
    "text": "could fine-tune from the last"
  },
  {
    "startMs": 205680,
    "text": "could fine-tune from the last generation. But I think there's like a"
  },
  {
    "startMs": 207350,
    "text": "generation. But I think there's like a"
  },
  {
    "startMs": 207360,
    "text": "generation. But I think there's like a lot of different reasons people do"
  },
  {
    "startMs": 208390,
    "text": "lot of different reasons people do"
  },
  {
    "startMs": 208400,
    "text": "lot of different reasons people do finetuning and in the long haul I think"
  },
  {
    "startMs": 211030,
    "text": "finetuning and in the long haul I think"
  },
  {
    "startMs": 211040,
    "text": "finetuning and in the long haul I think uh yeah this this is only going to"
  },
  {
    "startMs": 213110,
    "text": "uh yeah this this is only going to"
  },
  {
    "startMs": 213120,
    "text": "uh yeah this this is only going to become more common."
  },
  {
    "startMs": 215430,
    "text": "become more common."
  },
  {
    "startMs": 215440,
    "text": "become more common. So you may have heard of fine-tuning in"
  },
  {
    "startMs": 217110,
    "text": "So you may have heard of fine-tuning in"
  },
  {
    "startMs": 217120,
    "text": "So you may have heard of fine-tuning in the context of RL or SFT. And the"
  },
  {
    "startMs": 220390,
    "text": "the context of RL or SFT. And the"
  },
  {
    "startMs": 220400,
    "text": "the context of RL or SFT. And the question is why is this talk about RL"
  },
  {
    "startMs": 222550,
    "text": "question is why is this talk about RL"
  },
  {
    "startMs": 222560,
    "text": "question is why is this talk about RL and not SFT?"
  },
  {
    "startMs": 224470,
    "text": "and not SFT?"
  },
  {
    "startMs": 224480,
    "text": "and not SFT? Um"
  },
  {
    "startMs": 226390,
    "text": "Um"
  },
  {
    "startMs": 226400,
    "text": "Um I think the idea that I want to"
  },
  {
    "startMs": 228229,
    "text": "I think the idea that I want to"
  },
  {
    "startMs": 228239,
    "text": "I think the idea that I want to communicate here is that uh if you're if"
  },
  {
    "startMs": 230149,
    "text": "communicate here is that uh if you're if"
  },
  {
    "startMs": 230159,
    "text": "communicate here is that uh if you're if you're doing SFT you're basically"
  },
  {
    "startMs": 231670,
    "text": "you're doing SFT you're basically"
  },
  {
    "startMs": 231680,
    "text": "you're doing SFT you're basically training on traces recorded uh during"
  },
  {
    "startMs": 234550,
    "text": "training on traces recorded uh during"
  },
  {
    "startMs": 234560,
    "text": "training on traces recorded uh during production. Um but the problem is that"
  },
  {
    "startMs": 237830,
    "text": "production. Um but the problem is that"
  },
  {
    "startMs": 237840,
    "text": "production. Um but the problem is that agents in particular have this like"
  },
  {
    "startMs": 240630,
    "text": "agents in particular have this like"
  },
  {
    "startMs": 240640,
    "text": "agents in particular have this like branching structure uh based on the"
  },
  {
    "startMs": 242869,
    "text": "branching structure uh based on the"
  },
  {
    "startMs": 242879,
    "text": "branching structure uh based on the number of turns that they run. Um"
  },
  {
    "startMs": 247350,
    "text": "number of turns that they run. Um"
  },
  {
    "startMs": 247360,
    "text": "number of turns that they run. Um so in production if you train on this"
  },
  {
    "startMs": 249270,
    "text": "so in production if you train on this"
  },
  {
    "startMs": 249280,
    "text": "so in production if you train on this particular trace uh later on when you"
  },
  {
    "startMs": 251670,
    "text": "particular trace uh later on when you"
  },
  {
    "startMs": 251680,
    "text": "particular trace uh later on when you deploy that model it may see like a"
  },
  {
    "startMs": 253589,
    "text": "deploy that model it may see like a"
  },
  {
    "startMs": 253599,
    "text": "deploy that model it may see like a completely different branch in the kind"
  },
  {
    "startMs": 256390,
    "text": "completely different branch in the kind"
  },
  {
    "startMs": 256400,
    "text": "completely different branch in the kind of like session like the the tree of"
  },
  {
    "startMs": 258949,
    "text": "of like session like the the tree of"
  },
  {
    "startMs": 258959,
    "text": "of like session like the the tree of possible sessions and uh this is quite"
  },
  {
    "startMs": 261909,
    "text": "possible sessions and uh this is quite"
  },
  {
    "startMs": 261919,
    "text": "possible sessions and uh this is quite bad. SFT is quite brittle in in this"
  },
  {
    "startMs": 265189,
    "text": "bad. SFT is quite brittle in in this"
  },
  {
    "startMs": 265199,
    "text": "bad. SFT is quite brittle in in this regard. So kind of like why RL has"
  },
  {
    "startMs": 268310,
    "text": "regard. So kind of like why RL has"
  },
  {
    "startMs": 268320,
    "text": "regard. So kind of like why RL has emerged as this like dominant paradigm"
  },
  {
    "startMs": 270150,
    "text": "emerged as this like dominant paradigm"
  },
  {
    "startMs": 270160,
    "text": "emerged as this like dominant paradigm of fine-tuning is that um in RL the"
  },
  {
    "startMs": 273270,
    "text": "of fine-tuning is that um in RL the"
  },
  {
    "startMs": 273280,
    "text": "of fine-tuning is that um in RL the model actually explores all these"
  },
  {
    "startMs": 274629,
    "text": "model actually explores all these"
  },
  {
    "startMs": 274639,
    "text": "model actually explores all these branches uh during training time. At"
  },
  {
    "startMs": 277990,
    "text": "branches uh during training time. At"
  },
  {
    "startMs": 278000,
    "text": "branches uh during training time. At least that's kind of the hope and when"
  },
  {
    "startMs": 279430,
    "text": "least that's kind of the hope and when"
  },
  {
    "startMs": 279440,
    "text": "least that's kind of the hope and when people talk about RL like generalizing"
  },
  {
    "startMs": 281990,
    "text": "people talk about RL like generalizing"
  },
  {
    "startMs": 282000,
    "text": "people talk about RL like generalizing better that's often what they mean."
  },
  {
    "startMs": 286880,
    "text": "I think one reason why this trade-off"
  },
  {
    "startMs": 290629,
    "text": "I think one reason why this trade-off"
  },
  {
    "startMs": 290639,
    "text": "I think one reason why this trade-off has been hard for people is that they"
  },
  {
    "startMs": 292310,
    "text": "has been hard for people is that they"
  },
  {
    "startMs": 292320,
    "text": "has been hard for people is that they view like post- training and fine-tuning"
  },
  {
    "startMs": 294469,
    "text": "view like post- training and fine-tuning"
  },
  {
    "startMs": 294479,
    "text": "view like post- training and fine-tuning in RL as difficult. Um, but I think my"
  },
  {
    "startMs": 297510,
    "text": "in RL as difficult. Um, but I think my"
  },
  {
    "startMs": 297520,
    "text": "in RL as difficult. Um, but I think my claim here is actually that if you have"
  },
  {
    "startMs": 300550,
    "text": "claim here is actually that if you have"
  },
  {
    "startMs": 300560,
    "text": "claim here is actually that if you have an agent deployment in production, you"
  },
  {
    "startMs": 302469,
    "text": "an agent deployment in production, you"
  },
  {
    "startMs": 302479,
    "text": "an agent deployment in production, you probably already have the ingredients"
  },
  {
    "startMs": 304310,
    "text": "probably already have the ingredients"
  },
  {
    "startMs": 304320,
    "text": "probably already have the ingredients you need. Um, so if you're in"
  },
  {
    "startMs": 307270,
    "text": "you need. Um, so if you're in"
  },
  {
    "startMs": 307280,
    "text": "you need. Um, so if you're in production, you have an agent harness"
  },
  {
    "startMs": 308550,
    "text": "production, you have an agent harness"
  },
  {
    "startMs": 308560,
    "text": "production, you have an agent harness that that's deployed somewhere and"
  },
  {
    "startMs": 310550,
    "text": "that that's deployed somewhere and"
  },
  {
    "startMs": 310560,
    "text": "that that's deployed somewhere and hopefully you have eval."
  },
  {
    "startMs": 312710,
    "text": "hopefully you have eval."
  },
  {
    "startMs": 312720,
    "text": "hopefully you have eval. Maybe calling it eval is not necessarily"
  },
  {
    "startMs": 315430,
    "text": "Maybe calling it eval is not necessarily"
  },
  {
    "startMs": 315440,
    "text": "Maybe calling it eval is not necessarily required. It could be AB tests. It could"
  },
  {
    "startMs": 317270,
    "text": "required. It could be AB tests. It could"
  },
  {
    "startMs": 317280,
    "text": "required. It could be AB tests. It could be um basically any any way you do"
  },
  {
    "startMs": 319430,
    "text": "be um basically any any way you do"
  },
  {
    "startMs": 319440,
    "text": "be um basically any any way you do change management for this harness, for"
  },
  {
    "startMs": 320950,
    "text": "change management for this harness, for"
  },
  {
    "startMs": 320960,
    "text": "change management for this harness, for this deployment. Um but those two"
  },
  {
    "startMs": 323430,
    "text": "this deployment. Um but those two"
  },
  {
    "startMs": 323440,
    "text": "this deployment. Um but those two together give you the the tools you need"
  },
  {
    "startMs": 325270,
    "text": "together give you the the tools you need"
  },
  {
    "startMs": 325280,
    "text": "together give you the the tools you need to basically build out something that"
  },
  {
    "startMs": 327350,
    "text": "to basically build out something that"
  },
  {
    "startMs": 327360,
    "text": "to basically build out something that you can consider an environment, an RL"
  },
  {
    "startMs": 329029,
    "text": "you can consider an environment, an RL"
  },
  {
    "startMs": 329039,
    "text": "you can consider an environment, an RL environment. Um"
  },
  {
    "startMs": 331990,
    "text": "environment. Um"
  },
  {
    "startMs": 332000,
    "text": "environment. Um and why production traces are important"
  },
  {
    "startMs": 333670,
    "text": "and why production traces are important"
  },
  {
    "startMs": 333680,
    "text": "and why production traces are important for this is that you need to kind of get"
  },
  {
    "startMs": 336550,
    "text": "for this is that you need to kind of get"
  },
  {
    "startMs": 336560,
    "text": "for this is that you need to kind of get high quality tasks to train on in this"
  },
  {
    "startMs": 338950,
    "text": "high quality tasks to train on in this"
  },
  {
    "startMs": 338960,
    "text": "high quality tasks to train on in this environment from somewhere. And the best"
  },
  {
    "startMs": 340950,
    "text": "environment from somewhere. And the best"
  },
  {
    "startMs": 340960,
    "text": "environment from somewhere. And the best place often to do that is to"
  },
  {
    "startMs": 343189,
    "text": "place often to do that is to"
  },
  {
    "startMs": 343199,
    "text": "place often to do that is to synthetically generate these tasks from"
  },
  {
    "startMs": 345590,
    "text": "synthetically generate these tasks from"
  },
  {
    "startMs": 345600,
    "text": "synthetically generate these tasks from actual live production traces of things"
  },
  {
    "startMs": 347590,
    "text": "actual live production traces of things"
  },
  {
    "startMs": 347600,
    "text": "actual live production traces of things that your users are are doing um with"
  },
  {
    "startMs": 350790,
    "text": "that your users are are doing um with"
  },
  {
    "startMs": 350800,
    "text": "that your users are are doing um with with your product product and yeah kind"
  },
  {
    "startMs": 353749,
    "text": "with your product product and yeah kind"
  },
  {
    "startMs": 353759,
    "text": "with your product product and yeah kind of forming uh finding examples where"
  },
  {
    "startMs": 356150,
    "text": "of forming uh finding examples where"
  },
  {
    "startMs": 356160,
    "text": "of forming uh finding examples where it's failing, mining those and creating"
  },
  {
    "startMs": 358150,
    "text": "it's failing, mining those and creating"
  },
  {
    "startMs": 358160,
    "text": "it's failing, mining those and creating tasks around them so that fine-tuning"
  },
  {
    "startMs": 360469,
    "text": "tasks around them so that fine-tuning"
  },
  {
    "startMs": 360479,
    "text": "tasks around them so that fine-tuning actually gives you some chance of"
  },
  {
    "startMs": 361909,
    "text": "actually gives you some chance of"
  },
  {
    "startMs": 361919,
    "text": "actually gives you some chance of improving your product in those areas."
  },
  {
    "startMs": 365590,
    "text": "improving your product in those areas."
  },
  {
    "startMs": 365600,
    "text": "improving your product in those areas. And this is basically all you need to do"
  },
  {
    "startMs": 367270,
    "text": "And this is basically all you need to do"
  },
  {
    "startMs": 367280,
    "text": "And this is basically all you need to do RL."
  },
  {
    "startMs": 370230,
    "text": "RL."
  },
  {
    "startMs": 370240,
    "text": "RL. But then the next part part about it"
  },
  {
    "startMs": 371990,
    "text": "But then the next part part about it"
  },
  {
    "startMs": 372000,
    "text": "But then the next part part about it being hard is that like okay once just"
  },
  {
    "startMs": 373909,
    "text": "being hard is that like okay once just"
  },
  {
    "startMs": 373919,
    "text": "being hard is that like okay once just because you have an agent deployment"
  },
  {
    "startMs": 375110,
    "text": "because you have an agent deployment"
  },
  {
    "startMs": 375120,
    "text": "because you have an agent deployment doesn't mean that your team may be"
  },
  {
    "startMs": 376629,
    "text": "doesn't mean that your team may be"
  },
  {
    "startMs": 376639,
    "text": "doesn't mean that your team may be capable or maybe that's how you feel. Um"
  },
  {
    "startMs": 379590,
    "text": "capable or maybe that's how you feel. Um"
  },
  {
    "startMs": 379600,
    "text": "capable or maybe that's how you feel. Um and probably because RL has a reputation"
  },
  {
    "startMs": 382070,
    "text": "and probably because RL has a reputation"
  },
  {
    "startMs": 382080,
    "text": "and probably because RL has a reputation of just being difficult generally. Um,"
  },
  {
    "startMs": 384790,
    "text": "of just being difficult generally. Um,"
  },
  {
    "startMs": 384800,
    "text": "of just being difficult generally. Um, but I think over the last couple of at"
  },
  {
    "startMs": 387430,
    "text": "but I think over the last couple of at"
  },
  {
    "startMs": 387440,
    "text": "but I think over the last couple of at least over the last year, over the last"
  },
  {
    "startMs": 389110,
    "text": "least over the last year, over the last"
  },
  {
    "startMs": 389120,
    "text": "least over the last year, over the last several months, uh, like we have kind of"
  },
  {
    "startMs": 392950,
    "text": "several months, uh, like we have kind of"
  },
  {
    "startMs": 392960,
    "text": "several months, uh, like we have kind of figured out the science of of RL. And"
  },
  {
    "startMs": 396629,
    "text": "figured out the science of of RL. And"
  },
  {
    "startMs": 396639,
    "text": "figured out the science of of RL. And that's what I'm going to basically talk"
  },
  {
    "startMs": 398309,
    "text": "that's what I'm going to basically talk"
  },
  {
    "startMs": 398319,
    "text": "that's what I'm going to basically talk about today is that, uh, a particular RL"
  },
  {
    "startMs": 401029,
    "text": "about today is that, uh, a particular RL"
  },
  {
    "startMs": 401039,
    "text": "about today is that, uh, a particular RL recipe that we've had a lot of success"
  },
  {
    "startMs": 402790,
    "text": "recipe that we've had a lot of success"
  },
  {
    "startMs": 402800,
    "text": "recipe that we've had a lot of success with that I think simplifies and"
  },
  {
    "startMs": 405830,
    "text": "with that I think simplifies and"
  },
  {
    "startMs": 405840,
    "text": "with that I think simplifies and stabilizes a lot of what, you know, was"
  },
  {
    "startMs": 408790,
    "text": "stabilizes a lot of what, you know, was"
  },
  {
    "startMs": 408800,
    "text": "stabilizes a lot of what, you know, was difficult before."
  },
  {
    "startMs": 411590,
    "text": "difficult before."
  },
  {
    "startMs": 411600,
    "text": "difficult before. But first, okay, just to kind of recap"
  },
  {
    "startMs": 413590,
    "text": "But first, okay, just to kind of recap"
  },
  {
    "startMs": 413600,
    "text": "But first, okay, just to kind of recap how how does reinforcement learning"
  },
  {
    "startMs": 415189,
    "text": "how how does reinforcement learning"
  },
  {
    "startMs": 415199,
    "text": "how how does reinforcement learning work. Uh it's it's this loop"
  },
  {
    "startMs": 417909,
    "text": "work. Uh it's it's this loop"
  },
  {
    "startMs": 417919,
    "text": "work. Uh it's it's this loop mechanically. This is this is what's"
  },
  {
    "startMs": 419830,
    "text": "mechanically. This is this is what's"
  },
  {
    "startMs": 419840,
    "text": "mechanically. This is this is what's happening. You have a bunch of uh"
  },
  {
    "startMs": 422230,
    "text": "happening. You have a bunch of uh"
  },
  {
    "startMs": 422240,
    "text": "happening. You have a bunch of uh inference GPUs that are running your"
  },
  {
    "startMs": 424230,
    "text": "inference GPUs that are running your"
  },
  {
    "startMs": 424240,
    "text": "inference GPUs that are running your sampler aka the model that you're"
  },
  {
    "startMs": 425830,
    "text": "sampler aka the model that you're"
  },
  {
    "startMs": 425840,
    "text": "sampler aka the model that you're training. Um"
  },
  {
    "startMs": 428309,
    "text": "training. Um"
  },
  {
    "startMs": 428319,
    "text": "training. Um and those rollouts you you basically"
  },
  {
    "startMs": 430390,
    "text": "and those rollouts you you basically"
  },
  {
    "startMs": 430400,
    "text": "and those rollouts you you basically generate rollouts in your environment uh"
  },
  {
    "startMs": 432390,
    "text": "generate rollouts in your environment uh"
  },
  {
    "startMs": 432400,
    "text": "generate rollouts in your environment uh with your harness. Those rollouts get"
  },
  {
    "startMs": 434390,
    "text": "with your harness. Those rollouts get"
  },
  {
    "startMs": 434400,
    "text": "with your harness. Those rollouts get collected and fed to a trainer which is"
  },
  {
    "startMs": 437270,
    "text": "collected and fed to a trainer which is"
  },
  {
    "startMs": 437280,
    "text": "collected and fed to a trainer which is consuming all this data and updating the"
  },
  {
    "startMs": 439909,
    "text": "consuming all this data and updating the"
  },
  {
    "startMs": 439919,
    "text": "consuming all this data and updating the model uh with you know gradient descent"
  },
  {
    "startMs": 442550,
    "text": "model uh with you know gradient descent"
  },
  {
    "startMs": 442560,
    "text": "model uh with you know gradient descent and then uh that updated model you you"
  },
  {
    "startMs": 445430,
    "text": "and then uh that updated model you you"
  },
  {
    "startMs": 445440,
    "text": "and then uh that updated model you you broadcast those new weights to all the"
  },
  {
    "startMs": 447670,
    "text": "broadcast those new weights to all the"
  },
  {
    "startMs": 447680,
    "text": "broadcast those new weights to all the samplers and um over time you the"
  },
  {
    "startMs": 451029,
    "text": "samplers and um over time you the"
  },
  {
    "startMs": 451039,
    "text": "samplers and um over time you the rollouts you're generating are from this"
  },
  {
    "startMs": 452550,
    "text": "rollouts you're generating are from this"
  },
  {
    "startMs": 452560,
    "text": "rollouts you're generating are from this like continuously improving model. Uh so"
  },
  {
    "startMs": 454710,
    "text": "like continuously improving model. Uh so"
  },
  {
    "startMs": 454720,
    "text": "like continuously improving model. Uh so you're you're you're kind of like going"
  },
  {
    "startMs": 456550,
    "text": "you're you're you're kind of like going"
  },
  {
    "startMs": 456560,
    "text": "you're you're you're kind of like going back and forth between them and"
  },
  {
    "startMs": 457589,
    "text": "back and forth between them and"
  },
  {
    "startMs": 457599,
    "text": "back and forth between them and iterating and and that's how the model"
  },
  {
    "startMs": 459589,
    "text": "iterating and and that's how the model"
  },
  {
    "startMs": 459599,
    "text": "iterating and and that's how the model improves."
  },
  {
    "startMs": 461749,
    "text": "improves."
  },
  {
    "startMs": 461759,
    "text": "improves. Uh so it turns out that like the math of"
  },
  {
    "startMs": 464309,
    "text": "Uh so it turns out that like the math of"
  },
  {
    "startMs": 464319,
    "text": "Uh so it turns out that like the math of RL prefers this to be done sequentially."
  },
  {
    "startMs": 467029,
    "text": "RL prefers this to be done sequentially."
  },
  {
    "startMs": 467039,
    "text": "RL prefers this to be done sequentially. So first you do your rollouts then you"
  },
  {
    "startMs": 469670,
    "text": "So first you do your rollouts then you"
  },
  {
    "startMs": 469680,
    "text": "So first you do your rollouts then you train and update your model then you you"
  },
  {
    "startMs": 471830,
    "text": "train and update your model then you you"
  },
  {
    "startMs": 471840,
    "text": "train and update your model then you you take that model update send it back to"
  },
  {
    "startMs": 473670,
    "text": "take that model update send it back to"
  },
  {
    "startMs": 473680,
    "text": "take that model update send it back to your rollout side so that the new"
  },
  {
    "startMs": 475189,
    "text": "your rollout side so that the new"
  },
  {
    "startMs": 475199,
    "text": "your rollout side so that the new rollouts you generate are from that new"
  },
  {
    "startMs": 476550,
    "text": "rollouts you generate are from that new"
  },
  {
    "startMs": 476560,
    "text": "rollouts you generate are from that new version of the model. Uh but the problem"
  },
  {
    "startMs": 478469,
    "text": "version of the model. Uh but the problem"
  },
  {
    "startMs": 478479,
    "text": "version of the model. Uh but the problem is that this wastes a ton of idle"
  },
  {
    "startMs": 480869,
    "text": "is that this wastes a ton of idle"
  },
  {
    "startMs": 480879,
    "text": "is that this wastes a ton of idle compute. Um there are ways to like share"
  },
  {
    "startMs": 483990,
    "text": "compute. Um there are ways to like share"
  },
  {
    "startMs": 484000,
    "text": "compute. Um there are ways to like share the GPUs between the two but I think"
  },
  {
    "startMs": 486390,
    "text": "the GPUs between the two but I think"
  },
  {
    "startMs": 486400,
    "text": "the GPUs between the two but I think especially with agents. Um what's much"
  },
  {
    "startMs": 489430,
    "text": "especially with agents. Um what's much"
  },
  {
    "startMs": 489440,
    "text": "especially with agents. Um what's much better is that if you actually are"
  },
  {
    "startMs": 490710,
    "text": "better is that if you actually are"
  },
  {
    "startMs": 490720,
    "text": "better is that if you actually are continuously doing both in parallel"
  },
  {
    "startMs": 492469,
    "text": "continuously doing both in parallel"
  },
  {
    "startMs": 492479,
    "text": "continuously doing both in parallel asynchronously."
  },
  {
    "startMs": 493990,
    "text": "asynchronously."
  },
  {
    "startMs": 494000,
    "text": "asynchronously. Um this is better because you can like"
  },
  {
    "startMs": 496790,
    "text": "Um this is better because you can like"
  },
  {
    "startMs": 496800,
    "text": "Um this is better because you can like pack your GPUs, get high utilization"
  },
  {
    "startMs": 498629,
    "text": "pack your GPUs, get high utilization"
  },
  {
    "startMs": 498639,
    "text": "pack your GPUs, get high utilization from them and uh not waste money. But"
  },
  {
    "startMs": 501510,
    "text": "from them and uh not waste money. But"
  },
  {
    "startMs": 501520,
    "text": "from them and uh not waste money. But the downside is that um your rollouts"
  },
  {
    "startMs": 504230,
    "text": "the downside is that um your rollouts"
  },
  {
    "startMs": 504240,
    "text": "the downside is that um your rollouts that you generated were very likely from"
  },
  {
    "startMs": 505990,
    "text": "that you generated were very likely from"
  },
  {
    "startMs": 506000,
    "text": "that you generated were very likely from an old copy of the model. And this is"
  },
  {
    "startMs": 509350,
    "text": "an old copy of the model. And this is"
  },
  {
    "startMs": 509360,
    "text": "an old copy of the model. And this is what people refer to as staleness in"
  },
  {
    "startMs": 511189,
    "text": "what people refer to as staleness in"
  },
  {
    "startMs": 511199,
    "text": "what people refer to as staleness in asynchronous RL. Basically uh rollouts"
  },
  {
    "startMs": 514550,
    "text": "asynchronous RL. Basically uh rollouts"
  },
  {
    "startMs": 514560,
    "text": "asynchronous RL. Basically uh rollouts were gener generated by V1. uh the"
  },
  {
    "startMs": 517269,
    "text": "were gener generated by V1. uh the"
  },
  {
    "startMs": 517279,
    "text": "were gener generated by V1. uh the current model that you're training is"
  },
  {
    "startMs": 518389,
    "text": "current model that you're training is"
  },
  {
    "startMs": 518399,
    "text": "current model that you're training is V3. Are those rollouts still useful for"
  },
  {
    "startMs": 520630,
    "text": "V3. Are those rollouts still useful for"
  },
  {
    "startMs": 520640,
    "text": "V3. Are those rollouts still useful for V3 if the model's gotten better since"
  },
  {
    "startMs": 522870,
    "text": "V3 if the model's gotten better since"
  },
  {
    "startMs": 522880,
    "text": "V3 if the model's gotten better since then? Um and this is like one of the"
  },
  {
    "startMs": 525430,
    "text": "then? Um and this is like one of the"
  },
  {
    "startMs": 525440,
    "text": "then? Um and this is like one of the reasons why it was unstable. Another"
  },
  {
    "startMs": 527590,
    "text": "reasons why it was unstable. Another"
  },
  {
    "startMs": 527600,
    "text": "reasons why it was unstable. Another reason why it was unstable is actually"
  },
  {
    "startMs": 529269,
    "text": "reason why it was unstable is actually"
  },
  {
    "startMs": 529279,
    "text": "reason why it was unstable is actually purely at the systems level. There's a"
  },
  {
    "startMs": 530949,
    "text": "purely at the systems level. There's a"
  },
  {
    "startMs": 530959,
    "text": "purely at the systems level. There's a lot of details that can show up as"
  },
  {
    "startMs": 534550,
    "text": "lot of details that can show up as"
  },
  {
    "startMs": 534560,
    "text": "lot of details that can show up as differences in between what the trainer"
  },
  {
    "startMs": 537910,
    "text": "differences in between what the trainer"
  },
  {
    "startMs": 537920,
    "text": "differences in between what the trainer thinks it's training and what the"
  },
  {
    "startMs": 539590,
    "text": "thinks it's training and what the"
  },
  {
    "startMs": 539600,
    "text": "thinks it's training and what the samplers think we're training."
  },
  {
    "startMs": 541829,
    "text": "samplers think we're training."
  },
  {
    "startMs": 541839,
    "text": "samplers think we're training. uh basically at all these different"
  },
  {
    "startMs": 544150,
    "text": "uh basically at all these different"
  },
  {
    "startMs": 544160,
    "text": "uh basically at all these different levels of the stack there can be these"
  },
  {
    "startMs": 545750,
    "text": "levels of the stack there can be these"
  },
  {
    "startMs": 545760,
    "text": "levels of the stack there can be these like small differences in behavior and"
  },
  {
    "startMs": 549269,
    "text": "like small differences in behavior and"
  },
  {
    "startMs": 549279,
    "text": "like small differences in behavior and that means that like in in the math of"
  },
  {
    "startMs": 551030,
    "text": "that means that like in in the math of"
  },
  {
    "startMs": 551040,
    "text": "that means that like in in the math of RL I think the the idea is that both"
  },
  {
    "startMs": 553030,
    "text": "RL I think the the idea is that both"
  },
  {
    "startMs": 553040,
    "text": "RL I think the the idea is that both both sides of this think they're doing"
  },
  {
    "startMs": 554550,
    "text": "both sides of this think they're doing"
  },
  {
    "startMs": 554560,
    "text": "both sides of this think they're doing the same thing they think they're"
  },
  {
    "startMs": 555990,
    "text": "the same thing they think they're"
  },
  {
    "startMs": 556000,
    "text": "the same thing they think they're learning from the same data training"
  },
  {
    "startMs": 557910,
    "text": "learning from the same data training"
  },
  {
    "startMs": 557920,
    "text": "learning from the same data training like the same distribution but actually"
  },
  {
    "startMs": 559910,
    "text": "like the same distribution but actually"
  },
  {
    "startMs": 559920,
    "text": "like the same distribution but actually it's like slightly different and so the"
  },
  {
    "startMs": 562550,
    "text": "it's like slightly different and so the"
  },
  {
    "startMs": 562560,
    "text": "it's like slightly different and so the the solution that I'll present here is"
  },
  {
    "startMs": 565030,
    "text": "the solution that I'll present here is"
  },
  {
    "startMs": 565040,
    "text": "the solution that I'll present here is accounting for these differences um"
  },
  {
    "startMs": 567590,
    "text": "accounting for these differences um"
  },
  {
    "startMs": 567600,
    "text": "accounting for these differences um broadly with like kind of the simplest"
  },
  {
    "startMs": 569350,
    "text": "broadly with like kind of the simplest"
  },
  {
    "startMs": 569360,
    "text": "broadly with like kind of the simplest set of fixes that will uh correct for"
  },
  {
    "startMs": 572630,
    "text": "set of fixes that will uh correct for"
  },
  {
    "startMs": 572640,
    "text": "set of fixes that will uh correct for this mismatch without drowning out the"
  },
  {
    "startMs": 574710,
    "text": "this mismatch without drowning out the"
  },
  {
    "startMs": 574720,
    "text": "this mismatch without drowning out the signal of the the training loop."
  },
  {
    "startMs": 578870,
    "text": "signal of the the training loop."
  },
  {
    "startMs": 578880,
    "text": "signal of the the training loop. So yeah, what's the recipe? Here it is."
  },
  {
    "startMs": 580949,
    "text": "So yeah, what's the recipe? Here it is."
  },
  {
    "startMs": 580959,
    "text": "So yeah, what's the recipe? Here it is. Uh we start from gRPO. This is uh what"
  },
  {
    "startMs": 583430,
    "text": "Uh we start from gRPO. This is uh what"
  },
  {
    "startMs": 583440,
    "text": "Uh we start from gRPO. This is uh what DeepS introduced for Deepseek R1 and"
  },
  {
    "startMs": 586310,
    "text": "DeepS introduced for Deepseek R1 and"
  },
  {
    "startMs": 586320,
    "text": "DeepS introduced for Deepseek R1 and since then has become kind of a"
  },
  {
    "startMs": 587509,
    "text": "since then has become kind of a"
  },
  {
    "startMs": 587519,
    "text": "since then has become kind of a cornerstone of of doing RL on language"
  },
  {
    "startMs": 590550,
    "text": "cornerstone of of doing RL on language"
  },
  {
    "startMs": 590560,
    "text": "cornerstone of of doing RL on language models. Uh we add something called mass"
  },
  {
    "startMs": 593350,
    "text": "models. Uh we add something called mass"
  },
  {
    "startMs": 593360,
    "text": "models. Uh we add something called mass importance sampling. This is correcting"
  },
  {
    "startMs": 595750,
    "text": "importance sampling. This is correcting"
  },
  {
    "startMs": 595760,
    "text": "importance sampling. This is correcting for token-wise mismatch. And this"
  },
  {
    "startMs": 598230,
    "text": "for token-wise mismatch. And this"
  },
  {
    "startMs": 598240,
    "text": "for token-wise mismatch. And this mismatch is not just the mismatch from"
  },
  {
    "startMs": 600870,
    "text": "mismatch is not just the mismatch from"
  },
  {
    "startMs": 600880,
    "text": "mismatch is not just the mismatch from all the systems level stuff that I"
  },
  {
    "startMs": 602310,
    "text": "all the systems level stuff that I"
  },
  {
    "startMs": 602320,
    "text": "all the systems level stuff that I pointed out. Uh it's also from the"
  },
  {
    "startMs": 603910,
    "text": "pointed out. Uh it's also from the"
  },
  {
    "startMs": 603920,
    "text": "pointed out. Uh it's also from the staleness as well. So if an old version"
  },
  {
    "startMs": 606870,
    "text": "staleness as well. So if an old version"
  },
  {
    "startMs": 606880,
    "text": "staleness as well. So if an old version of the model would have sampled a"
  },
  {
    "startMs": 608230,
    "text": "of the model would have sampled a"
  },
  {
    "startMs": 608240,
    "text": "of the model would have sampled a different token here, uh the ratio"
  },
  {
    "startMs": 609910,
    "text": "different token here, uh the ratio"
  },
  {
    "startMs": 609920,
    "text": "different token here, uh the ratio between the two can be can be very"
  },
  {
    "startMs": 612150,
    "text": "between the two can be can be very"
  },
  {
    "startMs": 612160,
    "text": "between the two can be can be very different. And then uh this is a"
  },
  {
    "startMs": 613750,
    "text": "different. And then uh this is a"
  },
  {
    "startMs": 613760,
    "text": "different. And then uh this is a mismatch that we want to correct for"
  },
  {
    "startMs": 615509,
    "text": "mismatch that we want to correct for"
  },
  {
    "startMs": 615519,
    "text": "mismatch that we want to correct for somehow and and uh I won't go too much"
  },
  {
    "startMs": 618150,
    "text": "somehow and and uh I won't go too much"
  },
  {
    "startMs": 618160,
    "text": "somehow and and uh I won't go too much into details here about how these"
  },
  {
    "startMs": 619910,
    "text": "into details here about how these"
  },
  {
    "startMs": 619920,
    "text": "into details here about how these mechanisms work, but uh I will you know"
  },
  {
    "startMs": 622389,
    "text": "mechanisms work, but uh I will you know"
  },
  {
    "startMs": 622399,
    "text": "mechanisms work, but uh I will you know reference them. Um and then the second"
  },
  {
    "startMs": 625350,
    "text": "reference them. Um and then the second"
  },
  {
    "startMs": 625360,
    "text": "reference them. Um and then the second is score centering. So even though we're"
  },
  {
    "startMs": 627910,
    "text": "is score centering. So even though we're"
  },
  {
    "startMs": 627920,
    "text": "is score centering. So even though we're correcting for each token-wise mismatch,"
  },
  {
    "startMs": 630790,
    "text": "correcting for each token-wise mismatch,"
  },
  {
    "startMs": 630800,
    "text": "correcting for each token-wise mismatch, what ends up happening is over the"
  },
  {
    "startMs": 632310,
    "text": "what ends up happening is over the"
  },
  {
    "startMs": 632320,
    "text": "what ends up happening is over the course of training, whatever mismatch is"
  },
  {
    "startMs": 634790,
    "text": "course of training, whatever mismatch is"
  },
  {
    "startMs": 634800,
    "text": "course of training, whatever mismatch is still remaining accumulates and it kind"
  },
  {
    "startMs": 636710,
    "text": "still remaining accumulates and it kind"
  },
  {
    "startMs": 636720,
    "text": "still remaining accumulates and it kind of accumulates in the the weights"
  },
  {
    "startMs": 638870,
    "text": "of accumulates in the the weights"
  },
  {
    "startMs": 638880,
    "text": "of accumulates in the the weights themselves like the latest version of"
  },
  {
    "startMs": 640230,
    "text": "themselves like the latest version of"
  },
  {
    "startMs": 640240,
    "text": "themselves like the latest version of the model is slightly different than uh"
  },
  {
    "startMs": 642949,
    "text": "the model is slightly different than uh"
  },
  {
    "startMs": 642959,
    "text": "the model is slightly different than uh what it should be even though we've like"
  },
  {
    "startMs": 644710,
    "text": "what it should be even though we've like"
  },
  {
    "startMs": 644720,
    "text": "what it should be even though we've like corrected for the most obvious"
  },
  {
    "startMs": 646310,
    "text": "corrected for the most obvious"
  },
  {
    "startMs": 646320,
    "text": "corrected for the most obvious differences. Um and so this score"
  },
  {
    "startMs": 648949,
    "text": "differences. Um and so this score"
  },
  {
    "startMs": 648959,
    "text": "differences. Um and so this score centering approach is it basically"
  },
  {
    "startMs": 650790,
    "text": "centering approach is it basically"
  },
  {
    "startMs": 650800,
    "text": "centering approach is it basically removes this drift over the course of"
  },
  {
    "startMs": 652310,
    "text": "removes this drift over the course of"
  },
  {
    "startMs": 652320,
    "text": "removes this drift over the course of training uh not just at each token or at"
  },
  {
    "startMs": 655190,
    "text": "training uh not just at each token or at"
  },
  {
    "startMs": 655200,
    "text": "training uh not just at each token or at each step."
  },
  {
    "startMs": 656310,
    "text": "each step."
  },
  {
    "startMs": 656320,
    "text": "each step. um it kind of like builds up a term that"
  },
  {
    "startMs": 658389,
    "text": "um it kind of like builds up a term that"
  },
  {
    "startMs": 658399,
    "text": "um it kind of like builds up a term that accumulates with the drift and removes"
  },
  {
    "startMs": 660790,
    "text": "accumulates with the drift and removes"
  },
  {
    "startMs": 660800,
    "text": "accumulates with the drift and removes it. Um and then the last thing is top"
  },
  {
    "startMs": 663110,
    "text": "it. Um and then the last thing is top"
  },
  {
    "startMs": 663120,
    "text": "it. Um and then the last thing is top heat sampling uh with mass replay. So"
  },
  {
    "startMs": 666230,
    "text": "heat sampling uh with mass replay. So"
  },
  {
    "startMs": 666240,
    "text": "heat sampling uh with mass replay. So top p is is one of the parameters you"
  },
  {
    "startMs": 667990,
    "text": "top p is is one of the parameters you"
  },
  {
    "startMs": 668000,
    "text": "top p is is one of the parameters you use when you sample from a language"
  },
  {
    "startMs": 669110,
    "text": "use when you sample from a language"
  },
  {
    "startMs": 669120,
    "text": "use when you sample from a language model. Basically says only consider the"
  },
  {
    "startMs": 671350,
    "text": "model. Basically says only consider the"
  },
  {
    "startMs": 671360,
    "text": "model. Basically says only consider the top p of the of the distribution. P"
  },
  {
    "startMs": 674150,
    "text": "top p of the of the distribution. P"
  },
  {
    "startMs": 674160,
    "text": "top p of the of the distribution. P being like 0.98 0.95 basically only the"
  },
  {
    "startMs": 677269,
    "text": "being like 0.98 0.95 basically only the"
  },
  {
    "startMs": 677279,
    "text": "being like 0.98 0.95 basically only the tokens that have probability mass"
  },
  {
    "startMs": 679190,
    "text": "tokens that have probability mass"
  },
  {
    "startMs": 679200,
    "text": "tokens that have probability mass greater than 95 uh or greater than 5%."
  },
  {
    "startMs": 682470,
    "text": "greater than 95 uh or greater than 5%."
  },
  {
    "startMs": 682480,
    "text": "greater than 95 uh or greater than 5%. Um, and so it just removes a bunch of"
  },
  {
    "startMs": 684630,
    "text": "Um, and so it just removes a bunch of"
  },
  {
    "startMs": 684640,
    "text": "Um, and so it just removes a bunch of very rare, likely bad tokens that can"
  },
  {
    "startMs": 688150,
    "text": "very rare, likely bad tokens that can"
  },
  {
    "startMs": 688160,
    "text": "very rare, likely bad tokens that can just never be sampled from. And so if"
  },
  {
    "startMs": 690150,
    "text": "just never be sampled from. And so if"
  },
  {
    "startMs": 690160,
    "text": "just never be sampled from. And so if your sampler decides to only sample from"
  },
  {
    "startMs": 692230,
    "text": "your sampler decides to only sample from"
  },
  {
    "startMs": 692240,
    "text": "your sampler decides to only sample from these tokens, but your trainer decides"
  },
  {
    "startMs": 694710,
    "text": "these tokens, but your trainer decides"
  },
  {
    "startMs": 694720,
    "text": "these tokens, but your trainer decides unluckily to sample one of these rare"
  },
  {
    "startMs": 696389,
    "text": "unluckily to sample one of these rare"
  },
  {
    "startMs": 696399,
    "text": "unluckily to sample one of these rare tokens, this mass mismatch can be"
  },
  {
    "startMs": 698470,
    "text": "tokens, this mass mismatch can be"
  },
  {
    "startMs": 698480,
    "text": "tokens, this mass mismatch can be basically catastrophic if you don't"
  },
  {
    "startMs": 699750,
    "text": "basically catastrophic if you don't"
  },
  {
    "startMs": 699760,
    "text": "basically catastrophic if you don't correct for it. So we correct for it. Um"
  },
  {
    "startMs": 702310,
    "text": "correct for it. So we correct for it. Um"
  },
  {
    "startMs": 702320,
    "text": "correct for it. So we correct for it. Um and it turns out actually in the math of"
  },
  {
    "startMs": 703910,
    "text": "and it turns out actually in the math of"
  },
  {
    "startMs": 703920,
    "text": "and it turns out actually in the math of score centering that if you're doing top"
  },
  {
    "startMs": 705990,
    "text": "score centering that if you're doing top"
  },
  {
    "startMs": 706000,
    "text": "score centering that if you're doing top sampling like this that actually the the"
  },
  {
    "startMs": 708150,
    "text": "sampling like this that actually the the"
  },
  {
    "startMs": 708160,
    "text": "sampling like this that actually the the technique we we use to do score"
  },
  {
    "startMs": 709829,
    "text": "technique we we use to do score"
  },
  {
    "startMs": 709839,
    "text": "technique we we use to do score centering uh is exact instead of"
  },
  {
    "startMs": 711829,
    "text": "centering uh is exact instead of"
  },
  {
    "startMs": 711839,
    "text": "centering uh is exact instead of approximate because if you don't do top"
  },
  {
    "startMs": 713829,
    "text": "approximate because if you don't do top"
  },
  {
    "startMs": 713839,
    "text": "approximate because if you don't do top it's actually only an approximate"
  },
  {
    "startMs": 715430,
    "text": "it's actually only an approximate"
  },
  {
    "startMs": 715440,
    "text": "it's actually only an approximate correction for drift. So all this is to"
  },
  {
    "startMs": 717430,
    "text": "correction for drift. So all this is to"
  },
  {
    "startMs": 717440,
    "text": "correction for drift. So all this is to say that uh with this recipe uh things"
  },
  {
    "startMs": 720150,
    "text": "say that uh with this recipe uh things"
  },
  {
    "startMs": 720160,
    "text": "say that uh with this recipe uh things get much much better much more stable."
  },
  {
    "startMs": 722550,
    "text": "get much much better much more stable."
  },
  {
    "startMs": 722560,
    "text": "get much much better much more stable. Um"
  },
  {
    "startMs": 724150,
    "text": "Um"
  },
  {
    "startMs": 724160,
    "text": "Um and this is this is basically just an"
  },
  {
    "startMs": 726870,
    "text": "and this is this is basically just an"
  },
  {
    "startMs": 726880,
    "text": "and this is this is basically just an ablation that we did on quen 3.635b 6"
  },
  {
    "startMs": 729110,
    "text": "ablation that we did on quen 3.635b 6"
  },
  {
    "startMs": 729120,
    "text": "ablation that we did on quen 3.635b 6 35B of thee. Um, it's trained on the"
  },
  {
    "startMs": 733509,
    "text": "35B of thee. Um, it's trained on the"
  },
  {
    "startMs": 733519,
    "text": "35B of thee. Um, it's trained on the MIMO code environments with evalu"
  },
  {
    "startMs": 738800,
    "text": "subset. Um, basically what this shows is"
  },
  {
    "startMs": 741430,
    "text": "subset. Um, basically what this shows is"
  },
  {
    "startMs": 741440,
    "text": "subset. Um, basically what this shows is an ablation of these different"
  },
  {
    "startMs": 742550,
    "text": "an ablation of these different"
  },
  {
    "startMs": 742560,
    "text": "an ablation of these different techniques uh putting them all together"
  },
  {
    "startMs": 744470,
    "text": "techniques uh putting them all together"
  },
  {
    "startMs": 744480,
    "text": "techniques uh putting them all together showing like much more stable training"
  },
  {
    "startMs": 746870,
    "text": "showing like much more stable training"
  },
  {
    "startMs": 746880,
    "text": "showing like much more stable training uh than any of uh the alternatives"
  },
  {
    "startMs": 750389,
    "text": "uh than any of uh the alternatives"
  },
  {
    "startMs": 750399,
    "text": "uh than any of uh the alternatives that we've tested here. Um, so yeah, I"
  },
  {
    "startMs": 754389,
    "text": "that we've tested here. Um, so yeah, I"
  },
  {
    "startMs": 754399,
    "text": "that we've tested here. Um, so yeah, I think the that's all well and good, but"
  },
  {
    "startMs": 756550,
    "text": "think the that's all well and good, but"
  },
  {
    "startMs": 756560,
    "text": "think the that's all well and good, but I guess the question is like how far can"
  },
  {
    "startMs": 757670,
    "text": "I guess the question is like how far can"
  },
  {
    "startMs": 757680,
    "text": "I guess the question is like how far can we push this recipe? Does it actually"
  },
  {
    "startMs": 758949,
    "text": "we push this recipe? Does it actually"
  },
  {
    "startMs": 758959,
    "text": "we push this recipe? Does it actually scale? Does it does it work for broad"
  },
  {
    "startMs": 760470,
    "text": "scale? Does it does it work for broad"
  },
  {
    "startMs": 760480,
    "text": "scale? Does it does it work for broad problems? Before I talk about that, I"
  },
  {
    "startMs": 763030,
    "text": "problems? Before I talk about that, I"
  },
  {
    "startMs": 763040,
    "text": "problems? Before I talk about that, I think I want to talk briefly about why"
  },
  {
    "startMs": 765269,
    "text": "think I want to talk briefly about why"
  },
  {
    "startMs": 765279,
    "text": "think I want to talk briefly about why we care about RL at modal and like some"
  },
  {
    "startMs": 767829,
    "text": "we care about RL at modal and like some"
  },
  {
    "startMs": 767839,
    "text": "we care about RL at modal and like some things about doing RL on modal are"
  },
  {
    "startMs": 770069,
    "text": "things about doing RL on modal are"
  },
  {
    "startMs": 770079,
    "text": "things about doing RL on modal are different. I kind of want to just like"
  },
  {
    "startMs": 771110,
    "text": "different. I kind of want to just like"
  },
  {
    "startMs": 771120,
    "text": "different. I kind of want to just like get into that a little bit first. Um, so"
  },
  {
    "startMs": 773829,
    "text": "get into that a little bit first. Um, so"
  },
  {
    "startMs": 773839,
    "text": "get into that a little bit first. Um, so yeah, first why modal for agent RL in"
  },
  {
    "startMs": 775910,
    "text": "yeah, first why modal for agent RL in"
  },
  {
    "startMs": 775920,
    "text": "yeah, first why modal for agent RL in particular? Well, we do training. We"
  },
  {
    "startMs": 777990,
    "text": "particular? Well, we do training. We"
  },
  {
    "startMs": 778000,
    "text": "particular? Well, we do training. We just released uh clusters as GA last"
  },
  {
    "startMs": 781590,
    "text": "just released uh clusters as GA last"
  },
  {
    "startMs": 781600,
    "text": "just released uh clusters as GA last week I think or week before. Um we also"
  },
  {
    "startMs": 784389,
    "text": "week I think or week before. Um we also"
  },
  {
    "startMs": 784399,
    "text": "week I think or week before. Um we also do inference really well. We have a"
  },
  {
    "startMs": 785590,
    "text": "do inference really well. We have a"
  },
  {
    "startMs": 785600,
    "text": "do inference really well. We have a bunch of endpoints. Uh we've we've done"
  },
  {
    "startMs": 787190,
    "text": "bunch of endpoints. Uh we've we've done"
  },
  {
    "startMs": 787200,
    "text": "bunch of endpoints. Uh we've we've done a a lot of the hard work around doing LM"
  },
  {
    "startMs": 789030,
    "text": "a a lot of the hard work around doing LM"
  },
  {
    "startMs": 789040,
    "text": "a a lot of the hard work around doing LM inference at scale. Um and the"
  },
  {
    "startMs": 791829,
    "text": "inference at scale. Um and the"
  },
  {
    "startMs": 791839,
    "text": "inference at scale. Um and the environments you need they run in modal"
  },
  {
    "startMs": 793190,
    "text": "environments you need they run in modal"
  },
  {
    "startMs": 793200,
    "text": "environments you need they run in modal sandboxes. So like just naively we have"
  },
  {
    "startMs": 795750,
    "text": "sandboxes. So like just naively we have"
  },
  {
    "startMs": 795760,
    "text": "sandboxes. So like just naively we have all of the kind of infrared layers like"
  },
  {
    "startMs": 798629,
    "text": "all of the kind of infrared layers like"
  },
  {
    "startMs": 798639,
    "text": "all of the kind of infrared layers like or infrop primitives that you'd want to"
  },
  {
    "startMs": 800550,
    "text": "or infrop primitives that you'd want to"
  },
  {
    "startMs": 800560,
    "text": "or infrop primitives that you'd want to be able to compose into like a full"
  },
  {
    "startMs": 802470,
    "text": "be able to compose into like a full"
  },
  {
    "startMs": 802480,
    "text": "be able to compose into like a full production agent RL uh training system."
  },
  {
    "startMs": 806870,
    "text": "production agent RL uh training system."
  },
  {
    "startMs": 806880,
    "text": "production agent RL uh training system. The problem, the only problem actually"
  },
  {
    "startMs": 809750,
    "text": "The problem, the only problem actually"
  },
  {
    "startMs": 809760,
    "text": "The problem, the only problem actually uh because all that sounds great, but"
  },
  {
    "startMs": 811350,
    "text": "uh because all that sounds great, but"
  },
  {
    "startMs": 811360,
    "text": "uh because all that sounds great, but the only problem is that our capacity"
  },
  {
    "startMs": 813110,
    "text": "the only problem is that our capacity"
  },
  {
    "startMs": 813120,
    "text": "the only problem is that our capacity model is very different compared to"
  },
  {
    "startMs": 814710,
    "text": "model is very different compared to"
  },
  {
    "startMs": 814720,
    "text": "model is very different compared to other people. Um the reason for that is"
  },
  {
    "startMs": 817269,
    "text": "other people. Um the reason for that is"
  },
  {
    "startMs": 817279,
    "text": "other people. Um the reason for that is we aggregate tons of capacity across"
  },
  {
    "startMs": 819269,
    "text": "we aggregate tons of capacity across"
  },
  {
    "startMs": 819279,
    "text": "we aggregate tons of capacity across many different providers here. There's"
  },
  {
    "startMs": 821350,
    "text": "many different providers here. There's"
  },
  {
    "startMs": 821360,
    "text": "many different providers here. There's three pictured, but in reality it's like"
  },
  {
    "startMs": 823030,
    "text": "three pictured, but in reality it's like"
  },
  {
    "startMs": 823040,
    "text": "three pictured, but in reality it's like 15 or 20 at this point. Um we we run all"
  },
  {
    "startMs": 826069,
    "text": "15 or 20 at this point. Um we we run all"
  },
  {
    "startMs": 826079,
    "text": "15 or 20 at this point. Um we we run all the hyperscalers, but we also run on a"
  },
  {
    "startMs": 827670,
    "text": "the hyperscalers, but we also run on a"
  },
  {
    "startMs": 827680,
    "text": "the hyperscalers, but we also run on a bunch of NeoClouds. Um and we handle all"
  },
  {
    "startMs": 830790,
    "text": "bunch of NeoClouds. Um and we handle all"
  },
  {
    "startMs": 830800,
    "text": "bunch of NeoClouds. Um and we handle all of the kind of like GPU health and"
  },
  {
    "startMs": 832870,
    "text": "of the kind of like GPU health and"
  },
  {
    "startMs": 832880,
    "text": "of the kind of like GPU health and reliability concerns. Uh but"
  },
  {
    "startMs": 837430,
    "text": "reliability concerns. Uh but"
  },
  {
    "startMs": 837440,
    "text": "reliability concerns. Uh but this is great but uh it's great for"
  },
  {
    "startMs": 839030,
    "text": "this is great but uh it's great for"
  },
  {
    "startMs": 839040,
    "text": "this is great but uh it's great for reliability. It's great for kind of like"
  },
  {
    "startMs": 840629,
    "text": "reliability. It's great for kind of like"
  },
  {
    "startMs": 840639,
    "text": "reliability. It's great for kind of like fast and flexible scheduling. It gets us"
  },
  {
    "startMs": 842389,
    "text": "fast and flexible scheduling. It gets us"
  },
  {
    "startMs": 842399,
    "text": "fast and flexible scheduling. It gets us around compute shortages. Uh but it also"
  },
  {
    "startMs": 845670,
    "text": "around compute shortages. Uh but it also"
  },
  {
    "startMs": 845680,
    "text": "around compute shortages. Uh but it also fragments our compute pool. So uh if you"
  },
  {
    "startMs": 848710,
    "text": "fragments our compute pool. So uh if you"
  },
  {
    "startMs": 848720,
    "text": "fragments our compute pool. So uh if you want to schedule large training"
  },
  {
    "startMs": 849910,
    "text": "want to schedule large training"
  },
  {
    "startMs": 849920,
    "text": "want to schedule large training clusters, it's actually quite difficult"
  },
  {
    "startMs": 851750,
    "text": "clusters, it's actually quite difficult"
  },
  {
    "startMs": 851760,
    "text": "clusters, it's actually quite difficult in this like multi-provider multi-reion"
  },
  {
    "startMs": 854550,
    "text": "in this like multi-provider multi-reion"
  },
  {
    "startMs": 854560,
    "text": "in this like multi-provider multi-reion fleet. Uh we've engineered it so that we"
  },
  {
    "startMs": 857350,
    "text": "fleet. Uh we've engineered it so that we"
  },
  {
    "startMs": 857360,
    "text": "fleet. Uh we've engineered it so that we can schedule clusters pretty quickly up"
  },
  {
    "startMs": 859110,
    "text": "can schedule clusters pretty quickly up"
  },
  {
    "startMs": 859120,
    "text": "can schedule clusters pretty quickly up to like four nodes or eight nodes. Uh"
  },
  {
    "startMs": 861110,
    "text": "to like four nodes or eight nodes. Uh"
  },
  {
    "startMs": 861120,
    "text": "to like four nodes or eight nodes. Uh but once you get past that to 16 nodes"
  },
  {
    "startMs": 863030,
    "text": "but once you get past that to 16 nodes"
  },
  {
    "startMs": 863040,
    "text": "but once you get past that to 16 nodes or 32 nodes uh it gets pretty painful."
  },
  {
    "startMs": 865910,
    "text": "or 32 nodes uh it gets pretty painful."
  },
  {
    "startMs": 865920,
    "text": "or 32 nodes uh it gets pretty painful. It takes a really long time to find like"
  },
  {
    "startMs": 867590,
    "text": "It takes a really long time to find like"
  },
  {
    "startMs": 867600,
    "text": "It takes a really long time to find like a contiguous block of all these nodes."
  },
  {
    "startMs": 870310,
    "text": "a contiguous block of all these nodes."
  },
  {
    "startMs": 870320,
    "text": "a contiguous block of all these nodes. Um and in RL this is particularly bad"
  },
  {
    "startMs": 873350,
    "text": "Um and in RL this is particularly bad"
  },
  {
    "startMs": 873360,
    "text": "Um and in RL this is particularly bad because in RL we generally have"
  },
  {
    "startMs": 875430,
    "text": "because in RL we generally have"
  },
  {
    "startMs": 875440,
    "text": "because in RL we generally have especially for agent RL we have like two"
  },
  {
    "startMs": 877750,
    "text": "especially for agent RL we have like two"
  },
  {
    "startMs": 877760,
    "text": "especially for agent RL we have like two two inference GPUs for every training"
  },
  {
    "startMs": 879430,
    "text": "two inference GPUs for every training"
  },
  {
    "startMs": 879440,
    "text": "two inference GPUs for every training GPU. So 2/3ish"
  },
  {
    "startMs": 881910,
    "text": "GPU. So 2/3ish"
  },
  {
    "startMs": 881920,
    "text": "GPU. So 2/3ish of your cluster is actually just stuck"
  },
  {
    "startMs": 883829,
    "text": "of your cluster is actually just stuck"
  },
  {
    "startMs": 883839,
    "text": "of your cluster is actually just stuck doing inference. Um, so if you're"
  },
  {
    "startMs": 886150,
    "text": "doing inference. Um, so if you're"
  },
  {
    "startMs": 886160,
    "text": "doing inference. Um, so if you're scheduling an eight node cluster max,"
  },
  {
    "startMs": 888150,
    "text": "scheduling an eight node cluster max,"
  },
  {
    "startMs": 888160,
    "text": "scheduling an eight node cluster max, you're probably only using two or three"
  },
  {
    "startMs": 889750,
    "text": "you're probably only using two or three"
  },
  {
    "startMs": 889760,
    "text": "you're probably only using two or three of those nodes for for the trainer side"
  },
  {
    "startMs": 891509,
    "text": "of those nodes for for the trainer side"
  },
  {
    "startMs": 891519,
    "text": "of those nodes for for the trainer side and most of it's just spent doing"
  },
  {
    "startMs": 892629,
    "text": "and most of it's just spent doing"
  },
  {
    "startMs": 892639,
    "text": "and most of it's just spent doing inference."
  },
  {
    "startMs": 894629,
    "text": "inference."
  },
  {
    "startMs": 894639,
    "text": "inference. Um, it'd be really great if we could"
  },
  {
    "startMs": 896550,
    "text": "Um, it'd be really great if we could"
  },
  {
    "startMs": 896560,
    "text": "Um, it'd be really great if we could move these inference GPUs out of the"
  },
  {
    "startMs": 898629,
    "text": "move these inference GPUs out of the"
  },
  {
    "startMs": 898639,
    "text": "move these inference GPUs out of the cluster."
  },
  {
    "startMs": 901110,
    "text": "cluster."
  },
  {
    "startMs": 901120,
    "text": "cluster. And one reason why we think we can do"
  },
  {
    "startMs": 903750,
    "text": "And one reason why we think we can do"
  },
  {
    "startMs": 903760,
    "text": "And one reason why we think we can do this is because uh during RL the weight"
  },
  {
    "startMs": 906550,
    "text": "this is because uh during RL the weight"
  },
  {
    "startMs": 906560,
    "text": "this is because uh during RL the weight updates, the actual like gradients that"
  },
  {
    "startMs": 908710,
    "text": "updates, the actual like gradients that"
  },
  {
    "startMs": 908720,
    "text": "updates, the actual like gradients that you're applying to your model end up"
  },
  {
    "startMs": 910550,
    "text": "you're applying to your model end up"
  },
  {
    "startMs": 910560,
    "text": "you're applying to your model end up being incredibly sparse. And there's uh"
  },
  {
    "startMs": 912790,
    "text": "being incredibly sparse. And there's uh"
  },
  {
    "startMs": 912800,
    "text": "being incredibly sparse. And there's uh two reasons for that. But um yeah,"
  },
  {
    "startMs": 916389,
    "text": "two reasons for that. But um yeah,"
  },
  {
    "startMs": 916399,
    "text": "two reasons for that. But um yeah, essentially if you if you take the model"
  },
  {
    "startMs": 917590,
    "text": "essentially if you if you take the model"
  },
  {
    "startMs": 917600,
    "text": "essentially if you if you take the model at step t and subtract the the new model"
  },
  {
    "startMs": 920550,
    "text": "at step t and subtract the the new model"
  },
  {
    "startMs": 920560,
    "text": "at step t and subtract the the new model t+1 from it or sorry take the model at t"
  },
  {
    "startMs": 923670,
    "text": "t+1 from it or sorry take the model at t"
  },
  {
    "startMs": 923680,
    "text": "t+1 from it or sorry take the model at t plus1 subtract t uh you get this delta."
  },
  {
    "startMs": 926069,
    "text": "plus1 subtract t uh you get this delta."
  },
  {
    "startMs": 926079,
    "text": "plus1 subtract t uh you get this delta. And it turns out that even though the"
  },
  {
    "startMs": 927430,
    "text": "And it turns out that even though the"
  },
  {
    "startMs": 927440,
    "text": "And it turns out that even though the full model weights at each step are very"
  },
  {
    "startMs": 929750,
    "text": "full model weights at each step are very"
  },
  {
    "startMs": 929760,
    "text": "full model weights at each step are very dense and very large. Uh the updates"
  },
  {
    "startMs": 932230,
    "text": "dense and very large. Uh the updates"
  },
  {
    "startMs": 932240,
    "text": "dense and very large. Uh the updates themselves are incredibly sparse. 100x"
  },
  {
    "startMs": 934550,
    "text": "themselves are incredibly sparse. 100x"
  },
  {
    "startMs": 934560,
    "text": "themselves are incredibly sparse. 100x or a thousandx as sparse."
  },
  {
    "startMs": 938069,
    "text": "or a thousandx as sparse."
  },
  {
    "startMs": 938079,
    "text": "or a thousandx as sparse. So you can actually just feed this to"
  },
  {
    "startMs": 939430,
    "text": "So you can actually just feed this to"
  },
  {
    "startMs": 939440,
    "text": "So you can actually just feed this to offtheshelf compression. And it turns"
  },
  {
    "startMs": 941269,
    "text": "offtheshelf compression. And it turns"
  },
  {
    "startMs": 941279,
    "text": "offtheshelf compression. And it turns out that the amount of data you need to"
  },
  {
    "startMs": 943269,
    "text": "out that the amount of data you need to"
  },
  {
    "startMs": 943279,
    "text": "out that the amount of data you need to communicate from the train uh yeah from"
  },
  {
    "startMs": 945829,
    "text": "communicate from the train uh yeah from"
  },
  {
    "startMs": 945839,
    "text": "communicate from the train uh yeah from the trainer to the rollout engines goes"
  },
  {
    "startMs": 948389,
    "text": "the trainer to the rollout engines goes"
  },
  {
    "startMs": 948399,
    "text": "the trainer to the rollout engines goes down by a factor of a thousand. This"
  },
  {
    "startMs": 950629,
    "text": "down by a factor of a thousand. This"
  },
  {
    "startMs": 950639,
    "text": "down by a factor of a thousand. This means that you can actually move your"
  },
  {
    "startMs": 952470,
    "text": "means that you can actually move your"
  },
  {
    "startMs": 952480,
    "text": "means that you can actually move your inference GPUs outside of the cluster."
  },
  {
    "startMs": 955670,
    "text": "inference GPUs outside of the cluster."
  },
  {
    "startMs": 955680,
    "text": "inference GPUs outside of the cluster. And so actually for the for the run that"
  },
  {
    "startMs": 957749,
    "text": "And so actually for the for the run that"
  },
  {
    "startMs": 957759,
    "text": "And so actually for the for the run that I mentioned uh of coin 3.6 we ran the"
  },
  {
    "startMs": 961509,
    "text": "I mentioned uh of coin 3.6 we ran the"
  },
  {
    "startMs": 961519,
    "text": "I mentioned uh of coin 3.6 we ran the training cluster I think it it landed"
  },
  {
    "startMs": 962949,
    "text": "training cluster I think it it landed"
  },
  {
    "startMs": 962959,
    "text": "training cluster I think it it landed somewhere in GCP but the inference GPUs"
  },
  {
    "startMs": 965269,
    "text": "somewhere in GCP but the inference GPUs"
  },
  {
    "startMs": 965279,
    "text": "somewhere in GCP but the inference GPUs actually ran all across our fleet and"
  },
  {
    "startMs": 966710,
    "text": "actually ran all across our fleet and"
  },
  {
    "startMs": 966720,
    "text": "actually ran all across our fleet and all the different hyperscalers and"
  },
  {
    "startMs": 967829,
    "text": "all the different hyperscalers and"
  },
  {
    "startMs": 967839,
    "text": "all the different hyperscalers and neoclouds that we run across. Um and it"
  },
  {
    "startMs": 970710,
    "text": "neoclouds that we run across. Um and it"
  },
  {
    "startMs": 970720,
    "text": "neoclouds that we run across. Um and it turns out with this like sparse delta"
  },
  {
    "startMs": 972230,
    "text": "turns out with this like sparse delta"
  },
  {
    "startMs": 972240,
    "text": "turns out with this like sparse delta compression technique you can do this."
  },
  {
    "startMs": 974069,
    "text": "compression technique you can do this."
  },
  {
    "startMs": 974079,
    "text": "compression technique you can do this. Uh it works well and it does not"
  },
  {
    "startMs": 976949,
    "text": "Uh it works well and it does not"
  },
  {
    "startMs": 976959,
    "text": "Uh it works well and it does not introduce like bad performance. Um"
  },
  {
    "startMs": 981269,
    "text": "introduce like bad performance. Um"
  },
  {
    "startMs": 981279,
    "text": "introduce like bad performance. Um not only that we we actually fragment"
  },
  {
    "startMs": 983110,
    "text": "not only that we we actually fragment"
  },
  {
    "startMs": 983120,
    "text": "not only that we we actually fragment our rollout fleet across different G GPU"
  },
  {
    "startMs": 985430,
    "text": "our rollout fleet across different G GPU"
  },
  {
    "startMs": 985440,
    "text": "our rollout fleet across different G GPU types. So again because the we have"
  },
  {
    "startMs": 988550,
    "text": "types. So again because the we have"
  },
  {
    "startMs": 988560,
    "text": "types. So again because the we have fragmentation due to all the different"
  },
  {
    "startMs": 989990,
    "text": "fragmentation due to all the different"
  },
  {
    "startMs": 990000,
    "text": "fragmentation due to all the different providers we run on. Uh but we also have"
  },
  {
    "startMs": 992470,
    "text": "providers we run on. Uh but we also have"
  },
  {
    "startMs": 992480,
    "text": "providers we run on. Uh but we also have fragmentation because we have a lot of"
  },
  {
    "startMs": 993829,
    "text": "fragmentation because we have a lot of"
  },
  {
    "startMs": 993839,
    "text": "fragmentation because we have a lot of different GPUs that different customers"
  },
  {
    "startMs": 995350,
    "text": "different GPUs that different customers"
  },
  {
    "startMs": 995360,
    "text": "different GPUs that different customers want to use. Uh it'd be really great if"
  },
  {
    "startMs": 997269,
    "text": "want to use. Uh it'd be really great if"
  },
  {
    "startMs": 997279,
    "text": "want to use. Uh it'd be really great if we could leverage as many of them as"
  },
  {
    "startMs": 998629,
    "text": "we could leverage as many of them as"
  },
  {
    "startMs": 998639,
    "text": "we could leverage as many of them as possible for RL runs and that's that's"
  },
  {
    "startMs": 1000550,
    "text": "possible for RL runs and that's that's"
  },
  {
    "startMs": 1000560,
    "text": "possible for RL runs and that's that's what we've done. Um each of these GPU"
  },
  {
    "startMs": 1004230,
    "text": "what we've done. Um each of these GPU"
  },
  {
    "startMs": 1004240,
    "text": "what we've done. Um each of these GPU types they they increase the mismatch"
  },
  {
    "startMs": 1006310,
    "text": "types they they increase the mismatch"
  },
  {
    "startMs": 1006320,
    "text": "types they they increase the mismatch potentially. So if the train is running"
  },
  {
    "startMs": 1007590,
    "text": "potentially. So if the train is running"
  },
  {
    "startMs": 1007600,
    "text": "potentially. So if the train is running in B200 uh but your rollouts are running"
  },
  {
    "startMs": 1009990,
    "text": "in B200 uh but your rollouts are running"
  },
  {
    "startMs": 1010000,
    "text": "in B200 uh but your rollouts are running in H100 or A100 there are hardware level"
  },
  {
    "startMs": 1012629,
    "text": "in H100 or A100 there are hardware level"
  },
  {
    "startMs": 1012639,
    "text": "in H100 or A100 there are hardware level differences in the the kernels the math"
  },
  {
    "startMs": 1014790,
    "text": "differences in the the kernels the math"
  },
  {
    "startMs": 1014800,
    "text": "differences in the the kernels the math you can run that implement the model."
  },
  {
    "startMs": 1016550,
    "text": "you can run that implement the model."
  },
  {
    "startMs": 1016560,
    "text": "you can run that implement the model. And so this like increases more and more"
  },
  {
    "startMs": 1018470,
    "text": "And so this like increases more and more"
  },
  {
    "startMs": 1018480,
    "text": "And so this like increases more and more mismatch which traditionally should"
  },
  {
    "startMs": 1020949,
    "text": "mismatch which traditionally should"
  },
  {
    "startMs": 1020959,
    "text": "mismatch which traditionally should cause your training runs to destabilize."
  },
  {
    "startMs": 1023910,
    "text": "cause your training runs to destabilize."
  },
  {
    "startMs": 1023920,
    "text": "cause your training runs to destabilize. Um this is just kind of like a graph to"
  },
  {
    "startMs": 1026630,
    "text": "Um this is just kind of like a graph to"
  },
  {
    "startMs": 1026640,
    "text": "Um this is just kind of like a graph to show uh how many requests are being"
  },
  {
    "startMs": 1028870,
    "text": "show uh how many requests are being"
  },
  {
    "startMs": 1028880,
    "text": "show uh how many requests are being processed by each GPU type in the run we"
  },
  {
    "startMs": 1030870,
    "text": "processed by each GPU type in the run we"
  },
  {
    "startMs": 1030880,
    "text": "processed by each GPU type in the run we did over the course of training."
  },
  {
    "startMs": 1035919,
    "text": "So actually the recipe that I mentioned"
  },
  {
    "startMs": 1038470,
    "text": "So actually the recipe that I mentioned"
  },
  {
    "startMs": 1038480,
    "text": "So actually the recipe that I mentioned uh is capable of handling this amount of"
  },
  {
    "startMs": 1041750,
    "text": "uh is capable of handling this amount of"
  },
  {
    "startMs": 1041760,
    "text": "uh is capable of handling this amount of mismatch that we throw at it which uh"
  },
  {
    "startMs": 1045350,
    "text": "mismatch that we throw at it which uh"
  },
  {
    "startMs": 1045360,
    "text": "mismatch that we throw at it which uh yeah I think is like kind of"
  },
  {
    "startMs": 1046390,
    "text": "yeah I think is like kind of"
  },
  {
    "startMs": 1046400,
    "text": "yeah I think is like kind of unprecedented. I think this just goes to"
  },
  {
    "startMs": 1047990,
    "text": "unprecedented. I think this just goes to"
  },
  {
    "startMs": 1048000,
    "text": "unprecedented. I think this just goes to show how much we've figured out the"
  },
  {
    "startMs": 1050230,
    "text": "show how much we've figured out the"
  },
  {
    "startMs": 1050240,
    "text": "show how much we've figured out the science of RL over the past year or two."
  },
  {
    "startMs": 1053029,
    "text": "science of RL over the past year or two."
  },
  {
    "startMs": 1053039,
    "text": "science of RL over the past year or two. Um basically this shows that you have up"
  },
  {
    "startMs": 1057669,
    "text": "Um basically this shows that you have up"
  },
  {
    "startMs": 1057679,
    "text": "Um basically this shows that you have up to like four or five times as much"
  },
  {
    "startMs": 1059590,
    "text": "to like four or five times as much"
  },
  {
    "startMs": 1059600,
    "text": "to like four or five times as much mismatch. Uh but the training reward and"
  },
  {
    "startMs": 1062549,
    "text": "mismatch. Uh but the training reward and"
  },
  {
    "startMs": 1062559,
    "text": "mismatch. Uh but the training reward and gradient norm everything is is training"
  },
  {
    "startMs": 1065029,
    "text": "gradient norm everything is is training"
  },
  {
    "startMs": 1065039,
    "text": "gradient norm everything is is training as stably as it was uh in the"
  },
  {
    "startMs": 1068549,
    "text": "as stably as it was uh in the"
  },
  {
    "startMs": 1068559,
    "text": "as stably as it was uh in the homogeneous case which is basically"
  },
  {
    "startMs": 1070390,
    "text": "homogeneous case which is basically"
  },
  {
    "startMs": 1070400,
    "text": "homogeneous case which is basically running training on B200s and inference"
  },
  {
    "startMs": 1072630,
    "text": "running training on B200s and inference"
  },
  {
    "startMs": 1072640,
    "text": "running training on B200s and inference on B200s."
  },
  {
    "startMs": 1076240,
    "text": "And kind of a bonus, like a side effect"
  },
  {
    "startMs": 1077830,
    "text": "And kind of a bonus, like a side effect"
  },
  {
    "startMs": 1077840,
    "text": "And kind of a bonus, like a side effect of this is that uh the rollouts that we"
  },
  {
    "startMs": 1080710,
    "text": "of this is that uh the rollouts that we"
  },
  {
    "startMs": 1080720,
    "text": "of this is that uh the rollouts that we run for this full training run are"
  },
  {
    "startMs": 1081990,
    "text": "run for this full training run are"
  },
  {
    "startMs": 1082000,
    "text": "run for this full training run are actually 15% cheaper. And that's because"
  },
  {
    "startMs": 1084630,
    "text": "actually 15% cheaper. And that's because"
  },
  {
    "startMs": 1084640,
    "text": "actually 15% cheaper. And that's because the the demand for the the high-end GPUs"
  },
  {
    "startMs": 1087110,
    "text": "the the demand for the the high-end GPUs"
  },
  {
    "startMs": 1087120,
    "text": "the the demand for the the high-end GPUs like B300s uh means that the price is"
  },
  {
    "startMs": 1089590,
    "text": "like B300s uh means that the price is"
  },
  {
    "startMs": 1089600,
    "text": "like B300s uh means that the price is just generally higher. But if you can"
  },
  {
    "startMs": 1091510,
    "text": "just generally higher. But if you can"
  },
  {
    "startMs": 1091520,
    "text": "just generally higher. But if you can actually run your rollouts cheaply on on"
  },
  {
    "startMs": 1093270,
    "text": "actually run your rollouts cheaply on on"
  },
  {
    "startMs": 1093280,
    "text": "actually run your rollouts cheaply on on cheaper hardware that there's less"
  },
  {
    "startMs": 1094710,
    "text": "cheaper hardware that there's less"
  },
  {
    "startMs": 1094720,
    "text": "cheaper hardware that there's less demand for like the overall cost of"
  },
  {
    "startMs": 1096549,
    "text": "demand for like the overall cost of"
  },
  {
    "startMs": 1096559,
    "text": "demand for like the overall cost of doing this run is is significantly"
  },
  {
    "startMs": 1098230,
    "text": "doing this run is is significantly"
  },
  {
    "startMs": 1098240,
    "text": "doing this run is is significantly lower. This is actually without any"
  },
  {
    "startMs": 1100230,
    "text": "lower. This is actually without any"
  },
  {
    "startMs": 1100240,
    "text": "lower. This is actually without any inference tuning for each GPU type."
  },
  {
    "startMs": 1102390,
    "text": "inference tuning for each GPU type."
  },
  {
    "startMs": 1102400,
    "text": "inference tuning for each GPU type. we're just doing like the basic"
  },
  {
    "startMs": 1103830,
    "text": "we're just doing like the basic"
  },
  {
    "startMs": 1103840,
    "text": "we're just doing like the basic off-the-shelf inference recipe for each"
  },
  {
    "startMs": 1105590,
    "text": "off-the-shelf inference recipe for each"
  },
  {
    "startMs": 1105600,
    "text": "off-the-shelf inference recipe for each GPU. If you actually decided to uh you"
  },
  {
    "startMs": 1109350,
    "text": "GPU. If you actually decided to uh you"
  },
  {
    "startMs": 1109360,
    "text": "GPU. If you actually decided to uh you know optimize each individual deployment"
  },
  {
    "startMs": 1112549,
    "text": "know optimize each individual deployment"
  },
  {
    "startMs": 1112559,
    "text": "know optimize each individual deployment for each GPU type with for example"
  },
  {
    "startMs": 1114710,
    "text": "for each GPU type with for example"
  },
  {
    "startMs": 1114720,
    "text": "for each GPU type with for example different levels of quantization in the"
  },
  {
    "startMs": 1116150,
    "text": "different levels of quantization in the"
  },
  {
    "startMs": 1116160,
    "text": "different levels of quantization in the model um you could actually probably"
  },
  {
    "startMs": 1117990,
    "text": "model um you could actually probably"
  },
  {
    "startMs": 1118000,
    "text": "model um you could actually probably make this significantly more cheap. We"
  },
  {
    "startMs": 1119830,
    "text": "make this significantly more cheap. We"
  },
  {
    "startMs": 1119840,
    "text": "make this significantly more cheap. We just uh this is not the focus of this"
  },
  {
    "startMs": 1121590,
    "text": "just uh this is not the focus of this"
  },
  {
    "startMs": 1121600,
    "text": "just uh this is not the focus of this work but uh but yeah oh I did also maybe"
  },
  {
    "startMs": 1125590,
    "text": "work but uh but yeah oh I did also maybe"
  },
  {
    "startMs": 1125600,
    "text": "work but uh but yeah oh I did also maybe I forgot to mention but in this case"
  },
  {
    "startMs": 1128870,
    "text": "I forgot to mention but in this case"
  },
  {
    "startMs": 1128880,
    "text": "I forgot to mention but in this case there are cases where like for example"
  },
  {
    "startMs": 1130630,
    "text": "there are cases where like for example"
  },
  {
    "startMs": 1130640,
    "text": "there are cases where like for example H200 we have two different pools of"
  },
  {
    "startMs": 1132070,
    "text": "H200 we have two different pools of"
  },
  {
    "startMs": 1132080,
    "text": "H200 we have two different pools of H200s that's because we run two"
  },
  {
    "startMs": 1134390,
    "text": "H200s that's because we run two"
  },
  {
    "startMs": 1134400,
    "text": "H200s that's because we run two different uh quantizations of the model"
  },
  {
    "startMs": 1136789,
    "text": "different uh quantizations of the model"
  },
  {
    "startMs": 1136799,
    "text": "different uh quantizations of the model uh on H200. So we have one H200 pool"
  },
  {
    "startMs": 1139350,
    "text": "uh on H200. So we have one H200 pool"
  },
  {
    "startMs": 1139360,
    "text": "uh on H200. So we have one H200 pool doing it at BF16 which is like the the"
  },
  {
    "startMs": 1141669,
    "text": "doing it at BF16 which is like the the"
  },
  {
    "startMs": 1141679,
    "text": "doing it at BF16 which is like the the safest way to do it but we have another"
  },
  {
    "startMs": 1143430,
    "text": "safest way to do it but we have another"
  },
  {
    "startMs": 1143440,
    "text": "safest way to do it but we have another running at FP8. Uh and again like the"
  },
  {
    "startMs": 1146070,
    "text": "running at FP8. Uh and again like the"
  },
  {
    "startMs": 1146080,
    "text": "running at FP8. Uh and again like the the correction that we do can like drown"
  },
  {
    "startMs": 1148390,
    "text": "the correction that we do can like drown"
  },
  {
    "startMs": 1148400,
    "text": "the correction that we do can like drown out the mismatch that's introduced by"
  },
  {
    "startMs": 1150549,
    "text": "out the mismatch that's introduced by"
  },
  {
    "startMs": 1150559,
    "text": "out the mismatch that's introduced by even excessive quantization of the model"
  },
  {
    "startMs": 1152710,
    "text": "even excessive quantization of the model"
  },
  {
    "startMs": 1152720,
    "text": "even excessive quantization of the model and still learn from these quantized"
  },
  {
    "startMs": 1154150,
    "text": "and still learn from these quantized"
  },
  {
    "startMs": 1154160,
    "text": "and still learn from these quantized rollouts. Um so yeah I think the the"
  },
  {
    "startMs": 1158549,
    "text": "rollouts. Um so yeah I think the the"
  },
  {
    "startMs": 1158559,
    "text": "rollouts. Um so yeah I think the the point I'm trying to make here is that um"
  },
  {
    "startMs": 1161990,
    "text": "point I'm trying to make here is that um"
  },
  {
    "startMs": 1162000,
    "text": "point I'm trying to make here is that um like we've released this there's a blog"
  },
  {
    "startMs": 1163270,
    "text": "like we've released this there's a blog"
  },
  {
    "startMs": 1163280,
    "text": "like we've released this there's a blog post that where we go into like all the"
  },
  {
    "startMs": 1164789,
    "text": "post that where we go into like all the"
  },
  {
    "startMs": 1164799,
    "text": "post that where we go into like all the technical detail. We'll be releasing the"
  },
  {
    "startMs": 1167029,
    "text": "technical detail. We'll be releasing the"
  },
  {
    "startMs": 1167039,
    "text": "technical detail. We'll be releasing the source code for it very soon. uh it's"
  },
  {
    "startMs": 1169270,
    "text": "source code for it very soon. uh it's"
  },
  {
    "startMs": 1169280,
    "text": "source code for it very soon. uh it's based on miles and silang the the"
  },
  {
    "startMs": 1171510,
    "text": "based on miles and silang the the"
  },
  {
    "startMs": 1171520,
    "text": "based on miles and silang the the reinforcement learning framework in the"
  },
  {
    "startMs": 1172789,
    "text": "reinforcement learning framework in the"
  },
  {
    "startMs": 1172799,
    "text": "reinforcement learning framework in the inference engine and um I think the the"
  },
  {
    "startMs": 1176630,
    "text": "inference engine and um I think the the"
  },
  {
    "startMs": 1176640,
    "text": "inference engine and um I think the the takeaway is that um I do believe that if"
  },
  {
    "startMs": 1179830,
    "text": "takeaway is that um I do believe that if"
  },
  {
    "startMs": 1179840,
    "text": "takeaway is that um I do believe that if you want to be competitive in a world"
  },
  {
    "startMs": 1181350,
    "text": "you want to be competitive in a world"
  },
  {
    "startMs": 1181360,
    "text": "you want to be competitive in a world where a bunch of different competitors"
  },
  {
    "startMs": 1182710,
    "text": "where a bunch of different competitors"
  },
  {
    "startMs": 1182720,
    "text": "where a bunch of different competitors are running agent stacks you do need to"
  },
  {
    "startMs": 1185270,
    "text": "are running agent stacks you do need to"
  },
  {
    "startMs": 1185280,
    "text": "are running agent stacks you do need to some degree own the full stack yourself"
  },
  {
    "startMs": 1187110,
    "text": "some degree own the full stack yourself"
  },
  {
    "startMs": 1187120,
    "text": "some degree own the full stack yourself or at least like build competency in"
  },
  {
    "startMs": 1189430,
    "text": "or at least like build competency in"
  },
  {
    "startMs": 1189440,
    "text": "or at least like build competency in every area of the stack. Um I think a"
  },
  {
    "startMs": 1192950,
    "text": "every area of the stack. Um I think a"
  },
  {
    "startMs": 1192960,
    "text": "every area of the stack. Um I think a lot of work traditionally ends at this"
  },
  {
    "startMs": 1194549,
    "text": "lot of work traditionally ends at this"
  },
  {
    "startMs": 1194559,
    "text": "lot of work traditionally ends at this agent harness but actually um the"
  },
  {
    "startMs": 1197830,
    "text": "agent harness but actually um the"
  },
  {
    "startMs": 1197840,
    "text": "agent harness but actually um the vertical integration across all of this"
  },
  {
    "startMs": 1199270,
    "text": "vertical integration across all of this"
  },
  {
    "startMs": 1199280,
    "text": "vertical integration across all of this is just extremely important in a in a"
  },
  {
    "startMs": 1201669,
    "text": "is just extremely important in a in a"
  },
  {
    "startMs": 1201679,
    "text": "is just extremely important in a in a hyperco competitive uh industry. We see"
  },
  {
    "startMs": 1204470,
    "text": "hyperco competitive uh industry. We see"
  },
  {
    "startMs": 1204480,
    "text": "hyperco competitive uh industry. We see this again in certain industries like"
  },
  {
    "startMs": 1205990,
    "text": "this again in certain industries like"
  },
  {
    "startMs": 1206000,
    "text": "this again in certain industries like coding. Um but I think it we we also see"
  },
  {
    "startMs": 1209430,
    "text": "coding. Um but I think it we we also see"
  },
  {
    "startMs": 1209440,
    "text": "coding. Um but I think it we we also see it in other industries and it's only"
  },
  {
    "startMs": 1211270,
    "text": "it in other industries and it's only"
  },
  {
    "startMs": 1211280,
    "text": "it in other industries and it's only going to keep growing I think. Um, and"
  },
  {
    "startMs": 1213510,
    "text": "going to keep growing I think. Um, and"
  },
  {
    "startMs": 1213520,
    "text": "going to keep growing I think. Um, and of course like if you're building agent"
  },
  {
    "startMs": 1215830,
    "text": "of course like if you're building agent"
  },
  {
    "startMs": 1215840,
    "text": "of course like if you're building agent deployments, you own everything up the"
  },
  {
    "startMs": 1217750,
    "text": "deployments, you own everything up the"
  },
  {
    "startMs": 1217760,
    "text": "deployments, you own everything up the stack as well. And vertically"
  },
  {
    "startMs": 1219190,
    "text": "stack as well. And vertically"
  },
  {
    "startMs": 1219200,
    "text": "stack as well. And vertically integrating across all of this is"
  },
  {
    "startMs": 1221190,
    "text": "integrating across all of this is"
  },
  {
    "startMs": 1221200,
    "text": "integrating across all of this is extremely important uh for maintaining"
  },
  {
    "startMs": 1223669,
    "text": "extremely important uh for maintaining"
  },
  {
    "startMs": 1223679,
    "text": "extremely important uh for maintaining kind of competency and and competitive"
  },
  {
    "startMs": 1225669,
    "text": "kind of competency and and competitive"
  },
  {
    "startMs": 1225679,
    "text": "kind of competency and and competitive edge. Um, yeah, you should own your own"
  },
  {
    "startMs": 1228789,
    "text": "edge. Um, yeah, you should own your own"
  },
  {
    "startMs": 1228799,
    "text": "edge. Um, yeah, you should own your own agent RL and it's actually possible now"
  },
  {
    "startMs": 1230789,
    "text": "agent RL and it's actually possible now"
  },
  {
    "startMs": 1230799,
    "text": "agent RL and it's actually possible now because the science has been figured out"
  },
  {
    "startMs": 1232630,
    "text": "because the science has been figured out"
  },
  {
    "startMs": 1232640,
    "text": "because the science has been figured out and there are very good tools and very"
  },
  {
    "startMs": 1234149,
    "text": "and there are very good tools and very"
  },
  {
    "startMs": 1234159,
    "text": "and there are very good tools and very good infrastructure that lets you do"
  },
  {
    "startMs": 1235990,
    "text": "good infrastructure that lets you do"
  },
  {
    "startMs": 1236000,
    "text": "good infrastructure that lets you do that. Um, that's all I got. So thanks."
  },
  {
    "startMs": 1240230,
    "text": "that. Um, that's all I got. So thanks."
  },
  {
    "startMs": 1240240,
    "text": "that. Um, that's all I got. So thanks. We're we're working on hard problems."
  },
  {
    "startMs": 1242230,
    "text": "We're we're working on hard problems."
  },
  {
    "startMs": 1242240,
    "text": "We're we're working on hard problems. Come visit our booth and and uh talk to"
  },
  {
    "startMs": 1244230,
    "text": "Come visit our booth and and uh talk to"
  },
  {
    "startMs": 1244240,
    "text": "Come visit our booth and and uh talk to us."
  }
]