[
  {
    "startMs": 1309,
    "text": "[music]"
  },
  {
    "startMs": 12400,
    "text": ">> My name is Claire La Gory and I'm a"
  },
  {
    "startMs": 15320,
    "text": "senior principal engineer at AWS. I"
  },
  {
    "startMs": 18040,
    "text": "mostly work on Kuro, our agent encoding"
  },
  {
    "startMs": 20920,
    "text": "assistant, but today I want to talk"
  },
  {
    "startMs": 22920,
    "text": "about some of the practices we've been"
  },
  {
    "startMs": 24640,
    "text": "seeing inside of Amazon and Amazon teams"
  },
  {
    "startMs": 27840,
    "text": "where we've been seeing really exciting"
  },
  {
    "startMs": 29800,
    "text": "results of productivity increases that"
  },
  {
    "startMs": 33080,
    "text": "are step function improvements since"
  },
  {
    "startMs": 34800,
    "text": "what what we've been seeing with AI so"
  },
  {
    "startMs": 36640,
    "text": "far."
  },
  {
    "startMs": 38560,
    "text": "So, I've been working on agentic AI for"
  },
  {
    "startMs": 42480,
    "text": "over 3 years now and I've kind of seen"
  },
  {
    "startMs": 44440,
    "text": "the evolution that's happened in our"
  },
  {
    "startMs": 46600,
    "text": "industry when it comes to coding"
  },
  {
    "startMs": 48480,
    "text": "assistance with AI. First, we had this"
  },
  {
    "startMs": 51280,
    "text": "inline code completion helping us to"
  },
  {
    "startMs": 54960,
    "text": "write the next line, maybe the next"
  },
  {
    "startMs": 56280,
    "text": "function. We moved on to chat, asking"
  },
  {
    "startMs": 59160,
    "text": "questions about our code. Everybody"
  },
  {
    "startMs": 61560,
    "text": "started doing vibe coding sometime last"
  },
  {
    "startMs": 63800,
    "text": "year, but now we're starting to see kind"
  },
  {
    "startMs": 66480,
    "text": "of an early adopter phase of what we've"
  },
  {
    "startMs": 68480,
    "text": "been calling frontier development."
  },
  {
    "startMs": 71080,
    "text": "And completely anecdotally, based on my"
  },
  {
    "startMs": 73400,
    "text": "own experience, I've really only felt"
  },
  {
    "startMs": 76480,
    "text": "maybe 10 to 20% more productive with all"
  },
  {
    "startMs": 79960,
    "text": "of these phases that have come before."
  },
  {
    "startMs": 82520,
    "text": "But now inside of Amazon, we've been"
  },
  {
    "startMs": 84240,
    "text": "running pilots with different teams"
  },
  {
    "startMs": 86040,
    "text": "across the company and we've been seeing"
  },
  {
    "startMs": 89080,
    "text": "a median of 4.5x productivity"
  },
  {
    "startMs": 91880,
    "text": "improvement and sometimes more than 10x."
  },
  {
    "startMs": 94640,
    "text": "So, something has really changed here"
  },
  {
    "startMs": 96920,
    "text": "now that we're seeing these step"
  },
  {
    "startMs": 98120,
    "text": "function improvements in productivity."
  },
  {
    "startMs": 100840,
    "text": "And I like to"
  },
  {
    "startMs": 103560,
    "text": "define what we've been calling frontier"
  },
  {
    "startMs": 105560,
    "text": "developers inside of Amazon by three"
  },
  {
    "startMs": 108160,
    "text": "behaviors that I've been seeing. One is"
  },
  {
    "startMs": 111240,
    "text": "hands-off coding. Frontier developers"
  },
  {
    "startMs": 113840,
    "text": "write maybe 1 to 2% of the code that"
  },
  {
    "startMs": 116880,
    "text": "they produce. The rest is agents."
  },
  {
    "startMs": 120000,
    "text": "The second is that they interact with"
  },
  {
    "startMs": 121680,
    "text": "their agents infrequently. They'll aim"
  },
  {
    "startMs": 124440,
    "text": "to get their coding assistant to run for"
  },
  {
    "startMs": 126760,
    "text": "up to hours at a time without their"
  },
  {
    "startMs": 129160,
    "text": "intervention."
  },
  {
    "startMs": 130600,
    "text": "And third is that they minimize idle"
  },
  {
    "startMs": 132520,
    "text": "time."
  },
  {
    "startMs": 133520,
    "text": "These frontier developers tend to run"
  },
  {
    "startMs": 135760,
    "text": "multiple agents in parallel churning"
  },
  {
    "startMs": 138600,
    "text": "through a backlog of tasks."
  },
  {
    "startMs": 142240,
    "text": "The first time that I saw a frontier"
  },
  {
    "startMs": 144520,
    "text": "developer team was the Bedrock Mantle"
  },
  {
    "startMs": 147320,
    "text": "team. Bedrock is our model hosting"
  },
  {
    "startMs": 150200,
    "text": "service."
  },
  {
    "startMs": 151560,
    "text": "Hosts LLMs like Claude and GPT. And"
  },
  {
    "startMs": 156800,
    "text": "sometime last year we knew or I say we"
  },
  {
    "startMs": 160320,
    "text": "but the Bedrock team"
  },
  {
    "startMs": 162160,
    "text": "knew that they were going to need to"
  },
  {
    "startMs": 163720,
    "text": "build a new inference data plane. But"
  },
  {
    "startMs": 166320,
    "text": "they had estimated it at 30 people over"
  },
  {
    "startMs": 170080,
    "text": "18 months. This is a big big service and"
  },
  {
    "startMs": 173640,
    "text": "it was going to take time to build the"
  },
  {
    "startMs": 175440,
    "text": "new one, migrate customers over, migrate"
  },
  {
    "startMs": 178200,
    "text": "models over. They decided to take a step"
  },
  {
    "startMs": 180959,
    "text": "back. They took six people and they"
  },
  {
    "startMs": 183320,
    "text": "built [snorts] it in 76 days with Kiro."
  },
  {
    "startMs": 186160,
    "text": "So this was a huge achievement. This was"
  },
  {
    "startMs": 188600,
    "text": "the first time we've we'd seen anything"
  },
  {
    "startMs": 190680,
    "text": "of the kind inside of Amazon. So this"
  },
  {
    "startMs": 193040,
    "text": "was truly the pathfinder team that"
  },
  {
    "startMs": 195360,
    "text": "proved that it was possible to get up to"
  },
  {
    "startMs": 198640,
    "text": "20X improvement. Now they looked at"
  },
  {
    "startMs": 201680,
    "text": "commits and I'll talk about a couple of"
  },
  {
    "startMs": 203320,
    "text": "other ways that we are uh measuring"
  },
  {
    "startMs": 205760,
    "text": "productivity improvements."
  },
  {
    "startMs": 207840,
    "text": "But there was one problem with this"
  },
  {
    "startMs": 209440,
    "text": "story which was that yes, it was built"
  },
  {
    "startMs": 212360,
    "text": "with six people. It was built with some"
  },
  {
    "startMs": 215120,
    "text": "of the top engineers literally in the"
  },
  {
    "startMs": 217519,
    "text": "company including two distinguished"
  },
  {
    "startMs": 219840,
    "text": "engineers. So this was not just any team"
  },
  {
    "startMs": 222680,
    "text": "of six people. These were experts in"
  },
  {
    "startMs": 225040,
    "text": "distributed systems, experts at LLMs and"
  },
  {
    "startMs": 228920,
    "text": "their architecture."
  },
  {
    "startMs": 231000,
    "text": "So this the story was amazing and it"
  },
  {
    "startMs": 233920,
    "text": "kind of spread like wildfire across"
  },
  {
    "startMs": 235720,
    "text": "Amazon, but it was also very"
  },
  {
    "startMs": 237640,
    "text": "unachievable for a lot of teams. There"
  },
  {
    "startMs": 239480,
    "text": "were a lot of questions about can this"
  },
  {
    "startMs": 242080,
    "text": "actually be reproduced on another team?"
  },
  {
    "startMs": 245560,
    "text": "So, another experiment that I want to"
  },
  {
    "startMs": 247120,
    "text": "talk about is an experimental sprint"
  },
  {
    "startMs": 250000,
    "text": "that was done in the Prime Video"
  },
  {
    "startMs": 251880,
    "text": "organization."
  },
  {
    "startMs": 253360,
    "text": "They took a 10-day sprint and they did"
  },
  {
    "startMs": 256079,
    "text": "an experiment where they put, again, six"
  },
  {
    "startMs": 258200,
    "text": "engineers in a room and they let them go"
  },
  {
    "startMs": 261840,
    "text": "wild with Kiro."
  },
  {
    "startMs": 263640,
    "text": "Uh they brought down the project"
  },
  {
    "startMs": 266240,
    "text": "delivery time estimate from what was"
  },
  {
    "startMs": 269280,
    "text": "going to be 90 weeks down to 24 based on"
  },
  {
    "startMs": 272400,
    "text": "all of the progress they had made in"
  },
  {
    "startMs": 274440,
    "text": "this 10-day sprint. And they they looked"
  },
  {
    "startMs": 277640,
    "text": "at their commit history and they looked"
  },
  {
    "startMs": 280040,
    "text": "at what did they used to do prior to"
  },
  {
    "startMs": 282760,
    "text": "this 10-day sprint and how many commits"
  },
  {
    "startMs": 284600,
    "text": "did they produce just in this 10 days."
  },
  {
    "startMs": 288120,
    "text": "And so, this sprint really proved that"
  },
  {
    "startMs": 290560,
    "text": "we can achieve, again, at least"
  },
  {
    "startMs": 293120,
    "text": "something close to what the Bedrock"
  },
  {
    "startMs": 294560,
    "text": "Mantle team had uh had achieved with a"
  },
  {
    "startMs": 298520,
    "text": "different set of engineers."
  },
  {
    "startMs": 300800,
    "text": "But again, there was a challenge with"
  },
  {
    "startMs": 302680,
    "text": "this story, which was it was six"
  },
  {
    "startMs": 305160,
    "text": "engineers in a room, but they had no"
  },
  {
    "startMs": 307640,
    "text": "on-call duties, limited meetings, very"
  },
  {
    "startMs": 310440,
    "text": "few distractions, which we all know are"
  },
  {
    "startMs": 313720,
    "text": "regular in the lives of an engineer."
  },
  {
    "startMs": 316840,
    "text": "And the senior engineer on the team had"
  },
  {
    "startMs": 319000,
    "text": "spent the previous 3 weeks creating very"
  },
  {
    "startMs": 322240,
    "text": "detailed, small, well-scoped tasks with"
  },
  {
    "startMs": 325360,
    "text": "detailed requirements for these"
  },
  {
    "startMs": 328120,
    "text": "six [clears throat] engineers to just go"
  },
  {
    "startMs": 329400,
    "text": "churn on for those 2 weeks."
  },
  {
    "startMs": 332000,
    "text": "So, this was again not necessarily real"
  },
  {
    "startMs": 334560,
    "text": "life. This was a structured sprint, uh a"
  },
  {
    "startMs": 337680,
    "text": "a point in time that they were able to"
  },
  {
    "startMs": 339480,
    "text": "achieve this, but again, the question is"
  },
  {
    "startMs": 342120,
    "text": "is this achievable on real teams on"
  },
  {
    "startMs": 345520,
    "text": "day-to-day"
  },
  {
    "startMs": 347040,
    "text": "work?"
  },
  {
    "startMs": 348440,
    "text": "So, Amazon stores which encompasses"
  },
  {
    "startMs": 351320,
    "text": "amazon.com, all of our retail websites,"
  },
  {
    "startMs": 354840,
    "text": "as well as our physical stores,"
  },
  {
    "startMs": 356960,
    "text": "did a more structured pilot. They"
  },
  {
    "startMs": 359360,
    "text": "watched 50 teams that were totally"
  },
  {
    "startMs": 361880,
    "text": "normal normal distribution of um early"
  },
  {
    "startMs": 366200,
    "text": "career folks, mid-career, senior"
  },
  {
    "startMs": 368280,
    "text": "engineers, and that worked on existing"
  },
  {
    "startMs": 371040,
    "text": "systems. Nothing green field like the"
  },
  {
    "startMs": 373360,
    "text": "mantle team got to build from the ground"
  },
  {
    "startMs": 375400,
    "text": "up, but existing systems with existing"
  },
  {
    "startMs": 377640,
    "text": "code bases."
  },
  {
    "startMs": 379040,
    "text": "And they they watched them for the"
  },
  {
    "startMs": 381600,
    "text": "better part of last year, and they found"
  },
  {
    "startMs": 384400,
    "text": "something super interesting."
  },
  {
    "startMs": 386800,
    "text": "They found that there was a big"
  },
  {
    "startMs": 388280,
    "text": "difference in the productivity gains"
  },
  {
    "startMs": 390120,
    "text": "that they saw between half of the teams"
  },
  {
    "startMs": 392800,
    "text": "and the other half."
  },
  {
    "startMs": 394360,
    "text": "And in this case, they used a"
  },
  {
    "startMs": 396080,
    "text": "productivity metric of deployment"
  },
  {
    "startMs": 398440,
    "text": "velocity to production. So, not just"
  },
  {
    "startMs": 400880,
    "text": "commits, how many commits are they"
  },
  {
    "startMs": 403080,
    "text": "producing, but how quickly are we"
  },
  {
    "startMs": 405480,
    "text": "getting changes out to customers? How"
  },
  {
    "startMs": 407760,
    "text": "how quickly are we able to ship things?"
  },
  {
    "startMs": 410440,
    "text": "And they saw that for half of the teams,"
  },
  {
    "startMs": 412200,
    "text": "they achieved less than 3x increase."
  },
  {
    "startMs": 415640,
    "text": "And what they found that was the"
  },
  {
    "startMs": 417360,
    "text": "difference between seeing less than 3x"
  },
  {
    "startMs": 419600,
    "text": "productivity increase, these teams that"
  },
  {
    "startMs": 421560,
    "text": "saw a median of 4.5x, and and in some"
  },
  {
    "startMs": 424760,
    "text": "cases more than 10,"
  },
  {
    "startMs": 426560,
    "text": "was how they used the tools. 90% of"
  },
  {
    "startMs": 429840,
    "text": "these teams used Kiro, among other"
  },
  {
    "startMs": 431800,
    "text": "internal tools that we have, and what"
  },
  {
    "startMs": 434480,
    "text": "they found was it wasn't about the"
  },
  {
    "startMs": 436360,
    "text": "tools, it was about the way that they"
  },
  {
    "startMs": 438440,
    "text": "worked."
  },
  {
    "startMs": 439760,
    "text": "The teams that achieved step function"
  },
  {
    "startMs": 441880,
    "text": "improvements"
  },
  {
    "startMs": 443400,
    "text": "intentionally changed the way that they"
  },
  {
    "startMs": 445280,
    "text": "worked, and the other simply kind of"
  },
  {
    "startMs": 447440,
    "text": "sprinkled Kiro and some of the other"
  },
  {
    "startMs": 449640,
    "text": "tools that we have on top of their"
  },
  {
    "startMs": 451960,
    "text": "existing way of working. And for me at"
  },
  {
    "startMs": 454560,
    "text": "least, this was the big aha moment. That"
  },
  {
    "startMs": 457560,
    "text": "why I hadn't been feeling potentially"
  },
  {
    "startMs": 460720,
    "text": "the massive gains that productive that"
  },
  {
    "startMs": 463440,
    "text": "in in productivity that AI has promised."
  },
  {
    "startMs": 466000,
    "text": "It's about changing the way that we"
  },
  {
    "startMs": 467480,
    "text": "work."
  },
  {
    "startMs": 469480,
    "text": "So, across this pilot, they went and"
  },
  {
    "startMs": 471480,
    "text": "interviewed uh the teams that were"
  },
  {
    "startMs": 473520,
    "text": "involved in the pilot as well as some of"
  },
  {
    "startMs": 475080,
    "text": "these other teams on the Bedrock mantel"
  },
  {
    "startMs": 476800,
    "text": "team, on uh Prime Video, and they found"
  },
  {
    "startMs": 480680,
    "text": "five habits. And and I use the word"
  },
  {
    "startMs": 483600,
    "text": "habits very specifically because again,"
  },
  {
    "startMs": 485920,
    "text": "it's not about that one sprint. It's"
  },
  {
    "startMs": 488360,
    "text": "about doing this day-to-day. And it And"
  },
  {
    "startMs": 490840,
    "text": "what they found when they interviewed"
  },
  {
    "startMs": 492120,
    "text": "with these teams was that it really was"
  },
  {
    "startMs": 494720,
    "text": "habits that they had to build"
  },
  {
    "startMs": 496520,
    "text": "day-to-day. When we change our way of"
  },
  {
    "startMs": 498800,
    "text": "working, it's it's hard to build these"
  },
  {
    "startMs": 501280,
    "text": "habits. It takes time to build these"
  },
  {
    "startMs": 502960,
    "text": "habits."
  },
  {
    "startMs": 504040,
    "text": "So, let's go through each of these one"
  },
  {
    "startMs": 505440,
    "text": "by one."
  },
  {
    "startMs": 506960,
    "text": "Habit number one is investing in agent"
  },
  {
    "startMs": 508880,
    "text": "context. We have a lot of stuff in our"
  },
  {
    "startMs": 512039,
    "text": "head. We tend to transfer all of that"
  },
  {
    "startMs": 514280,
    "text": "stuff in our head to other people"
  },
  {
    "startMs": 515880,
    "text": "through Slack conversations, through"
  },
  {
    "startMs": 518560,
    "text": "onboarding, mentors, things like that,"
  },
  {
    "startMs": 520919,
    "text": "through code reviews, through"
  },
  {
    "startMs": 523440,
    "text": "stand-ups and sprint planning, and they"
  },
  {
    "startMs": 525880,
    "text": "had to write all of that down. And the"
  },
  {
    "startMs": 528200,
    "text": "habit that they built was every time the"
  },
  {
    "startMs": 531480,
    "text": "agent makes a mistake or does something"
  },
  {
    "startMs": 533640,
    "text": "not the way that you would have done it,"
  },
  {
    "startMs": 535760,
    "text": "what am I missing in my skills files?"
  },
  {
    "startMs": 537600,
    "text": "What am I missing in my steering files"
  },
  {
    "startMs": 540120,
    "text": "that the agent needed?"
  },
  {
    "startMs": 542000,
    "text": "But then, as we know, across last year,"
  },
  {
    "startMs": 544720,
    "text": "we saw leaps and bounds in models'"
  },
  {
    "startMs": 547120,
    "text": "abilities and their behaviors."
  },
  {
    "startMs": 549640,
    "text": "Uh the Sonnet 3.7 in the middle of last"
  },
  {
    "startMs": 552240,
    "text": "year had a lot of quirks that we had to"
  },
  {
    "startMs": 555000,
    "text": "put a lot of do nots in our uh in our"
  },
  {
    "startMs": 557560,
    "text": "steering files, and now we don't have to"
  },
  {
    "startMs": 559560,
    "text": "do that as much with Opus 4.5 as of last"
  },
  {
    "startMs": 562960,
    "text": "November, and then we've had 6 months"
  },
  {
    "startMs": 565320,
    "text": "more than 6 months of improvement since"
  },
  {
    "startMs": 567400,
    "text": "then"
  },
  {
    "startMs": 568360,
    "text": "uh with all of the new versions of"
  },
  {
    "startMs": 569680,
    "text": "models that have come out since then."
  },
  {
    "startMs": 571640,
    "text": "And so, the question, the new habit,"
  },
  {
    "startMs": 573960,
    "text": "again, is do I still need this in my"
  },
  {
    "startMs": 576200,
    "text": "steering files or is this just bloating"
  },
  {
    "startMs": 577960,
    "text": "context?"
  },
  {
    "startMs": 579240,
    "text": "The second one is slowing down to speed"
  },
  {
    "startMs": 581800,
    "text": "up. In almost every team that was"
  },
  {
    "startMs": 584520,
    "text": "interviewed, they reported that their"
  },
  {
    "startMs": 586280,
    "text": "productivity actually went down as they"
  },
  {
    "startMs": 589520,
    "text": "intentionally adopted a new way of"
  },
  {
    "startMs": 591320,
    "text": "working."
  },
  {
    "startMs": 592680,
    "text": "That's counterintuitive, right? You have"
  },
  {
    "startMs": 595120,
    "text": "to do intentional engineering work"
  },
  {
    "startMs": 597880,
    "text": "before you're going to see that hockey"
  },
  {
    "startMs": 599520,
    "text": "stick curve in productivity improvement."
  },
  {
    "startMs": 602520,
    "text": "Because we have to do real work in our"
  },
  {
    "startMs": 604480,
    "text": "code base first for agents to be"
  },
  {
    "startMs": 606440,
    "text": "successful there, especially in"
  },
  {
    "startMs": 608400,
    "text": "brownfield existing code bases. So they"
  },
  {
    "startMs": 610960,
    "text": "had to build that agent context up. They"
  },
  {
    "startMs": 613160,
    "text": "had to improve existing tools error"
  },
  {
    "startMs": 615080,
    "text": "messages so that the model knew what was"
  },
  {
    "startMs": 617480,
    "text": "going on when it failed. They built new"
  },
  {
    "startMs": 620040,
    "text": "tools, new MCP servers for helping that"
  },
  {
    "startMs": 623040,
    "text": "model to actually get done what it"
  },
  {
    "startMs": 625000,
    "text": "needed to get done. A lot of teams ended"
  },
  {
    "startMs": 627680,
    "text": "up restructuring their code base so that"
  },
  {
    "startMs": 629520,
    "text": "agents could actually navigate it more"
  },
  {
    "startMs": 631520,
    "text": "easily. And I've even seen drastic"
  },
  {
    "startMs": 634240,
    "text": "changes like changing the programming"
  },
  {
    "startMs": 636080,
    "text": "language of the code base."
  },
  {
    "startMs": 638200,
    "text": "Um often I've seen teams struggle with"
  },
  {
    "startMs": 640720,
    "text": "Python, with JavaScript because they're"
  },
  {
    "startMs": 643080,
    "text": "untyped languages. It's hard to test."
  },
  {
    "startMs": 646040,
    "text": "There's no compiler errors. So the model"
  },
  {
    "startMs": 648720,
    "text": "kind of guesses and give it gives it"
  },
  {
    "startMs": 650839,
    "text": "back to you. And so I've seen teams"
  },
  {
    "startMs": 653040,
    "text": "moving to TypeScript. Um Rust has become"
  },
  {
    "startMs": 656040,
    "text": "very popular inside of Amazon. The"
  },
  {
    "startMs": 657920,
    "text": "compiler gives great error messages."
  },
  {
    "startMs": 660880,
    "text": "Um you don't have to do that, but I've"
  },
  {
    "startMs": 662760,
    "text": "seen a lot of teams making those"
  },
  {
    "startMs": 664360,
    "text": "intentional changes for the productivity"
  },
  {
    "startMs": 666800,
    "text": "gains that they're able to see."
  },
  {
    "startMs": 669480,
    "text": "The third one is feeding agents, not"
  },
  {
    "startMs": 672160,
    "text": "babysitting agents. And for me this was"
  },
  {
    "startMs": 674960,
    "text": "one of those aha moments of why we're"
  },
  {
    "startMs": 677640,
    "text": "seeing this step function improvement in"
  },
  {
    "startMs": 679600,
    "text": "productivity."
  },
  {
    "startMs": 681160,
    "text": "If you are vibe coding, if you are"
  },
  {
    "startMs": 683839,
    "text": "having a back-and-forth conversation"
  },
  {
    "startMs": 685880,
    "text": "with your agent all day long, of course"
  },
  {
    "startMs": 688160,
    "text": "you're not going to see four to five x"
  },
  {
    "startMs": 691120,
    "text": "productivity improvements because you"
  },
  {
    "startMs": 693520,
    "text": "are in the loop the entire time. You're"
  },
  {
    "startMs": 695720,
    "text": "probably sitting there for 30 seconds to"
  },
  {
    "startMs": 697960,
    "text": "a minute waiting for it to generate code"
  },
  {
    "startMs": 700960,
    "text": "and come back to you with with the code"
  },
  {
    "startMs": 702880,
    "text": "to review."
  },
  {
    "startMs": 704600,
    "text": "If you're sitting there waiting for it,"
  },
  {
    "startMs": 706640,
    "text": "then you can't go off and do other"
  },
  {
    "startMs": 708040,
    "text": "stuff. It's really difficult to run"
  },
  {
    "startMs": 710040,
    "text": "agents in parallel. It's very difficult"
  },
  {
    "startMs": 713000,
    "text": "to get to to clone yourself into"
  },
  {
    "startMs": 715800,
    "text": "multiple agents. And so if your"
  },
  {
    "startMs": 718280,
    "text": "conversations look a bit like this on"
  },
  {
    "startMs": 720760,
    "text": "the left, then you're babysitting that"
  },
  {
    "startMs": 722480,
    "text": "agent. As opposed to the right side"
  },
  {
    "startMs": 725040,
    "text": "where you're feeding it what it needs to"
  },
  {
    "startMs": 727200,
    "text": "do and how it can self-validate. And"
  },
  {
    "startMs": 729920,
    "text": "that's really the key so that agents can"
  },
  {
    "startMs": 731720,
    "text": "self-correct and only come back to you"
  },
  {
    "startMs": 734280,
    "text": "when it meets a certain quality bar,"
  },
  {
    "startMs": 736000,
    "text": "when it when it actually runs and"
  },
  {
    "startMs": 738040,
    "text": "compiles and passes tests, when it's"
  },
  {
    "startMs": 740640,
    "text": "testable, when it it actually has high"
  },
  {
    "startMs": 743040,
    "text": "coverage. And of course the next level"
  },
  {
    "startMs": 745640,
    "text": "is put all of this content into your"
  },
  {
    "startMs": 747520,
    "text": "steering file so it does it every time"
  },
  {
    "startMs": 749240,
    "text": "without you having to prompt it."
  },
  {
    "startMs": 753560,
    "text": "The fourth habit is to make intent"
  },
  {
    "startMs": 756520,
    "text": "explicit. At Amazon we practice a lot of"
  },
  {
    "startMs": 759880,
    "text": "behavior-driven development. We've built"
  },
  {
    "startMs": 761440,
    "text": "that into the Q product and so it's very"
  },
  {
    "startMs": 764320,
    "text": "natural for Amazon engineers to adopt it"
  },
  {
    "startMs": 766920,
    "text": "in Q. Um what what I've typically seen"
  },
  {
    "startMs": 770720,
    "text": "with live coding as opposed to frontier"
  },
  {
    "startMs": 772720,
    "text": "engineering is giving a very high-level"
  },
  {
    "startMs": 776120,
    "text": "prompt, letting the agent generate a ton"
  },
  {
    "startMs": 779280,
    "text": "of code, and then having a"
  },
  {
    "startMs": 781880,
    "text": "back-and-forth conversation saying, \"Oh,"
  },
  {
    "startMs": 784640,
    "text": "that's not really what I meant. That you"
  },
  {
    "startMs": 787360,
    "text": "haven't you haven't exactly gotten the"
  },
  {
    "startMs": 789240,
    "text": "the requirements right. No, I didn't"
  },
  {
    "startMs": 791000,
    "text": "actually want to build it that way."
  },
  {
    "startMs": 792320,
    "text": "Here's a technical design.\" And it is"
  },
  {
    "startMs": 795440,
    "text": "less I find less productive to iterate"
  },
  {
    "startMs": 798320,
    "text": "with the agent on code when the intent"
  },
  {
    "startMs": 801440,
    "text": "itself was incorrect. So often will have"
  },
  {
    "startMs": 805480,
    "text": "will see Amazon engineers go through"
  },
  {
    "startMs": 808080,
    "text": "this process for for ambiguous complex"
  },
  {
    "startMs": 811120,
    "text": "features of writing the specification."
  },
  {
    "startMs": 814440,
    "text": "And in Kiro, of course, you don't have"
  },
  {
    "startMs": 815720,
    "text": "to write this whole specification. You"
  },
  {
    "startMs": 817800,
    "text": "can have the model generate it, but it's"
  },
  {
    "startMs": 820200,
    "text": "a lot easier to to iterate with the"
  },
  {
    "startMs": 823000,
    "text": "model in kind of a back and forth"
  },
  {
    "startMs": 824560,
    "text": "conversation about a document than it is"
  },
  {
    "startMs": 828480,
    "text": "about code that's code changes that are"
  },
  {
    "startMs": 830320,
    "text": "spread across a code base."
  },
  {
    "startMs": 833520,
    "text": "The fifth one is shift testing left. One"
  },
  {
    "startMs": 838120,
    "text": "of the keys here is to give the agent"
  },
  {
    "startMs": 840200,
    "text": "that fast feedback loop."
  },
  {
    "startMs": 842400,
    "text": "Because that's what lets it go off for"
  },
  {
    "startMs": 844440,
    "text": "hours at a time and self-correct. The"
  },
  {
    "startMs": 846840,
    "text": "agent is going to make mistakes and"
  },
  {
    "startMs": 848880,
    "text": "that's fine. But if you give it the"
  },
  {
    "startMs": 851400,
    "text": "right signals, it can self-correct and"
  },
  {
    "startMs": 853440,
    "text": "it can spend a while doing that."
  },
  {
    "startMs": 856520,
    "text": "So, I've seen teams adding linters,"
  },
  {
    "startMs": 859320,
    "text": "adding unit tests, integration tests,"
  },
  {
    "startMs": 861200,
    "text": "performance tests, security tests. These"
  },
  {
    "startMs": 863040,
    "text": "are all things we all know we should"
  },
  {
    "startMs": 864560,
    "text": "have been doing all along. This is good"
  },
  {
    "startMs": 867040,
    "text": "engineering hygiene and practices. But"
  },
  {
    "startMs": 869920,
    "text": "now the ROI is, I think, finally high"
  },
  {
    "startMs": 873240,
    "text": "enough for actually us to actually"
  },
  {
    "startMs": 874800,
    "text": "invest in it. Um one thing that I've"
  },
  {
    "startMs": 877200,
    "text": "been seeing a lot of teams do is mock"
  },
  {
    "startMs": 879760,
    "text": "out services. Often with integration"
  },
  {
    "startMs": 882360,
    "text": "tests, we would test kind of end-to-end"
  },
  {
    "startMs": 884480,
    "text": "an entire system including live"
  },
  {
    "startMs": 886920,
    "text": "services. But we've been investing a lot"
  },
  {
    "startMs": 889360,
    "text": "in in mock services that run entirely"
  },
  {
    "startMs": 891520,
    "text": "locally with deterministic responses"
  },
  {
    "startMs": 894080,
    "text": "because it lets the agent do everything"
  },
  {
    "startMs": 897320,
    "text": "locally. Um doing everything on your"
  },
  {
    "startMs": 900080,
    "text": "laptop without having to spin up a bunch"
  },
  {
    "startMs": 902200,
    "text": "of other services and and connect to"
  },
  {
    "startMs": 904280,
    "text": "cloud services makes everything a lot"
  },
  {
    "startMs": 907080,
    "text": "faster because the the more that your"
  },
  {
    "startMs": 910560,
    "text": "agent can get fast feedback means the"
  },
  {
    "startMs": 913240,
    "text": "more loops that it can can do and the"
  },
  {
    "startMs": 915839,
    "text": "more productive your own agent can be."
  },
  {
    "startMs": 919640,
    "text": "So, across all of these, these are some"
  },
  {
    "startMs": 921680,
    "text": "of the habits we've seen, but of course"
  },
  {
    "startMs": 923400,
    "text": "I would be remiss if I would tell you if"
  },
  {
    "startMs": 926440,
    "text": "you adopt all of these habits, you will"
  },
  {
    "startMs": 930240,
    "text": "achieve nirvana. You will be the most"
  },
  {
    "startMs": 931920,
    "text": "productive engineering organization the"
  },
  {
    "startMs": 934680,
    "text": "world has ever seen. Things are still"
  },
  {
    "startMs": 936480,
    "text": "hard. We are still very much in an early"
  },
  {
    "startMs": 938800,
    "text": "adopter phase and teams are still"
  },
  {
    "startMs": 942000,
    "text": "figuring it out."
  },
  {
    "startMs": 943840,
    "text": "So, one thing that we've been seeing"
  },
  {
    "startMs": 945640,
    "text": "across our teams just organizationally"
  },
  {
    "startMs": 948000,
    "text": "is the risk of burnout. I did not coin"
  },
  {
    "startMs": 951000,
    "text": "this term. I forget who did at what"
  },
  {
    "startMs": 953000,
    "text": "conference, but flow mat is real. We've"
  },
  {
    "startMs": 956120,
    "text": "been seeing engineers staying up late"
  },
  {
    "startMs": 958640,
    "text": "late at night"
  },
  {
    "startMs": 960120,
    "text": "trying to get that perfect prompt that's"
  },
  {
    "startMs": 962520,
    "text": "going to make their agent run for hours"
  },
  {
    "startMs": 964480,
    "text": "overnight so that they wake up in the"
  },
  {
    "startMs": 965960,
    "text": "morning with a code change ready."
  },
  {
    "startMs": 968120,
    "text": "The cognitive load increases as you run"
  },
  {
    "startMs": 971080,
    "text": "these multiple agents in parallel."
  },
  {
    "startMs": 973760,
    "text": "You're constantly shifting between"
  },
  {
    "startMs": 975560,
    "text": "terminal tabs."
  },
  {
    "startMs": 977240,
    "text": "And then we do see that reviewing AI"
  },
  {
    "startMs": 980000,
    "text": "output is often harder for some than"
  },
  {
    "startMs": 982080,
    "text": "than actually writing it, especially"
  },
  {
    "startMs": 984320,
    "text": "early in career."
  },
  {
    "startMs": 986240,
    "text": "Senior engineers have have already spent"
  },
  {
    "startMs": 988720,
    "text": "a large portion of their career"
  },
  {
    "startMs": 990120,
    "text": "reviewing others code."
  },
  {
    "startMs": 992400,
    "text": "But early career engineers don't have"
  },
  {
    "startMs": 995200,
    "text": "that muscle yet and so reviewing it can"
  },
  {
    "startMs": 998280,
    "text": "can feel like a lot more cognitive load"
  },
  {
    "startMs": 1001720,
    "text": "than they're used to and actually"
  },
  {
    "startMs": 1002920,
    "text": "writing it."
  },
  {
    "startMs": 1004520,
    "text": "The other one is organizational change."
  },
  {
    "startMs": 1007839,
    "text": "So, it's already hard to change the way"
  },
  {
    "startMs": 1010040,
    "text": "we work as engineers. The way that we"
  },
  {
    "startMs": 1011920,
    "text": "spend our entire day completely changes"
  },
  {
    "startMs": 1015080,
    "text": "when we're frontier engineers, but also"
  },
  {
    "startMs": 1017640,
    "text": "organizations have to change to enable"
  },
  {
    "startMs": 1020440,
    "text": "frontier engineering teams."
  },
  {
    "startMs": 1022960,
    "text": "One that I've seen very commonly is"
  },
  {
    "startMs": 1026079,
    "text": "accepting slowing down to speed up."
  },
  {
    "startMs": 1029120,
    "text": "And I've been guilty of this myself. My"
  },
  {
    "startMs": 1031280,
    "text": "my fellow leaders have been guilty of of"
  },
  {
    "startMs": 1033199,
    "text": "this of saying, \"Well, you have the AI"
  },
  {
    "startMs": 1035959,
    "text": "tools now and the models are so amazing"
  },
  {
    "startMs": 1038600,
    "text": "now. Why are you not going faster?"
  },
  {
    "startMs": 1042120,
    "text": "Um and that's because you have to take"
  },
  {
    "startMs": 1045079,
    "text": "those two months to invest in your code"
  },
  {
    "startMs": 1047720,
    "text": "base, to figure out the best practices"
  },
  {
    "startMs": 1049880,
    "text": "for your team, to make hard habit"
  },
  {
    "startMs": 1053080,
    "text": "changes on your team."
  },
  {
    "startMs": 1055120,
    "text": "Um and and if you're constantly"
  },
  {
    "startMs": 1057320,
    "text": "expecting"
  },
  {
    "startMs": 1058960,
    "text": "shipping features every month because"
  },
  {
    "startMs": 1060920,
    "text": "now we have these amazing models and"
  },
  {
    "startMs": 1062560,
    "text": "we're seeing um all of these these"
  },
  {
    "startMs": 1065800,
    "text": "companies on X saying how they're"
  },
  {
    "startMs": 1067600,
    "text": "shipping 20 PRs a day, um we have to"
  },
  {
    "startMs": 1071920,
    "text": "slow down to speed up."
  },
  {
    "startMs": 1074360,
    "text": "The second one is actually going too"
  },
  {
    "startMs": 1075840,
    "text": "broad in the organization too fast. I"
  },
  {
    "startMs": 1078240,
    "text": "think that if we had um expected all"
  },
  {
    "startMs": 1082080,
    "text": "teams in massive organizations to be"
  },
  {
    "startMs": 1084680,
    "text": "frontier teams immediately, we would not"
  },
  {
    "startMs": 1087960,
    "text": "have had the learnings that we had from"
  },
  {
    "startMs": 1090640,
    "text": "the Pathfinder, from the from the sprint"
  },
  {
    "startMs": 1093560,
    "text": "experiment, from the pilot uh teams"
  },
  {
    "startMs": 1096840,
    "text": "within Amazon. And now the challenge for"
  },
  {
    "startMs": 1099480,
    "text": "us is how do we scale it out? And that's"
  },
  {
    "startMs": 1101440,
    "text": "what 2026 is about for Amazon is how do"
  },
  {
    "startMs": 1103720,
    "text": "we scale this out to more and more"
  },
  {
    "startMs": 1105480,
    "text": "teams, to the next uh 2,000 teams"
  },
  {
    "startMs": 1108640,
    "text": "instead of uh 50 teams."
  },
  {
    "startMs": 1111040,
    "text": "Um and so I think that when you roll it"
  },
  {
    "startMs": 1113120,
    "text": "out too quickly, you have a lot of teams"
  },
  {
    "startMs": 1116600,
    "text": "who don't know what they're doing. You"
  },
  {
    "startMs": 1118200,
    "text": "haven't had time to find the best"
  },
  {
    "startMs": 1120320,
    "text": "practices for your own organizations,"
  },
  {
    "startMs": 1122480,
    "text": "the the context that your organization"
  },
  {
    "startMs": 1124840,
    "text": "needs."
  },
  {
    "startMs": 1125920,
    "text": "And the last one is that you're going to"
  },
  {
    "startMs": 1127480,
    "text": "find new bottlenecks."
  },
  {
    "startMs": 1129360,
    "text": "Previously, code writing code manually"
  },
  {
    "startMs": 1132520,
    "text": "was the bottleneck. Um I find that"
  },
  {
    "startMs": 1135760,
    "text": "within Amazon, we've found um the speed"
  },
  {
    "startMs": 1138720,
    "text": "of decision-making becomes a new"
  },
  {
    "startMs": 1140560,
    "text": "bottleneck. Um the more that you spend"
  },
  {
    "startMs": 1143760,
    "text": "reviewing the decision to actually build"
  },
  {
    "startMs": 1146040,
    "text": "a new product, the slower it is to build"
  },
  {
    "startMs": 1149200,
    "text": "the product now because the code only"
  },
  {
    "startMs": 1151200,
    "text": "takes 1 to two months to write."
  },
  {
    "startMs": 1153468,
    "text": ">> [snorts]"
  },
  {
    "startMs": 1153480,
    "text": ">> Um all of the review processes"
  },
  {
    "startMs": 1156160,
    "text": "associated with the launch of a product"
  },
  {
    "startMs": 1159240,
    "text": "become the bottleneck. When it used to"
  },
  {
    "startMs": 1161680,
    "text": "take 9 to 12 months to build a new"
  },
  {
    "startMs": 1164120,
    "text": "product, it didn't matter so much in the"
  },
  {
    "startMs": 1167200,
    "text": "in the overall wash of things if it took"
  },
  {
    "startMs": 1169600,
    "text": "two months to make the decision to build"
  },
  {
    "startMs": 1171560,
    "text": "the product and then two months to"
  },
  {
    "startMs": 1172960,
    "text": "approve the launch. But now those are"
  },
  {
    "startMs": 1176320,
    "text": "the bottlenecks. Those are the long"
  },
  {
    "startMs": 1177920,
    "text": "pole. And so you find all of these all"
  },
  {
    "startMs": 1181480,
    "text": "of these things that slow you down."
  },
  {
    "startMs": 1184040,
    "text": "Often I find that frontier engineering"
  },
  {
    "startMs": 1186360,
    "text": "teams spend more time making decisions"
  },
  {
    "startMs": 1189320,
    "text": "than they do writing code. And so the"
  },
  {
    "startMs": 1191400,
    "text": "more that you can make fast decisions,"
  },
  {
    "startMs": 1193240,
    "text": "especially ones that are easy to be"
  },
  {
    "startMs": 1195000,
    "text": "reversed, the better."
  },
  {
    "startMs": 1197880,
    "text": "So my one big takeaway for for everyone"
  },
  {
    "startMs": 1200920,
    "text": "here is that"
  },
  {
    "startMs": 1202800,
    "text": "frontier engineering is about"
  },
  {
    "startMs": 1204520,
    "text": "intentionally changing the way that you"
  },
  {
    "startMs": 1206360,
    "text": "work. And that is difficult. That takes"
  },
  {
    "startMs": 1209440,
    "text": "time. It is forming new habits and a new"
  },
  {
    "startMs": 1212880,
    "text": "way of working."
  },
  {
    "startMs": 1214400,
    "text": "And that goes across any engineering"
  },
  {
    "startMs": 1217200,
    "text": "team as well as your organization. Um so"
  },
  {
    "startMs": 1220600,
    "text": "I encourage you to think about"
  },
  {
    "startMs": 1223240,
    "text": "um how you're interacting with AI tools"
  },
  {
    "startMs": 1225800,
    "text": "and how that can change to free yourself"
  },
  {
    "startMs": 1229160,
    "text": "up from being in the loop."
  },
  {
    "startMs": 1231360,
    "text": "Um thanks. I'm going to I'll hang out uh"
  },
  {
    "startMs": 1233680,
    "text": "a little bit if anyone has questions in"
  },
  {
    "startMs": 1235720,
    "text": "the back. Um but thanks for the time"
  },
  {
    "startMs": 1237960,
    "text": "today."
  },
  {
    "startMs": 1254994,
    "text": ">> [music]"
  }
]