[
  {
    "startMs": 12560,
    "text": "Well, thank you for joining"
  },
  {
    "startMs": 14320,
    "text": "the final day of the conference here. I"
  },
  {
    "startMs": 15920,
    "text": "know it's been a long week."
  },
  {
    "startMs": 18480,
    "text": "But I'm excited to talk about a topic"
  },
  {
    "startMs": 21320,
    "text": "here that is very near and dear to me."
  },
  {
    "startMs": 24480,
    "text": "Software factories."
  },
  {
    "startMs": 26280,
    "text": "Um"
  },
  {
    "startMs": 27920,
    "text": "You this is probably not a new concept"
  },
  {
    "startMs": 30880,
    "text": "to any of you. I'm guessing if you're"
  },
  {
    "startMs": 31880,
    "text": "all here, you're very familiar with"
  },
  {
    "startMs": 33520,
    "text": "software factories. You know, Ramp"
  },
  {
    "startMs": 35880,
    "text": "blogged about their inspects"
  },
  {
    "startMs": 38840,
    "text": "several months ago. I think maybe"
  },
  {
    "startMs": 40080,
    "text": "December last year. I forget exactly"
  },
  {
    "startMs": 41800,
    "text": "when. It's been a little while. And that"
  },
  {
    "startMs": 43040,
    "text": "really kind of like excited the entire"
  },
  {
    "startMs": 45600,
    "text": "industry of thinking like, oh, we can"
  },
  {
    "startMs": 47800,
    "text": "actually build software very differently"
  },
  {
    "startMs": 49720,
    "text": "now that AI is extremely capable of"
  },
  {
    "startMs": 52480,
    "text": "writing code. Um a lot of other"
  },
  {
    "startMs": 55400,
    "text": "companies have followed suit. Um and"
  },
  {
    "startMs": 58120,
    "text": "there's kind of like a standard set up"
  },
  {
    "startMs": 60000,
    "text": "here. Uh you have a sandbox that you put"
  },
  {
    "startMs": 62160,
    "text": "your code into. Um you put a an AI agent"
  },
  {
    "startMs": 66680,
    "text": "in there. Maybe a Claude code section or"
  },
  {
    "startMs": 69760,
    "text": "an open code."
  },
  {
    "startMs": 71320,
    "text": "Um and with a prompt, it generates a PR"
  },
  {
    "startMs": 73760,
    "text": "and you merge it."
  },
  {
    "startMs": 75160,
    "text": "Um"
  },
  {
    "startMs": 75960,
    "text": "You might be building this already"
  },
  {
    "startMs": 77290,
    "text": "[clears throat] yourself."
  },
  {
    "startMs": 78920,
    "text": "We have taken a slightly different"
  },
  {
    "startMs": 81240,
    "text": "approach at WorkOS."
  },
  {
    "startMs": 83880,
    "text": "Um and the motivation for what we are"
  },
  {
    "startMs": 86800,
    "text": "building is that we see a lot of the"
  },
  {
    "startMs": 90120,
    "text": "kind of success metrics um really"
  },
  {
    "startMs": 93400,
    "text": "focusing on output, right? Like you'll"
  },
  {
    "startMs": 95880,
    "text": "see posts on X and blogs"
  },
  {
    "startMs": 98840,
    "text": "talking about the percentage of PRs or"
  },
  {
    "startMs": 101360,
    "text": "the number of pull requests or how much"
  },
  {
    "startMs": 103120,
    "text": "code uh is being generated by AI"
  },
  {
    "startMs": 107440,
    "text": "moving into production. Um and this is"
  },
  {
    "startMs": 109840,
    "text": "just really output metrics that"
  },
  {
    "startMs": 113000,
    "text": "um"
  },
  {
    "startMs": 113560,
    "text": "you know, this can actually disguise how"
  },
  {
    "startMs": 116760,
    "text": "well these systems are working."
  },
  {
    "startMs": 119080,
    "text": "Percentage of PRs or number of PRs"
  },
  {
    "startMs": 122280,
    "text": "may be lying in overall increase in PRs."
  },
  {
    "startMs": 124760,
    "text": "It's very hard to distinguish"
  },
  {
    "startMs": 126880,
    "text": "whether the output is actually driving"
  },
  {
    "startMs": 129679,
    "text": "outcomes."
  },
  {
    "startMs": 131120,
    "text": "And so that's kind of where we focus our"
  },
  {
    "startMs": 134880,
    "text": "efforts when we're thinking about a"
  },
  {
    "startMs": 135840,
    "text": "software factory. It's like if we're"
  },
  {
    "startMs": 137040,
    "text": "going to go put engineering time into"
  },
  {
    "startMs": 140480,
    "text": "building this level of automations and"
  },
  {
    "startMs": 142600,
    "text": "there's a"
  },
  {
    "startMs": 143360,
    "text": "bit of work that needs to go into this."
  },
  {
    "startMs": 145280,
    "text": "Full-time teams dedicated to this."
  },
  {
    "startMs": 147760,
    "text": "How do we want to think about the"
  },
  {
    "startMs": 150080,
    "text": "success of this and whether this is"
  },
  {
    "startMs": 152120,
    "text": "actually driving value for our"
  },
  {
    "startMs": 153920,
    "text": "organization, our engineering"
  },
  {
    "startMs": 155160,
    "text": "organization?"
  },
  {
    "startMs": 156480,
    "text": "And so we measure instead of just code"
  },
  {
    "startMs": 160640,
    "text": "output, we're looking at outcome"
  },
  {
    "startMs": 162560,
    "text": "metrics."
  },
  {
    "startMs": 164160,
    "text": "And that is really about like are we"
  },
  {
    "startMs": 166760,
    "text": "accelerating our ability to deliver"
  },
  {
    "startMs": 168880,
    "text": "features?"
  },
  {
    "startMs": 170040,
    "text": "We think that the dream of the software"
  },
  {
    "startMs": 171560,
    "text": "factory is that it gives each of our"
  },
  {
    "startMs": 174440,
    "text": "engineers a small engineering team for"
  },
  {
    "startMs": 176400,
    "text": "themselves. And so the consequence of"
  },
  {
    "startMs": 178840,
    "text": "that really needs to be that we're"
  },
  {
    "startMs": 180880,
    "text": "building more and we're shipping more"
  },
  {
    "startMs": 182720,
    "text": "and that as we embark on complex"
  },
  {
    "startMs": 184560,
    "text": "features, those can get built a lot more"
  },
  {
    "startMs": 187480,
    "text": "a lot more quickly."
  },
  {
    "startMs": 191239,
    "text": "So what does this actually look like?"
  },
  {
    "startMs": 192800,
    "text": "Well,"
  },
  {
    "startMs": 194280,
    "text": "we kind of started with the same thing"
  },
  {
    "startMs": 196920,
    "text": "you've seen in other other"
  },
  {
    "startMs": 199400,
    "text": "factories."
  },
  {
    "startMs": 201040,
    "text": "We start with a sandbox we built on top"
  },
  {
    "startMs": 202800,
    "text": "of Cloudflare. We put an open code model"
  },
  {
    "startMs": 206120,
    "text": "router in there. We're feeding it"
  },
  {
    "startMs": 207959,
    "text": "prompts."
  },
  {
    "startMs": 209280,
    "text": "Um,"
  },
  {
    "startMs": 209920,
    "text": "and we kind of very quickly kind of ran"
  },
  {
    "startMs": 212360,
    "text": "into this situation where this wasn't"
  },
  {
    "startMs": 214959,
    "text": "driving more outcomes. This was not an"
  },
  {
    "startMs": 217600,
    "text": "incremental increase or an exponential"
  },
  {
    "startMs": 219800,
    "text": "increase over engineers just driving"
  },
  {
    "startMs": 221880,
    "text": "cloud code on their laptops. It was"
  },
  {
    "startMs": 223320,
    "text": "actually pretty indistinguishable for"
  },
  {
    "startMs": 225000,
    "text": "us."
  },
  {
    "startMs": 227040,
    "text": "And so we really kind of started to"
  },
  {
    "startMs": 228280,
    "text": "think about, well,"
  },
  {
    "startMs": 229880,
    "text": "how can we embed our engineering"
  },
  {
    "startMs": 232840,
    "text": "processes into the factory itself? It's"
  },
  {
    "startMs": 236040,
    "text": "not sufficient that the factory is"
  },
  {
    "startMs": 238040,
    "text": "producing code. We actually want it to"
  },
  {
    "startMs": 240240,
    "text": "take a lot of the other work that our"
  },
  {
    "startMs": 242200,
    "text": "engineers do to produce products and"
  },
  {
    "startMs": 245800,
    "text": "automate that as well."
  },
  {
    "startMs": 248000,
    "text": "So, we kind of split this into two"
  },
  {
    "startMs": 249560,
    "text": "parts."
  },
  {
    "startMs": 250800,
    "text": "One is we have the system we call TARS."
  },
  {
    "startMs": 253840,
    "text": "This is a way for users to interact with"
  },
  {
    "startMs": 258359,
    "text": "the coding agent. And it is embedded"
  },
  {
    "startMs": 261280,
    "text": "into the tools that we use. So, not just"
  },
  {
    "startMs": 264160,
    "text": "Slack, but also Linear and GitHub."
  },
  {
    "startMs": 267440,
    "text": "We subscribe to webhooks through TARS so"
  },
  {
    "startMs": 269919,
    "text": "that TARS can actually track the"
  },
  {
    "startMs": 271680,
    "text": "progress of projects"
  },
  {
    "startMs": 274520,
    "text": "in addition to generating the code"
  },
  {
    "startMs": 276680,
    "text": "outputs."
  },
  {
    "startMs": 278720,
    "text": "And then we built a separate system"
  },
  {
    "startMs": 279880,
    "text": "called Horizon."
  },
  {
    "startMs": 282120,
    "text": "And this is the infrastructure"
  },
  {
    "startMs": 283240,
    "text": "orchestration layer. This is a lot more"
  },
  {
    "startMs": 285560,
    "text": "similar to"
  },
  {
    "startMs": 287520,
    "text": "Inspects and Minions and and some of the"
  },
  {
    "startMs": 289600,
    "text": "other systems that other companies have"
  },
  {
    "startMs": 291400,
    "text": "built."
  },
  {
    "startMs": 292640,
    "text": "But, it sits in front of an MCP gateway."
  },
  {
    "startMs": 296200,
    "text": "And I'm going to talk a little bit more"
  },
  {
    "startMs": 297640,
    "text": "about why that MCP gateway has been"
  },
  {
    "startMs": 301040,
    "text": "really transformational for us."
  },
  {
    "startMs": 307280,
    "text": "So, by"
  },
  {
    "startMs": 308760,
    "text": "feeding our webhooks, activities that"
  },
  {
    "startMs": 312040,
    "text": "are happening in other uh source code"
  },
  {
    "startMs": 314440,
    "text": "systems and project tracking systems,"
  },
  {
    "startMs": 317640,
    "text": "we're starting to get to that level of"
  },
  {
    "startMs": 318919,
    "text": "autonomy where"
  },
  {
    "startMs": 320480,
    "text": "our"
  },
  {
    "startMs": 321840,
    "text": "factory is performing product"
  },
  {
    "startMs": 323640,
    "text": "engineering work, not just code work."
  },
  {
    "startMs": 325720,
    "text": "So, what's an example of this?"
  },
  {
    "startMs": 327560,
    "text": "In Linear, we can define tickets that"
  },
  {
    "startMs": 330000,
    "text": "have dependencies. So, one ticket blocks"
  },
  {
    "startMs": 332440,
    "text": "another ticket. Pretty common for, you"
  },
  {
    "startMs": 334560,
    "text": "know, breaking down uh large units of"
  },
  {
    "startMs": 336520,
    "text": "work into smaller ones."
  },
  {
    "startMs": 338800,
    "text": "Because"
  },
  {
    "startMs": 340080,
    "text": "uh TARS is getting webhooks on ticket"
  },
  {
    "startMs": 342520,
    "text": "completion, it can automatically pick up"
  },
  {
    "startMs": 344960,
    "text": "the next ticket in a cycle. And so, we"
  },
  {
    "startMs": 347760,
    "text": "can do through our planning process"
  },
  {
    "startMs": 350080,
    "text": "construct like a map of how this plan we"
  },
  {
    "startMs": 353360,
    "text": "think this plan is going to get executed"
  },
  {
    "startMs": 355560,
    "text": "and TARS can start to execute on that"
  },
  {
    "startMs": 357360,
    "text": "autonomously. The other thing that we"
  },
  {
    "startMs": 359440,
    "text": "have is in between those steps when a"
  },
  {
    "startMs": 361160,
    "text": "ticket is completed, we can ask TARS"
  },
  {
    "startMs": 364000,
    "text": "\"Can you reevaluate the linear project"
  },
  {
    "startMs": 366320,
    "text": "and let me know if it's missing now"
  },
  {
    "startMs": 367960,
    "text": "tickets?\" Because as you do work, you're"
  },
  {
    "startMs": 370200,
    "text": "learning about where the gaps in your"
  },
  {
    "startMs": 372680,
    "text": "plan. And we want to continuously keep"
  },
  {
    "startMs": 376160,
    "text": "our our plan fresh by using the agent"
  },
  {
    "startMs": 379560,
    "text": "itself. Because the agent is determining"
  },
  {
    "startMs": 382480,
    "text": "that there may be missing pieces to what"
  },
  {
    "startMs": 385400,
    "text": "we initially planned for the project and"
  },
  {
    "startMs": 387160,
    "text": "the goal that we have in mind."
  },
  {
    "startMs": 391560,
    "text": "So, we run this product engineering"
  },
  {
    "startMs": 393320,
    "text": "culture at WorkOS. This is where"
  },
  {
    "startMs": 395320,
    "text": "engineers are responsible for a lot of"
  },
  {
    "startMs": 397560,
    "text": "the product functions. We don't have"
  },
  {
    "startMs": 399080,
    "text": "product managers on teams today."
  },
  {
    "startMs": 401440,
    "text": "Um and one of the rituals as part of"
  },
  {
    "startMs": 405120,
    "text": "this product engineering process is we"
  },
  {
    "startMs": 406840,
    "text": "create a hilltop document. It's a PRD"
  },
  {
    "startMs": 409880,
    "text": "and it is intended to"
  },
  {
    "startMs": 412400,
    "text": "both define what is the purpose of this"
  },
  {
    "startMs": 414919,
    "text": "project, but also incorporate"
  },
  {
    "startMs": 418200,
    "text": "um what are customers talking about?"
  },
  {
    "startMs": 420520,
    "text": "Like where are we seeing the need for"
  },
  {
    "startMs": 421919,
    "text": "this unit of work? Uh it looks at"
  },
  {
    "startMs": 424600,
    "text": "competitive analysis. Like are there"
  },
  {
    "startMs": 426240,
    "text": "similar products out in the market today"
  },
  {
    "startMs": 428360,
    "text": "that we can draw inspiration from? We"
  },
  {
    "startMs": 430440,
    "text": "start to bring design early design"
  },
  {
    "startMs": 432200,
    "text": "screens into this."
  },
  {
    "startMs": 433880,
    "text": "Um and we outline like what are the"
  },
  {
    "startMs": 435640,
    "text": "major milestones. And so, by kind of"
  },
  {
    "startMs": 438840,
    "text": "consolidating and canonicalizing this"
  },
  {
    "startMs": 441400,
    "text": "information into a document, this is"
  },
  {
    "startMs": 443360,
    "text": "something our product engineers have"
  },
  {
    "startMs": 444640,
    "text": "been doing uh throughout the entire uh"
  },
  {
    "startMs": 447520,
    "text": "company."
  },
  {
    "startMs": 448680,
    "text": "Um"
  },
  {
    "startMs": 449280,
    "text": "now we can ask give this resource to an"
  },
  {
    "startMs": 451120,
    "text": "agent and an agent can break this into"
  },
  {
    "startMs": 453640,
    "text": "units of work."
  },
  {
    "startMs": 454960,
    "text": "Uh and so, this has been a really uh"
  },
  {
    "startMs": 457480,
    "text": "powerful way for us to take existing"
  },
  {
    "startMs": 459480,
    "text": "processes"
  },
  {
    "startMs": 460760,
    "text": "and encode them into the factory"
  },
  {
    "startMs": 462920,
    "text": "themselves. Like I mentioned that it's"
  },
  {
    "startMs": 465040,
    "text": "listening for web hooks. I'll show an"
  },
  {
    "startMs": 466560,
    "text": "example of what this looks like, but it"
  },
  {
    "startMs": 468920,
    "text": "it can see that the hilltop document"
  },
  {
    "startMs": 471880,
    "text": "through linear tickets has been reviewed"
  },
  {
    "startMs": 474360,
    "text": "and approved and by virtue of it being"
  },
  {
    "startMs": 477480,
    "text": "approved can start work on this project"
  },
  {
    "startMs": 479600,
    "text": "automatically. So we don't need a human"
  },
  {
    "startMs": 481680,
    "text": "to be shepherding this agent through"
  },
  {
    "startMs": 483960,
    "text": "every single step of the life cycle."
  },
  {
    "startMs": 489920,
    "text": "So we created an agent specifically for"
  },
  {
    "startMs": 492080,
    "text": "this part of the process. We call it the"
  },
  {
    "startMs": 493800,
    "text": "PM."
  },
  {
    "startMs": 494960,
    "text": "And it is doing the first draft of the"
  },
  {
    "startMs": 497200,
    "text": "hilltop based on the brief specification"
  },
  {
    "startMs": 500520,
    "text": "that we give it."
  },
  {
    "startMs": 501800,
    "text": "It's adding context to that in addition"
  },
  {
    "startMs": 505000,
    "text": "to reading the human reviews and then it"
  },
  {
    "startMs": 507680,
    "text": "picks up the implementation and breaking"
  },
  {
    "startMs": 509360,
    "text": "that into tickets."
  },
  {
    "startMs": 511760,
    "text": "And there's an opportunity for humans to"
  },
  {
    "startMs": 513120,
    "text": "stay in the loop for each part of this"
  },
  {
    "startMs": 515520,
    "text": "process. I mean it's very common for a"
  },
  {
    "startMs": 518240,
    "text": "human to intervene or comment on a"
  },
  {
    "startMs": 519719,
    "text": "ticket that's generated by AI, give it"
  },
  {
    "startMs": 521880,
    "text": "further guidance or refinements."
  },
  {
    "startMs": 524039,
    "text": "Um but it's really about that like cold"
  },
  {
    "startMs": 526320,
    "text": "start problem where we can kind of get"
  },
  {
    "startMs": 529120,
    "text": "over the hump of creating all these"
  },
  {
    "startMs": 530720,
    "text": "different resources. Okay, so what does"
  },
  {
    "startMs": 532760,
    "text": "this actually look like?"
  },
  {
    "startMs": 535040,
    "text": "So here's a project I kicked off last"
  },
  {
    "startMs": 537320,
    "text": "month two months ago."
  },
  {
    "startMs": 539360,
    "text": "Um we're adding a new API to products I"
  },
  {
    "startMs": 541960,
    "text": "lead called Vaults."
  },
  {
    "startMs": 543720,
    "text": "And with a command here in"
  },
  {
    "startMs": 547280,
    "text": "um"
  },
  {
    "startMs": 548280,
    "text": "in Slack"
  },
  {
    "startMs": 549560,
    "text": "I can kick off this project with a"
  },
  {
    "startMs": 551680,
    "text": "simple description, usually a few"
  },
  {
    "startMs": 552960,
    "text": "sentences of what I'm trying to"
  },
  {
    "startMs": 554520,
    "text": "accomplish."
  },
  {
    "startMs": 556000,
    "text": "Tarsin goes in and creates all these"
  },
  {
    "startMs": 558120,
    "text": "resources for me. I don't need to go and"
  },
  {
    "startMs": 560200,
    "text": "create the project in linear. I don't"
  },
  {
    "startMs": 561839,
    "text": "need to create a draft of the notion"
  },
  {
    "startMs": 563839,
    "text": "document."
  },
  {
    "startMs": 565160,
    "text": "Um"
  },
  {
    "startMs": 565800,
    "text": "all of the decision logs and the open"
  },
  {
    "startMs": 568040,
    "text": "questions."
  },
  {
    "startMs": 569440,
    "text": "Um Tars is creating a first pass at"
  },
  {
    "startMs": 571560,
    "text": "that. And so this then gives me an easy"
  },
  {
    "startMs": 574000,
    "text": "framework for me to step into as the"
  },
  {
    "startMs": 577280,
    "text": "lead product engineer and start giving"
  },
  {
    "startMs": 579920,
    "text": "more specification, rounding out areas"
  },
  {
    "startMs": 582680,
    "text": "that aren't well defined with knowledge"
  },
  {
    "startMs": 585040,
    "text": "that I had know about what we're trying"
  },
  {
    "startMs": 586760,
    "text": "to accomplish with the project, and then"
  },
  {
    "startMs": 589040,
    "text": "hand it back to Tars uh for execution."
  },
  {
    "startMs": 593200,
    "text": "So, here's an example of uh the project."
  },
  {
    "startMs": 597160,
    "text": "Um and it's like I said, it sets up"
  },
  {
    "startMs": 598880,
    "text": "these first"
  },
  {
    "startMs": 600920,
    "text": "milestones around our product"
  },
  {
    "startMs": 602880,
    "text": "engineering process. So, we haven't done"
  },
  {
    "startMs": 605440,
    "text": "the hilltop review yet here. When that's"
  },
  {
    "startMs": 607640,
    "text": "done, we'll uh"
  },
  {
    "startMs": 609840,
    "text": "we'll we'll mark this ticket as complete"
  },
  {
    "startMs": 612200,
    "text": "and Tars will pick that up and move on"
  },
  {
    "startMs": 614000,
    "text": "to the next stage of project"
  },
  {
    "startMs": 615120,
    "text": "implementation."
  },
  {
    "startMs": 617800,
    "text": "So,"
  },
  {
    "startMs": 618840,
    "text": "what we found is like a lot of"
  },
  {
    "startMs": 621400,
    "text": "teams are now doing more project brief"
  },
  {
    "startMs": 624000,
    "text": "and hilltop work because our agent can"
  },
  {
    "startMs": 627920,
    "text": "seed all that information. It's that"
  },
  {
    "startMs": 629920,
    "text": "blank page problem with writing. If you"
  },
  {
    "startMs": 632320,
    "text": "can come into a document that already"
  },
  {
    "startMs": 633800,
    "text": "has information, and you know, these"
  },
  {
    "startMs": 635840,
    "text": "agents aren't perfect. There are times"
  },
  {
    "startMs": 637760,
    "text": "where it grossly overestimates what"
  },
  {
    "startMs": 640680,
    "text": "we're trying to accomplish with this"
  },
  {
    "startMs": 642040,
    "text": "project, and we have to cut out a lot of"
  },
  {
    "startMs": 643800,
    "text": "the scope that it comes up with. But,"
  },
  {
    "startMs": 645440,
    "text": "that's fine. That's a lot simpler for an"
  },
  {
    "startMs": 647520,
    "text": "engineer to add input into into rather"
  },
  {
    "startMs": 651120,
    "text": "than them spending time to set up all"
  },
  {
    "startMs": 652920,
    "text": "these primitives themselves."
  },
  {
    "startMs": 654880,
    "text": "And then we can continue to drive this"
  },
  {
    "startMs": 656600,
    "text": "work, you know, through Slack, through"
  },
  {
    "startMs": 658600,
    "text": "Linear,"
  },
  {
    "startMs": 659720,
    "text": "um and ultimately uh results in PRs in"
  },
  {
    "startMs": 662480,
    "text": "GitHub."
  },
  {
    "startMs": 663960,
    "text": "The other thing that this lets us do is"
  },
  {
    "startMs": 665480,
    "text": "we don't necessarily always need to use"
  },
  {
    "startMs": 667760,
    "text": "our coding agent to implement uh pieces"
  },
  {
    "startMs": 671520,
    "text": "of work. Uh Devin is very popular. We"
  },
  {
    "startMs": 673520,
    "text": "have a lot of folks that really like"
  },
  {
    "startMs": 674680,
    "text": "using Devin. They can point Devin to"
  },
  {
    "startMs": 677360,
    "text": "these tickets and the same documentation"
  },
  {
    "startMs": 680480,
    "text": "to give Devin the context it needs to go"
  },
  {
    "startMs": 682600,
    "text": "and implement different parts of the"
  },
  {
    "startMs": 683760,
    "text": "work. Uh same with the local cloud code"
  },
  {
    "startMs": 686000,
    "text": "if you're just using Opus in a local"
  },
  {
    "startMs": 688000,
    "text": "harness."
  },
  {
    "startMs": 689120,
    "text": "Um it can grab through MCP all of these"
  },
  {
    "startMs": 693400,
    "text": "uh"
  },
  {
    "startMs": 694280,
    "text": "all of these pieces of documentation and"
  },
  {
    "startMs": 698000,
    "text": "use that as context for its work."
  },
  {
    "startMs": 701520,
    "text": "And then, of course, we have people that"
  },
  {
    "startMs": 703320,
    "text": "collaborate in these project channels as"
  },
  {
    "startMs": 704760,
    "text": "well. You'll notice our security team"
  },
  {
    "startMs": 706640,
    "text": "will come in and look take a look at new"
  },
  {
    "startMs": 708480,
    "text": "projects, weigh in on the security"
  },
  {
    "startMs": 710520,
    "text": "implications, and so forth."
  },
  {
    "startMs": 719720,
    "text": "So, I alluded to we we kind of early on"
  },
  {
    "startMs": 723200,
    "text": "in the the development of our factory is"
  },
  {
    "startMs": 725160,
    "text": "created our own MCP gateway."
  },
  {
    "startMs": 727520,
    "text": "Um, we call it our context engine."
  },
  {
    "startMs": 730000,
    "text": "Um, this connects into all of our"
  },
  {
    "startMs": 731840,
    "text": "internal systems, but it also builds"
  },
  {
    "startMs": 735640,
    "text": "system prompts and context around what"
  },
  {
    "startMs": 738560,
    "text": "how to navigate these tools and when to"
  },
  {
    "startMs": 740560,
    "text": "use these tools."
  },
  {
    "startMs": 741960,
    "text": "So, it is connected into Snowflake,"
  },
  {
    "startMs": 745080,
    "text": "which is our main data lake. We have"
  },
  {
    "startMs": 748280,
    "text": "semantic tables that we've built in"
  },
  {
    "startMs": 750200,
    "text": "Snowflake that describe product"
  },
  {
    "startMs": 752160,
    "text": "utilization or customer conversations."
  },
  {
    "startMs": 755480,
    "text": "And then, we can provide an agent"
  },
  {
    "startMs": 758400,
    "text": "through our MCP server in the tool"
  },
  {
    "startMs": 760040,
    "text": "description"
  },
  {
    "startMs": 762240,
    "text": "list of here are the tables, this is the"
  },
  {
    "startMs": 764040,
    "text": "content they"
  },
  {
    "startMs": 765480,
    "text": "contain. If you're trying to answer"
  },
  {
    "startMs": 767520,
    "text": "questions about this type of content,"
  },
  {
    "startMs": 770440,
    "text": "write a query for these tables. And so,"
  },
  {
    "startMs": 772320,
    "text": "it gives us a little bit of orientation"
  },
  {
    "startMs": 774480,
    "text": "to an agent to understand how to use how"
  },
  {
    "startMs": 777360,
    "text": "we use our tools, how we've organized"
  },
  {
    "startMs": 779600,
    "text": "linear in Snowflake, and give some"
  },
  {
    "startMs": 783200,
    "text": "guidance on where to find information."
  },
  {
    "startMs": 786720,
    "text": "What has been really"
  },
  {
    "startMs": 788320,
    "text": "surprising about this is we actually now"
  },
  {
    "startMs": 790480,
    "text": "use this MCP server across a lot of"
  },
  {
    "startMs": 792600,
    "text": "other different internal tools."
  },
  {
    "startMs": 795000,
    "text": "We open it up for people to query"
  },
  {
    "startMs": 797080,
    "text": "directly from Slack"
  },
  {
    "startMs": 799200,
    "text": "to do data or customer analysis. And so,"
  },
  {
    "startMs": 801680,
    "text": "while we initially kind of envisioned"
  },
  {
    "startMs": 803280,
    "text": "that this would just be a way to connect"
  },
  {
    "startMs": 805400,
    "text": "our agent or coding agent to all of our"
  },
  {
    "startMs": 807600,
    "text": "systems. Uh this has been an incredible"
  },
  {
    "startMs": 810520,
    "text": "piece of leverage for a lot of our"
  },
  {
    "startMs": 812760,
    "text": "internal teams, and we're building a lot"
  },
  {
    "startMs": 814520,
    "text": "of other tools on top of this MCP"
  },
  {
    "startMs": 817120,
    "text": "gateway. So, if you're just getting"
  },
  {
    "startMs": 818880,
    "text": "started with thinking about a software"
  },
  {
    "startMs": 820640,
    "text": "factory, it's well worth your time to"
  },
  {
    "startMs": 823160,
    "text": "invest in an internal MCP gateway server"
  },
  {
    "startMs": 826640,
    "text": "that both connects all your tools,"
  },
  {
    "startMs": 829520,
    "text": "has descriptions that can tell the agent"
  },
  {
    "startMs": 831840,
    "text": "how to use those tools and how you"
  },
  {
    "startMs": 833240,
    "text": "specifically organize the information in"
  },
  {
    "startMs": 835720,
    "text": "them, and I think you'll find that that"
  },
  {
    "startMs": 838160,
    "text": "ends up being useful in a lot of other"
  },
  {
    "startMs": 840000,
    "text": "use cases."
  },
  {
    "startMs": 846600,
    "text": "Like I mentioned, we try to automate"
  },
  {
    "startMs": 849040,
    "text": "other parts of our software development."
  },
  {
    "startMs": 851880,
    "text": "Um we have like bug"
  },
  {
    "startMs": 854520,
    "text": "bug requests that come in through Slack."
  },
  {
    "startMs": 857200,
    "text": "Uh so, TARS through webhooks can listen"
  },
  {
    "startMs": 858840,
    "text": "to those and take a first pass at"
  },
  {
    "startMs": 861560,
    "text": "triaging and implementing opening PRs to"
  },
  {
    "startMs": 864240,
    "text": "fix bugs."
  },
  {
    "startMs": 865800,
    "text": "Uh we have a lot of our customers uh in"
  },
  {
    "startMs": 868240,
    "text": "shared channels in Slack, as well."
  },
  {
    "startMs": 870400,
    "text": "Uh we found TARS to be pretty useful to"
  },
  {
    "startMs": 873400,
    "text": "uh triage"
  },
  {
    "startMs": 874960,
    "text": "support requests that we're getting"
  },
  {
    "startMs": 876400,
    "text": "through them, because it can look at the"
  },
  {
    "startMs": 878400,
    "text": "code, it can often understand like where"
  },
  {
    "startMs": 880720,
    "text": "the customer might be running into"
  },
  {
    "startMs": 882560,
    "text": "problems with our product uh from the"
  },
  {
    "startMs": 884839,
    "text": "code perspective."
  },
  {
    "startMs": 886400,
    "text": "Um I showed you an example of how we"
  },
  {
    "startMs": 888800,
    "text": "kind of organize major features around"
  },
  {
    "startMs": 890880,
    "text": "this."
  },
  {
    "startMs": 891960,
    "text": "Uh and then the last place I think is"
  },
  {
    "startMs": 893400,
    "text": "really exciting, we're all trying to get"
  },
  {
    "startMs": 895240,
    "text": "to the self-improving software, or the"
  },
  {
    "startMs": 897320,
    "text": "self-driving software. So, we're using"
  },
  {
    "startMs": 899920,
    "text": "uh TARS to actually build out our own uh"
  },
  {
    "startMs": 902920,
    "text": "sandbox infrastructure."
  },
  {
    "startMs": 905040,
    "text": "So, we want to move off of some of the"
  },
  {
    "startMs": 907760,
    "text": "sandboxes as a service and own that"
  },
  {
    "startMs": 909480,
    "text": "infrastructure layer ourselves. Um"
  },
  {
    "startMs": 913280,
    "text": "Some of the reasons we have for that is"
  },
  {
    "startMs": 915040,
    "text": "we want really deep control over the"
  },
  {
    "startMs": 918240,
    "text": "session information, and"
  },
  {
    "startMs": 920880,
    "text": "uh be able to move workloads around"
  },
  {
    "startMs": 923080,
    "text": "different parts of our infrastructure."
  },
  {
    "startMs": 925520,
    "text": "Uh so what we're building out next is"
  },
  {
    "startMs": 927520,
    "text": "our memory layer and we want this to be"
  },
  {
    "startMs": 929880,
    "text": "an evergreen context uh about what every"
  },
  {
    "startMs": 934000,
    "text": "person in the company is working on,"
  },
  {
    "startMs": 936160,
    "text": "what team do they sit in, what products"
  },
  {
    "startMs": 938560,
    "text": "are they responsible for, and then at an"
  },
  {
    "startMs": 940600,
    "text": "organization level, what is the"
  },
  {
    "startMs": 942280,
    "text": "semantics about Work OS and how we do"
  },
  {
    "startMs": 944440,
    "text": "our work. And this we want to be able to"
  },
  {
    "startMs": 946440,
    "text": "plug into both our software factory, but"
  },
  {
    "startMs": 949400,
    "text": "lift that context out of our factory and"
  },
  {
    "startMs": 951640,
    "text": "into our other AI tools as well."
  },
  {
    "startMs": 957800,
    "text": "So, I think this is kind of our our"
  },
  {
    "startMs": 960080,
    "text": "major takeaway of where we've seen a lot"
  },
  {
    "startMs": 963160,
    "text": "of folks invest effort just around"
  },
  {
    "startMs": 965040,
    "text": "outputs and where we think we can"
  },
  {
    "startMs": 966440,
    "text": "actually get that human exponential"
  },
  {
    "startMs": 969520,
    "text": "value out of our factory."
  },
  {
    "startMs": 971840,
    "text": "Can we be actually delivering values to"
  },
  {
    "startMs": 974680,
    "text": "our customers? We think a lot about"
  },
  {
    "startMs": 976520,
    "text": "shipping and customer impact and so we"
  },
  {
    "startMs": 978880,
    "text": "want to encode that into their our"
  },
  {
    "startMs": 980880,
    "text": "software factory itself. Um certainly"
  },
  {
    "startMs": 984280,
    "text": "worried about, you know, introducing"
  },
  {
    "startMs": 986120,
    "text": "instability into our products. So, all"
  },
  {
    "startMs": 988839,
    "text": "right, can we measure"
  },
  {
    "startMs": 990520,
    "text": "uh defect rate and other"
  },
  {
    "startMs": 992800,
    "text": "um"
  },
  {
    "startMs": 993400,
    "text": "you know, time to recovery uh metrics?"
  },
  {
    "startMs": 996400,
    "text": "Um and then a lot of anecdotal, like we"
  },
  {
    "startMs": 998480,
    "text": "want to see our engineers using TARS as"
  },
  {
    "startMs": 1002600,
    "text": "a sign that this is giving them value"
  },
  {
    "startMs": 1004960,
    "text": "and accelerating their work. We want to"
  },
  {
    "startMs": 1007000,
    "text": "see that they're moving off electing to"
  },
  {
    "startMs": 1008800,
    "text": "move out of their local harness and into"
  },
  {
    "startMs": 1011600,
    "text": "a cloud sandbox."
  },
  {
    "startMs": 1018120,
    "text": "And then the real goal is we can use"
  },
  {
    "startMs": 1020560,
    "text": "this information because we own the"
  },
  {
    "startMs": 1021960,
    "text": "infrastructure, we can see what's"
  },
  {
    "startMs": 1023080,
    "text": "happening in the infrastructure. We use"
  },
  {
    "startMs": 1025160,
    "text": "this information to self-improve the"
  },
  {
    "startMs": 1026680,
    "text": "factory."
  },
  {
    "startMs": 1028079,
    "text": "We want it to learn, we want it to get"
  },
  {
    "startMs": 1029959,
    "text": "better about how it writes code."
  },
  {
    "startMs": 1033880,
    "text": "We want to see where um our engineers"
  },
  {
    "startMs": 1037160,
    "text": "can improve their skills. There's so"
  },
  {
    "startMs": 1039360,
    "text": "much moving quickly in AI. It's kind of"
  },
  {
    "startMs": 1042280,
    "text": "like a constant race to keep up with the"
  },
  {
    "startMs": 1044319,
    "text": "latest"
  },
  {
    "startMs": 1045839,
    "text": "um"
  },
  {
    "startMs": 1046520,
    "text": "you know, tips and techniques."
  },
  {
    "startMs": 1048560,
    "text": "We can actually point agents at sessions"
  },
  {
    "startMs": 1050720,
    "text": "and see where are the gaps and how"
  },
  {
    "startMs": 1052360,
    "text": "people are using"
  },
  {
    "startMs": 1054200,
    "text": "our factory. Where is there a skill that"
  },
  {
    "startMs": 1056320,
    "text": "we should be building because the agent"
  },
  {
    "startMs": 1058880,
    "text": "made a mistake. What skills are no"
  },
  {
    "startMs": 1060960,
    "text": "longer relevant? Maybe the code has"
  },
  {
    "startMs": 1063360,
    "text": "changed to a degree where a skill that"
  },
  {
    "startMs": 1065680,
    "text": "we wrote six months ago is now obsolete."
  },
  {
    "startMs": 1068280,
    "text": "So, that continues verification."
  },
  {
    "startMs": 1070760,
    "text": "Semi-online"
  },
  {
    "startMs": 1072400,
    "text": "is where we kind of see a big advantage"
  },
  {
    "startMs": 1074760,
    "text": "to actually owning all the"
  },
  {
    "startMs": 1076000,
    "text": "infrastructure ourselves."
  },
  {
    "startMs": 1082520,
    "text": "So,"
  },
  {
    "startMs": 1084360,
    "text": "say it for the third time, it's worth"
  },
  {
    "startMs": 1085560,
    "text": "repeating."
  },
  {
    "startMs": 1087200,
    "text": "Sandboxes are great."
  },
  {
    "startMs": 1089120,
    "text": "It's cool to run an agent and have it"
  },
  {
    "startMs": 1090800,
    "text": "open a PR."
  },
  {
    "startMs": 1092200,
    "text": "We really think about our software"
  },
  {
    "startMs": 1093720,
    "text": "engineering practices and processes and"
  },
  {
    "startMs": 1095720,
    "text": "we want to encode those in automation."
  },
  {
    "startMs": 1101120,
    "text": "I'm Ryan. I'm one of the engineers here"
  },
  {
    "startMs": 1102800,
    "text": "at WorkOS. We have a booth right"
  },
  {
    "startMs": 1104600,
    "text": "opposite the speaking area. If you are"
  },
  {
    "startMs": 1106600,
    "text": "building software factories, I would"
  },
  {
    "startMs": 1107760,
    "text": "love to talk to you. I would really love"
  },
  {
    "startMs": 1109760,
    "text": "to hear how you're handling"
  },
  {
    "startMs": 1110680,
    "text": "authorization. This is something I don't"
  },
  {
    "startMs": 1112760,
    "text": "talk about because we haven't figured it"
  },
  {
    "startMs": 1114680,
    "text": "out yet for ourselves, but maybe you all"
  },
  {
    "startMs": 1117000,
    "text": "you all have some insights. So, please"
  },
  {
    "startMs": 1118560,
    "text": "come by the booth, talk to me, would"
  },
  {
    "startMs": 1120880,
    "text": "love to chat. Thank you."
  },
  {
    "startMs": 1135783,
    "text": ">> [music]"
  }
]