[
  {
    "startMs": 36880,
    "text": "Uh hi everyone. Hey what's up? I'm"
  },
  {
    "startMs": 39440,
    "text": "Teresa. Uh and today I will be talking"
  },
  {
    "startMs": 42399,
    "text": "about software factory. uh everyone is"
  },
  {
    "startMs": 45120,
    "text": "talking about software factory but only"
  },
  {
    "startMs": 47760,
    "text": "few people are actually building one and"
  },
  {
    "startMs": 50079,
    "text": "even fewer people know what it actually"
  },
  {
    "startMs": 52320,
    "text": "takes to build one and how to define it."
  },
  {
    "startMs": 54800,
    "text": "So I will be talking about what is a"
  },
  {
    "startMs": 57199,
    "text": "software factory, should I build your"
  },
  {
    "startMs": 59199,
    "text": "own or outsource it, what works in"
  },
  {
    "startMs": 61520,
    "text": "production already, what are the main"
  },
  {
    "startMs": 63280,
    "text": "challenges and how much is this all"
  },
  {
    "startMs": 66400,
    "text": "going to cost?"
  },
  {
    "startMs": 68640,
    "text": "Uh my name is Theresa. I work at a"
  },
  {
    "startMs": 70640,
    "text": "company called factory.com and we've"
  },
  {
    "startMs": 73119,
    "text": "been building uh this concept of the"
  },
  {
    "startMs": 75439,
    "text": "software factory for quite a long time"
  },
  {
    "startMs": 77920,
    "text": "but finally uh the technology is"
  },
  {
    "startMs": 80640,
    "text": "catching up and it's possible to build"
  },
  {
    "startMs": 82400,
    "text": "this in production for enterprises like"
  },
  {
    "startMs": 84640,
    "text": "EY or Adobe uh I would define the"
  },
  {
    "startMs": 87840,
    "text": "software factory as the whole loop the"
  },
  {
    "startMs": 90320,
    "text": "whole life cycle of developing software"
  },
  {
    "startMs": 92960,
    "text": "with autonomy which doesn't mean just"
  },
  {
    "startMs": 95280,
    "text": "coding and generating code by that I"
  },
  {
    "startMs": 97600,
    "text": "mean collecting all the signals reacting"
  },
  {
    "startMs": 100000,
    "text": "to user feedback to logs, prioritizing"
  },
  {
    "startMs": 102720,
    "text": "what's important, then orchestrating it"
  },
  {
    "startMs": 105200,
    "text": "all, executing, validating, doing really"
  },
  {
    "startMs": 108240,
    "text": "good uh testing in production and then"
  },
  {
    "startMs": 110799,
    "text": "iterating on all this uh while also"
  },
  {
    "startMs": 113280,
    "text": "continuously improving in the process"
  },
  {
    "startMs": 115360,
    "text": "and gaining new knowledge and new"
  },
  {
    "startMs": 117600,
    "text": "skills. So this is just our dashboard"
  },
  {
    "startMs": 120320,
    "text": "how our software factory looks if you if"
  },
  {
    "startMs": 123200,
    "text": "you look at it and it's catching the"
  },
  {
    "startMs": 124960,
    "text": "whole cycle there. uh this is just some"
  },
  {
    "startMs": 128080,
    "text": "history and by history I mean 2023 in"
  },
  {
    "startMs": 131680,
    "text": "AI. So the software factory wasn't"
  },
  {
    "startMs": 134080,
    "text": "really possible before even though we"
  },
  {
    "startMs": 136560,
    "text": "always had this idea from the beginning"
  },
  {
    "startMs": 138720,
    "text": "of uh CH GPT lounge and others we had"
  },
  {
    "startMs": 140959,
    "text": "this idea of the continuous loop there"
  },
  {
    "startMs": 143360,
    "text": "was autog baby agi already with the"
  },
  {
    "startMs": 146160,
    "text": "concepts of iterating on the software"
  },
  {
    "startMs": 148720,
    "text": "but it didn't work before because the"
  },
  {
    "startMs": 151040,
    "text": "LMS were hallucinating there was problem"
  },
  {
    "startMs": 153440,
    "text": "with context length uh problem with the"
  },
  {
    "startMs": 156160,
    "text": "reasoning quality uh different problems"
  },
  {
    "startMs": 158560,
    "text": "like this or we were missing good"
  },
  {
    "startMs": 160720,
    "text": "environments where the agents can"
  },
  {
    "startMs": 162400,
    "text": "actually work in the isolation. So all"
  },
  {
    "startMs": 165120,
    "text": "this kind of led to why the software"
  },
  {
    "startMs": 167599,
    "text": "factory is the paradigm that's starting"
  },
  {
    "startMs": 171040,
    "text": "to be popular now and actually useful."
  },
  {
    "startMs": 175200,
    "text": "I like to define things by what they are"
  },
  {
    "startMs": 177680,
    "text": "not. So I want to say software factory"
  },
  {
    "startMs": 180080,
    "text": "is not just coding agent and it's not"
  },
  {
    "startMs": 182319,
    "text": "even a swarm of coding agents even"
  },
  {
    "startMs": 184720,
    "text": "thousands of agents because generating"
  },
  {
    "startMs": 187040,
    "text": "code writing code that's the easy part"
  },
  {
    "startMs": 190000,
    "text": "compared to all the others. uh engineers"
  },
  {
    "startMs": 192159,
    "text": "don't even spend most of the time uh"
  },
  {
    "startMs": 194239,
    "text": "just writing code. The challenges are"
  },
  {
    "startMs": 196480,
    "text": "the rest of it. Uh so this is what I"
  },
  {
    "startMs": 198720,
    "text": "want to enhance and it's also not just"
  },
  {
    "startMs": 200959,
    "text": "some consultancy or just some abstract"
  },
  {
    "startMs": 204159,
    "text": "strategy that someone will try to sell"
  },
  {
    "startMs": 206239,
    "text": "you and to your organization. Uh I think"
  },
  {
    "startMs": 209840,
    "text": "the approaches are of course different"
  },
  {
    "startMs": 211280,
    "text": "but we believe you should really rebuild"
  },
  {
    "startMs": 213760,
    "text": "your organization from ground up to be"
  },
  {
    "startMs": 216480,
    "text": "ready to become a software factory like"
  },
  {
    "startMs": 218879,
    "text": "you can't invite a consultancy and just"
  },
  {
    "startMs": 222159,
    "text": "throw something in the middle of your"
  },
  {
    "startMs": 224239,
    "text": "organization. You should really be"
  },
  {
    "startMs": 226000,
    "text": "mindful and rebuild it from scratch."
  },
  {
    "startMs": 230080,
    "text": "uh this is the approach I like uh"
  },
  {
    "startMs": 233599,
    "text": "showing to building the software factory"
  },
  {
    "startMs": 235680,
    "text": "and I think it's similar to building the"
  },
  {
    "startMs": 238080,
    "text": "real team of humans because there is a"
  },
  {
    "startMs": 240319,
    "text": "lot of agents as I said they will be"
  },
  {
    "startMs": 242400,
    "text": "testing validating iterating and it all"
  },
  {
    "startMs": 245760,
    "text": "can end up as a big chaos uh so you need"
  },
  {
    "startMs": 248400,
    "text": "to be mindful and I would say the three"
  },
  {
    "startMs": 250480,
    "text": "important things is being agnostic in"
  },
  {
    "startMs": 253360,
    "text": "the software factory so being"
  },
  {
    "startMs": 255280,
    "text": "independent of LLM choices uh of how you"
  },
  {
    "startMs": 258799,
    "text": "already work as the organization then"
  },
  {
    "startMs": 261040,
    "text": "you need to be autonomous. So you need"
  },
  {
    "startMs": 262639,
    "text": "to give the agents the right trusts and"
  },
  {
    "startMs": 264479,
    "text": "all this permissions and governance and"
  },
  {
    "startMs": 266800,
    "text": "you need to trust it to run really for a"
  },
  {
    "startMs": 268880,
    "text": "long time because the predictions are"
  },
  {
    "startMs": 270880,
    "text": "that agents will run even a year or more"
  },
  {
    "startMs": 273440,
    "text": "years without the human in the loop. Of"
  },
  {
    "startMs": 275759,
    "text": "course that sounds ambitious but for"
  },
  {
    "startMs": 277360,
    "text": "example our missions already our long"
  },
  {
    "startMs": 279600,
    "text": "runninging sessions already build uh"
  },
  {
    "startMs": 281759,
    "text": "things and run for weeks. So it's not"
  },
  {
    "startMs": 284160,
    "text": "that crazy. And third thing always"
  },
  {
    "startMs": 286240,
    "text": "improving. So same as human organization"
  },
  {
    "startMs": 289600,
    "text": "you need to onboard any new people in"
  },
  {
    "startMs": 291360,
    "text": "your team by giving them good uh code"
  },
  {
    "startMs": 294080,
    "text": "base understanding good structure"
  },
  {
    "startMs": 296240,
    "text": "documentation and also let them improve"
  },
  {
    "startMs": 298560,
    "text": "in the process and gain new knowledge"
  },
  {
    "startMs": 300800,
    "text": "and share it share it within the team."
  },
  {
    "startMs": 304160,
    "text": "So these are the three points and now"
  },
  {
    "startMs": 307039,
    "text": "let's talk about the agnostic one. Uh I"
  },
  {
    "startMs": 310400,
    "text": "think the important thing to point out"
  },
  {
    "startMs": 312000,
    "text": "if you are a builder and building"
  },
  {
    "startMs": 313520,
    "text": "different primitives of the software"
  },
  {
    "startMs": 315039,
    "text": "factory you need to be ready for how"
  },
  {
    "startMs": 317440,
    "text": "everyone is already working you need to"
  },
  {
    "startMs": 320160,
    "text": "work across environments like Slack,"
  },
  {
    "startMs": 322320,
    "text": "GitHub and connected to everything and"
  },
  {
    "startMs": 325280,
    "text": "you need to also allow people to already"
  },
  {
    "startMs": 327440,
    "text": "bring subscriptions they are using"
  },
  {
    "startMs": 329759,
    "text": "because there is every every time there"
  },
  {
    "startMs": 332000,
    "text": "is like new technology, new frontier"
  },
  {
    "startMs": 333759,
    "text": "models, uh new benchmarks and it's so"
  },
  {
    "startMs": 336000,
    "text": "difficult to catch up but also to"
  },
  {
    "startMs": 338000,
    "text": "predict what we'll be winning. So it's"
  },
  {
    "startMs": 340720,
    "text": "best to be agnostic for everyone."
  },
  {
    "startMs": 344000,
    "text": "Uh then there's the topic of being"
  },
  {
    "startMs": 346160,
    "text": "agnostic to models. Uh this is a topic"
  },
  {
    "startMs": 349440,
    "text": "of a lot of discussions recently and"
  },
  {
    "startMs": 351919,
    "text": "this chart is from this chart is from"
  },
  {
    "startMs": 354400,
    "text": "CEO of Coinbase uh who tweeted about how"
  },
  {
    "startMs": 357520,
    "text": "they started uh started saving money in"
  },
  {
    "startMs": 360720,
    "text": "their organ or organization on AI but"
  },
  {
    "startMs": 363280,
    "text": "without reducing the token spend. So the"
  },
  {
    "startMs": 365360,
    "text": "black line is how much they spent on"
  },
  {
    "startMs": 367199,
    "text": "tokens. They continue growing and token"
  },
  {
    "startMs": 369759,
    "text": "maxing but they stopped spending so much"
  },
  {
    "startMs": 372400,
    "text": "money. Uh how they did it uh different"
  },
  {
    "startMs": 375360,
    "text": "tricks like different default default"
  },
  {
    "startMs": 377600,
    "text": "models for people. So stop pushing"
  },
  {
    "startMs": 379680,
    "text": "people to use only the frontier as a"
  },
  {
    "startMs": 381520,
    "text": "default. Then caching of course uh to"
  },
  {
    "startMs": 385280,
    "text": "stop preferring the stuff every time."
  },
  {
    "startMs": 387840,
    "text": "And then no limits on spending but just"
  },
  {
    "startMs": 389919,
    "text": "needing to see the results if you spend"
  },
  {
    "startMs": 391600,
    "text": "a lot. And the last thing that's really"
  },
  {
    "startMs": 394319,
    "text": "important topic is routing. So smart"
  },
  {
    "startMs": 397360,
    "text": "routing between LLMs and just a token"
  },
  {
    "startMs": 400160,
    "text": "optimizing between uh the LMS instead of"
  },
  {
    "startMs": 403039,
    "text": "just spending so much tokens."
  },
  {
    "startMs": 406479,
    "text": "Uh we built uh a thing in factory called"
  },
  {
    "startMs": 409919,
    "text": "uh automatic model routing and there's"
  },
  {
    "startMs": 413600,
    "text": "been a lot of questions around it but"
  },
  {
    "startMs": 415360,
    "text": "what it does is it automatically routes"
  },
  {
    "startMs": 417840,
    "text": "between different models in your"
  },
  {
    "startMs": 419919,
    "text": "process. The important thing is it"
  },
  {
    "startMs": 422080,
    "text": "decides first what model is most optimal"
  },
  {
    "startMs": 424560,
    "text": "and then uh it can switch to different"
  },
  {
    "startMs": 427440,
    "text": "model if it's failing the task or in"
  },
  {
    "startMs": 429759,
    "text": "case of any troubles but it usually"
  },
  {
    "startMs": 431680,
    "text": "doesn't happen and important thing is it"
  },
  {
    "startMs": 434960,
    "text": "doesn't just help uh spend money. It"
  },
  {
    "startMs": 438319,
    "text": "also helps with reliability or with"
  },
  {
    "startMs": 440319,
    "text": "speed because open source models are"
  },
  {
    "startMs": 442560,
    "text": "often faster and uh if one LM provider"
  },
  {
    "startMs": 446319,
    "text": "fails you can just switch to another one"
  },
  {
    "startMs": 448319,
    "text": "automatically. So it really does more"
  },
  {
    "startMs": 450160,
    "text": "things than just saving the money. This"
  },
  {
    "startMs": 453599,
    "text": "uh is our benchmark which is very"
  },
  {
    "startMs": 455280,
    "text": "conservative. I prefer conservative"
  },
  {
    "startMs": 457599,
    "text": "benchmarks but I think the direction is"
  },
  {
    "startMs": 460160,
    "text": "clear. You can save for example 25% but"
  },
  {
    "startMs": 464080,
    "text": "even more probably."
  },
  {
    "startMs": 466479,
    "text": "Uh how the routing works uh there are"
  },
  {
    "startMs": 468960,
    "text": "four parts. First you just assign the"
  },
  {
    "startMs": 471199,
    "text": "task and you don't really need to do"
  },
  {
    "startMs": 473360,
    "text": "anything but as an organization you can"
  },
  {
    "startMs": 475440,
    "text": "for example give different permissions"
  },
  {
    "startMs": 477919,
    "text": "and different default models to"
  },
  {
    "startMs": 479360,
    "text": "different people which is very useful"
  },
  {
    "startMs": 481440,
    "text": "like marketing or sales or engineers can"
  },
  {
    "startMs": 483840,
    "text": "both uh have different default models"
  },
  {
    "startMs": 486639,
    "text": "and then you have the classification"
  },
  {
    "startMs": 488560,
    "text": "which is the important part. This is the"
  },
  {
    "startMs": 490160,
    "text": "magic of the routing. You need to really"
  },
  {
    "startMs": 492479,
    "text": "uh look at the structure of the prompt"
  },
  {
    "startMs": 494319,
    "text": "of the uh code base of how difficult the"
  },
  {
    "startMs": 497440,
    "text": "task is, what tools are being used, all"
  },
  {
    "startMs": 500000,
    "text": "these factors and make a classification"
  },
  {
    "startMs": 502240,
    "text": "of difficulty of the task. Uh after that"
  },
  {
    "startMs": 505919,
    "text": "you basically do a threshold of what is"
  },
  {
    "startMs": 508800,
    "text": "enough to accomplish the task and you"
  },
  {
    "startMs": 511360,
    "text": "choose the cheapest model above the"
  },
  {
    "startMs": 513360,
    "text": "threshold. So cheapest model that is"
  },
  {
    "startMs": 516240,
    "text": "predicted to accomplish your task and"
  },
  {
    "startMs": 518320,
    "text": "then you go."
  },
  {
    "startMs": 521039,
    "text": "Okay. So when we launched this we got a"
  },
  {
    "startMs": 522880,
    "text": "lot of questions like does it actually"
  },
  {
    "startMs": 525440,
    "text": "work? What if uh the model mis routes?"
  },
  {
    "startMs": 528560,
    "text": "Uh is it slower? Is it more expensive"
  },
  {
    "startMs": 530800,
    "text": "actually? What if the model can't"
  },
  {
    "startMs": 532480,
    "text": "accomplish the task or what if it needs"
  },
  {
    "startMs": 534880,
    "text": "to upgrade? And how do you handle"
  },
  {
    "startMs": 536399,
    "text": "caching? So I think these are all valid"
  },
  {
    "startMs": 539600,
    "text": "questions and that's why it's so"
  },
  {
    "startMs": 542000,
    "text": "difficult to build a good router like"
  },
  {
    "startMs": 543680,
    "text": "you need to basically classify it very"
  },
  {
    "startMs": 546160,
    "text": "well and the challenge is to not not to"
  },
  {
    "startMs": 548880,
    "text": "need to switch too often but even if"
  },
  {
    "startMs": 551040,
    "text": "you're switching in the middle of the"
  },
  {
    "startMs": 552320,
    "text": "task to difficult to more difficult"
  },
  {
    "startMs": 554399,
    "text": "model you still overall are faster"
  },
  {
    "startMs": 557920,
    "text": "probably because it's still worth"
  },
  {
    "startMs": 561120,
    "text": "uh this is just some overview of caching"
  },
  {
    "startMs": 564480,
    "text": "uh lot of questions we got on this was"
  },
  {
    "startMs": 566560,
    "text": "also So how do we handle caching? Do we"
  },
  {
    "startMs": 569600,
    "text": "do discounts for users? Uh because LLM"
  },
  {
    "startMs": 572880,
    "text": "LLM labs of course save a lot of money"
  },
  {
    "startMs": 574720,
    "text": "by caching and skipping all the context"
  },
  {
    "startMs": 577920,
    "text": "prefill all the time. Uh open models can"
  },
  {
    "startMs": 581040,
    "text": "do this as well. I think people"
  },
  {
    "startMs": 582560,
    "text": "sometimes forget this and you can just"
  },
  {
    "startMs": 584800,
    "text": "host open models as well uh on dedicated"
  },
  {
    "startMs": 587680,
    "text": "compute and you can take the same"
  },
  {
    "startMs": 589600,
    "text": "advantage of the caching. Uh so I want"
  },
  {
    "startMs": 592800,
    "text": "to enhance uh the final price for users."
  },
  {
    "startMs": 595760,
    "text": "is just a pricing decision. It's not a"
  },
  {
    "startMs": 598000,
    "text": "technical challenge because everyone can"
  },
  {
    "startMs": 600000,
    "text": "do caching. It's just what price you"
  },
  {
    "startMs": 602240,
    "text": "pass on the users and what deals you"
  },
  {
    "startMs": 604880,
    "text": "make with the API providers."
  },
  {
    "startMs": 607920,
    "text": "So this is uh being agnostic and now the"
  },
  {
    "startMs": 611120,
    "text": "autonomy which is the core of the"
  },
  {
    "startMs": 613200,
    "text": "software factory. Uh everyone is talking"
  },
  {
    "startMs": 616640,
    "text": "about loops. Uh loops I think have been"
  },
  {
    "startMs": 620000,
    "text": "here the whole time already in different"
  },
  {
    "startMs": 622560,
    "text": "context. They are just now leveling up"
  },
  {
    "startMs": 624800,
    "text": "and moving to the context of agents. You"
  },
  {
    "startMs": 627600,
    "text": "probably heard about the Ralph loop and"
  },
  {
    "startMs": 629440,
    "text": "specifying the tasks and splitting to"
  },
  {
    "startMs": 632000,
    "text": "subtasks for agents. Uh this is a known"
  },
  {
    "startMs": 635200,
    "text": "concept and uh the question is not the"
  },
  {
    "startMs": 638480,
    "text": "loop itself but the question is how how"
  },
  {
    "startMs": 640959,
    "text": "you define what it means to be done in"
  },
  {
    "startMs": 642959,
    "text": "the loop. This is one example from my"
  },
  {
    "startMs": 645760,
    "text": "colleague uh he he made a loop uh of"
  },
  {
    "startMs": 648720,
    "text": "agents building 3D printing of our logo."
  },
  {
    "startMs": 652320,
    "text": "uh I think here you can see what is the"
  },
  {
    "startMs": 654480,
    "text": "challenge of the loops because uh before"
  },
  {
    "startMs": 656560,
    "text": "in programming loops had clear criteria"
  },
  {
    "startMs": 658800,
    "text": "of what it means to be done while now"
  },
  {
    "startMs": 661519,
    "text": "the criteria become open-ended because a"
  },
  {
    "startMs": 664320,
    "text": "lot of the tasks are very"
  },
  {
    "startMs": 665440,
    "text": "nondeterministic it's basically open"
  },
  {
    "startMs": 667200,
    "text": "world you can do even real stuff like"
  },
  {
    "startMs": 669440,
    "text": "printing something and it's really"
  },
  {
    "startMs": 671279,
    "text": "difficult to define how do you verify"
  },
  {
    "startMs": 674079,
    "text": "that the loop was done that the task was"
  },
  {
    "startMs": 675839,
    "text": "accomplished so that's the difficult"
  },
  {
    "startMs": 677920,
    "text": "part"
  },
  {
    "startMs": 679680,
    "text": "uh yeah they just need to be verifiable"
  },
  {
    "startMs": 682240,
    "text": "This I call the scary chart. It"
  },
  {
    "startMs": 684160,
    "text": "basically shows that the tasks and how"
  },
  {
    "startMs": 686959,
    "text": "long they run autonomously with agents"
  },
  {
    "startMs": 688880,
    "text": "have been increasing. But still it's not"
  },
  {
    "startMs": 691760,
    "text": "that reliable. Even if you can run for a"
  },
  {
    "startMs": 694079,
    "text": "very long time, it doesn't mean that it"
  },
  {
    "startMs": 696399,
    "text": "will be reliable in production and you"
  },
  {
    "startMs": 698399,
    "text": "still probably need to need to iterate"
  },
  {
    "startMs": 700720,
    "text": "on it and uh provide feedback. So it's"
  },
  {
    "startMs": 703839,
    "text": "still not solved and there are problems"
  },
  {
    "startMs": 705279,
    "text": "like cheating. For example, if you write"
  },
  {
    "startMs": 707519,
    "text": "the what it means to be done in the"
  },
  {
    "startMs": 709760,
    "text": "wrong way, the agent can try to pass"
  },
  {
    "startMs": 712000,
    "text": "your tests but not really like not"
  },
  {
    "startMs": 715120,
    "text": "really verify what you need to do and"
  },
  {
    "startMs": 717040,
    "text": "accomplish but instead try to solve just"
  },
  {
    "startMs": 719839,
    "text": "passing your test. So cheating by that"
  },
  {
    "startMs": 723279,
    "text": "uh we have something called factory"
  },
  {
    "startMs": 724959,
    "text": "missions and this is like long"
  },
  {
    "startMs": 727360,
    "text": "longunning sessions of the agents. We"
  },
  {
    "startMs": 729279,
    "text": "just call it missions because we send"
  },
  {
    "startMs": 731040,
    "text": "the agent to the mission and uh the"
  },
  {
    "startMs": 733920,
    "text": "missions work in the loop and iterating"
  },
  {
    "startMs": 735839,
    "text": "on task until it's done and they can do"
  },
  {
    "startMs": 737760,
    "text": "it even for weeks and uh the main agent"
  },
  {
    "startMs": 740959,
    "text": "is orchestrator and then it's uh assigns"
  },
  {
    "startMs": 743680,
    "text": "work to workers agents and then"
  },
  {
    "startMs": 745920,
    "text": "validators who review the task. So this"
  },
  {
    "startMs": 748480,
    "text": "is one example uh real mission from our"
  },
  {
    "startMs": 751680,
    "text": "customers that run 16 hours and just"
  },
  {
    "startMs": 755120,
    "text": "just to see how important is the"
  },
  {
    "startMs": 756959,
    "text": "validation as well. it takes even 40% of"
  },
  {
    "startMs": 759760,
    "text": "the whole process. Uh to summarize, the"
  },
  {
    "startMs": 762639,
    "text": "orchestrator just decides and writes the"
  },
  {
    "startMs": 765600,
    "text": "conditions what it needs to be done."
  },
  {
    "startMs": 767120,
    "text": "Then it gives it to the agents called"
  },
  {
    "startMs": 769120,
    "text": "workers. The agents work on it. And what"
  },
  {
    "startMs": 772079,
    "text": "is interesting or worth noting is that"
  },
  {
    "startMs": 774160,
    "text": "they work in a sequence. So they don't"
  },
  {
    "startMs": 776320,
    "text": "work in a swarm or parallel. The agents"
  },
  {
    "startMs": 778880,
    "text": "work in sequence and everyone"
  },
  {
    "startMs": 780320,
    "text": "accomplishes something and passes it to"
  },
  {
    "startMs": 782560,
    "text": "the next one."
  },
  {
    "startMs": 783920,
    "text": "uh we actually found that if you do this"
  },
  {
    "startMs": 786160,
    "text": "you end up with more fresh context and"
  },
  {
    "startMs": 788639,
    "text": "kind of fresh head. Same with when with"
  },
  {
    "startMs": 790800,
    "text": "humans have like other colleagues"
  },
  {
    "startMs": 792880,
    "text": "verifying their code. So similar thing"
  },
  {
    "startMs": 795600,
    "text": "we just let the agents pass to the next"
  },
  {
    "startMs": 797360,
    "text": "one. But still every worker of the"
  },
  {
    "startMs": 800160,
    "text": "sequence can have parallel agents doing"
  },
  {
    "startMs": 802320,
    "text": "smaller tasks. So I don't know"
  },
  {
    "startMs": 804000,
    "text": "researching on the web or building files"
  },
  {
    "startMs": 807040,
    "text": "during that. So they are in sequence and"
  },
  {
    "startMs": 809040,
    "text": "everyone has sub agents as well. and"
  },
  {
    "startMs": 811279,
    "text": "validators they review the output and"
  },
  {
    "startMs": 813920,
    "text": "provide feedback and send it back to the"
  },
  {
    "startMs": 816480,
    "text": "beginning."
  },
  {
    "startMs": 818720,
    "text": "Uh important thing is uh the validators"
  },
  {
    "startMs": 821440,
    "text": "judge code that they didn't write. Uh"
  },
  {
    "startMs": 824639,
    "text": "there's something from the orchestrator"
  },
  {
    "startMs": 826320,
    "text": "agent called validation contract uh"
  },
  {
    "startMs": 829120,
    "text": "that's written before any code is done."
  },
  {
    "startMs": 831279,
    "text": "And first type is scrutiny validator."
  },
  {
    "startMs": 833600,
    "text": "Second is user testing validator. The"
  },
  {
    "startMs": 836079,
    "text": "scrutiny is the one who really verifies"
  },
  {
    "startMs": 838800,
    "text": "how the codebase looks. the liners types"
  },
  {
    "startMs": 841600,
    "text": "tests it's really rigorous check of the"
  },
  {
    "startMs": 844480,
    "text": "code but the second I think is really"
  },
  {
    "startMs": 846959,
    "text": "interesting is user testing validator"
  },
  {
    "startMs": 848880,
    "text": "and that's the one who is really in the"
  },
  {
    "startMs": 851120,
    "text": "arena trying the things so it doesn't"
  },
  {
    "startMs": 853839,
    "text": "care how it was made it just goes it"
  },
  {
    "startMs": 856639,
    "text": "works in its virtual computer and it"
  },
  {
    "startMs": 858639,
    "text": "clicks on the stuff and really checks if"
  },
  {
    "startMs": 860800,
    "text": "everything works I've seen one engineer"
  },
  {
    "startMs": 863519,
    "text": "migrating uh code bases with droid with"
  },
  {
    "startMs": 867199,
    "text": "our agent and uh it really needed the"
  },
  {
    "startMs": 870480,
    "text": "agent in the end to click through the"
  },
  {
    "startMs": 872000,
    "text": "stuff because with other products it"
  },
  {
    "startMs": 874240,
    "text": "just created created the product but it"
  },
  {
    "startMs": 876720,
    "text": "didn't work and was wasn't interactive."
  },
  {
    "startMs": 879440,
    "text": "It was just a dummy result. So our agent"
  },
  {
    "startMs": 882399,
    "text": "that really clicked on the stuff uh"
  },
  {
    "startMs": 885120,
    "text": "allowed to check that it's actually"
  },
  {
    "startMs": 887120,
    "text": "working and not just looking good in the"
  },
  {
    "startMs": 888880,
    "text": "code. I think this is one cool thing"
  },
  {
    "startMs": 890959,
    "text": "that was also allowed by the progress in"
  },
  {
    "startMs": 893600,
    "text": "computer use and the persistent"
  },
  {
    "startMs": 896079,
    "text": "environments virtual machines for agents"
  },
  {
    "startMs": 898560,
    "text": "which also wasn't here before or wasn't"
  },
  {
    "startMs": 901440,
    "text": "that great but now it's really great so"
  },
  {
    "startMs": 903360,
    "text": "it's allowing to go in the arena as an"
  },
  {
    "startMs": 906160,
    "text": "agent. Uh this is just one example of"
  },
  {
    "startMs": 909120,
    "text": "real mission from our users. I I like to"
  },
  {
    "startMs": 911839,
    "text": "just learn what users are building and"
  },
  {
    "startMs": 913519,
    "text": "get feedback and this is just uh how it"
  },
  {
    "startMs": 916720,
    "text": "can look in your dashboard. And this is"
  },
  {
    "startMs": 919519,
    "text": "how I visualize the missions. What is"
  },
  {
    "startMs": 922320,
    "text": "interesting is that it's just a loop"
  },
  {
    "startMs": 925519,
    "text": "with smaller loops because every of the"
  },
  {
    "startMs": 927519,
    "text": "agents are running in loops as well, but"
  },
  {
    "startMs": 930480,
    "text": "overall it's like one big loop. Okay,"
  },
  {
    "startMs": 932560,
    "text": "maybe this is a bit weird, but I just"
  },
  {
    "startMs": 935120,
    "text": "like that it's all loops with smaller"
  },
  {
    "startMs": 937360,
    "text": "loops."
  },
  {
    "startMs": 939040,
    "text": "Uh, okay. And the third part, always"
  },
  {
    "startMs": 941839,
    "text": "improving, always learning."
  },
  {
    "startMs": 944720,
    "text": "Uh there is this elephant in the room"
  },
  {
    "startMs": 946959,
    "text": "context with agents. A lot of questions"
  },
  {
    "startMs": 949519,
    "text": "which are valid are like how do you"
  },
  {
    "startMs": 951839,
    "text": "navigate the context blo because you"
  },
  {
    "startMs": 953680,
    "text": "work on these long missions in the"
  },
  {
    "startMs": 955440,
    "text": "software factory and how do you actually"
  },
  {
    "startMs": 957759,
    "text": "keep the context clean and that's very"
  },
  {
    "startMs": 960160,
    "text": "valid because enterprises especially"
  },
  {
    "startMs": 962079,
    "text": "they use hundreds tools they use Figma,"
  },
  {
    "startMs": 964800,
    "text": "notion, Gmail, drive, slack like"
  },
  {
    "startMs": 967759,
    "text": "hundreds tools on average. So all these"
  },
  {
    "startMs": 970399,
    "text": "have specification in the code. all"
  },
  {
    "startMs": 972399,
    "text": "these have schema and parameters and log"
  },
  {
    "startMs": 974720,
    "text": "descriptions. So, uh it can cause agent"
  },
  {
    "startMs": 978560,
    "text": "to really bloat with that and pick wrong"
  },
  {
    "startMs": 980240,
    "text": "tools if there are two tools that sound"
  },
  {
    "startMs": 982000,
    "text": "similar or to actually lose context"
  },
  {
    "startMs": 984959,
    "text": "because they fill up the the context"
  },
  {
    "startMs": 987120,
    "text": "window and need to compress. So, this is"
  },
  {
    "startMs": 989680,
    "text": "really dangerous. Uh for this we have"
  },
  {
    "startMs": 992240,
    "text": "something within the software factory"
  },
  {
    "startMs": 994560,
    "text": "called uh deferred context engine and we"
  },
  {
    "startMs": 997440,
    "text": "build it such that we just progressively"
  },
  {
    "startMs": 1000079,
    "text": "disclose what's in the context and what"
  },
  {
    "startMs": 1002320,
    "text": "tools to use. So we basically have a"
  },
  {
    "startMs": 1005600,
    "text": "surprise tools uh that help later only"
  },
  {
    "startMs": 1008480,
    "text": "if they are actually needed. So it first"
  },
  {
    "startMs": 1011279,
    "text": "just has short list of the tools and uh"
  },
  {
    "startMs": 1014639,
    "text": "only a short descriptions and when it's"
  },
  {
    "startMs": 1016320,
    "text": "needed actually in the code uh they can"
  },
  {
    "startMs": 1018560,
    "text": "call the tool and fully load it. So"
  },
  {
    "startMs": 1020959,
    "text": "important thing is nothing is actually"
  },
  {
    "startMs": 1022800,
    "text": "removed it's just hidden and not"
  },
  {
    "startMs": 1024880,
    "text": "reachable until needed. So it's just for"
  },
  {
    "startMs": 1027679,
    "text": "later and important thing is it actually"
  },
  {
    "startMs": 1030240,
    "text": "saves a lot of tokens uh at scale once"
  },
  {
    "startMs": 1032959,
    "text": "you use more and more tools. uh the more"
  },
  {
    "startMs": 1035679,
    "text": "actually you save it really scales and"
  },
  {
    "startMs": 1037600,
    "text": "you can save 50% of tokens or more."
  },
  {
    "startMs": 1042000,
    "text": "Okay. Uh another thing which is tricky"
  },
  {
    "startMs": 1044799,
    "text": "and was surprising for me for the first"
  },
  {
    "startMs": 1046720,
    "text": "time is that when adopting AI you either"
  },
  {
    "startMs": 1049840,
    "text": "succeed big or you can fail big the same"
  },
  {
    "startMs": 1052080,
    "text": "way it's a bit of power law. So if your"
  },
  {
    "startMs": 1055200,
    "text": "codebase is not ready, if you don't have"
  },
  {
    "startMs": 1057600,
    "text": "structured codebase and all important"
  },
  {
    "startMs": 1059679,
    "text": "things, then adopting or turning into"
  },
  {
    "startMs": 1062160,
    "text": "software factory can actually make you"
  },
  {
    "startMs": 1064480,
    "text": "end up worse and make your code"
  },
  {
    "startMs": 1066880,
    "text": "degrading. And uh there is a big and"
  },
  {
    "startMs": 1069840,
    "text": "growing gap in productivity between"
  },
  {
    "startMs": 1071679,
    "text": "those who just adopted AI versus those"
  },
  {
    "startMs": 1074480,
    "text": "who actually thought about it a bit"
  },
  {
    "startMs": 1076480,
    "text": "more."
  },
  {
    "startMs": 1078320,
    "text": "Uh there is another data from Stanford"
  },
  {
    "startMs": 1080559,
    "text": "about yeah without structured uh code"
  },
  {
    "startMs": 1083440,
    "text": "base and being ready and documenting"
  },
  {
    "startMs": 1085440,
    "text": "well the AI can make your code worse and"
  },
  {
    "startMs": 1087600,
    "text": "I think if you are engineers you"
  },
  {
    "startMs": 1089039,
    "text": "probably can agree that sometimes AI"
  },
  {
    "startMs": 1092240,
    "text": "really makes a mess and this can really"
  },
  {
    "startMs": 1094559,
    "text": "compound more and more and it's"
  },
  {
    "startMs": 1096240,
    "text": "difficult to go back."
  },
  {
    "startMs": 1099039,
    "text": "uh we have something as a part of the"
  },
  {
    "startMs": 1101280,
    "text": "software factory uh we have something"
  },
  {
    "startMs": 1103440,
    "text": "called agent readiness and it's a"
  },
  {
    "startMs": 1105919,
    "text": "framework I don't like the word"
  },
  {
    "startMs": 1107600,
    "text": "framework but uh take it as a hygiene"
  },
  {
    "startMs": 1110160,
    "text": "check of your codebase and things you"
  },
  {
    "startMs": 1112240,
    "text": "should do because there is actually nice"
  },
  {
    "startMs": 1114320,
    "text": "correlation between how your codebase is"
  },
  {
    "startMs": 1117120,
    "text": "looking and all the details there and"
  },
  {
    "startMs": 1118880,
    "text": "how good you're going to adopt the AI"
  },
  {
    "startMs": 1120559,
    "text": "and end up productive instead of uh with"
  },
  {
    "startMs": 1123520,
    "text": "a messy code base. So there are things"
  },
  {
    "startMs": 1125679,
    "text": "like how how reproducible is your"
  },
  {
    "startMs": 1128160,
    "text": "developer environment or if you have"
  },
  {
    "startMs": 1129919,
    "text": "written good tests if you have all"
  },
  {
    "startMs": 1132240,
    "text": "things well documented what's the style"
  },
  {
    "startMs": 1134480,
    "text": "of your code all the tests and llinters"
  },
  {
    "startMs": 1137840,
    "text": "uh everything that you would do also to"
  },
  {
    "startMs": 1140640,
    "text": "keep codebased screen. So these factors"
  },
  {
    "startMs": 1142640,
    "text": "actually show to be very useful and we"
  },
  {
    "startMs": 1145760,
    "text": "have our bigger customers actually go"
  },
  {
    "startMs": 1147919,
    "text": "through this agent readiness framework"
  },
  {
    "startMs": 1150080,
    "text": "and do all checks and then they can"
  },
  {
    "startMs": 1152160,
    "text": "follow up with recommended actions uh"
  },
  {
    "startMs": 1155120,
    "text": "how to fix that."
  },
  {
    "startMs": 1157280,
    "text": "Uh I think one more thing about the"
  },
  {
    "startMs": 1160400,
    "text": "context and the continuous learning in"
  },
  {
    "startMs": 1162960,
    "text": "the software factory is the most of the"
  },
  {
    "startMs": 1165840,
    "text": "time when working with AI or at least"
  },
  {
    "startMs": 1167919,
    "text": "from my experience is that you keep"
  },
  {
    "startMs": 1169840,
    "text": "repeating to agents how how do you want"
  },
  {
    "startMs": 1172000,
    "text": "the things done and you feel like the"
  },
  {
    "startMs": 1173919,
    "text": "agent doesn't really get you and I think"
  },
  {
    "startMs": 1176240,
    "text": "this is again nice parallel with human"
  },
  {
    "startMs": 1179200,
    "text": "organizations and teams because when you"
  },
  {
    "startMs": 1181520,
    "text": "join a new company there are a lot of"
  },
  {
    "startMs": 1183120,
    "text": "rules that are not really codified"
  },
  {
    "startMs": 1185200,
    "text": "anywhere like you just learn and observe"
  },
  {
    "startMs": 1187919,
    "text": "how things are done and there are a lot"
  },
  {
    "startMs": 1189840,
    "text": "of things between the lines that you"
  },
  {
    "startMs": 1192000,
    "text": "can't really learn anywhere else than"
  },
  {
    "startMs": 1194160,
    "text": "just observing. I think this is big"
  },
  {
    "startMs": 1196640,
    "text": "challenge for agents as well and one"
  },
  {
    "startMs": 1198880,
    "text": "thing we launched for that is plugins"
  },
  {
    "startMs": 1200799,
    "text": "which are like packaged reusable skills"
  },
  {
    "startMs": 1203760,
    "text": "and context and the things behind the"
  },
  {
    "startMs": 1206080,
    "text": "scenes that you can really codify or"
  },
  {
    "startMs": 1208720,
    "text": "auto which is automatically updating our"
  },
  {
    "startMs": 1211360,
    "text": "documentations and uh yeah and um"
  },
  {
    "startMs": 1215679,
    "text": "reviewing and documenting what you have."
  },
  {
    "startMs": 1219679,
    "text": "So the question is what will happen to"
  },
  {
    "startMs": 1221840,
    "text": "us humans if we build software factory"
  },
  {
    "startMs": 1224080,
    "text": "like this? Uh will humans just lose jobs"
  },
  {
    "startMs": 1227840,
    "text": "and go to permanent underclass? I think"
  },
  {
    "startMs": 1230640,
    "text": "they will not. And uh I want to be"
  },
  {
    "startMs": 1233360,
    "text": "positive. So I'm thinking of this"
  },
  {
    "startMs": 1235679,
    "text": "parallel of humans just moving up uh"
  },
  {
    "startMs": 1238640,
    "text": "levels up to the uh to the cooler tasks"
  },
  {
    "startMs": 1242159,
    "text": "than before. I think we actually had"
  },
  {
    "startMs": 1245200,
    "text": "before the same same experience of us as"
  },
  {
    "startMs": 1248320,
    "text": "humans abstracting some levels up and we"
  },
  {
    "startMs": 1251120,
    "text": "actually started ourselves as human"
  },
  {
    "startMs": 1253360,
    "text": "computers. We actually were the"
  },
  {
    "startMs": 1255280,
    "text": "computers doing all the most detailed"
  },
  {
    "startMs": 1257600,
    "text": "stuff at the beginning. Then we had"
  },
  {
    "startMs": 1259760,
    "text": "programming languages codifying some of"
  },
  {
    "startMs": 1261600,
    "text": "it, abstracting and then coding agents."
  },
  {
    "startMs": 1264320,
    "text": "So outsourcing some of it but uh still"
  },
  {
    "startMs": 1266960,
    "text": "being in the loop very closely uh and"
  },
  {
    "startMs": 1269440,
    "text": "very monitoring everything. And now we"
  },
  {
    "startMs": 1271120,
    "text": "are moving kind of up to software"
  },
  {
    "startMs": 1273679,
    "text": "factories where we manage these agents"
  },
  {
    "startMs": 1276240,
    "text": "as I said orchestrator agents and"
  },
  {
    "startMs": 1278400,
    "text": "workers and validators and all these"
  },
  {
    "startMs": 1280559,
    "text": "teams structured teams of agents that"
  },
  {
    "startMs": 1283039,
    "text": "work really continuously in the loop and"
  },
  {
    "startMs": 1285760,
    "text": "we just monitor and decide what to"
  },
  {
    "startMs": 1288720,
    "text": "build. Uh so I want to really end this"
  },
  {
    "startMs": 1291679,
    "text": "positively. We should be as humans"
  },
  {
    "startMs": 1293919,
    "text": "deciding what to build in the software"
  },
  {
    "startMs": 1296960,
    "text": "not how to build it. uh because that's"
  },
  {
    "startMs": 1299039,
    "text": "up for the agents and by this uh"
  },
  {
    "startMs": 1302080,
    "text": "continuous cycle of autonomous software"
  },
  {
    "startMs": 1305360,
    "text": "I think we can actually achieve that. Uh"
  },
  {
    "startMs": 1308559,
    "text": "this is one warning chart or if someone"
  },
  {
    "startMs": 1312080,
    "text": "thinks AI will take the cool cool stuff"
  },
  {
    "startMs": 1314480,
    "text": "from us. I believe actually it will take"
  },
  {
    "startMs": 1316640,
    "text": "the annoying stuff and in enterprises"
  },
  {
    "startMs": 1319120,
    "text": "and organizations you already spend a"
  },
  {
    "startMs": 1320880,
    "text": "lot on alignments on the meetings as I"
  },
  {
    "startMs": 1324400,
    "text": "mentioned basically the thing behind"
  },
  {
    "startMs": 1326400,
    "text": "between the lines that you really need"
  },
  {
    "startMs": 1328559,
    "text": "you need to get context from everyone"
  },
  {
    "startMs": 1330400,
    "text": "share your status sync on the things on"
  },
  {
    "startMs": 1332640,
    "text": "meetings so all these things could be"
  },
  {
    "startMs": 1334720,
    "text": "actually outsourced to your software"
  },
  {
    "startMs": 1336400,
    "text": "factory and you could be just talking"
  },
  {
    "startMs": 1338720,
    "text": "about the cool stuff okay so this is it"
  },
  {
    "startMs": 1342320,
    "text": "thank you so much go touch some grass"
  },
  {
    "startMs": 1344159,
    "text": "and let your agents uh built for you."
  },
  {
    "startMs": 1346720,
    "text": "Thank you."
  }
]