The 3 Pillars of Autonomy – Michele Catasta, Replit

Talk transcript

1. 0:00

   [on-hold music] So at Replit, we're building a coding agent for non-technical

2. 0:24

   users. It's a very peculiar challenge, I would say, compared to many people in this room. And what I'm gonna talk about today is why autonomy has become kind of the North Star that we keep chasing, you know, since we launched the very first version of Replit Agent September last year.

3. 0:41

   Let's start from this very interesting plot, in case my clicker worked, which now it does. Um, I'm sure you all have seen it. You know, it's the semi-async value of that published by Swiiks a few weeks ago, and it kind of clarified a bit the landscape, you know, for all of us, uh, agent builders.

4. 1:00

   On one hand, you have the low-latency interactions that really allow you to stay in the loop, you know, so you can do deep work and focus really on the, on the coding task at hand, but you need to be an expert.

5. 1:11

   You need to know exactly what to prompt the model for, and you need to understand quickly if you wanna accept the changes or not. Then, for several months, many of us, including Replit, we kind of live in this, I think, value of that where the agent wasn't autonomous enough to really delegate a task and come back and

6. 1:28

   see it com-accomplished. But at the same time, it run long enough not to keep in the zone, not to keep in the loop. Luckily, over time, we managed to go all the way on the right, and now we have agents that runs for several hours in a row.

7. 1:41

   What I'm gonna be arguing with today, and hopes Swiiks is not gonna stop inviting me to this event, is the fact that there is an additional dimension, like a third dimension to this plot that, you know, it hasn't been covered here.

8. 1:52

   And namely, the fact is, how do we build autonomous agents for non-technical users?

9. 2:00

   So what I'm gonna be arguing today is that there are two types of autonomy. One of which is more supervised. So think of the, you know, Tesla F as the example.

10. 2:09

    When you sit in a Tesla, you're still expected to have a driving license. You're gonna be sitting in front of the steering wheel. Perhaps ninety-nine percent of the time you're not gonna use it, but you're there in order to take care of the long-tail events.

11. 2:22

    And similarly, a lot of the coding agents that we have today require you to be technically savvy in order to use them correctly.

12. 2:30

    We at Replit and, you know, uh, other companies at this point are focusing on kind of the Waymo experience for autonomous coding agents. So you're expected to sit in the back.

13. 2:41

    You don't even have access to the steering wheel, and are expected basically not to need any driving license. Uh, why is this important? Because we want to empower every knowledge worker to create software, and I can't expect knowledge workers to know what kind of technical decisions an agent should be making.

14. 2:57

    We should offload completely the level of complexity away from them.

15. 3:02

    Of course, it took a while to get here, so I'm, I'm sure what I'm showing you here is something that all of you are very familiar with. It took several years to go from, I don't know, maybe a less than a minute feedback loop, constant supervision, and talking about completions and talking about assistance.

16. 3:18

    These are areas where the AI power ideas have really been pioneering this, this type of, uh, user interaction. Then we slowly climb through, you know, higher levels of autonomy.

17. 3:29

    So we had the first version of the agents based on or React. So we concocted autonomy, uh, with a very simple paradigm on top of LLMs. Then likely AI providers understood that tool calling was extremely important, poured a lot of effort on that, so we built the next version of agents with native tool calling.

18. 3:47

    And then I would say there is a third generation of agents, which I call autonomous, and that's when we started to break the barrier of, say, one hour of autonomy.

19. 3:54

    Basically, the, the agent being capable of running on long-horizon tasks and remaining coherent. It happens to be the case that those are also the versions of Replit Agent that we launched over the last year.

20. 4:05

    So the V3 is the one that we launched a couple of months ago, and it has exactly showcases those properties. So the question for today is, can we actually build fully autonomous agents, and how do we get there?

21. 4:19

    So I'm gonna try to redefine the definition of autonomy today. I think that oftentimes we conflate autonomy with the concept of something that runs for a, for a lot of time, and usually as a user, you lose control.

22. 4:32

    In reality, what the autonomy that I want to give to agents can be very specifically scoped. And what I mean by that is, especially with Replit Agent 3, what we accomplish is we, we make sure that our agent takes all the technical decisions.

23. 4:48

    Of course, that could lead to very long gap between the different user interactions and in case the agent again runs for several hours. But this happens if and only if the scope of the task you're giving to the agent is really broad.

24. 5:00

    And it turns out that in reality, you can have an agent that is really autonomous and is still fast, as long as you give it a very narrow scope for the task, you know, at hand.

25. 5:09

    So what we can accomplish in this way is that the user still maintains control on the aspects that they care about, and the user cares about what they're building, especially, again, our users, knowledge workers, they don't care about how something is being built.

26. 5:23

    They just wanna see their goals to be accomplished. So autonomy should not be basically conflated with long runtimes. And similarly, it shouldn't become a vanity metric. You know, a lot of us are talking about it as a, as a badge of honor, and it's definitely been exciting to see in the last few months that, you know, many

27. 5:40

    of us broke the, the barrier of, uh, running several hours in a row. But I think in terms of how to build agents that are gonna be more powerful and more scalable in the future, we kind of have to change a bit, uh, the, the target, the metric that we view, that we keep in mind.

28. 5:55

    So think about it in this way. Tasks have a natural level of complexity, and basically what we care about is that they have a minimum irreducible amount of work that they express.

29. 6:07

    What agents do is that they always go through this loop of planning, implementing, and testing. And of course, to make this happen and to make it work correctly, you want this work to be happening over a long-term trajectory.

30. 6:18

    So our goal is to maximize the reducible runtime of the agent. By reducible, I mean having a span of time where the user doesn't have to make any technical decisions, and the agent can accomplish the task again in full autonomy.

31. 6:32

    This is especially important for us because I can't trust our users to make technical decisions, so they, they need a proper technical collaborator by their side. I want to abstract away as much complexity as possible from the process of software creation.

32. 6:46

    And last but not least, I want the users to feel in control of what they're creating without stifling their creativity, because they have also to think about the technical decision that the agent is making.

33. 7:00

    So now, what are the pillars of autonomy? How are we making this happen? I would say there are three pillars that are extremely important to think about. The first one is, of course, the capabilities of frontier models, like the baseline IQ that we inject in the main agentic loop.

34. 7:15

    I'm gonna leave this as an exercise to the reader and to other people in the room. I'm really glad a lot of you are building amazing models that, you know, we use all the time at Replit.

35. 7:24

    So this is the pillar number one. The second pillar is verification. It's very important that we test for a lot of correctness of our agent at every step that it takes.

36. 7:35

    And the reason is fairly intuitive. If you are building on very shaky foundations, eventually the castle will topple down. So we brought verification in the loop to make sure that in a sense we are having, you know, nines of reliability.

37. 7:47

    We're avoiding the compounding errors that an agent will make unavoidably if, you know, you don't put any control on it. And last but not least, you heard it on stage even earlier, I'm sure you're gonna be hearing this, you know, the entire day or the entire duration of the conference, uh, the importance of context management.

38. 8:03

    So on one end, we want to have an agent that is capable of being globally coherent, so it's aligned with the intent of the user, the expectation of the user.

39. 8:11

    But at the same time, it is also to be capable of managing both the high-level goal and the single task that the agent is working on. I think we made amazing progress in the last months on context management, but I'm also excited to see, you know, where we're going as a field.

40. 8:25

    Let's start from the first pillar that we work at Replit, which is verification.

41. 8:32

    So why do we focus on this? Over the, you know, last year, we realized something that I think each one of you has experienced. So without testing, agents build a lot of painted doors.

42. 8:44

    In our case, the painted doors are very visible because we create a lot of web applications. So you end up basically trying to click on a button and the handler is not hooked up, or some of the data they were showing is actually mock data and it's not coming, it's not coming from a database.

43. 8:58

    But in general, this phenomenon spans, you know, across every type of component we're building. Being it front-end or back-end, a lot of components are actually not fully fleshed, uh, by the agent.

44. 9:09

    So we ran some evaluations internally. We found out that more than thirty percent of the individual features happen to be broken, you know, the first time that they are hooked by the agent.

45. 9:18

    And that also means that almost every application has at least one broken feature or painted door. They are hard to find. The reason is users are not gonna spend time testing every single button, every single field.

46. 9:33

    And this is also probably one of the reasons why a lot of our users, especially the non-technical ones, still can't trust coding agents very much. They are shocked when they find that there is a painted door out there.

47. 9:45

    So how do we solve this problem? Fundamentally, we need... an agent must gather all the feedback that they need from their environment, right? It's easier said than done. Um, again, non-technical users not only cannot make technical decisions, but also they cannot provide the technical feedback that, you know, an agent is required to make progress.

48. 10:06

    And most of what they can do is basic, you know, quality assurance testing. They can literally go around the UI, click, interact with the application. I'm, I'm sure you have tried it in your life.

49. 10:16

    This is extremely tedious to do, and it leads to a very bad user experience. And even though we relied on that with our first release of the agent last year, quickly we found out that users don't want to spend time doing testing.

50. 10:28

    So we had to find a complete, you know, orthogonal solution to that, which is autonomous testing. And it solves several different issues. The first one is it breaks the feedback bottleneck.

51. 10:40

    Even if, again, we ask feedback to the user, we were not given enough of that. Now we don't have to wait anymore for human feedback. We have a way to e- e- elicit as much information as possible from the app autonomously.

52. 10:52

    We also want to prevent the accumulation of small errors. What I was saying before, we don't want to have compounding errors while the agent is building. And last but not least, we have to overcome the laziness of frontier models.

53. 11:03

    So we need to verify that whenever a model tells us that a task has been completed, that is actually the truth and that result is not being hallucinated.

54. 11:12

    There is a wide spectrum of code verification that, you know, you, you can accomplish. I think we all started from the very left. You know, you have basic static code analysis with LSPs.

55. 11:22

    We have been executing the code since we had basic LLMs that were capable of debugging, and then we slowly started to move towards the right. So generating unit tests and running them, it has a limitation.

56. 11:33

    It's limited only to functional correctness. Uh, unit testing, um, is not very powerful to do, like, proper integration testing by definition. We started also to do now API testing, but it's only limited to API code.

57. 11:45

    So you can test endpoint of an applications. You can't really test how a web app functions and looks like. And for this reason, in the last few months, Haas and other companies are pouring a lot of effort in really creating autonomous testing based on the browser, you know, in case the app that we're building is a web

58. 12:03

    application. There are two main categories here. One is computer use. It's a one-to-one mapping with user interface, so the model is directly interacting with the application. It requires screenshots It tends to be fairly expensive and fairly slow.

59. 12:17

    I'm sure you, you tested it yourself. A good way in mid-in the middle is browser use, where we simulate the user interface. You can then interact with the browser and with the web application, and it relies on basically accessing the DOM through abstractions.

60. 12:33

    So how do we, how do we make this work at Replit? Um, what we do is that we generate applications that are amenable to testing, and we sort of merge everything together from the previous slides that I showed you.

61. 12:46

    So we allow the, our testing agent to interact with an application and gather screenshots in case nothing has worked, so we have a fallback to computer use. But the vast majority of times, what we do is that we have programmatic interactions with the applications.

62. 13:01

    So we interact with the database, we read the logs, we do API calls. We literally click on the app and, and get back all the information that we need.

63. 13:09

    And by putting all of this together, we collect enough feedback that allows our agent both to make progress and also to fix all the painted doors that it encounters.

64. 13:21

    Just a, a s-short technical deep dive on how we accomplish this. I'm sure you've seen a lot of the tool-based, uh, browser use. There are amazing libraries out there.

65. 13:33

    Uh, first one that comes to my mind is StageAnd. And the idea is that you have an agent that has a few very generic tools exposed, so now, you know, the agent can create a new tab, can click, can fill forms, et cetera, et cetera.

66. 13:46

    The limitation here is that it's difficult to enumerate all the different type of interactions you could be having with a browser. The problem of testing is very similar to the Tesla analogy I was making before.

67. 13:57

    Maybe this cardinality of tools available is enough for ninety-nine percent of the interaction types, but then there is always a long tail of idiosyncratic interactions that a user makes with the, with a web application that are hard to map into these tool, these different tool calls.

68. 14:13

    So what we do, uh, in our case at Replit is we directly write Playwright code. And Playwright code is, first of all, very manageable for LLMs. LLMs are kind of amazing at writing Playwright.

69. 14:25

    You know, this is the experience that we had, uh, since we started to work on this project. It's also very powerful and expressive. So in, in a sense, it's a super set of what you can express, uh, on the-- compared to the left on the tools, uh, testing.

70. 14:39

    And last but not least, there is beauty in creating Playwright code because you can reuse those tests. If... The moment you write a test in script, then you can rerun it as many times as you want.

71. 14:49

    So in a sense, the moment you created a test, you're also creating a regression test suite that you can keep running in the future. And all these kind of, uh, tricks that I explained to you right now, they helped us to create something that is roughly a whole other magnitude cheaper and faster compared to computer use.

72. 15:07

    And we'll go back later on how important latency is.

73. 15:11

    The second thing, though, the second pillar that I wanted to talk about today, of course, is context management, and I'm gonna go very fast here because I think you're gonna be hearing a lot of talks today about it.

74. 15:21

    The, the high-level message here is that long context models are not needed to work on coherent long trajectories. Uh, from experience, we found that most of the tasks, even the more ambitious one, can be accomplished within the two hundred thousand tokens.

75. 15:36

    So we're still not in a world where working with models that have ten million or a hundred million, uh, context windows is necessary to actually run autonomous agents. And we accomplish this by means of learning how to do context management correctly.

76. 15:51

    So first of all, there are several different ways to maintain state which don't imply chucking all the state into your context window. You can do that, for example, by using the code base itself to maintain state.

77. 16:04

    So you can write documentation while the agent is creating new code. You can also include the plan description and all the different task lists that the agent is working on.

78. 16:13

    You can persist them on the file system. So even there, like, you have a lot of ways to offload your memories. And last but not least, and this is something I think, you know, Anthropic has been, uh, really evangelizing about, um, you can even dump directly your memories in the file system and then making sure that your

79. 16:29

    agent decides when to write them back the moment they become relevant to your work. So for this reason, we have been seeing a lot of announcements in the last couple of months.

80. 16:38

    Uh, just pick this one from Anthropic. You know, we, with Claude Sonnet four point seven, so I wish, four point five, uh, they've been able to run, uh, focused tasks for more than thirty hours in a row.

81. 16:48

    We have seen similar results from OpenAI on the math problems. So I think we, we kind of broke the barrier of running for long and, you know, being able to have coherent tasks.

82. 17:00

    I would say the key ingredient to make this happen has been how good models and us as agent builders have become in doing subagent orchestration. Subagents basically work by means of they are invoked in the core loop.

83. 17:12

    So it's a completely-- It's starting from a blank slate, uh, from a completely fresh context. You, as an agent builder, decide what subset of the context to inject when the subagent starts.

84. 17:23

    And it's a concept that is very similar, I think, to everyone who's been writing software, you know, in the last decades. It's separation of concerns. So you decide what your subagent is gonna be working on.

85. 17:32

    You give it the least possible amount of context. You allow it to run to completion. You only get the output, the results. You inject them back into the main loop, and you keep running in this way.

86. 17:42

    Of course, this significantly improves the number of memories per compression. I just brought this plot from, directly from Replit Agent run in production. The moment we kicked in our new subagent orchestrator on the x-- on the y-axis, you can see the number of memories per compression.

87. 17:59

    So we went from roughly thirty-five to forty-five, fifty recently. So big improvement in terms of how often we are recompressing our context just because we kind of float a lot of the context pollution by means of using subagents.

88. 18:16

    I'm gonna give an example where this made the difference for us. You know, the-- what I'm showing you here is more kind of a cost optimization in a sense.

89. 18:23

    Like, you're compressing less, you also have separation of concerns, which definitely make your agent be smarter. In the case of testing, working with subagent was almost mandatory for us, and basically, we started to work on automated testing even before we were very advanced in terms of subagent orchestration.

90. 18:41

    And what we found out is, of course, again, as I was saying before, it makes things easier, better cost, less pollution. But when you allow the main loop not only to create code but also to, to do browser acti-browser actions, to put back the observation of your browser actions into the main loop, you tend to confuse the

91. 19:00

    a-the agent loop very much 'cause at this point, there is a lot of heterogeneity in terms of the action that your main loop is looking at. So in order to make this work, not only we have to build all the Playwright framework that I was showing to you before, but we also have to move our entire architecture

92. 19:15

    into subagents. So at this point, you can see very clearly why there is a separation of concern here. We have the main agent loop running. We decide at a certain point that it's time to verify if the output of the agent has been correct.

93. 19:27

    We make this happen all within a subagent, then we scratch the context window of that subagent. We just return back the last observation to the agent loop, and then we keep running in that way.

94. 19:37

    So if you're having issues today making your subagents, uh, work correctly, this is one of the reasons why that you wanna take a look at.

95. 19:46

    So I think we covered the high level of how to create more and more powerful, uh, autonomous agents over time, and I only see us as a field becoming even more proficient than that in the next months.

96. 19:58

    There is one additional ingredient, though, that is gonna make the difference, and it's parallelism. And I will argue that parallelism is important, not because it's gonna make agents more powerful per se, but rather because it's gonna make the user experience more exciting.

97. 20:13

    So of course, it is great to have an agent that is capable of running autonomously for long, but at the same time, it comes with the price of making the user experience less thrilling.

98. 20:25

    You are not in the zone anymore. What you do is that you write a very long prompt, it's translated into a task list, uh, and then you go to have lunch with your colleagues, and then you come back, and you hope that the agent is done.

99. 20:36

    That is not the kind of experience that most of the productive people want to have in life. You know, you want to see as much work as done as possible in the shortest span of time.

100. 20:45

     So what we do as a, as a field at this point has been to create parallel agents. It's a very common trade-off, which, by the way, doesn't only apply to agents.

101. 20:54

     It a- it applies to computing in general. And for parallel agents, what you do is that you, you trade off basically extra compute in exchange for time. Why there is this trade-off?

102. 21:05

     So first of all, when you're running agents in parallel, you're gathering the same context in multiple context windows. So every single parallel agent you would be running probably shares, say, eighty percent of the context across the board.

103. 21:18

     So of course, you are just putting more compute at work because you're running those agents in parallel. There's also another cost that is kind of intangible for a lot of you here in the room because I'm sure you're all, uh, expert software developers, but what do you do with the output of multiple parallel agents at the end?

104. 21:36

     Oftentimes, you need to resolve merge conflicts. So as a reminder, my users don't even know what's the concept of merge conflicts. It's something that I have to figure out on our own.

105. 21:46

     So the current way in which we think of parallel agents in the, in the space doesn't really apply to Replit. Now, at the same time, I still wanna very much to accomplish this.

106. 21:55

     There are so many interesting features that we can enable with parallelism. Aside from the fact that you can get more work done, uh, at times you want to... you want testing to be running in parallel with the agent that creates code.

107. 22:07

     Testing, no matter how much we optimize it, is still very slow. If an agent is only spending time on testing, users are not gonna be engaging with your application anymore.

108. 22:16

     Um, at the same time, it's also great to have a synchronous process running while your agent is running because you can inject useful information back into the main code loop.

109. 22:25

     And last but not least is a very common technique that we know boosts performance if you have enough budget to do so. You should be sampling multiple trajectories at the same time.

110. 22:36

     So a lot of perks are coming with parallel agents, but the, the way in which we implement them today, which I call, basically call user as an orchestrator, is the fact that tasks, the parallel tasks that you want to run are determined by you, by the user.

111. 22:51

     And each task is dispatched in its own thread, so there's a bit of manual process. Even the task decomposition, in a sense, is happening in your mind while you're thinking about which agents you want to run.

112. 23:02

     And then the moment you get back all the results, you need to go through the problem of merge conflicts, and oftentimes, this is not trivial at all, no matter how, how many amazing tools are out there.

113. 23:13

     So what we're working on today for our next version of the agent is having the code loop as the orchestrator. So the key difference here is the fact that the s- the sub-tasks that we're gonna be working on are not determined by the user, but they are determined by the code agent loop.

114. 23:30

     And the parallelism is basically decided on the fly. The agent does the task decomposition on behalf of the user, and this comes with a couple of advantages. First of all, again, there is no cognitive burden to, for the user to understand how they should be decomposing the task.

115. 23:46

     At the same time, also, there are ways in which you can create tasks that sort of mitigate the problem of merge conflicts. I'm not claiming that we're gonna be able to mitigate it a hundred percent.

116. 23:58

     There are so many corner cases in which merge conflict will still represent a problem. But there are a lot of different techniques known in software engineering to make sure that you can try to have multiple sub-agent not stepping on each other toes.

117. 24:10

     So the code loop as an orchestrator is gonna be the, our main bet for the next few months.

118. 24:18

     And in case you're passionate about these topics,

119. 24:21

     I'm always hiring at Replit. Thank you. [outro music]