Building your own software factory

Talk transcript

1. 0:00

   [upbeat music] Um, okay, so we're starting five minutes early.

2. 0:17

   Um, hey, everyone. I'm Eric. Uh, I'm an engineer at Cursor, and I mostly work at, at developer experience and product. And today, I kinda wanted to talk to you about

3. 0:27

   my experiences, like working at Cursor, dogfooding the product, and like getting to a place where you can build your own like software factory, and like what that kinda like takes, and the practical steps getting there.

4. 0:39

   To be honest, I don't think we're really there yet. Like sub-parts of the product and sub-parts of the company are running like fairly autonomously. Um, but building a software factory takes a lot of work.

5. 0:50

   I mean, like look at like real-life, uh, factories producing like hardware. There's a lot of assembly lines. There's a lot of people that goes into this, a lot of managing, observability, and all that, and there's a lot of concepts we can borrow from that world and put into the, uh, software world.

6. 1:04

   So anyway, here goes my observations from doing this. Um, but first, um, the agenda. I wanna talk about like levels of autonomy, uh, precursor to factory, pun intended, um, building the factory, running the factory, and then scaling the factory.

7. 1:21

   And I wanna finish with some Q&A for any kind of questions.

8. 1:26

   Okay, so for the levels of autonomy, Dan Shapiro put out this blog post, I think in January or, uh, February, uh, explaining like six different stages of, uh, autonomy, uh, throughout like s- automated software.

9. 1:41

   Uh, Carpathia has also like previously used Cursor as example of like going from tab to agent and all that. But I think this kinda like encapsulates this really, really well.

10. 1:50

    So we have this spicy autocomplete, uh, at the start, and this is kinda like where Cursor started in '22, '23, like ages ago at this point. Um, and we g- kinda like gradually moved up the ladder in making the software creation more autonomous and letting the agents do more work.

11. 2:07

    And I think most people, uh, adopting the AI tools, um, are like at somewhere between level two and level three, where you have a pair programmer, where essentially just going back and forth with the agent, asking questions, um, getting suggestions, asking the agent to do work, um, and eventually, like finishing their tasks.

12. 2:29

    And the step above that would be, uh, having the AI generate the majority of the code, um, which we can see like here in the developer level three, um, where you as a human more kinda like reviews it, um, kinda like in the loop, following traces and all that.

13. 2:43

    But as you further progress, you're like becoming more and more of a manager, and we'll talk about, uh, this more later. But eventually, like level four, I think this is where I'm, uh, at, at this point, like for most like software projects, where I'm like delegating as much work as possible to agents, and probably like reviewing the

14. 3:01

    outputs before I actually review the code, um, 'cause I still look at the code sometimes. [chuckles]

15. 3:07

    Um, and lastly, we have the software factory, which is essentially like a black box. Um, Dan Shapiro calls it like the dark factory, where you don't really have an insight.

16. 3:16

    It's just like agents going around, doing their thing, uh, shipping the code, testing the code, building the code, all that, and you as a manager just provides like the intent and the instructions, um, and like the goal, uh, from what you want out of the factory.

17. 3:32

    Okay. Um, yeah. So like why do you even wanna create a factory? Um, first of all, like throughput. You probably wanna create more code with like less resources. Um, you can run agent twenty-four/seven.

18. 3:44

    You don't have to, uh, rely on humans that need sleep and food and eat and all that. Um, you can just like have more agents. Um, another like thing with the factory is like you have assembly lines, and assembly lines produces, um, consistent outputs.

19. 4:00

    So if you build your factory right, you can probably have very consistent output. Um, but at some point,

20. 4:07

    you... Initially, you feel like if you don't have a right setup, you might feel like the agents are getting more and more probabilistic, and like you're losing a lot of determinism, um, because they just go off and do random things, um, which is probably a sign that you need to like build more guardrails for the factory.

21. 4:22

    And I think this is a function of the model capa- capabilities as well. Like, as the models get better, they can follow instructions better and just execute on whatever you want them to do.

22. 4:32

    Um, and thirdly, um, you might wanna have a factory because you can leverage your taste better. Um, you can like get more out of your creativity out, um, instead of just like waiting for you as a human to create them and produce this, uh, software that you're creating.

23. 4:48

    Um, and then obligatory like then and now. This is what it used to look like. This is like a Tesla factory from a couple years ago. Uh, and this is like kinda what we're getting after here.

24. 4:59

    Okay, let's get straight into it. So to build a factory, what do you actually need? Um, I like to think of this as primitives and patterns. So just like how do you structure the code?

25. 5:12

    Um, is it like a modularized code base? Um, do you have this scattered all over the place? Is it co-located code, et cetera?

26. 5:20

    Um, because the, like, um, the distance in, um, in locating, like if you have an agent like LSing a folder, uh, it can like discover all the relevant files at once instead of having to grep and search all over the code base.

27. 5:37

    It can just like be very isolated to work within one, um, single part of the code base. And this goes the same with humans. Like, if you have an easy time like onboarding yourself to a new code base, an agent probably will have that too.

28. 5:51

    Um, the second thing is like usage patterns. Do you have specific like methods and services for authenticating a user? Do you have like startup scripts? Do you have a way to like write tests, et cetera?

29. 6:02

    Do you have this boilerplate in place? Um, because if you do, you can point the agent to like existing references and just asking this to reproduce, uh, over time.

30. 6:12

    So those are like some of the like primitives and structures of the code base.

31. 6:17

    Um, the second one would be guardrails. So like, you might-- you wanna let the agents free, but not too free. Uh, so you wanna have some rules and, and checks and, and hooks in place.

32. 6:28

    Um, for example, um, a hook you might wanna have is, uh, touching a specific part of the codebase. Uh, maybe the agent should not be able to change like the most sensitive, like, encryption of sensitive data or authentication or anything like that, where, uh, a mistake could be like very, very, very costly, um, for the company or

33. 6:52

    for you as a human, et cetera. Um, rules, um, rules is probably the most misunderstood, um, concept since we launched Cursor rules. Um, there's, uh, Cursor Directory, which launched a good collection of different rules.

34. 7:09

    Um, and the assumption was usually that you should just install every rule that you can, depending on like what, um, uh, software stack you're using. For example, if you're using Next.js, maybe you should have Next.js rules.

35. 7:20

    Um, but what I found and what I'm seeing amongst our users and internally is that rules should just like emerge dynamically. Like, if you're finding agents going off the rails, you should probably create a rule for that.

36. 7:31

    And it should kinda like be sort of like an S-SOP, uh, to showing like the agents what they can do and cannot do. And again, the models are getting so good at following specific rules that they usually don't go off the rails anymore.

37. 7:46

    And I think that just kinda like extrapolate over time as well.

38. 7:51

    Um, and of course, tests, like can the agent, uh, verify its own work, and can it run tests to know like, "Oh, I messed something up," or, um, "I made a change," depending in like, in the specific area of the code, but it still passes.

39. 8:06

    I can, I can still run the code, and, uh, the check looks good. Um,

40. 8:11

    and lastly, which I think is probably most exciting, is the enablers. Like what can you allow the agents to do to actually let them be free? Um, skills is good for this, um, just giving the agents more capabilities, skills, and MCPs.

41. 8:25

    Um, accessing like external context, um, getting like understanding of how to implement a certain thing. Uh, I'm gonna show you some later, uh, in the Cursor codebase, uh, what we are doing.

42. 8:37

    For example, like feature flagging. Can we give, uh, the agent the skill to add a feature flag, so when we launch them autonomously, they can just flag the actual changes made and merge the PR and come back to us like, "Hey, uh, if you wanna try this, just turn on this flag.

43. 8:52

    Um, if you don't like it, we'll just revert to PR. If you like it, we can like expand it to more users."

44. 8:58

    Um, and lastly, like what kind of environment are you letting the agents run in? Um, can your agents, um, start your dev environment? Can you just ask them like, "Hey, um, start my project," um, and let them do that without having to like have any human in the loop?

45. 9:18

    Um, 'cause if that's the case, you can probably like have them run, um... You can scale it up like infinitely on separate VMs. Um, and then this checklist is like what I'm usually following, um, when thinking of like building the actual like, uh, the factory.

46. 9:33

    Um, and part of that is like, is it runnable? Um, there's a typo in here. I blame my Swedish. Uh, [chuckles] there's, is it accessible, like the context that the agents needs to have?

47. 9:45

    Um, can they interface with Linear or Notion or Datadog or Slack, et cetera, just to understand and like see what's, what is like the broader context of this, the intent that the user have.

48. 9:57

    And lastly, which I think people should be spending a lot more time is like building verifiable systems. How can, um, the agents themselves like verify their own work, whether that's through, um, unit tests or integration tests or, uh, UI tests, like actually clicking around in the DOM and like trying to reproduce things that's actually happening for the

49. 10:20

    end user. Um, this is arguably easier for, uh, backend systems where there's like no UI really happening, and you can have like clearer contracts and boundaries of what should work and what shouldn't.

50. 10:33

    Uh, whereas for, for web and UI and all that, you actually need to click around and making sure things work. The buttons actually have a loading spinner, et cetera.

51. 10:42

    Okay, so this is like part of building the factory. So if we switch over to Cursor here, um, I'm not sure if you've seen this, but this is Cursor, uh, 3.

52. 10:53

    Uh, we launched this a couple of weeks ago, and it's a complete rewrite of Cursor. There's no VS Code anymore. Uh, most of you are probably familiar with this type of cursor, um,

53. 11:06

    where you have files and sidebars and, uh, a lot of different things, [chuckles] uh, whereas this is a bit more streamlined for like an agent-first, uh, workflow.

54. 11:17

    Uh, and we'll get to like why we created this as well, so, at a later point. But I wanted to show you some parts of, um, some rules, et cetera.

55. 11:26

    Let's see where I put them. Um, so for example, I built this music agent, uh, project, and if you've used Ableton before, you probably recognize this. Yeah.

56. 11:37

    It's like really small. We can't see.

57. 11:39

    Yeah. Yeah, I'll expand it. More? Good? Okay.

58. 11:51

    Um, yeah, so if you've used Ableton or any like music production software, um, you probably recognize this interface. Oops. Uh, it's not really working in this size. [chuckles]

59. 12:04

    Um, but what I, uh, essentially asked the agent to do here is like, "Can you start a local dev server?" And we can see that it worked for a while.

60. 12:14

    Um, it explored some files, read package.json, and based on this, uh, there is a start script. So- Like package.json and all these dependency files are so in distribution of the models that they know, like, we should immediately go to package.json, uh, if there's a JS product or if it exists, um, to look for a start script.

61. 12:34

    And this is, like, a good example of having, like, a pattern that is predefined and, like, making your code base more, like, in distribution, uh, in that way. Um, because now it's like, it's super easy for the agent to understand, like, "Oh, I should just go in here and start the server."

62. 12:49

    So it started the server. Um, it's running on localhost:3000. Um, and let's see here. We can see that we have this agents.md file. Um, so agents.md is, like, Cursor rules.

63. 13:05

    Uh, it's, uh, across for many different harnesses. Um,

64. 13:11

    and what I wanted to accomplish with this project is essentially, like, building a factory around this idea of building, like, a online music, uh, creation tool.

65. 13:21

    Um, and to do that, I, like, I forced myself never to write any code, um, myself. Try not to look at the code that much either and just, like, try to figure out, like, what is the systems and the structures I need around this.

66. 13:35

    Um, and, um, immediately it became pretty clear that we need a way to start the project. Um, we need a way for the agent to, like, verify its own work.

67. 13:46

    So the agent created this, uh, end-to-end tests, um, using Playwright, so it can just spawn browsers, um, go to a route, et cetera, click around and get by test ID, uh, and making sure, like, for every different change I make, um, for example, the play button still works, or I can add notes to this project here, uh,

68. 14:06

    without, um, anything breaking. Um, so these are, like, some examples of, um, how you can create, like, verifiable outputs like that. Um,

69. 14:19

    okay, uh, we have V test, we have this, et cetera. So let's see here. If you go back...

70. 14:29

    Um, oh yeah, an-another option here. Oh, casual scrolling of Twitter.

71. 14:35

    Um, a different way to verify the work is using, um, like an automation to rec-code review. Um, you can ask the agent to just review the changes it made, um, or you can use like, um, a more, like, integrated tool like Bugbot that we have in Cursor that just looks at, uh, different PRs, uh, in GitHub and

72. 14:55

    reviews them and comes back. Um, and this is, like, also, like, one piece of the whole, like, factory that you should have multiple different stages where you pr- you plan it, you produce it, you review it, and you essentially follow the whole, uh, SDLC, uh, but you, like, automate and codify, uh, this work.

73. 15:16

    Um, let's see here. Yes. Um, I di- I wanna show you this as well. Um, so we launched, uh, updated cloud agents, um, in the last couple of weeks, where we gave, uh, each agent their separate VM, and you can have them, like, create this very reproducible environment in the cloud.

74. 15:40

    And this essentially allows you to scale, like, infinitely. Uh, but we also gave the agent a tool to test its own work, um, by controlling the computer. So for example, we have Glass here, uh, which is the interface, and I asked the agent to...

75. 15:54

    Let's see here. Uh, Glass agents, still a bit rough with the keyboard, Control+Tab, et cetera, like better, uh, accessibility and, um, using the keyboard to navigate the agents. Um, and I asked it to, uh, make the change and then record this with the full editor, because the first one was just a sidebar.

76. 16:16

    So what we got back here is just a video of the agent actually testing its own work. So we can see that it has this highlighted row. I'm not sure if you can see that.

77. 16:25

    Um, but just some context for me as a human to verify the work. Um, and then, uh, it actually clicking around and using the keyboard to, to navigate. So with this, we're like, we're getting kind of far in, like, the factory, like where we're at.

78. 16:41

    Like, a lot of the things are automated, like review is automated. Um, the testing is automated. Um, uh, we have some rules to, like, steer the agents, et cetera.

79. 16:52

    Um, but there's still a lot, still a lot more to do. Um, so I think when you have this in place, the most important thing you can do is, like, shift your mindset.

80. 17:03

    Like, you are going to look way less at code. So you are going to go from, like, worker to manager, um, where instead of just doing the work yourself, you're overseeing a lot of agents doing the work, uh, for you.

81. 17:15

    Um, so this also means going from sync to async, because most of the work is gonna happen in the background, and you can still tap in and see what's going on, uh, for different agents.

82. 17:26

    But the more agents you spawn over time, the harder time you're gonna have to, like, understand what's going on in each of them. So then you need a way to aggregate these changes, like, upwards.

83. 17:37

    Um, and it's just-- I think it's so interesting that it's just the same as, like, in human organization. Like, all the same principles kind of follow. You still have, uh, you start with a very small team, and then you add more and more people because you need to get more throughput, and all of a sudden, you need

84. 17:53

    a manager to, like, oversee things, and then you add more managers, and then you need a manager of the manager. And this is essentially what's gonna happen with agents too, but you are just gonna k- like, keep on going up the label-levels of abstraction.

85. 18:08

    So when you're a manager, you need to start thinking of, like, how do you scope and parallelize the work? Uh, because you wanna get, like, higher throughput. Um, but some things are not necessarily, um...

86. 18:21

    It's not good to make all the changes at once. For example, if you have two different tasks working on the same part of the code base, you're gonna get merge conflicts.

87. 18:29

    So you need to still, like, plan out, scope, and parallelize, uh, the work. Um, and one, like, one unit of work can always be one agent. Um, so then, like, how do you take a long, long list of things you wanna do and actually, like, make the most out of that, um, and run the most amount of

88. 18:46

    agents that you can do? And to do this, I think it's important that you c- preserve, um, like tribal knowledge of the code base. Like, you still understand what's going on in the different systems.

89. 18:58

    Um, you know, like, how data flows, uh, what the users want, um, which part are critical, which part are don't. Um, so not outsourcing too much, uh, to the agents, but, like, very-- be very, like, direct, um, in, in managing, managing them pretty well.

90. 19:15

    And when you're going from sync to async, you are going to need to trust the agents a lot more, um, because

91. 19:25

    you are going to send them off and doing longer and longer tasks. And when you do that, you need to, like, get more context upfront. So you kinda, like, front load, uh, the context to the agents, either through, like, a plan or a long spec, and then you send them off, and then you let them go.

92. 19:40

    And once you start doing this regularly, you're gonna under- like, start to feel the agents. You're gonna, like, understand the models, and you're gonna see, like, these are the weaknesses, these are the strengths, and you are gonna create, like, this alignment with the models so you know, like, how to prompt them and what intent to give them.

93. 19:59

    And again, as the models keep getting better, you have to give them shorter or less and less prompts as you used to, uh, before. But you still gotta provide the intent and be very clear, like, what, uh, with the change you want the agents to do.

94. 20:13

    Um, and there's, like, no, there's no shortcut to this, uh, from what I've found and from the-- what the team has found. You just gotta, like, spawn a shitload of agents and just, like, let them do the work and see what happens.

95. 20:24

    And as long as you have good safety guardrails, you can just let them do that. Um, so you probably shouldn't let them push to prod, like, straight away.

96. 20:33

    Sorry, one question. Do you, do you multiply the, the working environments as well, or do you let them, all the agents work in parallel on the same development environment?

97. 20:46

    Um-

98. 20:48

    On the same code-

99. 20:48

    Yeah

100. 20:48

     ... base as well?

101. 20:49

     So this kinda comes down to, like, um, personally, I'm always using isolated environments through-- in different VMs. Um, I just tweeted about this actually, 'cause on one hand, if you're sharing the workspace, you can have, like, Git worktrees, where you, like, have diff- shallow copies essentially of the code base on the same machine, and you can reuse

102. 21:08

     services. But you're still gonna have to branch every, like, database or cache or user management to have, like, reproducible and separate environments. Like, if you are going to make, uh, a lot of changes at once, you need to-- you wanna know that they are pure, and they're not, like, having side effects to the other branches.

103. 21:28

     And that's why I found, like, just using cloud agents, uh, where I spawn a VM, and this VM can run a database, uh, uh, internal tooling, databases, other stuff, uh, and the Cursor app itself, and then have the agent just work in that isolated environment to be much better.

104. 21:44

     Um, it is more expensive. It's gonna take a lot more work to set up your, like, factory or your environment to support this. But once you have it set it up properly, you can scale this to, like, a hundred or a thousand agents.

105. 21:57

     I'm not sure how many we are running today, but I bet it's, like, multiple thousands a day, um, just agents running in the same or, like, copies of the code base.

106. 22:07

     Um, so that's what I would recommend. Um, yeah. So [chuckles] when you're a manager, like, your job changes quite a bit. Um, so, um,

107. 22:20

     you have to, like, look at your system as a whole. You gotta, like, think of where is the human in the loop needed. For example, do you have a log service like Datadog, and do you need to copy-paste the logs and go into the code base and paste them and it's like run the agents to identify and,

108. 22:37

     and trace down issues? Or do you have user feedback that you need to copy-paste from Twitter into somewhere else and let the agents do something with that? Um, do you have, like, a Notion thing, uh, where you have all your specs and you need to copy-paste the Notion or export them into Markdown and then to agents?

109. 22:54

     There's probably a way to, like, automate all these different things. Uh, either it's, like, skills for MCPs or either-- or, or separate automations.

110. 23:04

     So think of, like, where is the human in loop needed and try to, like, automate that away.

111. 23:09

     Um, the second thing is, like, catch-- where-- how can you catch agents going, like, off, not doing what you actually want it to do? Um, and this is, like, the-- this, this is, like, the perfect flywheel for improving your factory as well.

112. 23:25

     If you can see agents, like, um, creating, like, wrong, uh, schemas in your database because they're not following naming conventions, et cetera, that's probably a rule somewhere. Um, or if they are, um, just producing really ugly UI, there's probably a way for you to create a design system and let the agents be aware of the design systems,

113. 23:47

     where they can, uh, incorporate that and, uh, use it for the next kind of, like, iteration you do.

114. 23:54

     And yeah, then you take all these learnings, and, uh, you use it to actually improve the factory.

115. 24:03

     And thirdly, uh, it comes to, like, scaling the factory. So now we have, like, your environment set up. You know how to, uh, be a manager, to, like, manage a fleet of agents.

116. 24:12

     You scope the task, and you do all this. Um, so how do you, like, actually take it from, like, five agents to ten agents to fifty to a hundred, uh, agents?

117. 24:23

     And, um, the thing is, again, um, not looking at code is gonna be a real thing if the models gets better, and they are getting better. So observing the outcomes, um, kinda like the same thing as previously, like, where they go off the rail, uh, what are they producing, what are the artifacts, et cetera, um- How can

118. 24:42

     you make it so that the agents also can verify their own work and verify the outcome that they produce?

119. 24:50

     Um, you should set up automations. You should look again at the things you're doing repetitively. Um, so one thing we could do, for example, here is if we go to Cursor and we go to, uh, this music agent again, uh, I can ask, uh, "Looking at my, uh, chat history, what repetitive tasks am I doing?"

120. 25:13

     Um, so we can ask the agent to, like, look at this and identify potential opportunities.

121. 25:21

     Uh, so it's searching the agent transcripts, and it's producing

122. 25:26

     some kind of artifact of this. Um, yeah, we'll see how this goes. Um, I actually built this into a plugin. Oh, let's see here. Uh, plan execution loops, restarting the product direction.

123. 25:41

     Um, let's see here. Ableton-like UI iteration. I should probably, like, put this in a rule saying, like, make it look like Ableton. Um,

124. 25:51

     tooling, housekeeping, et cetera, et cetera. So this product is very short-lived, but if you're looking at an actual production thing where you have prompted a lot over time, you're probably gonna find things that you are doing recurrently.

125. 26:03

     And I wanna show you some things that we are doing at Cursor, um, that we are automating. Um, and some of these are not that obvious all the time, but one is, for example...

126. 26:16

     Let's see here. Oh, not this one. Uh, let's see here. For example, daily review.

127. 26:25

     So I have this, um, automation for checking my own daily review. So this is going to, um, look at Slack, it's going to look at GitHub, um, and it's going to send me a summary of the things I've done, um, over the last day.

128. 26:44

     So I would previously have done this, like writing down my notes maybe, um, thinking of like, what did I get done today, uh, or like running an agent with access to MCP, but now I can just put this on a schedule and do this automatically for me.

129. 26:58

     Um, I wanna show you a different one. Uh, for example, read merged PR comments. Um, this is also like a way for you to, uh, learn over time. So for all the PRs that we merge in our main repository, we can look at the comments and we can look at what did humans actually review here, uh, and

130. 27:17

     what did they say about the changes I made? Uh, because if it's, if a human actually goes in and reviews a PR and leaves a comment, there's probably like high, high value and high signal and high intent, uh, in that comment, and we can then store that later, uh, in order for the agents to actually learn over

131. 27:32

     time. Um, we have another one, uh, which

132. 27:38

     I can show you here. Uh, this one. Yeah, a- again, the code owners.

133. 27:47

     Um, so this one allows us to... We essentially had this problem where, um, we had code owners in our code base, and they were kind of right most of the time, like 80% of the time, but for at least 20% of the time, they caused a lot of bottlenecks for us internally.

134. 28:05

     Like, we were blocked on merging the PR. We needed someone else to, um, to review it for us, and maybe they were in a different time zone perhaps. So what we started doing was building this agentic code owner thing, and what it essentially does is look in at PRs and check in like, first of all, what's the

135. 28:24

     risk of this? What's the risk level? Can we-- Is it just like changing a variable name or is it changing a constant that's changing, like, how long a trial subscription is or something like that?

136. 28:34

     Um, and if it is low risk, it can just approve the PR because we don't really, we don't wanna block, uh, our own engineers, uh, on these things. But if it is, um, we can see that it is a high-risk PR, and then we can find, like, okay, who, who made changes to this previously, and can we,

137. 28:52

     like, pull in their feedback, um, in making the most out of this? And like, um, first of all, making the code safe, um, and not breaking any systems, but also for the user that actually did the initial change, keep them in the loop and, like, keeping them up to date on and refreshing their context of what's going

138. 29:10

     on here. So it kinda like it goes both ways. Um,

139. 29:16

     and, and yeah, multiple, uh, value adds from doing this.

140. 29:21

     Um, let's see if there's one more review. No, I think that was pretty much it. Um, or yeah, I have this one more thing called continual learning. Um, so continual learning is another type of automation, um, that I created, um, a couple of weeks ago as well, and it essentially does what we did with the agent.

141. 29:40

     We look at the previous transcripts we have, and we can then extract, like, memories and learnings from what we said previously. Like, if we're correcting the agent to do, uh, a certain thing, like, um, use this component instead of that component, or, um, always, uh, refer to me as, uh, like, always, like, have very, like, verbose, uh,

142. 30:01

     descriptions of things that you're doing. Instead of me, like, every time going in and, and, um, asking the agent to do this, I can create a rule, but I'm kinda lazy, so I don't really, um, remember to create a rule.

143. 30:14

     So instead, we can have this continual learning plugin that looks, looks through, uh, the transcripts and store this as a rule, uh, for you instead. Um, so these are all examples of, like, systems to automate yourself away and to automate, like, things that the agent can do for you.

144. 30:31

     Um, and I think that's the important part of, like, building these factories. Like, how can you identify, uh, the flywheels and loops where you can, uh, automate yourself away by building systems?

145. 30:43

     Um, okay. Um, and yeah, you are going to move up abstractions. So now you're managing five to 10 agents, but tomorrow you might be managing, uh, an agent managing other agents.

146. 30:57

     Um, and that is just gonna grow. Like, you're gonna have a lot of sub-agents, like, under you working for you.

147. 31:05

     Um, cool. So yeah, what I want you to take away from this is be very clear about the intent and, like, really think about what's the actual problem to solve here, what do we g- wanna get out of this.

148. 31:18

     Um, don't outsource important decisions. Like, make sure you're staying in the loop for important decisions, um, whether this is like, uh, safety or security or databases or payments and authentication.

149. 31:31

     Um, some things are really important and should not be made, [chuckles] um, um... should not be decided by agents, but by humans.

150. 31:41

     Um, build tools and systems. Try to find these flywheels and, like, codify them and get them in, uh, your systems and let the agent have access to them.

151. 31:53

     Um, store context for later, whether that is like agent transcripts or artifacts of things you think look good, um, 'cause this is gonna help the agent to, like, know what good and bad looks like over time, and it's gonna change.

152. 32:06

     Um, so storing the context and building the tools and, like, keeping them up to date is, is more important than actually doing the work because this is gonna provide, like, the framework and the guardrails, uh, for the agents.

153. 32:20

     Um, and lastly, like, let the agents be free. Like, think of what do they need. Um, I have a friend at Lovable, uh, he mentioned that they set up a Slack channel, or he gave the agent a tool, a vent tool, so the agent can complain about things, uh, when it was running.

154. 32:36

     And the agent started complaining about like, "Hey, um, I can't, like, access this image. Um, I'm, like, very frustrated about this." And then it posted straight into a Slack channel.

155. 32:46

     And they, they set it up as a joke, but then they started scrolling through and like, "Oh, this actually is very valuable. Like, we should probably, like, [chuckles] give the agent access to reading images."

156. 32:54

     And they did, and then the agent started complaining about something else that was a problem with the harness. Um, so find ways to let the agents be free. I think that's, um, a very important thing.

157. 33:07

     Um, okay. That's kind of it. Uh, and that's kind of like a direction of-- and things we have found, like building Cursor and, like, taking Cursor towards a software factory.

158. 33:21

     Um, I hope you learned a thing or two and can take away, um, some of this. I'm happy to take any questions about anything Cursor. Yeah. Or actually, now we have the microphones coming here.

159. 33:39

     Uh, thank you very much. I have a question about, um, uh, code quality or architecture quality. So when agents ship, uh, tons of code, uh, and, uh, you barely can review them, uh, how you, uh, ensure their,

160. 33:59

     the code is extensible and so on? I mean, um, you can, uh, establish hooks, uh, or guardrails for measurable things like, I don't know, uh, number of lines in the file should not be more than something.

161. 34:17

     But, uh, the architecture is not, uh, measured this way, so, um... And agents, uh, they have this completion bias. They want to finish task, uh, as soon as possible and, uh, they, uh, don't think ahead, and they don't have their, uh, picture of the future, how code will evolve.

162. 34:40

     They just want to finish task now and, uh, yeah.

163. 34:46

     Thank you.

164. 34:46

     Yeah. It's a good question. Um, I think we as humans have the same problem, but it just takes a lot more time, uh, for us to, like, discover them.

165. 34:56

     Um, one pattern-- Like, the good thing about agents and models being like, um, essentially like completion machines is that they will just look at existing references and just continue forward with that same path.

166. 35:11

     So if you have existing things you can point them to, I think that's very important. If you don't, I think there's a case where you let the agents do, um, one-off implementations here and there, and then eventually you have another agent like refactoring, like we do as humans as well.

167. 35:26

     So like, one to generalize, um, and build abstractions and all these things. Um, so like, how-- can you build like a system to, like, detect this and verify that the abstractions that are getting built is also good and in line with what you wanna do?

168. 35:40

     Um, but I think it's gonna be like a lot more architectural review for humans, um, and, and scoping and, like, planning of what the architecture should look like and system design.

169. 35:51

     Um, but yeah, it's, it's a tough problem. [chuckles]

170. 35:56

     Thank you.

171. 35:58

     Thank you.

172. 36:00

     Hello, Eric. Thank you for the talk.

173. 36:01

     Yeah.

174. 36:01

     Um, when it comes to the activities of building the factory, one thing that I observe, for example, when it comes to building things like rules in a, in a team, is that because it's so new, almost everybody feels, "Oh, this is a rule for me, and I don't want to inflict it on other people."

175. 36:19

     Mm.

176. 36:20

     And I notice this creation of silos where each engineer ends up having their own separate different-

177. 36:26

     Mm.

178. 36:26

     -factory. Do you have any advice on how to bring it to the point where the whole team is contributing to the creation of the factory?

179. 36:34

     Uh, it's a, it's a great question. Um, I think it's hard. I think it's very cultural as well. Um, I mean, like, we... Developers have always created our own tools, and, like, we wanna have our own custom setup.

180. 36:45

     Uh, but at some points, like, we have to- Unify and like, uh, on a certain structure. So I think historically we have like had PR reviews and all these kind of things as a ceremony to like align on the code that's being produced and making sure it's consistent.

181. 37:01

     I think we gotta take the same principles and apply that to the tools we're building as well, and like the, the guardrails and enablers and primitives. Um, so I think,

182. 37:12

     I don't know, establishing some kind of, um, a forum where you can discuss these things and like s- plan like what do we want the factory to look like?

183. 37:21

     What are the components we need? Like, what are the integrations we need? Do you have any examples of like specific things that people... Is it like flavor or is it more bigger changes that the agents are doing?

184. 37:31

     Um, what, what I notice when, when it comes to rules, they, they create like, oh, I want to-- Like one person wants to write the test first, and they create the rule to write the test first, but they know that somebody else doesn't want to do it that way.

185. 37:43

     Mm.

186. 37:44

     So then they have the rules only on their machine. They, they don't share it because it is too unique to what they are. So they're collaborating, the whole team is collaborating on creating the code base-

187. 37:56

     Mm

188. 37:57

     ... but the collaboration in creating the factory in thinking, well, are we deciding now that the factory writes the test first or not?

189. 38:06

     Mm.

190. 38:06

     That is a big decision that is hard to align everybody and accept that.

191. 38:11

     Like with all of these rules, not everybody's going to be completely on board, and in most cases it doesn't matter when-

192. 38:17

     Yeah

193. 38:17

     ... when you defer a little bit, but it is hard to.

194. 38:20

     Yeah. I guess it's, it's, it's a human problem and a human change that needs to be made. [chuckles]

195. 38:24

     Yeah. This is a human, yeah.

196. 38:25

     But it's, uh... It's a good question. I'll think a lot about it. Thank you.

197. 38:31

     Okay, cool.

198. 38:34

     Thanks for the talk. Um, a lot of the patterns, uh, resonate. Um, I was wondering what is needed, what kind of patterns can you suggest to take it to the next level if you work on enterprise brownfield mission-critical systems-

199. 38:48

     Mm

200. 38:49

     ... that cannot fail, they cannot be insecure. If you look at the recent supply chain attacks and you give your agent sandboxes, maybe that's not even enough.

201. 38:57

     Mm.

202. 38:58

     Um, [lip smacks] so the humans remain accountable-

203. 39:01

     Yeah

204. 39:02

     ... and we can't say, uh, "Oh, uh, uh, it's not my fault. My agent did that." So do you have any extra patterns that, um,

205. 39:11

     um... Or i- or is it just inherently we have to keep reading the code, which may feel like reading assembly lines in the '80s or something? [chuckles]

206. 39:21

     Um, I think if you can spend a lot of compute and tokens upfront before you as a human actually like needs to be involved, I think that's a pattern that we found be- to be pretty successful.

207. 39:34

     Um, so one thing is like manually writing tests for very critical parts of the systems, um, and then just letting the agents like run them, uh, a lot. Um, the second part is like

208. 39:47

     building automation to like, um... Our security team, they built like the Security Sentinel, which is an automation that like s- looks specifically for very, very, uh, specific invariants, um, of the system, and they run like ten of these on like certain PRs that changes certain files.

209. 40:04

     Um, and then, yeah, I think, I think it's a bit contextual as well, but yeah, just spending a lot of tokens before, uh, and trying to like find different variants and like almost red teaming.

210. 40:19

     Um-

211. 40:19

     So one thing I did is, um, instead of focusing on velocity and throughput, I focus on quality.

212. 40:28

     Sorry, what?

213. 40:29

     I, I use AI to focus on quality-

214. 40:32

     Mm-hmm

215. 40:32

     ... and just improve the tests and just make it-

216. 40:34

     Nice

217. 40:34

     ... completely AI ready.

218. 40:36

     Yeah. I think that's very good. Um, 'cause if you as a human trust the tests, you probably are trusting the output even though you don't have to look at the code.

219. 40:45

     Um, and that's kinda like where we're going.

220. 40:49

     Trying to think.

221. 40:58

     Hi. So, uh, thanks for a great presentation. Uh, I fi- I find myself kinda like lacking and slacking in using guide rails, uh, especially like rules and hooks. Uh, partly because historically the, the, the knowledge of how to do that properly was very scattered and decentralized across, uh, whole web.

222. 41:20

     So you would have this like exotic GitHub repos-

223. 41:23

     Mm-hmm

224. 41:23

     ... who would, uh, try to like centralize this knowledge, or maybe you would have some like Medium articles, or maybe Cursor would-- Cursor company would do a blog post on this, right?

225. 41:34

     But still it was very evolving, and also the capabilities of models themselves on, especially on instruction following, uh, they are also evolving and they are getting better on that and, and, and it always felt like kinda like duct taping-

226. 41:50

     Mm

227. 41:50

     ... uh, to me. So I'm wondering, uh, basically can we have AI to help us with that? Meaning that could Cursor, for example, give us like proactive agents or maybe some new setup, uh, or maybe wizards kinda, uh, setups where we could identify our workflow and then help AI build us

228. 42:16

     rules and, and guardrails and all those like rules artifacts, uh, for us. So maybe just like a proactive agent, uh, so, so maybe we, we would have like an agent that would scan our workflow globally-

229. 42:31

     Yeah

230. 42:32

     ... and then help us build those artifacts. What do you think about it, and do, do you guys think about this in the company? Maybe do you work on that?

231. 42:40

     Yeah, totally. Um, I think now there's like two places where you can do this. One is like in the product itself with the whole, um, uh, with the like continual learning prod-- Um, let's see here

232. 42:55

     Oh, I don't have it installed. Uh, we can go to Marketplace. Yeah, with the continual learning kind of, uh, plugin to actually, like, look at your, um, transcripts and, like, extracting rules and memories and all that.

233. 43:07

     That's, like, one way to do it. [lip smack] Then there's, like, another world where, um,

234. 43:13

     you, like, change the weights of the model depending on, like, what your codebase looks like and what, like, your engineers are doing, like, in a specific team. Uh, and you, like, you reconcile them.

235. 43:24

     And it's like, it's, like, true continual learning, uh, not this [laughs] hacky plugin. Um, and you, like, actually bake this into the model, uh, so they actually know what your preferences are, et cetera. [lip smack]

236. 43:37

     Um, but totally, like, memory and rules and all that, I think that's gonna become more and more important over time, um, because that's kinda like what's lacking. That's kinda like what's preventing me from having a lot of trust in agents sometimes.

237. 43:51

     Because, like, I say something and they forget about it, uh, but they're just, like, stateless machines, so how do we capture this knowledge? Um, so I think we should put a lot more, like, time and effort into, um, building these systems.

238. 44:05

     If, if I may just follow up on that. So, so you, you say that you seem to first do-- to start a project or, or do-- to dive into the project that's already existing in the codebase, and then to build rules on top of that.

239. 44:19

     How about we, we first have rules and we want to start a new codebase, new project.

240. 44:24

     Mm.

241. 44:24

     How to, how to actually have those good rules for us? Do, do you think that humans should, humans should still do that, or can we also automate that? Can-- Do we have a new best workflows for- [chuckles]

242. 44:38

     ... that?

243. 44:38

     I think it's hard because, like, my perspective of rules is, like, the bridge between, uh, the model behavior and the, like, the human behavior, and, like, how do we steer the models in a way that they follow me as a human, what I wanna do.

244. 44:51

     Um, and in a new product, I'm not really sure what I wanna do. Like, I kinda wanna, like, outsource that to the model, like, see what are they doing here.

245. 44:58

     Can I run different models? Do I wanna, like, combine them, or do I wanna scrap everything? Um, [lip smack]

246. 45:04

     so I think it's, it's hard. Like, one-- Like, the best example of a rule that I can think of internally is for Bugbot. Um, so when we're doing database migrations, uh, we're not really using foreign keys on a database, um, for performance reasons.

247. 45:19

     And the models, like, the right way to do this is use f- foreign keys, right? Um, so they will always add a foreign key. Um, but when it, like, hits GitHub and, and there's a PR created, we have Bugbot looking at this and reviewing this, like, "Oh, I have this rule saying, like, we should never use foreign

248. 45:35

     keys." So then it flags this. Um, so that's, like, the gap between the human and the model and what we want-- the desired, like, intent we have versus what they have.

249. 45:45

     So I think rules should, like, emerge dynamically over time. Um, and before that, you should probably introduce this ephemeral, like, specs and plans. Um, oh yeah. There's, like, one over-

250. 45:57

     Yeah.

251. 45:57

     Oh, yeah. Oh, yeah.

252. 46:00

     It doesn't work?

253. 46:01

     Oh.

254. 46:01

     It, it worked. Uh, so thank you, Eric, um, for the talk. Um, as evalua- evaluation and trust is a big point, I'd like to know how you effectively do, uh, GUI testing and, uh, user acceptance testing-

255. 46:16

     Mm.

256. 46:17

     ... automated.

257. 46:18

     Yeah.

258. 46:18

     If you could show, show, like, uh, something of your workflow, that would be great.

259. 46:22

     Um, totally. The best or, like, the main way I do it is using... Let's see here. Oh yeah, I have this one, for example. Um, the main way I do it is using, uh, the Cursor Cloud Agent with the computer use that we have.

260. 46:36

     So I'm gonna publish this... Oh no. That's bad. [laughs]

261. 46:43

     Uh, I guess we're not doing that. Um, I have this website where it's running a, uh... I have, like, seven components, um, like a button, a dropdown, et cetera, et cetera, web components, and then I'm generating each of these components with a different model, and...

262. 47:02

     'Cause I wanna, like, compare, like, what does a Composer dropdown look like versus a GPT 4 dropdown look like. And I put this in a grid. But when I created this website, there was an error where I had this, like, view code button, so I could actually see the generated code.

263. 47:16

     It was not working because the model didn't, uh, bundle the actual code. So I went to Cursor, and I clicked-- when clicking View Code on that component, it says it cannot load a code.

264. 47:27

     And it's like, it's a very, like, short description. So what the agent did, uh, you can see here, it spawned my local server.

265. 47:36

     Um, it started, like, clicking around and pressing Enter. Uh, we can see the cursor up here. [lip smack]

266. 47:42

     And it's creating this, like, Screen Studio-esque, uh, recording where it's, like, um, chopping and speeding up and zooming in, et cetera.

267. 47:52

     Um, so here it's taking a while because computer use is fairly slow. Um, it's consuming a lot of tokens. And we can see we have this View Code button, and now we can actually see it's working too.

268. 48:04

     Um, so since this is a very, like, much of a side product for me, I'm not really gonna look at the code. I'm just gonna, like, see that this works, and I'm gonna merge it.

269. 48:14

     Um, but you can keep on prompting the model to do very specific things for you, like, "Can you follow these, like, specific instructions?" Um, like a login flow, for example.

270. 48:24

     You should click the button. You should log in. Um, the models-- Th- this, like, login steps are probably so much in distribution that you can probably just prompt the model to say, like, "Go to this URL and click Log In," or, like, "Log in," and it's gonna, like, understand which steps it needs to take.

271. 48:40

     But then you can ask the model to, like, uh, input a wrong password or input a wrong email and see, uh, what are the results from the website. And maybe the res- uh, website is giving, like, wrong credentials, and then the agent would understand, like, "Oh, I need to, like, put in the right credentials."

272. 48:56

     Um, so just like you would Um, like hire a consultant, like a QA consultant, and giving them instructions. You would just give the same instructions, uh, to the agent.

273. 49:09

     Um, so this is like one way to do it. Um, I guess the other way would do like more, uh, playwright/puppeteer and just automating like a browser thing, uh, which is a bit more deterministic as you can review it, um, and check it in and like have other people reuse it.

274. 49:26

     Is it-- Does that answer the question?

275. 49:30

     My question was going more into, uh, like user acceptance testing to check, does this thing actually look right? Because, like, uh, testing a login, you can do-- you can automate that.

276. 49:42

     Mm-hmm.

277. 49:43

     You need an agent for that. But like, does the, does the website, uh, look right? Is it consistent through all the pages that are generated-

278. 49:52

     Mm.

279. 49:52

     ... stuff like that?

280. 49:53

     Yeah. Yeah. Then I've-- Then I, I use cloud agents for that a lot. Um,

281. 49:59

     there was one ca- I can't remember now, but I think it was, I did some changes in the docs, and I just asked it to like open every single instance where this, where it is referenced, uh, take a screenshot, and give it back to me.

282. 50:12

     So then I could just like look at all different screenshot, everything looked good, and then I could merge the code. Um, so letting like the agents do, uh, the navigation and clicking around and, uh, the testing for you, um, I think it works surprisingly well.

283. 50:27

     Like, this was like a very much an AGI moment for me, uh, when we launched this in last year sometime internally. So have you, have you had the chance to try cloud agents in Cursor?

284. 50:38

     No.

285. 50:40

     You should. [chuckles] Curious to get your feedback.

286. 50:42

     How expensive was to run this? I know you are-- you have sponsored one.

287. 50:47

     Uh, which one? This one?

288. 50:48

     No, no, the agent like spawning all the VMs and giving us the walkthrough.

289. 50:54

     Uh, what, what was the initial question? How long it took or?

290. 50:57

     Yeah. No, I, I see. But how expensive wa- it would be for like a typical user?

291. 51:03

     Ah. Um, very straightforward. Like I-- for this one, I did no specific setup. Um, for like our own repository, uh, where we have like-- When running Cursor, so like we can actually like reproduce.

292. 51:16

     Like this demo here is running all the back-end services for Cursor, it's running all the front-end things, um, and this is like a lot, a lot of different things.

293. 51:25

     Um, so the VM is quite beefy. Um, but as long as you give the right instructions, it's working really well. What we did was creating this internal CLI that the agent could use to sort like, uh, we call it, um,

294. 51:40

     like Cursor dev tool. Cursor dev tool back-end start, Cursor dev tool front-end start. Um, and that is abstracting everything away, um, that actually needs to get to like, uh, Orb Stack, to running, uh, ClickHouse and Postgres and Redis, and then the front-end running like, um, Electron and then, uh, Glass here.

295. 52:02

     But then they just like coexist the two different processes. Um, and the agent have access to everything, like just as a human would do. Um, and you can have like the agent be authenticated if you store like a snapshot where you are authenticated, et cetera.

296. 52:16

     Yeah. What, what I meant, how expensive was it in dollars? Like-

297. 52:20

     Oh, sorry, sorry, sorry. Okay, okay, okay. [laughs] Uh, my bad, my bad. Um, yeah, this one I don't really have the... I could probably look it up. I would guess this is like,

298. 52:31

     mm, one dollar, something like that. Um-

299. 52:35

     Just one term?

300. 52:37

     There's like for one term-- Probably like this initial one would be one dollar. Uh, and the other ones, I just asked them to re-record a bunch of different things.

301. 52:46

     Um.

302. 52:50

     Mm-hmm. Yeah.

303. 52:51

     Something like... I can look it up later. Totally.

304. 52:53

     I, I jump back and forth between Cursor and other like, like Cloud Code environments.

305. 52:57

     Mm.

306. 52:58

     I, I'm not sure.

307. 53:00

     Yeah. And I guess depends on which model you're using too.

308. 53:05

     Okay. Okay. Uh, my question is about, uh, uh, hand, handoff between humans and-

309. 53:13

     Mm.

310. 53:13

     -and agents wh- whenever you are using different tools. So in my current setup, I have a product owner and a functional analyst that they, they work on Cloud Code, and they prototype very fast, uh, with basically without, uh, uh, so much thinking about, uh, oh, the back end, the architectural choices or whatever.

311. 53:35

     And then they pass the, the control down to the delivery team that uses Cursor and has to make that stuff work, actually work.

312. 53:44

     Mm-hmm.

313. 53:45

     Uh, which best practices do you suggest in order to enforce a proper workflow between people just not knowing basically what they are doing-

314. 53:55

     Mm.

315. 53:55

     -uh, on a technical point of view, of course, uh, and the people that needs to bring that thing that maybe has, um, okay, some poor choices such as, okay, use that database or then, uh, Cloud Code changed the idea and they moved from a super base to, uh, Turso to a, any other kind of fancy database that

316. 54:15

     actually is in that, uh, in that environment, and then bring that into some sound architectural choices-

317. 54:22

     Mm.

318. 54:22

     -moving from Cloud Code to, to Cursor.

319. 54:25

     Mm, mm. I think what we're doing internally is like we have like one or two PMs, and they are building a lot of different prototypes. Um, sometimes it's actually in the real like product itself.

320. 54:38

     Uh, they're using maybe cloud agents and just prompting them. They're getting like a video like this back of the changes, and it's like, "Oh, it kind of looks like I wanted to," and then they tweak the designs a bit.

321. 54:48

     But the code might be really bad or like not following best practices, um, which if they had a, if we had a good factory, then it probably would. Um, but if that's the case, uh, we hand off like a link to the cloud agents.

322. 55:01

     We just copy the link and just send it to the, the like engineers. Like, "Hey, this is like something that we wanna build." Um- Does this make sense? Like, can we do this?

323. 55:11

     Um, and then you have a lot of intent already expressed. Um, but the other case is like having the PMs, they have a separate repo called like Prototypes, and it's just like an HTML file, like a mega HTML file, uh, reproducing like the Cursor UI or the dashboard.

324. 55:28

     Yeah. The, the problem is the migration. So, uh, just a particular use case, I had my PO and functional team, uh, build out a very fancy demo using, uh, Prisma and Turso and-

325. 55:41

     Mm.

326. 55:42

     ... whatever database, and then storing data on Vercel, uh, blob storage. And then my delivery team had to migrate that to use SQL Server and, uh-

327. 55:54

     Mm

328. 55:54

     ... uh, C# and Aspire for the backend. And the migration was really painful.

329. 55:59

     Yeah.

330. 55:59

     Even because, uh, when they use the agent freely with no constraint, uh, the agent, mm, sometimes decided to use, say, Next.js, some other times decided to use Vite.

331. 56:11

     Mm.

332. 56:11

     Another time it, it decided to use Svelte. And, uh, putting constraints in form of rules within that agent-

333. 56:20

     Mm

334. 56:20

     ... shaped that down the path. But the problem is that, uh, we need to, uh, write a lot of rules and make them consistent. Uh, and it is not easy to, to manage all the workflow.

335. 56:33

     So, uh, we, we are shifting a lot of effort from, uh, having people to write code to having people to write guardrails and the rules and whatsoever, and make all the pieces talk to each others.

336. 56:46

     Mm. I see. Yeah, yeah. Yeah. I guess, um, if, if the POs and PMs can't have access to the actual code base, just like handing off an artifact is like the minimum viable intent, uh, which could be like an interactive...

337. 57:02

     Like back in the days, it used to be like Figma prototypes, right? You can click around, and you get like a feeling for it. Now you can have them even higher fidelity, where you have an interactive prototype using like web technology without like touching anything on the backend stuff.

338. 57:17

     Or it doesn't have to be like a working thing for real if it's just a prototype internally. Uh, but just enough to like your engineers can understand like, "Oh, this is like the intended thing.

339. 57:27

     If I click this thing, that should happen." Um, or, "If I like enter some text here and click send, a row should show up here." Um, and I think all that can just be done, um, in the front end, kind of like a hackathon.

340. 57:41

     You don't need to migrate the, the prototype into something that becomes production-ready, but rather, uh, rewrite that.

341. 57:49

     Yeah, I think so. I think rewriting. Um, and I think like, um, setting like clear expectations from, from the engineers to the PMs and the POs, like what engineers kind of want from the product organization and like what's most helpful for them.

342. 58:02

     So maybe not like vibe coding complete SaaS products is the most efficient thing.

343. 58:07

     Okay. Thank you.

344. 58:13

     Hi, Zak. Thank you for that presentation. Uh, my question is, we're building more and more agent, and it become part of our time-critical processes. How do you see the brownouts and blackouts as, as a, as a, as a new risk?

345. 58:27

     And, um, what's your op- uh, what's your view how it can be mitigated and, and the impact reduced?

346. 58:35

     Yeah, it's a great question. Um, it's a really good question. I think it comes down to what we talked, like the humans are still accountable for the things that's being shipped.

347. 58:45

     Um, so the humans need to build like systems and observability and monitoring around the changes that's being made. Um, and I think that still like comes down to understanding which are like system critical areas of the code base, making sure you have good like observability and understanding of everything that goes on.

348. 59:03

     Maybe like every line should be human rewritten in these critical things, or at least like always humanly reviewed by one or two people. Um, and yeah, it's, it's close to vibe-- It's easy to vibe code close to the sun and fly too close.

349. 59:20

     Um, so I think it's also like a cultural thing where you have to make sure that the humans are still like accountable for, for the things getting shipped. Uh, but yeah, setting up good systems to understand, um, the changes being made, I think that's important, and tests.

350. 59:43

     Hey, Eric. Thanks a lot for the talk. And I'm assuming you're probably one of the people around the world that has the best understanding of how to use these technologies.

351. 59:52

     So this question takes a step back about from the technology and thinks about processes and how do you manage yourself in your workdays. And I wonder, how long are these tasks or how, how long do you get to be away from your agents without babysitting them?

352. 1:00:10

     And how do you actually invest this time? Uh, let's say you have five, 10, 15 minutes. How do you make the best out of your time? And maybe how many agents do you have in parallel, like-

353. 1:00:20

     Mm

354. 1:00:20

     ... mental processes, and how do you manage yourself? Thanks a lot.

355. 1:00:23

     Yeah. It's a great question. And I think like once you... Like there's like two levers to pull. Uh, one is like the scope of the, of the change. Like the larger the scope is, the longer the agents are gonna run.

356. 1:00:35

     And if you want them to run for a really long time, uh, you want to have like verifiable, um, systems so like they can check their own work, et cetera.

357. 1:00:43

     Um, and the other thing is like how much can you parallelize? Like how many of these agents can you spawn off? Um, and I think the sad reality [chuckles] in some sense is that there's gonna be a lot of context switching.

358. 1:00:55

     Um, I probably work in four different repos for like four different areas of the code base at the same time. Uh, whether that is like through a like single like feature that requires front end, back end, database, um, testing, yada yada.

359. 1:01:12

     Uh, or if that's like five completely different things. It could be like docs, it could be like, uh, side projects I'm exploring, it could be fixing a bug from a Twitter user.

360. 1:01:22

     Um, but I usually-- they range from like, um, probably five to 10 agents, five agents like asynchronously running in the cloud at all times. And while I'm waiting for these, I'm either like scrolling Twitter or [chuckles]...

361. 1:01:39

     It's true. I also have the browser in Cursor now, so I can just stay in here and do it. [laughs]

362. 1:01:45

     Or I have like a synchronous task going where like I'm a bit back and forth. Uh, maybe that's like fixing a small thing in the code base, or maybe that's like planning the next thing.

363. 1:01:55

     Maybe I'm like sourcing in Notion and Slack and just like creating a spec in Cursor using a model. So I love to like plan synchronously and then just execute the plans like asynchronously.

364. 1:02:07

     And then once that is done, one of my cloud agents is probably done as well, so I can come back and like review that, keep on prompting it a bit, maybe merging.

365. 1:02:15

     Um, in some parts I still like need to test manually. Like maybe I need to download a copy of Glass or Cursor 3, um, test it manually and like, "This looks good to me.

366. 1:02:25

     Uh, let's go ahead and merge."

367. 1:02:37

     Thank you. A quick question. This factory building leaves us with a scattered ecosystem of a lot of markdown files. Is there an easy way to organize these files and to keep an overview of the factory you have actually built, as maintaining a factory would require you to have an overview of the processes you want your coding agents

368. 1:02:56

     to go through? What tools do you use? What methods do you recommend? How do you keep a mental map of the factory you have built, and how do you maintain it?

369. 1:03:05

     Yeah. It's, it's a really good question. I think it's somewhat unsolved as well. Um, one of the reasons we rebuilt Cursor to look like this instead of like the traditional IDE is the fact that we are using more agents, and we need like a better control panel where you can like see all the agents and manage them

370. 1:03:23

     and spawn them, et cetera. Um, so what's gonna happen with like Cursor 3, um, this is like the first stab at like multi-agent orchestration. Uh, what's gonna happen is that these are gonna be like nested agents.

371. 1:03:37

     So you're gonna have like opening this one up, and you're gonna have like 10 agents in here. Um, so you can still like introspect them and see what's going on and following the traces.

372. 1:03:47

     But you're probably al- also gonna have like somewhere here, like some kind of project view where you can see like an aggregated status update. So like, here's what everyone is working on, and here's like the latest, here's what you as a human need to review.

373. 1:04:02

     Um, [lip smack] so I think these are product things that we are gonna build into Cursor. Um, but to like set the spec for the factory, I would probably like have a folder in your code base, um,

374. 1:04:15

     where you like outline how certain things should work. Um, maybe that's like just markdown files of saying, "Here are some best practices." Uh, maybe it's probably rules, um, and establishing some kind of council to decide on like what goes into the factory and what doesn't, and like what are we lacking to like improve the factory.

375. 1:04:35

     Um, so as long as it's something that the agent can understand and read, which is files, um, that's probably what I would do, and just store them as, uh, yeah, in your code base that's checked in somewhere.

376. 1:04:49

     Thank you. Um, I, I'm just thinking about like teams of the future. Uh, so, you know, a, a year or two ago, it's like very reasonable to have, you know, an engineering team that might be several hundred people, several thousand people.

377. 1:05:04

     Um, what does this do to that, and, uh, what roles-- a-a-and kind of like roles in a engineering team, right? This is kind of akin to almost becoming somewhere between like a product manager and like a, an architect.

378. 1:05:22

     Um, so what roles do engineers have?

379. 1:05:24

     Yeah, I think that, I think that's very accurate. Um, it's hard to predict like what are the like second, third, fourth order effects of, of this happening. And it's definitely like writing less code, looking at less code, um, spawning more agents.

380. 1:05:40

     Um, it's gonna be like how do you take--

381. 1:05:45

     'cause like we're still building software for humans mostly. So like how do we know what other humans want? Like how do we talk to our customers? How do we market what we're building?

382. 1:05:54

     How do we do all these things and bring them into the actual like factory? Um, who sets the direction? What's the intent? Um, all these things are coming from somewhere.

383. 1:06:03

     Either it's like creativity from someone else's head, or it's actually like a user demand. Um,

384. 1:06:10

     so having someone like doing that is gonna be very important. Having someone like s- like aligning that between the different humans in the org, I think is gonna be important.

385. 1:06:21

     Um, having people building, uh, the scaffolding for the other agents and like, um, just, uh, pr- building the assembly lines where the agents can actually run, I think that's also gonna be, uh, important.

386. 1:06:35

     But like to what magnitude and how many people it's gonna be like in-- Yeah, I don't know. It's really hard. Um, you can do a lot with the models right now with a very like small team if you have the right setups in place. [lip smack]

387. 1:06:49

     And like, yeah, depending on the domain you're working in. I don't know. Do you have any predictions?

388. 1:06:56

     Um, I, I, I see issues with kind of like, uh, from like a labor perspective. Um, if you're, if you're working in an incredibly agentic environment, what's your need to like-- like what happens to training new grads, hiring new grads?

389. 1:07:15

     Um, and kind of like the, the future from that perspective, what happens with office politics and, like, land grabbing, right? Because-

390. 1:07:24

     Mm.

391. 1:07:24

     ... basically, your, your value now becomes in your ability to configure and set up your own kind of like agentic team, not in your ability to kind of, well, like program and be productive anymore.

392. 1:07:39

     Mm.

393. 1:07:39

     Like, the 10X engineer is no longer about, you know, words per minute. It's, like, prompting.

394. 1:07:46

     Yeah.

395. 1:07:48

     Yeah. Token, to-token usage.

396. 1:07:51

     Yeah.

397. 1:07:51

     Am I, am I paid in tokens? Am I, am I-

398. 1:07:55

     Leaderboards.

399. 1:07:55

     Yeah, leaderboard, you know. Um-

400. 1:07:57

     Gotta be token maxing

401. 1:07:58

     ... am I paid an amount, and then, like, my token usage takes away from that? [laughing] You know, so how do you, how do you optimize, you know, for that?

402. 1:08:05

     We've gotta train the models to be more political, I think.

403. 1:08:08

     Yeah.

404. 1:08:08

     That's the solution, right?

405. 1:08:09

     We, we need more, like, you know, watercooler talk.

406. 1:08:13

     I guess we're gonna have more of that if the agents are doing our work.

407. 1:08:18

     Hi, Erik.

408. 1:08:19

     Hey.

409. 1:08:19

     Thanks for your talk. Um, I was wondering, um, probably, uh, you are using, uh, uh, at Cursor, uh, some kind of, uh, uh, issue tracking, uh, tools like Atlassian or Jira, Jira.

410. 1:08:36

     Okay. Um, are you using, uh-- I was wondering if you are using, uh, um, um, agents to check, automatically check, uh, and, uh, um, read tasks directly from, uh, uh, Jira, for example, and spawn, um, uh,

411. 1:09:00

     sub-agents to perform the work. Or if there is always a, a human that, uh, um, start work using Cursor.

412. 1:09:13

     Uh, so we're using Linear for issue management, and, uh, we have this first-party integration as well. So for every ticket that's getting created in Linear, we spawn in the cloud agent.

413. 1:09:23

     Um, so like one-- Where I interface with this the most is, like, if we have a feature flag for a specific thing that's rolled out, and if it's rolled out for two weeks with 100%, um, the system kind of like s-signals us like, "Hey, uh, you can-- It's a stable feature flag at this time.

414. 1:09:40

     Uh, you can remove it." So then we have this to create an automatic issue in Linear, and since that is hooked up with Cursor, it triggers a cloud agent to remove, uh, the feature flag.

415. 1:09:52

     So it's kinda like completely automatic once the system knows that it's rolled out to everyone. And I can just like, I can probably look at a code and it's like, "Okay, we can merge this.

416. 1:10:01

     The feature is no longer active." Um, and we do this for like everything. So once you post something in Slack, uh, we either have a Linear Slack agen- Slack agent look at it, or we have a Cursor automation to like, um, look at the message that was posted and, uh, triage it and like look for duplicates or

417. 1:10:21

     like if it's determined to be easy, like start to implement the fix for it immediately. Um, and this is like an example of where a human is like in the loop where it might not have to be.

418. 1:10:32

     It could be like me going on Twitter and like seeing a tweet like something is broken with, um, the plan mode, uh, button dropdown. I can copy that into Slack and then having the agent, uh, perform the work.

419. 1:10:47

     But there's probably a way of where we can just source this feedback immediately without me having to like scan it and triage it and copy-paste it. Um, so that's kinda like a bit how we work with, uh, Linear and issue management.

420. 1:11:00

     Um, but yeah, we, we're also like-- Yeah, since we're spawning a cloud agent for every single thing, it provides a good way for us to dog food the product and like test it out.

421. 1:11:10

     But I'm not sure if I would recommend that for, for everyone because it, it can be quite costly.

422. 1:11:19

     Thank you.

423. 1:11:20

     Oh, yeah.

424. 1:11:20

     Hi. My name is [inaudible]. Um, as cloud agents are a little expensive, uh, do you have something in roadmap to run something locally? Like, I'm, I'm just thinking of an alternative called dev containers and opening in that.

425. 1:11:40

     But do you have something planned in the roadmap for that?

426. 1:11:44

     Um, [smacks lips] what-- I think the closest thing you can do is probably just prompt the agent to run for a really long time. Um, it's kinda like the same thing with like running local models, um, and the reason like for-- I've tried it, like I've probably tried it like once a month running like the best open source local

427. 1:12:02

     model and like seeing how it works in Cursor. But it's never the same experience as running like, um, GPT or Claude or Composer. Um, and the same thing with like running really long things locally.

428. 1:12:16

     I found it to not work that well as if it's running for a long time, it's probably gonna use your, your local database, your other local stuff, um, and it's gonna prevent you from doing other work locally unless you like create a VM on your own machine.

429. 1:12:31

     Um, um, and it-- and if you do, you could probably...

430. 1:12:37

     Wait, never mind. Just re-- Ignore everything I said. [laughs] We launched Cursor Workers. So Cursor Worker is, um... [smacks lips]

431. 1:12:47

     Uh, we launched it like yesterday. Uh, [laughs] it's a way for you to run the same, uh, infrastructure, uh, and orchestration layer as we do for cloud agents, but on any machine you might have.

432. 1:12:58

     Um, so you can do like, not right now, uh, we can do cd dev, uh, or...

433. 1:13:07

     Let me zoom in.

434. 1:13:08

     Can you zoom in for us?

435. 1:13:08

     Yeah. So you can do agent. So we have the agent CLI, and there's now a worker, and you can call worker start. Uh, so from here, we have a worker running.

436. 1:13:19

     Um, and this worker is gonna show up in here. Let's see So we can do self-hosted.

437. 1:13:28

     Let's see here. [keyboard clicking] Oh, I don't think it's hooked up yet. It's a different, uh, account I'm running it on. But eventua- essentially, you can run this on any kind of machine, and you can get access to this, um, from, like Cursor Cloud.

438. 1:13:49

     So you can spawn multiple of these on your own machine, or you can run like a Mac Mini, or you can have a VM, um, in any like cloud platform provider.

439. 1:13:59

     Right. Um, just a follow-up on that. So you, you, you are saying that we can have isolated environments in the local itself using this command?

440. 1:14:11

     Yeah. So it's, it's-

441. 1:14:11

     Can we call the open s- still call the frontier models or composer models?

442. 1:14:17

     Yes, exactly.

443. 1:14:18

     Okay.

444. 1:14:18

     So this is gonna like leverage the Cursor harness. Um, but it's gonna run on wherever you're spawning this, uh, daemon.

445. 1:14:26

     Yeah, that's interesting. Thank you.

446. 1:14:28

     So I like, I built this like Cursor claw thing, uh, where I have one running on my Mac Mini, and that has access to iMessage and Calendar and all these kind of other things.

447. 1:14:39

     And, um, yesterday we launched automations as well, so I can get like, um, like a daily report or a weekly report of everything that's going on in my machine, uh, that I might like wanna know on a specific cadence.

448. 1:14:51

     And since it's running like the agent daemon,

449. 1:14:56

     you will get access to this in like Slack and the web and the mobile app that's coming, um, at some point, not too far out. [clapping] [laughing]

450. 1:15:07

     For iPad too? [laughs]

451. 1:15:09

     Wait, sorry, what?

452. 1:15:09

     For iPads?

453. 1:15:10

     Sorry, what?

454. 1:15:11

     For iPads. A lot-

455. 1:15:13

     Ooh

456. 1:15:13

     ... a lot of time people wanted to have IDEs on iPads.

457. 1:15:17

     It's gonna use SwiftUI, so it's probably gonna be compatible with, uh, iPads as well.

458. 1:15:24

     Right.

459. 1:15:28

     Cool.

460. 1:15:30

     I think that the two versions of iOS and iPadOS are two different things actually, so-

461. 1:15:35

     Probably

462. 1:15:35

     ... you have to really, yeah, design for... That's why we didn't have... You, you can use like GitHub, uh, workspaces-

463. 1:15:42

     Mm

464. 1:15:42

     ... uh, on, on iPadOS, and it kind, kind of works, but it's not really the same thing as Mac.

465. 1:15:49

     Got it. Yeah. Nice. Uh, yeah, one more.

466. 1:15:53

     I just wanna ask quite a simple question. Like, when you have obviously more than one developer in your, where you're working in your company, and you're spawning hundred and hundred of agents to do a lot of different kind of work, how do you ensure you don't step on each other toes doing the same kind of work twice?

467. 1:16:11

     And even high, like you're running internally, do you still, do you still, do you use Scrum or still agile ways of work, you know, even that has already kind of gone out of the window already?

468. 1:16:21

     Mm. [laughs] Um, yeah. What are we doing? We're not really following any like traditional methodologies in that sense. Uh, we do have like monthly goals and, of things we wanna get shipped.

469. 1:16:35

     Uh, but I think since everyone has so much like power at their fingertips with agents, uh, this like causes people to have like extreme ownership over certain things. Um, so for the longest time, there was like one guy building like MCP and rules and like all kind of extensibility, uh, by himself.

470. 1:16:53

     Um, and now we have like maybe one person focusing on MCP, uh, but they can own everything around MCP, and they don't need to, really need to interact that much with other teams.

471. 1:17:03

     Um, but at some point, that's gonna break too. Um, and like so far in the like history of Cursor, we have like found ways to like go in around this.

472. 1:17:12

     The like agentic code owner thing was probably one place where we stepped on each other toes, where the code owners were like misconfigured. So we could just like, instead of having a deterministic thing, can we just pull in the relevant people at relevant time?

473. 1:17:25

     Um, so like something like that is probably gonna happen with other like problems that we're gonna surface in the fut- future.

474. 1:17:32

     Thank you.

475. 1:17:34

     One question about the, the self-hosted agents. So do we get all the goodies that we get with cloud agents, these video walkthroughs? Do, do we also have them?

476. 1:17:45

     I think computer use is the one thing that's like still in, uh, early access, I think. We're, I think we haven't shipped to GA yet, but it's coming for sure.

477. 1:17:55

     So this should be like completely on parity with the cloud agent.

478. 1:18:01

     Yeah.

479. 1:18:01

     Can you describe the, the profile of these, can you describe the profile of these kind of like, uh, mix between product managers and engineers that, that take this, this ownership?

480. 1:18:12

     Mm. Yeah. So I guess the archetypes we have, it's like a PM. Um, they talk a lot internally in, at Cursor. Like, they talk with, uh, go to market, with sales.

481. 1:18:25

     Um, they talk with engineers. They talk with users. They just product manage and product manage and just keep everything together in a way, and also like shield like engineers, uh, from various things.

482. 1:18:38

     Um, and then we have designers. Um, designers work, I would say like 50/50 in Figma and code at this point. Like, all of them do code. Uh, all of them like do push to production.

483. 1:18:51

     Um, but it's a lot of like exploratory work. Like what should... Like what does it look like when it had like 10 nested sub-agents? Um, and you can't really feel that in Figma.

484. 1:19:01

     Like you gotta actually like develop and prototype that. Um, [lip smack] and they work, um, they work with PMs, and then we have engineers of course. Um, but I think Cursor is very fortunate to build like, um, developer products.

485. 1:19:18

     So developers are building the developer product, and it's kinda like they have good taste. They know what good and bad look like. They know like what developers want and don't want.

486. 1:19:27

     Um- And I think because of that, they can take such, like, ownership, and they can, like, go with a concept and go really, really far. Um, whereas, so, like, the PM might be setting more of the business and, uh, like, the overall ch- overarching, like, direction, and then the engineers and designers, like, collaborate on, like, what does

487. 1:19:46

     this actually look like in code? But also, like, how should it feel and how should it look, um, for a developer.

488. 1:19:54

     Makes sense. Uh, are there, like, analysts in this mix as well, or is that done by the product managers?

489. 1:20:00

     Mm. Oh, yeah, that's a good... Yeah. So we have a data, data team as well, uh, data scientists, analysts, and they are also working closely with, um, the PMs of course, and, like, understanding, like, how users are using the product, where the bottlenecks are.

490. 1:20:14

     Uh, but also with, like, with engineers and, like, instrumenting the code in the right way and, like, understanding feature flags and why certain users hit certain paths and some don't.

491. 1:20:23

     Um, so everyone is, like, just working together. Um, and we have, like... I think the way we've structured the team is, like, pretty much, um, domain. Like, extensibility might be one team.

492. 1:20:35

     Um, cloud mu- might be one team. Um, and clouds should still be extensible, so then they have to also work together. Um, but we try to, like, keep it, like, um, modularized and not to ship our organization that much.

493. 1:20:49

     Thanks.

494. 1:20:51

     Cool. I guess one final question if there is one.

495. 1:20:55

     Okay.

496. 1:21:00

     So, um, from time to time, I messed up and started a cloud agent in a wrong repo or something where just, like, went out on tangents, came back to it an hour later where it was desper- desperately trying to get access to that repo.

497. 1:21:14

     Um, are there any way to catch these agents that just don't provide any value? They just, just continue doing stuff, but they're not really making progress.

498. 1:21:25

     Hmm. Yeah, I think that's, that's on us for sure. Um, over the last year we have made a lot of improvements to the cloud agents where initially they were...

499. 1:21:35

     Like, when they were to, like, work, they were extremely useful, but most of the time they weren't. Um, so, like, again, cloud agents also come from this, like, internal need of us just wanting to, like, run things asynchronously.

500. 1:21:48

     Um, and because of that, we have also, like, put a lot, a lot of effort into making our own code base work really well in cloud agents. So maybe some, like...

501. 1:21:59

     We, like, have to sometimes, like, create new projects and jump into other projects and talk to our customers to understand, like, where these things fall short. And we try to have, like, instrumentation of, like, does the agent run for X amount of hours or minutes, and, like, does it touch any files at all, or, like, is it

502. 1:22:17

     going in circles, and loop detection and these kind of things. Um, and this is, like, part of the observability I was talking about before. Um, most of that should happen on our side.

503. 1:22:27

     Uh, but there are always gonna be, like, very specific, uh, contextual things where, um, like, if you are the, uh, the code base owner need to, like, set up certain things.

504. 1:22:37

     Um, but yeah, we're, we're working on that, improving it. And if you have any examples, like, please come to me and I'll try to take a look.

505. 1:22:45

     I think the worst was when I started it on a wrong repo and it just, like-

506. 1:22:50

     Hmm

507. 1:22:51

     ... called out to Slack MCP and tried to get access in te- 10 different ways, and it failed.

508. 1:22:58

     Yeah, yeah. Yeah, we could make that better. [laughs]

509. 1:23:03

     Good that you're working on it.

510. 1:23:06

     All right. Thanks everyone for coming. Um- [audience applauding] I'll be around for the next two days as well, so please grab me if you wanna discuss anything Cursor or anything at all, actually. [laughs] [upbeat music]