Building Cursor Composer

Talk transcript

1. 0:00

   [on-hold music] Great to be back in New York, and I'm very excited to be here and talk on behalf of all

2. 0:25

   of our engineering and research teams at Cursor about building Cursor Composer, our first agent model. And my colleague Sasha actually gave a version of this talk recently, so I'm excited to give my own, uh, my own take on it.

3. 0:37

   So Cursor Composer is a model designed for real-world, real-world software engineering, and it tries to be both fast and smart. So as we've measured it against our own benchmarks, it's better than the best open source models.

4. 0:50

   It's, like, up against recent frontier models, but kind of slightly below the latest frontier with Sonnet 4.5, GPT-5.1 Codex. But where it really shines is it's about four times more efficient at token generation than models at a similar level of intelligence.

5. 1:05

   So we're trying to mesh speed as well as intelligence. So why did we build this model? I mean, obviously Cursor has an IDE. Why are we getting into the model space?

6. 1:15

   Why do we care about this? Well, our research and product teams have been building a model called Tab, which you can use for autocomplete. Maybe some of you use that inside of Cursor, and we wanted to take that same approach for a very low latency model and apply it to coding with agents.

7. 1:29

   But honestly, we weren't really sure if it would work. So [chuckles] we started prototyping some early versions of what this model could look like, started to put it out and get some feedback from users, and w- we were pretty surprised that this cheetah slug we released for this model, people actually really liked it.

8. 1:45

   Uh, they really liked the speed, but the feedback we got was it's not really smart enough yet to be a daily driver for a lot of their coding. So we needed it to be smart and fast.

9. 1:55

   Definitely needed to be smart. So we really worked on making this internal benchmark that represented our usage on our own repos and how we actually built software. Like, if we had a model that was both fast and smart and a checkpoint that our developers would use every single day to build the product and to build all of

10. 2:10

    our software, then we knew that we would be onto something. And for example, one big change here that helped actually push this towards a level where we had a checkpoint where people would use it was being able to call tools in parallel and being able to very effectively use our semantic search tool, and we'll talk about that

11. 2:26

    a little bit more here later. So if you haven't seen it, uh, here's Cursor and, uh, Cursor 2.0 and our new view, and we're going to use the Composer One model.

12. 2:35

    And you'll notice that it is doing a lot of things very quickly. It's calling a bunch of tools in parallel, like grep, so reading a lot of files. It's making shell commands.

13. 2:44

    Uh, it's making file edits. It's writing and managing, uh, a list of to-dos. And you can kind of very quickly work through tasks in the foreground here. Uh, in this case, I'm investigating an issue in an open source repo.

14. 2:57

    And I don't know about y'all, but this has been a quite different programming experience for me, uh, having working with coding agents for a little bit of time now, versus kind of firing off an agent and waiting, let's call it twenty minutes for it to complete where you can kind of context switch away.

15. 3:11

    This really does help keep you in the flow and is a kind of a different style of programming, I think. So I wanna talk about how we did this in a way that's hopefully accessible for you all.

16. 3:20

    I, I'm not a machine learning researcher, but I do really enjoy this stuff. Uh, what we learned, some of the infrastructure challenges, and then, uh, a little bit on where we're going, uh, moving forward.

17. 3:29

    So in Cursor, a user s- kind of submits a query to our back end. The agent reads that query and then decides to make a series of tool calls.

18. 3:37

    And our agent has about ten tools, uh, give or take, but we're gonna focus on five here. So reading files, editing files, searching your code base, looking at lints, and then also running terminal or shell commands.

19. 3:48

    And the agent then is able to autonomously decide, do we call these serially or do we run these in parallel? And our goal with reinforcement learning here is to try to mirror the Cursor production environment as close as we possibly can.

20. 4:01

    So this data that we have in training, we wanna kinda pretend like we're actually calling real Cursor queries. Uh, so to do that, we are running a series of rollouts.

21. 4:10

    Um, for example, in this rollout, we're calling a series of tools like reading files and editing files, and when we run more rollouts, we can start from that same initial starting point, but we might call a completely different set of tools.

22. 4:22

    So in this one, we're also doing code-based search. So we score the output, we decide which one is better, and then we update the parameters of our model based on that change.

23. 4:32

    So conceptually, a pretty simple idea. The challenges come from when you take the simple idea and then you try to scale it up to a very large amount, and so there's kind of three challenges.

24. 4:41

    The first one is trying to match the training and inference environment, so when the model's actually being used in the product. Um, in this case, with Composer, we're training a large mixture of experts model, and it's being parallelized across thousands of GPUs, and if we don't speed that up, it's gonna take forever to train the thing.

25. 4:58

    So we wanna make it really fast and match the training and kinda sampling version to be as close as possible. The second challenge is that the rollouts can get pretty complex when you start to look at real world data here.

26. 5:10

    So models are gonna use hundreds of thousands to millions of tokens. They're gonna make hundreds of different tool calls, and each of these rollouts could take a, you know, a pretty different amount of time.

27. 5:20

    One might make a lot of tool calls, one might make not as many, and they'll complete at different times, so we have to figure out how to deal with that challenge.

28. 5:27

    And finally, there's this challenge of consistency. If we want to mimic the production Cursor environment as close as possible, we need to use exactly the same tool format and the tool response.

29. 5:37

    But in training, we have this really bursty amount of compute, basically. We're, like, doing all of this training all at once, which is different than at production. So it is really an infrastructure challenge.

30. 5:49

    We have these three machine learning challenges, and all of the solutions, coincidentally, are actually infrastructure problems. So let's talk through a few of these problems and how we solved it at the infrastructure layer.

31. 6:00

    So our architecture is probably familiar for some of you who have been involved in this space a little bit, but I still think it's really interesting to talk about at kind of a high level.

32. 6:09

    Uh, we have three different servers. We have an inference server. We have kind of the standard ML stack with PyTorch. We have an inference server, so the rollouts that I just talked about, um, that's where we use Ray, and then we have environment servers, and these are the ones where we're kind of simulating that Cursor environment that

33. 6:24

    I talked about. And all these servers talk to each other. So for example, the inference server can basically send these advantages back to the trainer, which is like nudging it up or down, uh, b- based on the rollout, and then updating the model and getting new parameters.

34. 6:40

    So this, this one is a bit more on the ML side, but we're, we're trying to train a model that's very, very large and to do it as fast as possible.

35. 6:47

    And one way that our team was able to do this on the research side was to develop a library of custom kernels that allowed for very low-precision training. And basically, this just allows us to just speed up the training process in a big way and also make it much easier to ship to our inference server.

36. 7:03

    So if you're the type of person who loves this, we wrote a blog post going way in-depth on all of this. It talks about our custom kernels. Uh, if you're interested, the TLDR here is we found for the mixture of experts layer was about three and a half times faster, uh, a speed-up on NVIDIA Blackwell chips.

37. 7:18

    So it made a pretty significant, uh, impact on our training runs. So once we update the weights, we need to send them back over to the inference server, uh, during this training process, and the inference server is the one that's doing all the rollouts that I talked about, calling the tools and kinda managing, um, what we sent.

38. 7:35

    The challenge here, uh, [chuckles] is that they all complete at different times. So kind of a naive version of this, there will be a lot of wasted time. So what we were able to do is do load balancing across the different threads and processes to basically shift the work around and, and not have a bunch of idle time.

39. 7:52

    So if one rollout, for example, makes a ton of tool calls, maybe it installs some packages, installs some library, we're not just sitting there waiting for all of the other ones to finish.

40. 8:01

    The inference server is spending all this time going back and forth, making the tool calls to the environment, uh, and getting the tool results back, so again, communicating between these servers, and we want that environment to be as close as possible to the Cursor product.

41. 8:15

    One thing that's nice about having both the coding agent, the IDE, as well as what we're doing with the model research and training our own models, is we can kind of co-design these things together.

42. 8:24

    So as we were building out a lot of our RL work for this model, we were also building our Cloud Agents product. Um, this is how you can run a Cursor agent kind of offline.

43. 8:34

    You can run it from your phone or on the web or kick it off from Slack, for example. And to do this, we spin up virtual machines in the cloud.

44. 8:41

    So each one of these VMs loads up the user's code. Uh, it allows the agent to kinda like make file changes, run tools, and edit code in a secure sandbox.

45. 8:50

    And coincidentally, this is the perfect infra for RL and our use in training. So we have this like fleet of cloud VMs, and we have an environment that very closely matches the production Cursor environment, and we can then use that for training.

46. 9:05

    This does still have some challenges, though. I kinda talked about how the training workload is very spiky, and it's different than the kind of standard inference when you're running the Cloud Agents product.

47. 9:14

    So we needed to build infrastructure to support all of these VMs and orchestrating between them. So, you know, we have many different clusters, hundreds of thousands of VMs here, and you can see behind me one of the internal dashboards we built, uh, with Composer actually, to visualize, uh, all of the different VMs in the fleet.

48. 9:33

    So why spend all this time trying to match the environment to be as close as possible to Cursor production? I've kinda mentioned that a few times. We could mock it.

49. 9:43

    We could simulate it out. Um, but one of the really nice benefits is we get to give the model, uh, specific tools that we think are very valuable inside of the agent.

50. 9:51

    So one of those is that we've trained our own embedding model that allows you to do semantic search. So when you use Cursor, we go and index your code base, and then it allows the agent to make natural langua- natural language queries to find files that it might wanna edit.

51. 10:07

    And we did some research on this recently. We found that semantic search not only helped basically every single model inside of the Cursor agent harness, but it was particularly helpful with Composer, which kinda makes sense when you think about it.

52. 10:19

    Like, we trained Composer in the exact same environment that we're using at inference time, and so the model kind of becomes a power user of this tool, which is really effective.

53. 10:30

    So let's talk about, uh, how the release has been going and kind of where we're going next. Um, as we were doing the training process, we kind of knew that RL was working when we were able to continuously improve the model and start to see more and more improvements after more and more rollouts.

54. 10:47

    So we started about kinda the same performance as the best open model, and then as we trained and kind of threw more compute at it, the performance continued to increase, and to a point today where we're close to the frontier in terms of kind of the best coding agents that are available.

55. 11:02

    And personally, I think this is a great sign just for being able to take and scale RL and apply it to these very hard specialized tasks, like in our example, coding, but it could be applied to other domains as well.

56. 11:14

    Uh, RL also allowed us to kind of change properties of the model in a way that was very useful for the Cursor product. We wanted the model to be both kind of fast at generating tokens, but also the end-to-end experience of getting a result that's helpful.

57. 11:29

    So for example, instead of reading a file one by one, you can read 10 files in parallel with tool calling. And as you saw in the demo earlier, it makes Composer feel much faster when you have that.

58. 11:39

    And we think this is kind of just the start. There's a lot more we can do in this area to speed up the model. Uh, and the second one is the model learned how to behave better as an agent.

59. 11:48

    So in the beginning, the model was, was kinda making too many edits. Sometimes the edits were made unnecessarily. But as we trained more and more, the model actually got surprisingly better at learning to search and read files more.

60. 12:01

    So it would go and find the right thing before it tried to make edits, overall just being, uh, uh, you know, a bit more effective.

61. 12:08

    So we released Composer last month in Compers-- uh, Cursor two point O, and so far, it seems like people seem to like it. Has anyone here tried the model by chance?

62. 12:17

    Okay, that's pretty great. That's more than I expected, so that's great to hear. I think from my perspective, using this model and, and using coding agents for some time, I kinda describe this problem as, like, airplane Wi-Fi.

63. 12:28

    So when you're on airplane Wi-Fi, uh, it works, but it's kind of frustrating. You really want to do whatever you're trying to do, but it's just, it's a little slow, almost to where sometimes you wish that you just didn't have Wi-Fi at all. [laughs]

64. 12:39

    And I think for some of us who adopted coding agents very early, it kinda feels like airplane Wi-Fi sometimes, 'cause if it's taking ten or twenty minutes, you're in this weird, I think [REDACTED:origin] called it semi-async valley of death, where you either want something that's really fast, or you want the most powerful, most intelligent model that can

65. 12:57

    run for, you know, a significantly long amount of time, maybe in the background, maybe, you know, thirty minutes, hours, days. And I think when you're stuck in the middle, that's, that's very, very painful.

66. 13:06

    So for me, Composer, and I think other people, it's brought a lot of joy back to coding with agents that felt more like when you were writing code by hand, where you're very in the loop, very synchronous.

67. 13:17

    So I'm excited to see more people exploring this space as well. For me daily, uh, I'm writing a lot of plans with kind of the latest, uh, model, like the, the highest frontier, so GPT-5.1 Codex is, is really great for plans.

68. 13:29

    Uh, and then I'm using Composer to actually take that plan, kinda like what Dex talked about, like, take the context engineering work, and then actually go and build the thing with it.

69. 13:38

    So, uh, a few reflections from our research and products team on building Composer. The first is that RL can work surprisingly well for training very specific models and, you know, giving it this high-quality data and a decent amount of compute.

70. 13:55

    You know, at, at Cursor, we're not trying to build general intelligence. We're not trying to build AGI. We're trying to build very good coding models, and our hu- RL has worked surprisingly well for that.

71. 14:06

    The second one is, uh, how much tools, AI tools like Cursor, it doesn't have to be Cursor, but like Cursor, really help speed up research and development. You know, of course, our entire team uses Cursor to help them write code and debug code more efficiently, but that speed up, that increase really compounds across all of our engineering

72. 14:25

    efforts, so we're able to try more ideas, ship product faster, try new research, um, so it's been really, really helpful there. And the last one that's, you know, personally pretty interesting for me is that [laughs] it was interesting to see how much of the ML work in the training process was actually also an infrastructure problem.

73. 14:43

    They were very correlated. And going back to my time at Vercel, we saw a very similar thing where a lot of the magic moments that you can have in working in frameworks in the JavaScript or Python space, you also need to think a little bit about the infrastructure of where they're actually deployed.

74. 14:57

    So these things are, are more related than people might think. So those are some of our reflections. Uh, sounds like some of you have tried it out. If this is something that you're interested in and working on, we're hiring pretty much across the board at Cursor right now.

75. 15:08

    We just opened up an office in New York, if you're here based in New York, and we'd love to talk to you about building the best coding models in the world.

76. 15:15

    Thank you. [audience applauding] [upbeat music]