Replacing 12K LoC with a 200 LoC Skill

[upbeat music] Hi, everyone.

How you all doing? Thank you for, uh, coming today. Um, I'm going to be talking about how Markdown is basically the new code. Uh, as Teja's already sort of previewed, um, we recently replaced a lot of code in the Cursor application with just Markdown, just a skill.

And in today's talk, I'm going to share a bit of the journey of going from a full-blown feature with a lot of code, a lot of dependencies, a lot of complexity and tests into a much more lightweight, trimmed-down version of the same feature effectively, but just with a single skill.

Um, before I start though, I have to give you guys a little recap of Git worktrees and how they work in Cursor. Now, if you haven't heard of worktrees in Git, they're effectively like, um, separate checkouts, and I'm sorry for the wide screen.

Um, but they're effectively like, like separate checkouts of your repos that allow you to work in parallel. So different agents can be working on the same task at the same-- or on different tasks at the same time without, um, interfering with each other.

If you've never used this feature before in Cursor, the way it works is that you can spin up an agent on an individual worktree, um, and you will see, for example, the same file in two different worktrees, and you can see that they look different because then the agent is doing some work on, on the worktree, but

not on your primary checkout. And any time the agent runs commands or lints or anything it does, will be isolated and scoped to that Git worktree. Um, with this feature, you can also, um, work even in parallel.

At the same time on the screen, you can have, like, these grids of agents working for you. Um, and f- if you say, "Hey, open a PR," the agent will open a pull request from that worktree with the changes that it produced inside that worktree.

And one of the coolest things about this feature is that it allows you to give the same task even to different models at the same time, and then compare what different models do on the same prompt.

So if you haven't heard of this, we call it Best-of-N, and it's effectively a way for you to compete on, on diff-- have, have different models compete on the same task.

And then you can even preview the changes. If it's a front-end, um, project you're working on, uh, you can, um, compare all the different visual implementations and then choose the one you prefer.

Now, if you have never heard about this, all, all-- everything I'm talking about today, um, I will also just say that it all came out in around October of last year or alongside Cursor two point o.

Um, and when we initially shipped that, it came with a lot of complexity. Um, we had to write all the code for creating worktrees, managing these worktrees, fitting them into the agent as context.

We also had to make sure that the agents were scoped and isolated, and they could not escape the worktree they were working on. Uh, we also have something called setup scripts, which users can configure and run, uh, and, and have Cursor run them anytime an agent starts operating on a given workstream.

We also have the judging. So I didn't show you this before, but, uh, there's a little thumbs up icon on one of the models. That's just a, a judge that we run, um, that tells you which implementation looks the best based on, um, different criterion.

Uh, and then we also had to make some changes to the harness, uh, and introduce some system reminders to help the agents stay on track in these worktrees. And then finally, there's, there's some cleanup complexity as well because people like to spin up hundreds of these worktrees, and then their disk sizes blow up, and we have to

help them by cleaning up the, um, the, the worktrees that stay behind. Now, in our new implementation, the one that I'm gonna be talking about today, we were able to get rid of most of these things.

And in fact, I recently opened a PR, uh, removing this entire feature from Cursor, and it was a massive, like, deletion of, of, of code. Like, I think it was around fifteen thousand lines of code deleted.

The new implementation of the feature is almost as good as the previous one, um, and it is much, much more lightweight in terms of us to maintain it, um, and it even has some benefits compared to the previous implementation that I'll be talking about today.

So how were we able to replace an entire feature with a skill?

We decided that there are two primitives that we could use to effectively allow Cursor users to use worktrees by simply leveraging two primitives. One is agent skills and the other are sub-agents.

So both of these are existing Cursor features. You can learn more about them in our docs. Uh, we have a page for skills, and we have a page for sub-agents.

We realized that if we took these two things together, we could basically re-implement both the Cursor Worktrees feature as well as the Cursor Best-of-N feature with just Markdown. And this is a little video of how it works.

So I can now, as a user, say /worktree, and then I'll give it some task. I'll say, "Fix a typo in the footer of the website." And this agent will run in an isolated worktree and do its work there.

So the way the skill is written is actually really simple. I can show you most of it.

Uh, it doesn't fit on the screen, but it's basically a set of instructions telling the model Um, how to create worktrees and, um, to run the setup scripts that the user might have configured, and then to stay on that checkout, right?

We want to make sure that when the agent is operating on a worktree, it is staying in that checkout. Um, the Best-of-N skill is very similar. It's, um, actually even smaller.

The entire skill fits on the screen here with, with a small, uh, font. Um, and what we're doing here is we're instructing the parent agent to go and create sub-agents for each model, and then spin up a worktree for each...

Uh, so a- have each sub-agent create its own worktree, and work inside that worktree. Um, and then we also tell it to wait for all the sub-agents, and when they're done, please provide some commentary.

Please let the user know, um, what, um, the different implementations by the different sub-agents look like. Maybe you can grade them, maybe you can make some, uh, criticism of them, and maybe you can help the user choose which one is the best.

Um, and, and please give that to the user in some nice table format or something. But again, it's only around 40 lines of code, and it's all Markdown. Like, it's not even code.

And the previous version of this was maybe four thousand lines of code.

Uh, some of the considerations we have to have in this, in this skill is that the skill must be cross-platform compatible. Like, we have Windows-specific instructions, and we have Linux and macOS instructions as well.

We also instruct the parent model to run the setup scripts for each worktree that the user might have configured. And then, and this is the hardest part, we'll spend a bit of time on this on the talk today, we have to instruct the model to stay on that worktree, right?

We have to really say, "Hey, do not ever work outside this and do not ever, um, escape," right? Um,

and we, we do that with some aggressive prompting, effectively. So, the new commands are /worktree and then /best-of-n to do the basic- basically, like, um, the, to start agents in isolated worktrees and to start multiple agents on the same task.

And then we also have apply worktree and delete worktree to bring over changes from the side worktree into your primary checkout, and delete worktree just does, uh, what you would expect.

Uh, a little note is that these are not actually skills in Cursor. They're actually commands, but the way these commands work in Cursor is extremely similar to how skills work in that they're-- the prompts only get loaded into the context if the user chooses to load them.

Um, and the only reason we did it as commands and not as skills is so that the prompts for them can be controlled in our servers, in our backend.

This means I can iterate on these prompts, um, without you having to update your Cursor version. Um, if I do some improvements to these prompts, the next time you use them, you're gonna have-- you're gonna get the latest version of the prompts.

But effectively, they work like skills. Um, this is a demo of the Best-of-N, um, skill or command where I'm giving the same task to Kimi, Groq, Composer, GPT, and Opus.

And w- what you will see is that the parent agent starts by spinning up five sub-agents on the five different models that I specified, and each one is gonna have its own worktree.

Each, each one has its own context. And then Opus takes a little longer, as expected. And then at the end, the parent model, as instructed, will do that comparison ac- a- across all the different sub-agents.

It'll say, um, "These two models did basically the same thing. This one did something that none of the others did." And you can even talk to the parent agent and you can say, "Oh, I like this part that Opus did, and I like this part that GPT did.

Can you, can you mash them together?" And the, the, the parent agent will do that for you.

Um, so let's talk about some of the pros of the new implementation, and then I'll talk about some of the, some of the, the, the cons, some of the things we lost, um, with this refactor.

So the main pro of reimplementing this entire feature as a skill is that I have a lot less code to maintain. [laughs]

Uh, selfishly, um, I'm going to be spending a lot less time maintaining this feature. And this is an a- an advanced feature, right? We're not talking about a feature that is used by ninety percent of Cursor's users.

Far from it. Worktrees are kind of an advanced thing, um, and so only the Cursor power users that love parallelizing and having these grids of agents are using worktrees.

So it's not the kind of feature where we want to be spending a lot of time with maintenance.

Another advantage is that our users can now switch into a worktree halfway through a chat. It was not possible before. Um, we didn't want to pollute the prompt UI too much with all these, like, dropdowns and settings.

And so now that it's just a slash command, it's much easier for, for users to switch to a worktree halfway through a chat. They can start talking about something, and then if they decide they wanna work on the side, they can do that with /worktree.

Another big advantage is that the previous implementation did not work if you were working on multiple repos at the same time. So it's very common to have a multi-repo setup where maybe your front-end and your back-end are separate repos.

In the past, you could not do worktrees in this kind of setup. It was just disabled. With the new /worktree command, everything works fine. The agent will make sure to create a worktree on each repo, and then if you open a PR, it'll open two PRs, one for each repo.

It works quite well. Another advantage of the new skill implementation is that the judging experience at the end of knowing what model did which for Best-of-N is far superior.

The parent now has a lot more context over what each of the sub-agents did, and the user can even ask the agent to stitch together little different piece, pieces and bits from the different implementations, which was not possible before.

In the previous implementation, you had to choose one

Sub-agent or one model and just s-stick with that.

Now let's talk about some of the cons. And if you're curious, um, we have a forums link here where we're actually getting some mixed feedback on the new implementation.

Like, some people were really accustomed to the old way of how the feature used to work. Um, and if you're curious, you can go and see that not everyone is happy with the change, at least for now, but we're, we're tracking.

What are the problems? Number one, it's very hard for the agent to stay on track.

With our previous approach, um, the agent had to stay on track. Like, it-- We didn't let the model ever touch any files outside its worktree. It was physically impossible for it to do so.

Now we're trusting the model, so it's... You could say it's a bit vibes-based because we're basically saying, "Hey, operate on this directory," and, and, and then, like, you know, "Knock on wood, please, please don't forget about this."

And especially over long sessions, it's quite possible that the model will forget where it should be operating. And sometimes these models, especially the worst models, will kind of hallucinate or they'll go a bit haywire, and they'll start doing things they shouldn't.

But we're, we're working on this. Um, another con is that it feels slower because you're, you're seeing the agent create the worktree, and you're seeing that in your chat.

It's not actually slower, but it does feel like the agent is kind of, like, wasting time doing something that should be done for it in advance. Um, we're also looking at some improvements here.

And then finally, this is much harder to find the feature now, right? Like, before, whenever you opened Cursor, you had this dropdown that would show you, "Do you wanna run this task locally, or do you wanna run it in the cloud, or do you wanna run it in a worktree?"

Now that entire dropdown is gone, and so if you want to use worktrees, you have to know the feature exists, so you can actually type /worktree. So the discoverability is a bit worse, but as I mentioned before, this is an advanced power user feature, um, which we're personally okay-- We're, we're, we're okay with being less discoverable in

general. So how can we make the skill better? Um, as I mentioned, the biggest problem right now is that the agent is not really always staying on track. Uh, there's two ways that we're gonna improve this.

One is with evals, and then using those evals to improve the prompts, and then the other one is through RL and training. So at Cursor, we train our own model called Composer.

And for Composer 2, our-- the latest version of this model, we didn't have any RL tasks with these prompts. We di- we didn't have any tasks in all of the many, many thousands of tasks that we, um, used for RL actually operating in this type of environment.

So we're working on adding a bunch of these tasks into our RL pipeline so that by the time we launch Composer 3 or 4 or 5, um, at least our own model will be much better at this.

Obviously, we cannot improve the models that the other companies develop, but we've been sharing feedback with all the other labs and model providers on this kind of thing.

And for evals, uh, I've been working on some evals for this feature, and it was actually my first time... Or not my first time, but one-- I'm, I'm fairly, um, early in my, um, my writing evals, uh, journey, and I was actually very surprised.

If you use something like Braintrust, and shout-out to Braintrust, they've been super helpful, uh, writing these kinds of evals are s- is actually super, super easy. You don't have to know almost anything about evals, and you can just prompt the agent, and it'll do everything for you.

Um, effectively, what I'm doing is I spin up the Cursor CLI. It's headless, so it's great for evals. Um, and then I have two scorers, one that checks to see if the model did any work in its worktree as expected, and then another one which is the reverse of that, which is did the model do any work

in the primary checkout where it shouldn't be doing any work? Uh, and so far, the evals I've got are pretty simple, so I actually haven't been,

um, able to simulate extremely long sessions, which is when the models start performing worse. But even so far, I've already understood that not all models are equally good at this.

So for example, Haiku, which is a smaller, less intelligent model, will very often deviate and start working in the primary checkout. But the other models that I've been testing, such as Composer and Groq, um, are doing much better.

So I still have to improve these evals a lot more to make them more complicated. But the hope is that as soon as I can start to find patterns here, I can actually go and improve the prompts.

And then another thing we can do is have better system reminders to the models, uh, instructing them to stay on track and to not deviate from the worktree that they are supposed to be working in.

Okay. So what's next? Um, the first thing is we're actually going to take a, a small step back here, and we're actually going to have a much more

complete and native worktrees implementation in the new Cursor agent window. If you're, uh-- If you've been following, we recently announced Cursor 3.0. Part of 3.0 is a more agentic interface for coding, where you can still edit code and you can still see code, but the UI and the UX are much more optimized around the agent and the

chat interface. We believe this kind of interface is the right place for a proper worktrees implementation. The kind of person who is more likely to be, uh, doing a bunch of local parallelization is usually the same type of person that is more likely to use this type of UI.

So we're taking a small step back there and building a proper worktrees, uh, implementation that is more native, not so much agentic in the new UI. Also, we're improving the skills, um, as I mentioned, through this continued work on evals and then RL and other training work.

And then finally, we are actually looking into other parallelization primitives that are not Git worktrees. So if you've used Git worktrees, you might know that, uh, they can be a bit slow to create, um, and also to, uh...

they also use up a lot of disk space on your computer. Um, and then finally, uh, they only work in Git repos. So if you're using something other than Git, there's really no local parallelization primitive in Cursor.

Um, in the near future, we hope to, uh, share more about this, but we're looking into some other solutions for local parallelization that don't involve Git and don't involve Git worktrees.

Um, so yeah, stay tuned for that. Um, thank you all for coming to the talk today. Um, I'm sure many of you have questions, and I'm gonna be around all day.

Uh, feel free to grab me anytime and, uh, um, I'm happy to chat with anyone. Thank you. [applause] [outro jingle]