AI Engineer Europe 2026
RAG is dead, right?? — Kuba Rogut,
Read the talk
RAG is dead—so why do coding agents still need retrieval?
Coding agents need more than a single vector lookup. Cursor’s indexing strategy shows how reusable semantic search fits inside an iterative search, read, and reason loop.
Before you start: Familiarity with LLM context windows, embeddings, and basic code search will help you follow the examples.
What is supposed to be dead?
If agentic file search makes RAG obsolete, what exactly has it replaced? Kuba Rogut approaches that question as a deployed engineer at turbopuffer, a full-text and vector search database built on object storage. His starting point is the growing role of hybrid retrieval tools in agentic search.
The opening contrast is familiar: social-media posts declare that RAG is dead and agentic file search is all anyone needs. Yet the Google search-interest chart Rogut presents rises in 2023, levels off through 2024, and accelerates around the middle of 2025. That chart establishes continuing interest, not retrieval quality or adoption. The more useful question is whether the debate has confused RAG with one particular implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Retrieval methods and the agentic loop
The narrow version of RAG embeds a corpus, searches it with a query embedding, and passes the returned content to an LLM. But retrieval-augmented generation does not prescribe vector search. Retrieval can use semantic similarity, BM25 full-text search, grep, glob patterns, regular expressions, or ordinary filters. The generation step receives whatever relevant context those methods find.
Agentic search has a similar naming problem. Claude Code and Codex make filesystem search a visible part of an agent’s work, so the term often becomes shorthand for grepping files. Rogut’s broader definition is an agent using tools to progressively find and reason over external context.
| Concept | Narrow interpretation | Broader mechanism |
|---|---|---|
| RAG | One vector lookup | Retrieve useful context, then generate |
| Agentic search | Filesystem grep | Choose tools and iteratively investigate |
These definitions can coexist: an agent can retrieve semantic matches as one step in a larger investigation.
The loop is straightforward:
- Search for potentially relevant material.
- Read what the search found.
- Assess whether it answers the question or leaves a gap.
- Search again if necessary, then continue the task when the context is sufficient.
The distinguishing feature is the feedback between reading and searching. A result changes what the agent knows, which changes what it should look for next.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Cursor reuses codebase indexing work
Cursor, which Rogut identifies as one of turbopuffer’s earliest customers, provides a concrete example. Its Securely indexing large codebases article describes how opening a codebase or branch prepares that code for semantic search: parse the files, split them into chunks, and compute embeddings.
Doing all that work independently for every developer would repeat much of the same computation. Imagine a team of a hundred engineers working across a few repositories. Their checkouts overlap substantially, even when their branches differ. Re-chunking, re-embedding, and uploading every shared file each time wastes that overlap.
The reuse mechanism separates detecting changes from finding an existing index worth reusing. Rogut summarizes this through Merkle trees, hash trees that expose differences in file contents. Cursor’s published implementation distinguishes Merkle-tree change detection from a derived simhash used to select a similar existing index. Once an index is reused, changed content is updated and unchanged chunks can retain their cached embeddings.
Security matters because a reusable team index may contain files a particular client does not possess. Rogut credits turbopuffer’s role in secure reuse; Cursor’s account explains the access boundary more specifically through content proofs that exclude results for files the client cannot prove it possesses. Reuse therefore avoids repeated work without making every indexed file available to every client.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What semantic search contributed
Why maintain this infrastructure? Cursor’s Improving agent with semantic search evaluates what happens when semantic search is available alongside traditional search tools. On its internal Cursor Context Benchmark, Cursor reports 12.5% higher answer accuracy on average across models when semantic search is available. The published figure resolves Rogut’s spoken uncertainty between 12.5% and 13.5%. This is a vendor-reported internal evaluation, not a public benchmark.
Rogut highlights an almost 24% answer-accuracy improvement for the earlier Composer model, before Composer 2. That model-specific attribution comes from his presentation; the accessible article text reports a maximum of 23.5% without identifying the model. These are reported percentage improvements, not percentage-point changes.
The online A/B test measures different outcomes. Both groups used the same model, and the published report gives the following results:
| Outcome | Reported result | Condition |
|---|---|---|
| Code retention | 2.6% higher with semantic search | Repositories with at least 1,000 files |
| Dissatisfied requests | 2.2% higher without semantic search | Online A/B test |
The dissatisfaction result uses the group with semantic search as its baseline. It should not be restated as an identically sized decrease when the tool is enabled.
The smaller online effects do not contradict the larger codebase-question gains. Giving an agent a semantic search tool does not mean every request uses it, or that every request benefits from it. An aggregate test includes requests for which the added tool makes little difference, diluting the effect across the whole population.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Embeddings as cached compute
Claude Code supplies a different design choice. In the historical account Rogut presents, Boris Cherny says early versions tried RAG with a local vector database and did not retain that approach. The displayed exchange describes agentic search as working better for Claude Code. It does not establish that semantic retrieval is unsuitable for every coding agent, nor does Rogut provide a detailed technical explanation for the decision.
The comparison becomes more useful when embeddings are understood as cached compute. Some work that an agent would otherwise perform while discovering a codebase can be moved into reusable preprocessing. Rogut illustrates this with Claude Code-like and Cursor-like traces: repeated runtime discovery on one side, upfront indexing followed by retrieval on the other. These are illustrative traces, not a controlled benchmark of the two products.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Finding how metadata filtering works
Consider the question: how does metadata filtering work in this codebase? A filesystem-based agent starts by finding candidate files, reading them, and deciding whether it has enough information. In a TypeScript repository, a first search might look like this:
bash
grep -Rni --include='*.ts' -- 'metadata' src/
The output locates occurrences of a word; the agent still has to read the surrounding implementation and determine which matches explain filtering. It may need another search before it can answer.
That discovery repeats across sessions. Different developers can ask the same question on different days, and each agent may perform much the same search, read, and assessment cycle. Rogut describes roughly 6,000 tokens for the discovery sub-step in his illustration, not for an entire task or as a universal cost.
The indexed path moves some of that work earlier. Parsing, chunking, and embedding prepare the codebase for a lightweight runtime query such as How is metadata filtered?. The query returns relevant chunks for the agent to inspect, reducing the amount of discovery it must do from scratch.
| Work | Per-session discovery | Indexed retrieval |
|---|---|---|
| Preparation | Begin searching at runtime | Parse, chunk, and embed upfront |
| Runtime | Search, read, assess, repeat | Query the index and inspect results |
| Reuse | Discovery may repeat each session | Reuse preprocessing; update changed content |
The initial index is reusable, not maintenance-free. Its potential payoff is less repeated discovery, with corresponding token, latency, and cost savings; the example supplies no universal speedup or break-even threshold.
Rogut also reports that some turbopuffer teammates had started switching from Claude Code to Cursor because it felt faster, associating that experience with Composer 2 and semantic search. That is a team anecdote, separate from both the illustrative traces and Cursor’s reported evaluations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From one lookup to retrieval during reasoning
The pattern being displaced is the simplest early RAG pipeline: perform one vector search, put its results in the context window, and generate an answer. Rogut associates that pattern with 2023 and early 2024. Among more sophisticated customers, he now sees agents making many retrieval calls across several reasoning steps, with performance gains and new product possibilities that he describes qualitatively.
In this design, semantic search and full-text search are tools the agent chooses as needed. It fetches context for the current question, reasons over the results, and uses what it learns to guide subsequent searches. Retrieval becomes part of reasoning rather than a fixed prelude to it. The system can therefore narrow or redirect its investigation instead of assuming that its first lookup found everything required.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Larger context still requires selection
Large context windows do not remove the selection problem. Rogut closes by recalling Jeff Dean’s discussion of long-context models: even an enormous context capacity leaves a need for staged retrieval, using lightweight mechanisms to narrow a much larger collection to the relevant material.
Rogut’s formulation of Dean’s point is that the goal is the right million tokens, not a trillion tokens at once. He reports customers with trillions of tokens represented in turbopuffer, while the useful subset for a particular task might be ten thousand, a hundred thousand, or a million tokens. Those are different quantities: the corpus available to search and the context worth supplying to the model. As context windows grow, the retrieval system still has to decide what belongs in them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Cursor's evaluation of semantic search alongside traditional search tools, covering codebase questions and online user outcomes.
How Cursor reuses team indexes, synchronizes changed files, and filters results using proofs of local file possession.
Further reading
A practical guide to combining vector and BM25 retrieval, merging rankings, and optionally reranking results.
Podcast recording and transcript discussing long context, staged document selection, and the relationship between search infrastructure and language models.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hi, welcome everyone.
- 0:15
Uh, thanks for coming out. I see it's a full room, so appreciate everyone coming out. Uh, so welcome to the talk about RAG is Dead, right? Um, so my name is Kuba.
- 0:24
I'm a deployed engineer [REDACTED:username] [REDACTED:username]. So for those that don't know what [REDACTED:username] is, we are a full-text search and vector search database, uh, built from first principles on top of object storage.
- 0:34
Um, if you would love to learn more, uh, just come find me after the talk if you have any questions. So let's get started. So, um, this talk I get-- is a, [clears throat] sorry, about how RAG is dead, how hybrid tool, tool-rich retrieval is becoming a default for serious agentic search.
- 0:49
So, if you guys have been on Twitter or other social media platforms, or I guess X as they call it now, um, you might have seen a lot of tweets like this about how RAG is dead.
- 0:57
You can see there's lots of tweets, especially in the last, you know, end of twenty twenty-five and in the early of this year about how, you know, RAG is dead, agentic file search is all we need, and there's kind of a, a lot of tweet and a lot of kind of content about this now.
- 1:11
But, you know, interestingly, if we're to look [REDACTED:username] something like the Google, uh, Google search volume over the last two years or the last couple of years, you can see that, you know, in twenty twenty-three, kind of as AI starts, we have this kind of this, this, uh, you know, increase, kinda caps out a little bit in
- 1:25
twenty twenty-four, settles down for about a year, and then about midway through twenty twenty-five, we hit this new inflection point where search volume just goes through the roof. Um, so take that Twitter.
- 1:37
Um, so let's clarify first. What is RAG and what is agentic search? These are kind of the two terms a lot of people are throwing out these days. Um, so RAG.
- 1:45
What a lot of people think RAG is, is just simple vector search. They just think that this is, um, just simply, you know, embedding a bunch of, you know, a corpus of content, passing an embedding vector and getting it back, passing it through your LLM.
- 1:57
And what [REDACTED:username] [REDACTED:username], what we think this actually means, you know, if we break down RAG into retrieval-augmented generation, you know, retrieval is not just vector search. It's a lot of different things.
- 2:07
It could be vector search, uh, full-text search using stuff like BM25, grepping, globbing, using regex, using other just basic filters. Um, and then the augmented generation is obviously just passing it into your LLM cho- of choice.
- 2:20
And then agentic search. This is kind of the terms people are throwing around a lot these days. And generally, when people start talking about agentic search, what they usually talk about is essentially just file system grep.
- 2:31
So if you guys are familiar with something like Claude Code, um, and kind of that's what-- or Claude Code Codex, um, you know, a lot of people call this agentic search, and this just essentially is grepping through your file system, and this is kind of why these terms are so correlated.
- 2:45
Um, and what we actually believe it is and, you know, kind of the definition we wanna bel- we wanna give it is, it's really giving the agents a set of tools to kind of progressively and iteratively find and reason over context.
- 2:55
So with Claude Code, you can, you know, if you guys have-- are familiar with it, it can read your file, you can start grepping through your file system and read a file, decide that it hasn't found what it needed, what it, what it needed to actually complete the task, and it will, you know, find something again, and
- 3:10
it keep doing this until it's happy, you know, it's reached a happy state where it can continue on with the task. So we're gonna take a step back, um, and talk about one of the companies that use [REDACTED:username], uh, that we believe is doing an excellent job with agentic search.
- 3:23
This is a company called Cursor. You might have heard of them. Um, fun fact, they're actually one of, uh, [REDACTED:username]'s very first customers. Um, and they have this excellent blog post that came out in the beginning of twenty twenty-six about how they index codebases.
- 3:35
So for those unaware, when you open up a new codebase or a new branch in Cursor, what happens is that Cursor will start embedding your codebase. So what they'll do is, you know, chunk out your parse, chunk and embed your codebase, and make it available for semantic search.
- 3:47
And this blog post goes into an excellent, um, kind of excellent technical detail of how they do this. Um, just to give you the gist, essentially the cool thing they do, um, is that they found that, you know, most people working on a team, let's say there's a hundred engineers, uh, when they open up codebases, they're normally
- 4:03
the same codebase, you know, ninety-nine percent of the time because you can have a team of a hundred people most of the time working on one, two, maybe a few codebases, right?
- 4:11
Um, and it's really expensive to have to, like, re-chunk, re-embed, and re-upload these codebases every single time. Uh, so they essentially use like Merkle trees, which essentially is this crypto hash tree, um, to calculate similarities between codebases people open on the same team.
- 4:26
And if they're similar enough, uh, they will essentially copy over the data and then only update the, uh, and re-chunk and re-embed the files that have changed and use [REDACTED:username] in order to make sure this is done securely.
- 4:38
And, um, yeah, it's j-just excellent blog post. They, they do some really cool stuff. Uh, and you may think like, "This is a lot of work. Uh, why do they do this?"
- 4:46
Well, the reason they do this is also covered in a, in a different blog post, uh, about how they use semantic search. Again, they use [REDACTED:username], [REDACTED:username] for this.
- 4:54
Uh, and what they find is on average across models, I think it's like a twelve and a half or thirteen and a half percent increase in answer accuracy. Um, this is across, across their internal Cursor Context Bench mark.
- 5:05
Um, so, you know, not, not a public benchmark, but, uh, you can trust the numbers they give us. And you can see on, on the right side, uh, their Composer model, so this is, this is before Composer 2, uh, they had a almost a twenty-four percent increase in answer accuracy.
- 5:19
So giving semantic search to these tools and, uh, to these models is really can drive real performance gains. And you can see on the, on the bottom right, um, this is from an online A/B test they did, which is also covered in their thing, uh, in, in their blog post, um, about how it's almost like a two
- 5:36
point six percent retenti- uh, code retention in large codebases, and there's a two point two percent decrease in dissatisfied user request. And you may be thinking like, "Oh, well, these numbers aren't that big, like two point six percent, two point two percent, not that large."
- 5:49
But they also cover that, um, semantic search isn't used in every single query. So in their online A/B test, you know, if you give it-- if you give these tools to a hundred quer- a hundred random queries, not every hundred query will actually benefit from the existence of a semantic search tool, so that's why these numbers look
- 6:06
kinda small Um, and now let's talk a little bit about Claude Code. Uh, so Claude Code doesn't use vector search, as covered, uh, by this tweet from Boris Cherny.
- 6:15
Uh, so those unfamiliar with Boris, he's essentially the founding father of Claude Code. Um, and he says that in early iterations of Claude Code, they actually did use RAG in a local vector DB, but they found that it just didn't really work out for them.
- 6:30
But this is something that, uh, is important to understand, and something we like-- we've kind of like taken on a lot internally, uh, understanding, um, here [REDACTED:username] [REDACTED:username], is this idea that like embeddings and semantic search are kind of cached compute.
- 6:44
Um, and you may be thinking like, "Cached compute?" Like, kind of throwing out a lot of terms [REDACTED:username] me right now, like I don't know what exactly what that means.
- 6:50
Um, and I think it's like best to walk through an example, um, of essentially almost like a Claude Code-looking trace and a cursor-looking trace, um, of how some, how these agents will understand your code base.
- 7:03
So on the left is kind of a per session discovery of Claude Code. So for example, if we were to ask the agent to understand how metadata filtering works, what it would have to do is grep, read, assess, and repeat, and try to find the files it needs in order to gain this understanding on a per session
- 7:20
basis. So what this means is you could have, you know, ten agents on ten different days, um, across ten developers, and you, they can be asking the same question multiple times, you know, in day, uh, every day.
- 7:32
And every time, the agent's gonna have to kinda repeat these same exact steps, uh, to gain kind of the like same understanding of, of this code base. Um, and this could, you know, cost quite a few tokens.
- 7:42
Uh, you know, six thousand doesn't seem like a lot here, but just remember this is like one sub-step of an agent. And then on the right is kind of like a more cursor-looking trace where there's this upfront cost of indexing, but then we're able to allow for this like lightweight tool to help the agent kind of retrieve
- 7:57
this information [REDACTED:username] runtime. Um, so obviously there's this like upfront cost of, uh, parsing a code base, embedding it, uh, and making it available, but this is like a one-time cost.
- 8:06
Uh, and then [REDACTED:username] runtime, the agent can just query something like, "How is metadata filtered?" Uh, it can get some simple results, and it would save a lot of tokens, a lot of time, and just a lot of money.
- 8:16
Um, and this just helps the agent to become a lot faster. Um, you know, a lot of people on the team now that maybe were big Claude Code users here [REDACTED:username] [REDACTED:username], um, they've, you know, they've actually started switching to Cursor just because of how fast it's becoming, uh, especially with their Composer 2 models and also the
- 8:32
semantic understanding. Uh, it's just become we're, we're finding really, really good.
- 8:38
So from RAG to agentic retrieval. Um, so what we're finding now is that a lot of people are no longer doing the simple RAG, you know, the, the Twitter quote, unquote, "RAG" of just, um, doing a vector search once and throwing it into the context windows.
- 8:55
Uh, what we're finding, um, is that this worked, you know, back in twenty twenty-three or early twenty twenty-four [REDACTED:username] kind of the beginnings of AI, but a lot of the more sophisticated customers are doing agentic search and it's giving like real, real big performance gains, uh, and kind of un-unlocking like new products.
- 9:12
Um, and it, what we're finding is, you know, they're doing a ton of calls. They're reasoning, these agents are reasoning through several steps. They're searching semantically or through full text, et cetera, as needed, and they're only fetching what's needed for that specific, uh, specific use case.
- 9:28
But the important thing to know is that, you know, retrieval is no longer just this like simple one-time call to a vector DB. It's becoming super iterative, and these agents are really understanding what they're searching and searching to understand more in a sense.
- 9:42
Um, and it's kind of this like interesting loop. Uh, you know, Google's Jeff Dean, he went on a, I, I forget if it was a show or a podcast or whatever, and he had this really good quote that, that we like to use, um, that w- we also we thought was super interesting.
- 9:55
Uh, he was talking a little bit, I believe, about how Gemini's models were kind of having these really big context windows. Uh, and I forget the exact question the, the host asked him, uh, but he was saying, you know, big context windows, it doesn't matter if you get to a trillion context window size, um, what you really
- 10:11
need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions [REDACTED:username] a time. Um, and like the exact quote is, "You don't need a trillion [REDACTED:username] once.
- 10:20
You need the right million." Um, this is something we think a lot about turb-- uh, here [REDACTED:username] [REDACTED:username]. Um, you know, we have customers that embed, you know, have trillions of tokens, um, inside [REDACTED:username], and as we see, like the really important part is just getting down to this right hundred thousand, right ten thousand, right million, uh,
- 10:40
in order to pass into these context windows.
- 10:43
Uh, that's about it for a talk. Um, if you have any questions about any specifics, I'd love to, you know, either ask, have them asked now, or you can find me after the talk.
- 10:52
Uh, but appreciate you guys coming out. [audience applauding] [upbeat electronic music]