# RAG is dead, right?? — Kuba Rogut, AI Engineer Europe 2026 [Talk page](https://ai.engineer/talks/UM6sFg_jdlE-rag-is-dead-right-kuba-rogut) · [Video](https://www.youtube.com/watch?v=UM6sFg_jdlE) ## RAG is dead, right?? Coding agents need more than a single vector lookup. Cursor’s indexing strategy shows how reusable semantic search fits inside an iterative search, read, and reason loop. Familiarity with LLM context windows, embeddings, and basic code search will help you follow the examples. ### What is supposed to be dead? [0:24](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=24s) If agentic file search makes RAG obsolete, what exactly has it replaced? Kuba Rogut approaches that question as a deployed engineer at [turbopuffer](https://turbopuffer.com/docs), a full-text and vector search database built on object storage. His starting point is the growing role of hybrid retrieval tools in agentic search. The opening contrast is familiar: social-media posts declare that RAG is dead and agentic file search is all anyone needs. Yet the Google search-interest chart Rogut presents rises in 2023, levels off through 2024, and accelerates around the middle of 2025. That chart establishes continuing interest, not retrieval quality or adoption. The more useful question is whether the debate has confused RAG with one particular implementation. ![A Google Trends line chart under the heading “RAG is dead”? shows a low early baseline, a plateau, and a steep rise at the right.](https://ai.engineer/api/talk-essay-media/97d360602be164ee6a4fd9bd809bc8c88eac5d73361ea4dae0e955b86a0b6be1) Search interest rises sharply on the “RAG is dead”? slide. ### Retrieval methods and the agentic loop [1:37](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=97s) The narrow version of RAG embeds a corpus, searches it with a query embedding, and passes the returned content to an LLM. But **retrieval-augmented generation** does not prescribe vector search. Retrieval can use semantic similarity, BM25 full-text search, `grep`, glob patterns, regular expressions, or ordinary filters. The generation step receives whatever relevant context those methods find. Agentic search has a similar naming problem. Claude Code and Codex make filesystem search a visible part of an agent’s work, so the term often becomes shorthand for grepping files. Rogut’s broader definition is an agent using tools to progressively find and reason over external context. | Concept | Narrow interpretation | Broader mechanism | |---|---|---| | RAG | One vector lookup | Retrieve useful context, then generate | | Agentic search | Filesystem `grep` | Choose tools and iteratively investigate | These definitions can coexist: an agent can retrieve semantic matches as one step in a larger investigation. ![Two panels contrast vector search with retrieval methods including BM25, grep, glob, regex and filters, and filesystem grep with tools that progressively find and reason over external context.](https://ai.engineer/api/talk-essay-media/19cc00dfe14daaeaa3f2ea30c0e6c82893bc4ea04d4f7b984dc576e047ff090e) RAG and agentic search: common assumptions versus broader definitions. The loop is straightforward: 1. Search for potentially relevant material. 2. Read what the search found. 3. Assess whether it answers the question or leaves a gap. 4. Search again if necessary, then continue the task when the context is sufficient. The distinguishing feature is the feedback between reading and searching. A result changes what the agent knows, which changes what it should look for next. ### Cursor reuses codebase indexing work [3:23](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=203s) Cursor, which Rogut identifies as one of turbopuffer’s earliest customers, provides a concrete example. Its [Securely indexing large codebases](https://cursor.com/blog/secure-codebase-indexing) article describes how opening a codebase or branch prepares that code for semantic search: parse the files, split them into chunks, and compute embeddings. Doing all that work independently for every developer would repeat much of the same computation. Imagine a team of a hundred engineers working across a few repositories. Their checkouts overlap substantially, even when their branches differ. Re-chunking, re-embedding, and uploading every shared file each time wastes that overlap. The reuse mechanism separates detecting changes from finding an existing index worth reusing. Rogut summarizes this through **Merkle trees**, hash trees that expose differences in file contents. Cursor’s published implementation distinguishes Merkle-tree change detection from a derived **simhash** used to select a similar existing index. Once an index is reused, changed content is updated and unchanged chunks can retain their cached embeddings. Security matters because a reusable team index may contain files a particular client does not possess. Rogut credits turbopuffer’s role in secure reuse; Cursor’s account explains the access boundary more specifically through content proofs that exclude results for files the client cannot prove it possesses. Reuse therefore avoids repeated work without making every indexed file available to every client. ### What semantic search contributed [4:46](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=286s) Why maintain this infrastructure? Cursor’s [Improving agent with semantic search](https://cursor.com/blog/semsearch) evaluates what happens when semantic search is available alongside traditional search tools. On its internal Cursor Context Benchmark, Cursor reports 12.5% higher answer accuracy on average across models when semantic search is available. The published figure resolves Rogut’s spoken uncertainty between 12.5% and 13.5%. This is a vendor-reported internal evaluation, not a public benchmark. Rogut highlights an almost 24% answer-accuracy improvement for the earlier **Composer** model, before **Composer 2**. That model-specific attribution comes from his presentation; the accessible article text reports a maximum of 23.5% without identifying the model. These are reported percentage improvements, not percentage-point changes. The online A/B test measures different outcomes. Both groups used the same model, and the published report gives the following results: | Outcome | Reported result | Condition | |---|---|---| | Code retention | 2.6% higher with semantic search | Repositories with at least 1,000 files | | Dissatisfied requests | 2.2% higher without semantic search | Online A/B test | The dissatisfaction result uses the group with semantic search as its baseline. It should not be restated as an identically sized decrease when the tool is enabled. The smaller online effects do not contradict the larger codebase-question gains. Giving an agent a semantic search tool does not mean every request uses it, or that every request benefits from it. An aggregate test includes requests for which the added tool makes little difference, diluting the effect across the whole population. ### Embeddings as cached compute [6:06](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=366s) Claude Code supplies a different design choice. In the historical account Rogut presents, Boris Cherny says early versions tried RAG with a local vector database and did not retain that approach. The displayed exchange describes agentic search as working better for Claude Code. It does not establish that semantic retrieval is unsuitable for every coding agent, nor does Rogut provide a detailed technical explanation for the decision. ![A slide titled “claude uses grep” shows Ethan Lipnik’s question about embeddings and Boris Cherny’s reply that early Claude Code used RAG with a local vector database but agentic search worked better.](https://ai.engineer/api/talk-essay-media/d1b17b2b6d22f0c3e6e3c081e3f402c1cdf55f643b07dc922cb353433b628c2c) Boris Cherny describes Claude Code’s move away from a local vector database. The comparison becomes more useful when embeddings are understood as **cached compute**. Some work that an agent would otherwise perform while discovering a codebase can be moved into reusable preprocessing. Rogut illustrates this with Claude Code-like and Cursor-like traces: repeated runtime discovery on one side, upfront indexing followed by retrieval on the other. These are illustrative traces, not a controlled benchmark of the two products. ### Finding how metadata filtering works [7:03](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=423s) Consider the question: how does metadata filtering work in this codebase? A filesystem-based agent starts by finding candidate files, reading them, and deciding whether it has enough information. In a TypeScript repository, a first search might look like this: ```bash grep -Rni --include='*.ts' -- 'metadata' src/ ``` The output locates occurrences of a word; the agent still has to read the surrounding implementation and determine which matches explain filtering. It may need another search before it can answer. That discovery repeats across sessions. Different developers can ask the same question on different days, and each agent may perform much the same search, read, and assessment cycle. Rogut describes roughly 6,000 tokens for the discovery sub-step in his illustration, not for an entire task or as a universal cost. The indexed path moves some of that work earlier. Parsing, chunking, and embedding prepare the codebase for a lightweight runtime query such as `How is metadata filtered?`. The query returns relevant chunks for the agent to inspect, reducing the amount of discovery it must do from scratch. | Work | Per-session discovery | Indexed retrieval | |---|---|---| | Preparation | Begin searching at runtime | Parse, chunk, and embed upfront | | Runtime | Search, read, assess, repeat | Query the index and inspect results | | Reuse | Discovery may repeat each session | Reuse preprocessing; update changed content | The initial index is reusable, not maintenance-free. Its potential payoff is less repeated discovery, with corresponding token, latency, and cost savings; the example supplies no universal speedup or break-even threshold. ![Two panels compare per-session grep and file reading totaling 6,314 tokens with indexing once and retrieving ranked chunks totaling 424 tokens.](https://ai.engineer/api/talk-essay-media/17603e09a7c3d184fa7e10108dfcb15f2e4c6d4d954bfc5cfbadd5b4e56e9173) “Embeddings are cached compute” contrasts repeated discovery with indexed retrieval. Rogut also reports that some turbopuffer teammates had started switching from Claude Code to Cursor because it felt faster, associating that experience with Composer 2 and semantic search. That is a team anecdote, separate from both the illustrative traces and Cursor’s reported evaluations. ### From one lookup to retrieval during reasoning [8:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=518s) The pattern being displaced is the simplest early RAG pipeline: perform one vector search, put its results in the context window, and generate an answer. Rogut associates that pattern with 2023 and early 2024. Among more sophisticated customers, he now sees agents making many retrieval calls across several reasoning steps, with performance gains and new product possibilities that he describes qualitatively. In this design, semantic search and full-text search are tools the agent chooses as needed. It fetches context for the current question, reasons over the results, and uses what it learns to guide subsequent searches. **Retrieval becomes part of reasoning rather than a fixed prelude to it.** The system can therefore narrow or redirect its investigation instead of assuming that its first lookup found everything required. ### Larger context still requires selection [9:42](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=582s) Large context windows do not remove the selection problem. Rogut closes by recalling Jeff Dean’s discussion of long-context models: even an enormous context capacity leaves a need for staged retrieval, using lightweight mechanisms to narrow a much larger collection to the relevant material. Rogut’s formulation of Dean’s point is that the goal is the right million tokens, not a trillion tokens at once. He reports customers with trillions of tokens represented in turbopuffer, while the useful subset for a particular task might be ten thousand, a hundred thousand, or a million tokens. Those are different quantities: the corpus available to search and the context worth supplying to the model. As context windows grow, the retrieval system still has to decide what belongs in them. ## Resources - [Improving agent with semantic search](https://cursor.com/blog/semsearch) — Cursor's evaluation of semantic search alongside traditional search tools, covering codebase questions and online user outcomes. - [Securely indexing large codebases](https://cursor.com/blog/secure-codebase-indexing) — How Cursor reuses team indexes, synchronizes changed files, and filters results using proofs of local file possession. - [Hybrid Search](https://turbopuffer.com/docs/hybrid) — A practical guide to combining vector and BM25 retrieval, merging rankings, and optionally reranking results. - [Owning the AI Pareto Frontier — Jeff Dean](https://www.latent.space/p/jeffdean) — Podcast recording and transcript discussing long context, staged document selection, and the relationship between search infrastructure and language models. ## Slides ### AI Engineer Europe [0:00](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=0s) [AI Engineer Europe logo in white on black.](https://assets.ai.engineer/sites/aiecode2025/sha256/1139a3ab9cd4b0c7a7efb7ea1dfcefad1d3267abf6e38bf57a10e00ec0d05614.webp) AI Engineer EUROPE ### Presenting sponsor [0:05](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=5s) [Google DeepMind presenting sponsor card.](https://assets.ai.engineer/sites/aiecode2025/sha256/c355b6bbae3b4d6b4965fd4a157f49d1fb3d85c96baf548458d985f3e27fedc9.webp) PRESENTING SPONSOR Google DeepMind ### Platinum sponsors Braintrust, WorkOS and OpenAI are the platinum sponsors. [0:10](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=10s) [Braintrust, WorkOS and OpenAI platinum sponsor logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/9c9583ff6399a11312051adba3b99b5a74e5226f9bd4bfb123325e4b839a837a.webp) PLATINUM SPONSORS Braintrust WorkOS OpenAI ### RAG is dead, right? Kuba Rogut introduces a talk on how hybrid, tool-rich retrieval is becoming the default for serious agentic search. [0:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=20s) [Talk title slide introducing hybrid, tool-rich retrieval for agentic search, presented by Kuba Rogut.](https://assets.ai.engineer/sites/aiecode2025/sha256/5764d6844a32ffe13451b3e9a241e3e8a28bdbdacc1631e6e49ccd0c1b7c4766.webp) turbopuffer Rag is dead, right?? How hybrid, tool-rich retrieval is becoming the default for serious agentic search Kuba Rogut Deployed Engineer April 9, 2026 AIE Google DeepMind ### Interest in retrieval-augmented generation U.S. Google search interest in retrieval-augmented generation rose in 2023, leveled off through 2024, then climbed sharply from mid-2025 into early 2026. [1:11](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=71s) [Google Trends chart shows U.S. search interest in retrieval-augmented generation rising sharply through early 2026.](https://assets.ai.engineer/sites/aiecode2025/sha256/28c22403dfed5634f8bba8177c04dd850f3f8ee9dd7c7576dbeaca970cd94132.webp) Interest over time United States · Mar 1, 2021 - Feb 27, 2026 Google Trends Y-axis ticks: 0, 20, 40, 60, 80, 100 Timeline: 2022, 2023, 2024, 2025, 2026 Note Interface text: Retrieval Augmented Generation Search term United States Mar 1, 2021 - Feb 27, 2026 Web Search AIE AI Engineer EUROPE ### RAG vs agentic search RAG can use vector search, BM25, grep and other methods to retrieve context for generation. Agentic search gives agents tools to find and reason over context iteratively, searching again when a result falls short. [2:17](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=137s) [RAG combines retrieval methods with generation; agentic search gives agents tools to progressively find and reason over external context.](https://assets.ai.engineer/sites/aiecode2025/sha256/94a142982d32174f70c2599d77af2d92bf18b18decb174d772bf647102f70200.webp) Let’s clarify: RAG vs Agentic Search turbopuffer RAG What people *think* it is: ↳ Vector search What it *actually* is: Retrieval Augmented Generation vector bm25 grep glob regex filter... LLM harness Agentic Search What people *think* it is: ↳ Filesystem grep What it *actually* is: giving agents a set of tools to progressively find and reason over external context AIE Engineering the future of AI ### Securely indexing large codebases Cursor uses Merkle trees to compare codebases, reusing an index when they are sufficiently similar and updating changed files. Clones of the same codebase average 92% similarity across users within an organization. [4:16](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=256s) [Cursor article describes reusing codebase indexes, with diagrams of Merkle-tree synchronization and secure index sharing.](https://assets.ai.engineer/sites/aiecode2025/sha256/3589e21c897cbd19f7e00c3eb5b2049b6310c0966ce78f6f40480522e12ac7ad.webp) cursor.com/blog/secure-codebase-indexing CURSOR Securely indexing large codebases Jan 27, 2026 by Jeremy Stribling in Research Semantic search is one of the biggest drivers of agent performance. In our recent evaluation, it improved response accuracy by 12.5% on average, produced code changes that were more likely to be retained in codebases, and raised overall request satisfaction. To power semantic search, Cursor builds a searchable index of your codebase when you open a project. For small projects, this happens almost instantly. But large repositories with tens of thousands of files can take hours to process if indexed naively, and semantic search isn't available until at least 80% of that work is finished. We looked for ways to speed up indexing based on the simple observation that most teams work from near-identical copies of the same codebase. In fact, clones of the same codebase average 92% similarity across users within an organization. This means that rather than rebuilding every index from scratch when someone joins or switches machines, we can securely reuse a teammate's existing index. This cuts time-to-first-query from hours to seconds on the largest repos. Merkle trees enable fast, incremental sync Client | Repository service /docs #[illegible] | /docs #[illegible] compare hashes of entries guide.md #[illegible] | sync | guide.md #[illegible] diagram.py #[illegible] | no updates needed | diagram.py #[illegible] No matching on client | delete | config.json #[illegible] Secure index sharing enables fast, safe onboarding Client build and compare Merkle trees Repository service if hashes differ Indexing worker → Vector DB Embedding model → DynamoDB cache localCodebase DB Secure index copies if hashes match Interface text: turbopuffer AIE Engineering the future of AI ### Claude uses grep Boris Cherny says early Claude Code used RAG and a local vector database, but the team found agentic search generally worked better, was simpler, and did not have the same security, privacy, staleness, and reliability issues. [6:17](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=377s) [A post says early Claude Code versions used a local vector database but found agentic search generally worked better.](https://assets.ai.engineer/sites/aiecode2025/sha256/52f378cfba47af65978fa92d4a7fc136ba17741c40654f06e5cca8c1d5485a84.webp) claude uses grep turbopuffer Ethan Lipnik @EthanLipnik · Jan 30 Does anyone know why Codex and Claude doesn't use cloud-based embeddings like Cursor to quickly search through the codebase? Boris Cherny @bcherny 👋 Early versions of Claude Code used RAG + a local vector db, but we found pretty quickly that agentic search generally works better. It is also simpler and doesn’t have the same issues around security, privacy, staleness, and reliability. 9:56 PM · Jan 31, 2026 · 1.1M Views Interface text: 68 replies · 8 reposts · 658 likes · 266K views AIE Engineering the future of AI ### Embeddings are cached compute In this example, repeated code discovery costs 6,314 tokens, while retrieving indexed chunks costs 424. Indexing requires upfront work, which can be reused across sessions. [6:28](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=388s) [An example compares repeated code discovery at 6,314 tokens with indexed retrieval at 424 tokens.](https://assets.ai.engineer/sites/aiecode2025/sha256/c27f82a01b3664525175abbc8d769bb49c3b6dcde8f956159aa9cf06190ac6cf.webp) embeddings are cached compute per-session discovery Agent greps, reads, assesses, repeats grep “metadata filter” src/ +820 tokens read indexing.py (wrong section) +2,106 tokens grep “ingest pipeline” src/ +640 tokens read api_client.ts:280-400 +1,450 tokens read types.ts (for context) +1,298 tokens Repeated every session by every agent, across every task → 6,314 tokens ⚠️ amortized understanding index once, retrieve at runtime Index time: - Codebase chunked, embedded, indexed - Semantic meaning encoded upfront - Model has already “read” every file - One-time cost, amortized across all sessions Runtime: Agent query: “How is metadata filtered?” Results(ranked chunks): - indexing.py +142 tokens - api_client.ts +188 tokens - types.ts +94 tokens → 424 tokens ✅ Interface text: AIE Engineering the future of AI [8:04](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=484s) [A comparison shows 6,314 tokens for repeated code discovery versus 424 tokens for retrieved chunks.](https://assets.ai.engineer/sites/aiecode2025/sha256/dd6bd7145005f34b7335c30c8c08041046b55967d4e67e3f8ac90ffac14ca1ad.webp) per-session discovery Agent greps, reads, assesses, repeats grep “metadata filter” src/ +820 tokens read indexing.py (wrong section) +2,106 tokens grep “ingest pipeline” src/ +640 tokens read api_client.ts:280-400 +1,450 tokens read types.ts (for context) +1,298 tokens Repeated every session by every agent, across every task → 6,314 tokens ⚠️ amortized understanding index once, retrieve at runtime Index time: - Codebase chunked, embedded, indexed - Semantic meaning encoded upfront - Model has already “read” every file - One-time cost, amortized across all sessions Runtime: Agent query: “How is metadata filtered?” Results(ranked chunks): - indexing.py +142 tokens - api_client.ts +188 tokens - types.ts +94 tokens → 424 tokens ✅ Interface text: AIE AI Engineer EUROPE ### From RAG to agentic retrieval Agents reason and search in steps, fetching useful context as needed. A post cites Jeff Dean on staged retrieval: narrowing a trillion tokens to 10 million, then to the million needed. [9:14](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=554s) [A comparison contrasts one-time retrieval with reasoning and searching iteratively; an adjacent post discusses staged retrieval.](https://assets.ai.engineer/sites/aiecode2025/sha256/5a43836f437b766d1a13e3040f7c6a72da9edd8fbc588ad14c22a51d523e32df.webp) from RAG to agentic retrieval turbopuffer pattern From RAG → Agentic Retrieval old - Retrieve once - Stuff the prompt - Cross fingers new - Reason in steps - Search as needed - Fetch what is useful Retrieval is now iterative with tools Haider. @slow_developer Google Jeff Dean says bigger context windows alone are not enough What matters is staged retrieval: lightweight mechanisms that narrow a trillion tokens down to 10 million, then to the million you actually need “you don't need a trillion at once, you need the right million” Interface text: Subscribe @slow_developer AIE Engineering the future of AI ### Thank you [10:42](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=642s) [A thank-you slide with pixel-art mascots.](https://assets.ai.engineer/sites/aiecode2025/sha256/fe89f5fb4000cc22b2c35fb9d1be97e10b60f74c2706669cbc422147b3d08f88.webp) thank you @turbopuffer @rogutkuba AIE AI Engineer EUROPE ## Full transcript [0:00](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=0s) [upbeat music] Hi, welcome everyone. [0:15](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=15s) Uh, thanks for coming out. I see it's a full room, so appreciate everyone coming out. Uh, so welcome to the talk about RAG is Dead, right? Um, so my name is Kuba. [0:24](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=24s) I'm a deployed engineer [REDACTED:username] [REDACTED:username]. So for those that don't know what [REDACTED:username] is, we are a full-text search and vector search database, uh, built from first principles on top of object storage. [0:34](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=34s) Um, if you would love to learn more, uh, just come find me after the talk if you have any questions. So let's get started. So, um, this talk I get-- is a, [clears throat] sorry, about how RAG is dead, how hybrid tool, tool-rich retrieval is becoming a default for serious agentic search. [0:49](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=49s) So, if you guys have been on Twitter or other social media platforms, or I guess X as they call it now, um, you might have seen a lot of tweets like this about how RAG is dead. [0:57](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=57s) You can see there's lots of tweets, especially in the last, you know, end of twenty twenty-five and in the early of this year about how, you know, RAG is dead, agentic file search is all we need, and there's kind of a, a lot of tweet and a lot of kind of content about this now. [1:11](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=71s) But, you know, interestingly, if we're to look [REDACTED:username] something like the Google, uh, Google search volume over the last two years or the last couple of years, you can see that, you know, in twenty twenty-three, kind of as AI starts, we have this kind of this, this, uh, you know, increase, kinda caps out a little bit in [1:25](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=85s) twenty twenty-four, settles down for about a year, and then about midway through twenty twenty-five, we hit this new inflection point where search volume just goes through the roof. Um, so take that Twitter. [1:37](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=97s) Um, so let's clarify first. What is RAG and what is agentic search? These are kind of the two terms a lot of people are throwing out these days. Um, so RAG. [1:45](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=105s) What a lot of people think RAG is, is just simple vector search. They just think that this is, um, just simply, you know, embedding a bunch of, you know, a corpus of content, passing an embedding vector and getting it back, passing it through your LLM. [1:57](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=117s) And what [REDACTED:username] [REDACTED:username], what we think this actually means, you know, if we break down RAG into retrieval-augmented generation, you know, retrieval is not just vector search. It's a lot of different things. [2:07](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=127s) It could be vector search, uh, full-text search using stuff like BM25, grepping, globbing, using regex, using other just basic filters. Um, and then the augmented generation is obviously just passing it into your LLM cho- of choice. [2:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=140s) And then agentic search. This is kind of the terms people are throwing around a lot these days. And generally, when people start talking about agentic search, what they usually talk about is essentially just file system grep. [2:31](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=151s) So if you guys are familiar with something like Claude Code, um, and kind of that's what-- or Claude Code, Codex, um, you know, a lot of people call this agentic search, and this just essentially is grepping through your file system, and this is kind of why these terms are so correlated. [2:45](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=165s) Um, and what we actually believe it is and, you know, kind of the definition we wanna bel- we wanna give it is, it's really giving the agents a set of tools to kind of progressively and iteratively find and reason over context. [2:55](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=175s) So with Claude Code, you can, you know, if you guys have-- are familiar with it, it can read your file, you can start grepping through your file system and read a file, decide that it hasn't found what it needed, what it, what it needed to actually complete the task, and it will, you know, find something again, and [3:10](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=190s) it keep doing this until it's happy, you know, it's reached a happy state where it can continue on with the task. So we're gonna take a step back, um, and talk about one of the companies that use [REDACTED:username], uh, that we believe is doing an excellent job with agentic search. [3:23](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=203s) This is a company called Cursor. You might have heard of them. Um, fun fact, they're actually one of, uh, [REDACTED:username]'s very first customers. Um, and they have this excellent blog post that came out in the beginning of twenty twenty-six about how they index codebases. [3:35](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=215s) So for those unaware, when you open up a new codebase or a new branch in Cursor, what happens is that Cursor will start embedding your codebase. So what they'll do is, you know, chunk out your parse, chunk and embed your codebase, and make it available for semantic search. [3:47](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=227s) And this blog post goes into an excellent, um, kind of excellent technical detail of how they do this. Um, just to give you the gist, essentially the cool thing they do, um, is that they found that, you know, most people working on a team, let's say there's a hundred engineers, uh, when they open up codebases, they're normally [4:03](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=243s) the same codebase, you know, ninety-nine percent of the time because you can have a team of a hundred people most of the time working on one, two, maybe a few codebases, right? [4:11](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=251s) Um, and it's really expensive to have to, like, re-chunk, re-embed, and re-upload these codebases every single time. Uh, so they essentially use like Merkle trees, which essentially is this crypto hash tree, um, to calculate similarities between codebases people open on the same team. [4:26](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=266s) And if they're similar enough, uh, they will essentially copy over the data and then only update the, uh, and re-chunk and re-embed the files that have changed and use [REDACTED:username] in order to make sure this is done securely. [4:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=278s) And, um, yeah, it's j-just excellent blog post. They, they do some really cool stuff. Uh, and you may think like, "This is a lot of work. Uh, why do they do this?" [4:46](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=286s) Well, the reason they do this is also covered in a, in a different blog post, uh, about how they use semantic search. Again, they use [REDACTED:username], [REDACTED:username] for this. [4:54](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=294s) Uh, and what they find is on average across models, I think it's like a twelve and a half or thirteen and a half percent increase in answer accuracy. Um, this is across, across their internal Cursor Context Benchmark. [5:05](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=305s) Um, so, you know, not, not a public benchmark, but, uh, you can trust the numbers they give us. And you can see on, on the right side, uh, their Composer model, so this is, this is before Composer 2, uh, they had a almost a twenty-four percent increase in answer accuracy. [5:19](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=319s) So giving semantic search to these tools and, uh, to these models is really can drive real performance gains. And you can see on the, on the bottom right, um, this is from an online A/B test they did, which is also covered in their thing, uh, in, in their blog post, um, about how it's almost like a two [5:36](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=336s) point six percent retenti- uh, code retention in large codebases, and there's a two point two percent decrease in dissatisfied user request. And you may be thinking like, "Oh, well, these numbers aren't that big, like two point six percent, two point two percent, not that large." [5:49](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=349s) But they also cover that, um, semantic search isn't used in every single query. So in their online A/B test, you know, if you give it-- if you give these tools to a hundred quer- a hundred random queries, not every hundred query will actually benefit from the existence of a semantic search tool, so that's why these numbers look [6:06](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=366s) kinda small Um, and now let's talk a little bit about Claude Code. Uh, so Claude Code doesn't use vector search, as covered, uh, by this tweet from Boris Cherny. [6:15](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=375s) Uh, so those unfamiliar with Boris, he's essentially the founding father of Claude Code. Um, and he says that in early iterations of Claude Code, they actually did use RAG in a local vector DB, but they found that it just didn't really work out for them. [6:30](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=390s) But this is something that, uh, is important to understand, and something we like-- we've kind of like taken on a lot internally, uh, understanding, um, here [REDACTED:username] [REDACTED:username], is this idea that like embeddings and semantic search are kind of cached compute. [6:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=404s) Um, and you may be thinking like, "Cached compute?" Like, kind of throwing out a lot of terms [REDACTED:username] me right now, like I don't know what exactly what that means. [6:50](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=410s) Um, and I think it's like best to walk through an example, um, of essentially almost like a Claude Code-looking trace and a Cursor-looking trace, um, of how some, how these agents will understand your code base. [7:03](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=423s) So on the left is kind of a per session discovery of Claude Code. So for example, if we were to ask the agent to understand how metadata filtering works, what it would have to do is grep, read, assess, and repeat, and try to find the files it needs in order to gain this understanding on a per session [7:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=440s) basis. So what this means is you could have, you know, ten agents on ten different days, um, across ten developers, and you, they can be asking the same question multiple times, you know, in day, uh, every day. [7:32](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=452s) And every time, the agent's gonna have to kinda repeat these same exact steps, uh, to gain kind of the like same understanding of, of this code base. Um, and this could, you know, cost quite a few tokens. [7:42](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=462s) Uh, you know, six thousand doesn't seem like a lot here, but just remember this is like one sub-step of an agent. And then on the right is kind of like a more Cursor-looking trace where there's this upfront cost of indexing, but then we're able to allow for this like lightweight tool to help the agent kind of retrieve [7:57](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=477s) this information [REDACTED:username] runtime. Um, so obviously there's this like upfront cost of, uh, parsing a code base, embedding it, uh, and making it available, but this is like a one-time cost. [8:06](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=486s) Uh, and then [REDACTED:username] runtime, the agent can just query something like, "How is metadata filtered?" Uh, it can get some simple results, and it would save a lot of tokens, a lot of time, and just a lot of money. [8:16](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=496s) Um, and this just helps the agent to become a lot faster. Um, you know, a lot of people on the team now that maybe were big Claude Code users here [REDACTED:username] [REDACTED:username], um, they've, you know, they've actually started switching to Cursor just because of how fast it's becoming, uh, especially with their Composer 2 models and also the [8:32](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=512s) semantic understanding. Uh, it's just become we're, we're finding really, really good. [8:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=518s) So from RAG to agentic retrieval. Um, so what we're finding now is that a lot of people are no longer doing the simple RAG, you know, the, the Twitter quote, unquote, "RAG" of just, um, doing a vector search once and throwing it into the context windows. [8:55](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=535s) Uh, what we're finding, um, is that this worked, you know, back in twenty twenty-three or early twenty twenty-four [REDACTED:username] kind of the beginnings of AI, but a lot of the more sophisticated customers are doing agentic search and it's giving like real, real big performance gains, uh, and kind of un-unlocking like new products. [9:12](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=552s) Um, and it, what we're finding is, you know, they're doing a ton of calls. They're reasoning, these agents are reasoning through several steps. They're searching semantically or through full text, et cetera, as needed, and they're only fetching what's needed for that specific, uh, specific use case. [9:28](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=568s) But the important thing to know is that, you know, retrieval is no longer just this like simple one-time call to a vector DB. It's becoming super iterative, and these agents are really understanding what they're searching and searching to understand more in a sense. [9:42](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=582s) Um, and it's kind of this like interesting loop. Uh, you know, Google's Jeff Dean, he went on a, I, I forget if it was a show or a podcast or whatever, and he had this really good quote that, that we like to use, um, that w- we also we thought was super interesting. [9:55](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=595s) Uh, he was talking a little bit, I believe, about how Gemini's models were kind of having these really big context windows. Uh, and I forget the exact question the, the host asked him, uh, but he was saying, you know, big context windows, it doesn't matter if you get to a trillion context window size, um, what you really [10:11](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=611s) need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions [REDACTED:username] a time. Um, and like the exact quote is, "You don't need a trillion [REDACTED:username] once. [10:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=620s) You need the right million." Um, this is something we think a lot about turb-- uh, here [REDACTED:username] [REDACTED:username]. Um, you know, we have customers that embed, you know, have trillions of tokens, um, inside [REDACTED:username], and as we see, like the really important part is just getting down to this right hundred thousand, right ten thousand, right million, uh, [10:40](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=640s) in order to pass into these context windows. [10:43](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=643s) Uh, that's about it for a talk. Um, if you have any questions about any specifics, I'd love to, you know, either ask, have them asked now, or you can find me after the talk. [10:52](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=652s) Uh, but appreciate you guys coming out. [audience applauding] [upbeat electronic music]