Where RL Will Take Search — Maximilian-David Rumpf, SID.ai

Finding the right documents can consume much of an agent’s effort before it starts the work the user requested. Maximilian-David Rumpf, founder and CEO of SID.ai, opens with the comparison driving his research: agents are about twice as likely to find the right documents as classical search, but cost roughly 100–1,000 times more per search and take minutes rather than milliseconds. Better context comes with a substantial bill.

Rumpf estimates that searching uses 30–50 percent of an agent’s tokens, usually at the beginning of a task. The agent needs to gather context before it can act on it. His proposed division of work follows that dependency: the main agent delegates searching to a sub-agent trained specifically for retrieval, then uses the returned material to perform the task. Reinforcement learning is the proposed way to make that specialist faster and cheaper.

The target spans legal, finance, knowledge bases, science and email, using a mixture of academic and internal benchmarks. Rumpf presents vector-only retrieval and reranking as lower-performing baselines, while frontier models improve retrieval by spending minutes on each question. The goal is to retain high recall—the ability to find the relevant documents—while reducing the time and cost of the search phase. These comparisons concern the workloads he presents; they do not establish the same improvement for every corpus or question.

The classical pipeline Rumpf describes follows a fixed route: a question enters, an optional LLM rewrites it, a search backend retrieves candidates, and an optional reranker orders them before the system returns results. These are chained, locally optimized models. Improving a component can make its particular operation better, but the sequence still determines which operations happen and how much computation each question receives.

The reranker exposes the problem most clearly. It may recognize that the retrieved documents do not adequately answer the question. Yet it can only reorder the candidates it already has; it cannot obtain better ones by launching another search. The system has detected insufficient evidence without giving that detection an action that could repair the failure. It returns the results anyway.

Unexpected questions then produce a long tail of failures. Developers add special cases and tweaks for questions the original design did not anticipate, but no finite collection of exceptions covers every future request. Rumpf’s objection is to making retrieval decisions once, during system design, and applying them uniformly even when the results reveal that a different approach is needed.

Rumpf motivates learned search strategies through two earlier changes in machine learning:

• Computer vision: his progression moves from handwritten edge detection to narrow object detectors—the generation of models that could put a box around a dog—and then to more general vision-language models.
• Chess: his analogy moves from human-authored rules, through systems combining approaches, toward learned strategies in AlphaZero and MuZero.

He expects search to follow a similar progression: established algorithms such as BM25 and PageRank, then specialized vector models and rerankers, then a model trained to make retrieval decisions throughout the search.

Results become input to the next decision

The proposed mechanism is a loop between one model and a database. The model searches, reads what came back, and decides what to do next. It can search again, set metadata filters during the process or constrain the search before eventually returning a ranked list of documents. The results of an unsuccessful query can therefore change the next retrieval action. That is the capability missing from the fixed pipeline.

The loop also lets effort vary with difficulty. A difficult question can receive more computation and more iterations than an easy one. Rumpf wants the model to discover useful strategies rather than follow a prescribed sequence. His description of removing baked-in decisions applies to the model’s search behavior: the system still supplies actions such as searching and filtering, and training still defines rewards. The learned policy decides how to use those opportunities and when to return results.

Reinforcement learning needs feedback that distinguishes successful attempts from unsuccessful ones, plus enough opportunities to attempt the task. Rumpf identifies two useful properties of search:

• Verifiable outcomes: for a training question, the system can check whether the model found the correct document. The retrieval outcome supplies a concrete reward target.
• Repeatable attempts: the model can try the task many times in a search environment. Rumpf describes thousands of attempts per second during a training run.

This makes it possible to train search behavior through repeated outcomes rather than write a rule for every retrieval situation.

Specialization supplies another route to efficiency. Rumpf argues that strong search does not require all the capabilities of a general-purpose language model. He compares CPUs, GPUs and ASICs: a general device may be able to do the work, while a specialized one performs it much more effectively. The analogy motivates narrowing the model’s job. It does not specify which language-model components can be removed or what architecture produces the savings.

Reward successful retrieval and efficient searching

In an earlier training task, Rumpf reports that search quality increases predictably with compute, alongside vector-only and reranker-only baselines. Training can also include latency rewards, so the model has a reason to find documents efficiently as well as correctly. The desired behavior is discovered through training rather than imposed as a fixed sequence of queries.

Rumpf says the team has not yet seen a ceiling to this approach, but he explicitly leaves open whether one exists. Continued improvement in the observed training regime supports further scaling experiments; it does not establish unlimited improvement across all domains.

Rumpf returns to the initial comparison with SID-1, including a configuration with parallel execution. He reports approximately 20 times faster searching: around five seconds on average instead of around two minutes, at roughly one hundredth of the cost of a frontier model on the task. Both the times and the speedup are rounded comparisons. Benchmark composition, retrieval quality and execution configuration matter when deciding whether those gains will transfer to another workload.

The specialist still takes longer than a vector-and-reranker pipeline. Rumpf expects that gap to narrow, but classical retrieval latency remains a future target. The decision therefore involves three measures together: retrieval quality, latency and cost. A large improvement over frontier-model searching does not erase the simpler pipeline’s speed advantage.

Keep exploration in the search agent’s context

The production example adds a second reason to delegate. When the main agent searches directly, it encounters both useful documents and bad results. All of that exploratory material enters the context it carries into the rest of the task. A search sub-agent can do the reading, thinking and iteration separately, then return selected results. The main agent gets the useful output without carrying the full search history.

Rumpf expects cleaner context to make the main agent more likely to be correct. That benefit depends on the search specialist finding and selecting the right documents; a separate context window cannot guarantee that its output is good. The mechanism reduces exposure to unsuccessful exploration, while retrieval quality determines whether the material that replaces it actually helps.

Delegation also moves the search work associated with that initial 30–50 percent token share onto the cheaper model. This is a saving within the task: making the search phase about 100 times cheaper does not make the complete task 100 times cheaper, because the main agent still performs the work after retrieval. The economic benefit grows with the amount of search the task would otherwise require.

Rumpf closes by forecasting arbitrarily good search across domains as reinforcement learning scales. He also expects faster and cheaper models to make iterative search practical in voice and e-commerce. These are proposed destinations beyond the reported results. His earlier acknowledgment that the method’s ceiling is unknown remains an important qualification on the prediction of unlimited improvement.

The larger opportunity, in his view, is knowledge work over information that public web search cannot supply. He argues that the web contains only a small part of the data available and uses J.P. Morgan as his example: the knowledge needed to run the organization sits deep inside its databases. His claim concerns the operational knowledge held within an organization, rather than information a public search engine can readily retrieve.

That ending gives the search specialist a concrete purpose. The main agent needs the right context before it can do useful work, and much of the relevant context may be inside an organization. A model that can inspect results, refine its search and return selected documents could make more of that knowledge usable without forcing the main agent to spend its own expensive context on every attempt.