Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread

Why did Intel put so much LPDDR5 memory in a new GPU when it could have used HBM? An equity analyst has a good place to start. The fund’s data providers have sent research reports, interviews, and podcasts that might explain the design decision. The difficulty for an agent is figuring out which source addresses the choice. Plenty of people mention Intel, GPUs, and memory without saying why Intel chose one kind of memory for this card.

Give a coding agent local access to those files and it can use a terminal tool such as grep to look for the terms. A long list of hits gives it little guidance about what to read first. Meanwhile, a scanned PDF or a figure embedded in a report may hold the explanation and yield no searchable text at all. The agent can spend tokens reading through every hit and still miss the evidence. Shakir estimates that on a sufficiently large collection, even a basic question answered this way could cost $10 to $20.

A conventional retrieval-augmented generation pipeline tries to make that collection searchable before the analyst asks anything. It transcribes audio, runs optical character recognition (OCR) on PDFs, divides the text into chunks, embeds each chunk as a vector, and stores the vectors in a database. At query time it retrieves candidates and reranks them for the agent.

Consider what has to survive that trip for a financial table to be useful. OCR might recover the digits but lose the relationship between a value, its row label, and the quarter above its column. Even if the parser gets those relationships right, a chunk boundary can separate a figure from its explanation. The embedding model then has to represent what remains as a single point. A reranker can reorder retrieved chunks, but it has no way to recover a column heading that disappeared during parsing.

Each step also brings a procurement and evaluation decision. A team has to choose the OCR engine, chunking strategy, embedding model, vector database, and reranker, then get vendors through security review. Shakir estimates that assembling what Mixedbread calls the “Frankenstein stack” can occupy four or five people for six to eight months.

Single-vector search is easy to picture with Shakir’s bread example. Embed a sentence about loving bread, embed a question about who loves bread, and search the neighborhood around the question’s vector. Now try that with Nvidia earnings reports from 1996, 2000, and 2026. All three discuss the same company and metrics. A search can land in exactly the right neighborhood for Nvidia earnings while returning net income for the wrong year.

Shakir says customers have seen respectable quality on tens or hundreds of documents, followed by a substantial decline as similar material accumulates in production. An agent can compensate by reformulating the query, launching tens of subagents, and pulling more reports into its context window. Sometimes it gets the answer. The fund then pays for that investigation every time an analyst asks, and more often still if agents start issuing questions of their own.

That experience led Shakir to treat access to evidence as a separate engineering problem. A fund should be able to choose its reasoning model and give it a search interface that can already distinguish one filing, period, or metric from another.

Shakir traces text retrieval from BM25 keyword ranking in the 1990s to the embedding models he regarded as a significant advance in 2022. In his account, the subsequent gains were gradual. Mixedbread, which began as a retrieval research lab, took a different approach with a late-interaction model it was already running for customers at the time of the talk.

The model produces multiple embeddings for a document and multiple embeddings for a query. Search can match finer details represented on each side. In the Nvidia example, net income and revenue can have distinct representations. Because the model processes the page in one pass, those representations also carry the surrounding context that identifies the company. Shakir attributes the improvement to this design together with a larger model and better training data.

Mixedbread also indexes a page as it looks, allowing the model to use visual relationships directly. The company built its own search models, database, and ingestion pipelines for material including PDFs, spreadsheets, images, video, and audio across more than 100 languages. For financial customers, Shakir also describes a bring-your-own-bucket option and support for zero data retention.

In an Intel earnings deck, a revenue figure illustrates the value of indexing a screenshot. Reading a chart for the second quarter of 2025 requires connecting the number with the right period and series. The visual model can use those relationships to find the page. Once it does, the application can pass the screenshot to a multimodal model or use Mixedbread’s OCR text and layout information with a text model.

Shakir asks for Intel Foundry’s segment operating margin in Q2 2025. The search retrieves an Intel Foundry page containing the quarter and segment financials. Layout coordinates let the application draw a bounding box around supporting material, so an analyst can trace a cited claim to a particular part of the page. The demonstration establishes where to look without reporting a numeric margin. Shakir says search normally takes under 200 milliseconds, although the live connection was slower.

Asking whether Intel Foundry’s profitability is improving, and what is driving it, changes the job. One quarter’s margin cannot establish a trend or explain its cause. Mixedbread routes the question to Toast One, its trained agentic search model, which breaks it into multiple queries and gathers evidence for a language model to interpret. Applications can expose grep, ordinary search, and agentic search separately and let their agent choose among them. Shakir describes Toast One as matching the comparison models at 40 times the speed and lower cost, without giving the benchmark details.

The customer behind the headline was a large hedge fund with over a billion documents and an internal agent platform for analysts and fund managers. Its existing system used OCR, chunking, embeddings, and a vector database. Human experts had prepared questions with known answers. Against those answers, the agent was correct 36% of the time. Shakir says the fund onboarded with Mixedbread in about a week and a half and measured 98% answer accuracy after the switch.

The billion documents describe the fund’s holdings. Shakir does not give the number of evaluation questions. He reports a particularly large improvement on questions involving multimodal material and says language-model spending fell to about one-third of its previous level. His explanation links the two results. When retrieval supplies the relevant evidence early, the agent has fewer documents to parse and read before it can answer.