The Search Engine for the Agentic Web — Will Bryk, Exa

A coding agent looking up documentation and a sales agent assembling a company list both need search, but they need different results. Will Bryk opens with a forecast about how common those calls will become: he expects AI-issued searches to exceed human searches in 2026 and reach a thousand times human volume over the next few years. That is his projection, rather than a measured traffic total established here. It frames Exa’s product decision: build retrieval for software that searches while doing other work.

Bryk reports that Exa serves over five thousand companies and over four hundred thousand developers. The examples span several kinds of agent:

• Coding: Cursor agents use Exa to retrieve technical documentation or news.
• Go-to-market: HubSpot uses it to help users find companies to sell to.
• Finance: Financial agents need financial data to support their work.

In each case, retrieval feeds an application’s next step. The useful search result depends on what that step requires.

Bryk’s motivation starts with the distance between available information and usable understanding. He describes a web on the order of a trillion pages: company websites, blog posts, images, tweets and much else. People cannot absorb that collection themselves. They need tools that filter or synthesize it. Google handles familiar requests, such as finding the Costco homepage, well; his objection concerns requests with conditions that matching vocabulary alone cannot satisfy.

An exclusion changes the meaning

His simplest example is shirts without stripes, which returns shirts with stripes. A striped shirt shares the query’s subject and vocabulary while violating its exclusion. The word without determines whether the result qualifies. Bryk uses this failure to describe mainstream search as a recommendation engine: it offers related material without necessarily returning precisely what the request permits. This example illustrates his criticism; it does not establish that every query behaves that way.

The next request asks for everyone in Singapore who works on AI search, together with any blog posts or research papers they have written. Answering it requires connecting people, location, field of work and authorship. Documents containing those words may help, but the requested result is a set of matching people linked to their work. Bryk’s hypothetical 412 matches illustrate what a complete answer would look like; they are not a reported census.

Missing results are hard to notice

A job seeker can find plausible biotech employers and still have no idea which suitable companies were omitted. Someone researching a region faces both that discovery problem and a trust problem: finding information does not make it reliable. Bryk extends the concern to finding important US news across media, where receiving a few recommendations leaves the reader uncertain about coverage.

These gaps explain his unusually large stakes for search. He imagines a dystopian San Francisco in 2035 to express the danger of powerful technology combined with people who cannot understand what is happening. He believes poor information makes manipulation, conflict and bad decisions more likely, and calls solving information the most important neglected problem. The imagined city is a warning scenario, not a demonstrated consequence of retrieval failures.

In 2021, transformers gave the team a reason to attempt a new search engine. Bryk remembers GPT-3 as a model that could understand a paragraph in a way their search experience did not. His founding thought experiment was straightforward: give a language model one complex query and one document, then ask whether they match. If that judgment works well, repeating it across the collection suggests a path toward near-perfect retrieval.

The difficulty is paying for that judgment at web scale. Bryk estimates roughly ten million dollars per search to evaluate a trillion documents. His engineering question becomes how to reduce the cost by a billionfold or a trillionfold while preserving useful understanding. Exhaustive evaluation is the motivating ideal, with its quality still dependent on the model’s matching judgments; it is not the deployed procedure he describes.

Move document processing ahead of the query

Embeddings offer a way to reuse work. The system preprocesses each document into a representation that captures some of a neural network’s understanding. Later searches use those representations, rather than running the full network over every document again for every query. The timing change matters: expensive document processing can serve many searches instead of being repeated across the whole collection each time.

Bryk retains two complementary ingredients:

• Keywords: Efficient matching that can handle simple queries.
• Embeddings: Precomputed representations that bring learned understanding into retrieval for more complex requests.

Embeddings have problems of their own, so he says combining them with keywords is often useful. He reports that Exa can handle the shirts-without-stripes request, but does not describe how the system enforces that exclusion. Precomputed representations explain an important cost reduction, not every detail of Exa’s search behavior.

The team committed real money to this direction. After raising a couple million dollars, it spent half on a GPU cluster and pursued years of research. Bryk describes the decision through the Bitter Lesson meme and the idea of stacking more layers: the team was willing to invest heavily in neural computation despite being new to search. His later account makes clear that the resulting product grew beyond embedding search into a combination of systems.

The company launched its first search engine in 2022 under the name Metaphor, roughly a year and a half after starting. Bryk recalls ChatGPT arriving two weeks later. The team had been trying to build a better consumer search engine than Google, with monetization left for later. A message on Twitter asking for API access introduced a product they had not planned to sell.

At first, there was no API to offer. More requests followed, including one from Bryk’s downstairs roommate. They revealed the business opportunity: language models need a programmatic way to retrieve information. Bryk’s argument is that even a gigantic model remains small compared with the internet. Increasing model size does not remove the need to connect it to information outside its weights.

He contrasts a human request such as SpaceX news with an agent’s much larger appetite for information. A search engine designed around short queries and human browsing need not be optimal for software making repeated retrieval calls. An API lets the application request information as part of its own work, without a person initiating and inspecting each search.

The pivot also fit the founding goal. Bryk says agents want something like a database of the world’s information, with no need for advertisements or pages whose appeal comes from search optimization. Selling retrieval to those systems gave Exa a business model for the search engine it already wanted to build. Over the following years, the system became more complex than embedding search alone; this account does not specify all of its components.

Research can take minutes; conversation cannot

Bryk’s present-day research example asks for every YC-funded startup working in AI, including its batch and status. He says Exa can assemble complex lists of companies, people, blog posts or news articles, though the work may take minutes. Such a request needs both matching entities and information about each one. The example describes the capability he offers, without measuring whether every qualifying startup is found.

At the other end of the latency spectrum, he describes a 200-millisecond search endpoint. A voice agent explains why this matters: search must finish before the language model processes the retrieved information and the system produces reply audio. Every delay in retrieval adds to the user’s wait. The endpoint figure concerns search, not the entire voice response, and Bryk’s claim that it is the fastest search API comes without a comparative benchmark.

Send the model what it needs

Retrieval also determines how much text the downstream model must consume. Bryk describes selecting roughly a hundred important tokens from ten documents instead of passing along all their content. This can reduce downstream LLM costs by shrinking the input before model processing. He does not specify whether the token quantity applies to the set or each document, or quantify the savings. Selection is useful only insofar as it retains the evidence the task needs.

Structured output serves a different next step. A recruiting agent might look for engineers who recently left a major lab and request each person’s most-cited paper, college and graduation year. Returning those fields gives the application a result it can consume directly, rather than excerpts it must interpret first. Bryk explains the requested shape, but leaves handling of unavailable or conflicting fields unspecified.

Let the customer define useful search

These differences explain Bryk’s description of five thousand search engines for five thousand customers. He is emphasizing customization, with controls over several independent choices:

• Speed and depth: Prefer a fast response, or accept minutes for the highest possible quality.
• Source scope: Search only a chosen set of a thousand domains, or exclude a set of a thousand domains.
• Time window: Restrict results to the period the task requires.
• Page type: Exclude product pages when they do not belong in the desired result.

The customer specifies what information is useful and which tradeoffs fit the application.

Bryk attributes Exa’s quality to years of research into search for agents and claims it performs better than Google for that use. No evaluation method or results establish the scope of that advantage here. The concrete design lesson is narrower and useful: query complexity, latency, source restrictions and output format all affect whether retrieval helps the application.

Exa Connect brings providers into retrieval

An agent’s question may require information that public-web search cannot supply. Exa Connect lets data providers partner with Exa so developers can obtain their data for agents. Providers decide what their content is worth; developers choose which data they want and pay for access. The marketplace gives providers a way to earn money from agent use and developers a way to reach additional information sources.

Bryk’s example combines public information about companies with monthly website visitor information from Similarweb that he describes as nonpublic. One research request can draw on both kinds of source. This explains the purpose of the marketplace, while leaving its authentication, billing granularity and treatment of conflicting information open.

A year of research in a second

The future target joins depth and speed: any information query should work, however complex, and feel as though someone had spent a year researching it before returning the result in a second. That is Bryk’s ambition. The current capabilities he describes still involve choices between research that may take minutes and a fast search endpoint. He also says the team now makes as much progress in one or two quarters as in its first five years, without defining a measurement for that comparison.

Bryk connects that ambition to people gaining a deeper understanding of events, including his hope that better information access will help people stay informed ahead of the 2028 presidential election. He encourages others to work on the problem as essential infrastructure. The intended social benefit extends beyond the retrieval capabilities demonstrated in his examples.

He then returns to the thousandfold search forecast and supplies its proposed driver. Humans initiate searches a few times a day, in his account; assistants and AI features throughout software could ground ordinary interactions in retrieval. A user would not have to explicitly ask for every search. Applications would issue them while working, multiplying query volume and making retrieval quality matter across many more decisions. That adoption pattern motivates the forecast; it does not establish its eventual magnitude.

The ending returns to coordination. Bryk sees increasingly capable technology alongside failures to make sensible decisions together, using San Francisco as his example. He believes the information people consume affects their ability to coordinate, and treats better search as his contribution to improving that information. Retrieval alone does not demonstrate a solution to coordination, but the connection explains the company’s ambition: help people understand enough to act together.