The Search Engine for the Agentic Web — Will Bryk, Exa

How's everybody doing? Good. Ooh.

Whoo.

All right. Pretty crazy times we live in. Ooh, my microphone's here. Um, okay, I am very excited to tell you all about perfect search built for AI agents. Uh, and I'm gonna say a lot of crazy things in this talk, so bear with me. Uh, they are all true. And I will tell you what is Exa, uh, the story of Exa, so how we got here, and then where we're going. Okay, cool. If you take away anything from this talk, it is this, uh, slide. Uh, this is showing web searches per day over the past

thirty years, uh, from humans and AIs. Obviously, it was all humans until around, uh, twenty twenties, and now we're in twenty twenty six, and actually this year, well, we expect the number of searches from AI systems to exceed that of humans. Pretty crazy. And then in the next few years, it should be a thousand times more. So AIs, AI systems, AI products, whatever AI system you use, some together will, will search a thousand times more than humans. That's pretty, uh, pretty crazy world that, that we're getting into, and the entire ecosystem of

search is changing because of it. So Exa is the search engine for AI agents. You know, we're the first ones to be like-- the-- we're the first search engine for AI, and now things are getting, uh, kind of wild. Uh, we now serve over five thousand companies, over four hundred thousand developers. I see some customers in the audience. Uh, we serve, you know, a very diverse set of agents from coding agents like Cursor. If you, if you use Cursor, at some point, the Cursor agent decides to search for the latest technical documentation or the news, it'll be using Exa under the hood. We serve go-to-market agents like

HubSpot, so, uh, we help, uh, their users get, you know, really high-quality lists of companies to sell to. We serve all sorts of financial agents. Uh, I was in New York a few weeks ago, and pretty much everyone there is now building financial agents, and they need the best financial data. Um, yeah, and really just a, a huge diversity of agents from, uh, from labs to, to, to YC startups. Okay. But how'd we get here? So what's the why of Exa? To me, that's always been the most important. And there are a lot of ways to say the problem, but, uh, the short way is just misinformed

anarchy. This is what we're trying to avoid. We wanna create the opposite of this. Uh, so what, what does this mean? Well, this is the Internet, or it's a visual depiction of the Internet. You know, you've got a bunch of pages, you got blog posts, you got company websites, you got images, you got tweets, you got all sorts of things. Uh, and the Web is really, really big, right? This is showing a few thousand pages. The Web is, you know, on the order of a trillion pages, so, uh, you know, a million times bigger than this. And it contains a huge amount of the world's information. Uh, and that means that, you

know, if this is just readily available, you could fi-- if you could go to any link and just get all the world's information, then I'm sure we all walk around with, like, deep understanding of everything, right? Obviously not. Uh, it's messy. Uh, and so, uh, and it's crazy, and it can't fit in our heads. So we need information tools that could help synthesize this or filter it into the, the things you need to know. And, you know, w-we have a tool called Google, and it, it's a pretty solid tool. It, it could get you things like the Costco homepage or information about Taylor Swift or whatever you wanna search. But it's not perfect, right? And I love this example where you

type shirts without stripes, uh, and if you notice, you get shirts with stripes. Why is it doing that? Well, it's not, it's not trying to be a database of the world's information that gives you exactly what you want. It's kind of like a recommendation engine. Um, and, and, and you can see this with more examples. Um, so find, find me everyone in Singapore who works on AI search and any blog post or research paper they've written. I bet you've never typed in anything like this to Google because you know it's not gonna work, right? You're gonna get, like, uh, links and documents that, uh, contain some of those words, but not, like, actually a database result of all, you know, four hundred and

twelve people who match. A-a-and it gets pretty serious, right? Like, I'm a citizen. I'm trying to be informed about what's going on in the world. I wanna find the most important US news across all media or, you know, US news articles, whatever it is. You, you just don't trust Google to give you that, right? It's just gonna get-- It's, it's just gonna recommend some things. It's, it's almost akin to, like, social media in the sense. It's, it's kind of like a recommendation engine. Okay. So that means that no one really has a complete understanding of anything. I, I actually-- When I walk around, like, SF or wherever I'm walking around and I see people, I often think, like,

"No one knows what's going on in the world." Everyone's like this, uh, including myself. You know, uh, you can imagine, uh, this person, uh, on the right on their phone, like, trying to find a new job. They're looking for biotech companies to work for. Uh, are they gonna get, like, all the possible biotech companies that match? No. Uh, so there, there, there, there's always gonna be, like, this unknown of what's out there. Or, uh, you know, maybe this other person with the headphones, uh, maybe they're trying to-- Yeah, try-- they wanna understand what's going on, uh, in, in, in, in, in, in some region of the world. They're just not gonna have a deep understanding. They, they can't-- Not only can they not find the information, th-they might not be able to trust the

information. So we basically are in a world where we live without this, like, key information infrastructure that is so critical. And I think this is extremely important, so important that if we don't fix this problem, I believe we get a world that looks like this, a dystopia. I'm not kidding. Uh, this is AI-generated, uh, version of San Francisco in twenty thirty-five. Um, and it's basically a world where no one, no person really understands what's going on. If people don't understand what's going on, then as we have this crazy AI technology that we all are talking about today that's coming, and the world is getting way more powerful,

there's gonna be conflict and all these things. If, if people don't know what's going on in the world, this is very bad. Like, we will be manipulated. We will make really bad decisions, b- as individuals and as a society. And I think this-- I-if, you know, if we could fix this problem, it'll be way better. Um, and when I think about all the possible problems that there are that are really important and neglected, to me, like, solving information is the most important and neglected problem. Okay, so that's the why of Exa. Uh, quick story of how we got here. So I, I, I've been thinking about this problem for a very long time, way before even twenty twenty-one when we started, uh, even since high school.

Uh, but I think what was really cool, uh, is that in twenty twenty-one, it suddenly became possible in our eyes to build a new type of search engine because transformers had gotten really good. So this is a time when, like, GPT-3 had recently come out. GPT-3 was, like, magical. You know, you type in a, a paragraph of text, and it fully understands you. At the same time, you know, as we saw with Google, it's like it doesn't fully understand you. And so what if you could combine, uh, the power of GPT-3 with a search engine? And maybe you could have perfect search over the world's information. And actually, the thought experiment that always drove me was like, you know, if we take a query, a complex

query and a document, and we run GPT-3 over it and we say, "Does this match?" It'll be a-- It'll do a really good job of saying, "Does it match?" Now, do that over a trillion documents for every search, and you get a perfect search engine or near perfect. Uh, the problem is that would cost like ten million dollars per search. So it really becomes an interesting optimization problem. How do you like billion X or trillion X reduce the cost of that? So that's kinda like the ideas that started Exa. This is actually the first day of Exa, twenty twenty-one. It's, it's actually, by the way, our, our five-year anniversary as of, uh, as of, as of yesterday. So, um, yeah, been a great time. Um, yeah. I wish I took a better selfie here,

but this was the first, first day. Um, and, and, uh, it-- Exa was basically built on the idea that, look, like, uh, traditional search engines, they use keywords. Keywords are, are, are very efficient and, a-a-and they get you-- they can handle simple queries. But if you wanna handle more complex queries, you just need to, to, to use neural networks. A-a-and particularly embeddings are a way of like encoding. You can't run a, like I said, like a neural network over every document, uh, for every query, but you can pre-process every document into some sort of structure like an embedding. Uh, and then

you could use-- and then that, that captures a lot of the intelligence of a neural network, and then you can use those embeddings. Of course, embeddings have also their own problems, and often you wanna combine embeddings and keywords, but certainly embeddings are a big part of, of the picture. And that's how you can handle shirts without stripes. We can handle this kind of query. Uh, another way of saying it is just stack more layers. Um, very bitter lesson pill. This is-- We are very early on to being a bitter lesson pilled. I don't know if you know this meme. If you don't, it probably looks really weird. Okay. Um, but anyway, so we were very bitter lesson pilled, so we did some crazy things, right? We, we, you know, raised a couple million dollars. We spent half of it on a GPU cluster. That

was crazy at the time. Um, we did a huge amount of research for, for, for really years, just heads down. And, and we did a lot of-- We were very new to search, to be honest. Like, we're just really obsessed with the problem, and so we invented a lot of new stuff that I still haven't seen, uh, uh, even today. Um, and so just, uh, yeah, history of Exa. So twenty twenty-two, so this is like a year and a half after starting, uh, we la-- we were called Metaphor at the time. Um, some of you might know it. Uh, we launched our first, uh, search engine to, to the world, um, a-and it was pretty exciting. Like, it was a new way of doing search. A lot of people were really

excited about it. The next big thing that happened two weeks later was ChatGPT came out, uh, and that really changed the world. Uh, and like this is what San Francisco looked like at the time, if you remember. Um, and this is the Exa team at the time. Uh, all right. Um, but, uh, everything was saved when we got this, uh, this, this message, message on, on Twitter from this person, whatever, uh, who wanted API access to our search engine. And that was really weird because we were never thinking that, oh, this is gonna be an API. We were just trying to build a better search engine than Google. We'll

figure out how to make money later. Uh, then someone asked us for an API. We were like, "No, we don't have an API, sorry." Uh, but then, like we started getting, uh, re- more requests for API access, including, uh, you know, my roommate, uh, who lived downstairs. And then we very quickly realized, wait a sec, like there's a business model. Like, okay, what we realized was AIs need search because like the, the argument is basically, look, even GB5, gigantic model, it's tiny in comparison to the internet, right? So like these systems always need to search. You're never gonna have GB6, GB7. It's not gonna be able to like just know everything

about the world. It needs to be connected to a retrieval engine. Uh, and, and that was a really interesting insight because th-these things now need a search API, uh, right? And so we pretty quickly realized, okay, wait, AI is gonna search the web. In fact, they're gonna search the web way more than humans. Um, and they're gonna search in very different ways. Um, so, uh, this is example of what humans, uh, this, this looks like when humans search, right? They search simple queries. This is what Google was made for, like SpaceX news. It's good at that. Um, but an AI system is very different, right? It kinda looks like this information

guzzler creature that's like insane and, and like it would be crazy if the same search engine that was optimal for humans was also optimal for these AI systems. So anyway, uh, a-a you know, we realized, okay, uh, if we build a search, uh, API for these AI agents, uh, or it wasn't called AI agents at the time, it was just AIs for LLMs, uh, then, uh, that will-- we can make money from that. Uh, that's a nice business model, and we think it's gonna grow really fast. Uh, and also the beautiful thing is it matches our initial-- our, our mission, which is perfect search, right? Like AI systems really

want perfect search. They don't want SEO. Uh, they don't want ads. They just want almost like a database of the world's information, which is what we were always trying to build. So we built the first search for LLMs, and yeah. Actually, like in twenty twenty-three, we said soon AIs will search more than humans. Uh, you know, uh, three years later, it's now happening. Um, okay. Yeah, and so then the next couple of years, we, we built a lot of really crazy stuff. Uh, it's way more complex than just embedding search. It combines all sorts of systems, some of which are included here. And now, you know, we're a much bigger team.

Um, and that's how we got here. Okay, cool. So just, uh, quick like what can you do with Exa? And then I'll talk about where, where we're going, how we're gonna get the perfect search. So the present. So right now, yeah, we're the highest quality information f-for AI agents, um, and you could do all sorts of things. So for example, uh, a lot of people like really complex queries. Uh, so you wanna find every startup funded by YC work in AI, give me their batch and status. You could do that with Exa now. Uh, and you could make this like arbitrarily complex. Like a lot of people don't-- aren't aware of this, but you, you could just use Exa

a-and just find r-really like any, uh, list of companies, people, uh, like, like blog posts, news articles that you want. Uh, it will take some time. So it'll take, you know, maybe not seconds, it might take minutes, but you'll get the information you want. At the same time, we also have the fastest search API in the world. So we have a two hundred millisecond search endpoint, and that's what it feels like. So it's super fast. It's way too fast for humans, right? But we're not serving humans. Uh, we're serving AI systems. And so, like for example, we, we serve some voice agents. And, uh, if you're a voice agent and you talk to the voice agent and

it wants to do a search underneath the hood, uh, you know, every millisecond counts. You want it to do a search really fast so that it could go, you know, process it with an LLM and then output the best, uh, audio back to the customer. We also have cool things like, uh, super efficient token extraction. So everyone's talking about, uh, the compute crunch and how everyone's spending way too much, uh, on LLMs. Well, actually, Exa could help there, uh, because, you know, when the LLM makes a query, it wants to get the-- just the information it needs, just the tokens it needs. And so we take, uh, the documents.

We give you ten documents, and then we'll give you only the most important, like hundred tokens from those documents. And that will save you a lot of downstream LLM costs.

We also ha- you know, some people, they don't necessarily want s- uh, you know, snippets from each document. They actually want structured output. So, hey, uh, you know, like, let's say you're, you're building a recruiting AI agent and you wanna find, you know, all the engineers who recently left their, uh, big lab job. Give me like, uh, you know, uh, the, the most cited, uh, uh, p- paper that they've written. Give me the, the college they went to and, and the year they graduated, and we'll just give you that as structured output. It makes it really easy. And yeah, I think, uh, one takeaway here is we're not building one search engine. Like, perfect search is not one

thing. It's actually we have five thousand search engines for each of our five thousand customers, right? Uh, we wanna build our system so it's super flexible because we want every business to be super optimized. And that's, I think, a beautiful thing because, like, we don't wanna declare what is the perfect search. We want you to almost, uh, you know, tell us what exactly do you want. Do you want super fast? Do you want, uh, you know, the highest possible quality even though it takes minutes? Do you wanna search only over these thousand domains? Do you wanna never search over those thousand domains? Do you wanna search over-- w-within this time window? Do you wanna never get product pages? Some people ask us for that. Like, so there's all sorts of

things that you could do, uh, with Exa. It's very flexible, customizable.

And yeah, I mean, our, our search quality, uh, i-is really good, uh, for these AI agents because we've spent years, you know, doing research, uh, into how do you build a new type of search engine for agents. Uh, it's even better than Google, which was built for humans, which, which makes sense.

Okay. A, a cool thing that we released recently, uh, was Exa Connect. Uh, so, you know, agents, they don't really care whether the information is from the public web or from private data sources. They just want the truth, right? And so it's always been obvious to us that, you know, we wanna assemble all the world's information. It's perfect search over all the world's information, not just the web. And so now we have a system where, uh, data providers can actually, uh, partner with Exa so that developers can then get the data from those providers. So we're, we're basically creating like a new market, uh, a new economy, uh, for the-- for agents,

uh, where, you know, if you have high valuable data, you could get paid, uh, from all the developers who want their agents to access that data. So we're creating this beautiful marketplace. I think it's really cool. It's like a, it's like a free market. Like, the, the, uh, the, the data providers can decide how much they think their content is worth, and then developers can decide what data they want.

And then you could do cool things like this. Uh, research AI info companies, monthly website visitors, Similarweb. Oh, I guess it went too fast. But you could get-- Like, basically, this is combining, uh, information from the public web and also information from Similarweb, uh, which is not publicly available. So you could do queries like that right now. Okay. So that's, uh, where Exa is right now. But the future has always been super exciting to me and, and the goal has always been perfect information, and n-now we know it's for AI agents. So instead of a world like this-- No, bad.

Uh, we want the-- We want search to kind of feel like this. It's actually really hard to, to describe what perfect information feels like. Uh, best way you could say is, like, literally any information query you have, it just works no matter how complex that is. Um, another way you could think about it is, like, it's as if you did a year of research in a second. So imagine no matter what you're looking for, whether it's people or, or companies or news, imagine you did-- you spent a whole year. You spent all of twenty twenty-six just doing research for it. You get that in a second. That, that should give you a sense of what perfect, uh, information feels like, and

you do that for every search, all the, you know, crazy number of searches that AI agents are gonna make. And so, yeah, we wanna move really fast at Exa. Like, we're, we're moving extremely-- Like, basically every-- Like, basically a quarter or two quarters now, we have as much progress as we did the past five years, and it keeps being like that. It's exponential growth. And so, yeah, our ambitions for twenty twenty-seven are pretty crazy. Uh, we want, uh, the world to be like this, where everyone walks around with, like, deep understanding of what's going on. I think it's particularly important because, you know, things like the twenty twenty-eight presidential election are coming out, are, are coming soon, and I would love for,

you know, the entire world to have access to near perfect information so that everyone is very informed going into that election. You, you kind of can feel the gravity o-o-o-o-of what we're doing here of, of, of perfect information. I encourage others to try to do it too. Um, it's very important for the world. It's like key information. Uh, it's key infrastructure. And yeah, you have to-- Like, that, that slide I showed at the beginning, it's not the whole picture, right? If you play it out, we're talking a thousand times more searches from AI systems than humans. This-- You can't even capture that on a graph. Like, this is not-- That's like twenty times. A thousand times would be all the way up there. It's like the top

of a building or something, right? So it, it's crazy w-what's coming and it's, it's really happening. Um, basically, like, you know, humans on average search a few times a day on Google. But when everyone has, you know, AI assistants and every software product you use has AIs in it, every interaction you're doing is gonna be grounding itself in, in search. So it's gonna be a huge number of searches. And if each of those searches are, you know, as, as true as possible, as near perfect, uh, then, then, then the world looks like this in twenty thirty-five.

Um, a-a-and yeah, I, I do think that if w-- I, I do think we're-- basically our, our future is limited by ourselves. Like, we're basically getting to a world where our technology is so good, it's really just a matter of, like, can we coordinate and just decide together that, like, on sensible things, right? Like, if you look at San Francisco, there's amazing things happening here, and there's really stupid things happening here at the same time. Like, that's, that's just coordination problems, and, and coordination comes from the information we consume. Uh, so that's what we're working on. You all have a role to play i-in also getting to this world. Uh, so thank you all for, for, for working on whatever passion you're working

on, and thanks for listening to me. All right. Thank you.