Pinecone 2.0 — Edo Liberty, Pinecone

All right. Welcome, everyone.

I wanna tell you a little bit about, uh, the new, the new, uh, knowledge layer, and, uh, I'll start with, uh, telling you a little bit about theory of mind. Uh, how many of you, uh, know what theory of mind is? All right, so a few folks. It's a, uh, term in psychology about, uh, our capacity to understand, uh, and have a mental state or a model of a mental state, uh, for other people. What do they know? What do they believe in? How do they act? That's, uh, what allows us to socialize, that's what

allows us to lie to each other, and so on. Uh, but it's a, a crucial developmental stage for humans, uh, and a necessary function. I'll, I'll sort of explain it in the best possible way with a funny story just to break the ice. Uh, about fifteen, sixteen years ago, I, uh, worked at a company called Yahoo, um, that had a product called Yahoo Answers, which was k-sort of like a Q&A forum. Uh, we used, uh, then AI, which is the Stone Age of AI, you know. It was like,

uh, NLP and other statistical models to answer questions. But the funny thing is people would come to, uh, Yahoo and ask questions like, "Am I fat?" Which, uh, is funny because obviously, uh, uh, you know, a Q&A forum would-wouldn't have any clue whether you're, you're too fat or not. Uh, but this is a funny way to say, yeah, people have a very bad theory of mind on sort of what the products that they use, what information they have and what they don't

have. And I wanna argue that in some sense, we are making that mistake with AI today in a more subtle and less sort of like, uh, silly way, but in a just as a fundamental of a mismatch. And so I will categorize roughly three kinds of knowledge. I think there are, there are many more, but, you know, in broadstro-- in broad, broad strokes, this is not a bad way to categorize. First of is, is, uh, general knowledge. Think about that as what's available in the public

domain, things that models are trained on, and we expect an LLM to just know out the gate. Uh, things about just, you know, uh, improving your, uh, Rust code, uh, asking you like, rec-- you know, explaining stuff about the legal terms or, uh, sort of exploring use cases for, for products. Um, w-we know how to do that. Models are really good at that. When we use our agents, they have that information. There is specific knowledge the companies... Of course, the models wouldn't

have, uh, any clue about whether we've, uh, implemented something specific in our code base, what a contract of a specific customer has. Of course, this is information that's private to the company, uh, but we know how to deal with that. And RAG and search and of course, uh, Pinecone, VectorDBs, and so on, uh, solved these problems a long time ago. There's still a lot to go, but we by and large know how to do it. And then I would say there is tribal knowledge, okay? Tribal knowledge

is the sort of general knowledge inside the company. This is the kind of information that, uh, an emp-- a seasoned employee has that a new hire does not, okay? It's roughly how we do things, what our processes are, who's in charge of what, uh, what are the guiding principles, what are the-- you know, sort of how we get stuff done in this company, okay? Um, that is not a s-- that's not general knowledge because it's, it's specific to your company, but it's also not a s-specific piece

of knowledge because you can't just get it by searching. You can't just get it by plucking it off one place and say, "Yes, this document says, uh, that this is our culture." It's the sum total of a lot of, uh, things. Um, and so if our theory of mind of most of our agents in big companies today should be something like a new hire. They're brilliant, they have tools, they have-- we have very high expectations of them, but they are by and large clueless, okay? They have no idea about your

company's goals, culture, priorities, processes. Um, uh, they have to read a bunch of documents, uh, to be able to get any semblance of understanding of where they are in the world. Um, and really, they start most tasks from scratch, okay? They really start-- kinda wake up, and every time you start y-your session, uh, they're like a new hire on their first day on the job.

So I would argue that what they need is not necessarily to be smarter, to have better tools, but to have something new, which I would call a knowledge layer going forward. Um, and, uh, I have to explain to you what I think a knowledge layer has to actually do. But just I'll define it generally before I explain to you what we do. Um, so at the very least, it needs to be persistent. We talked about this as the company knowledge, processes, uh, capabilities, and, uh, goals, and so on.

It has to be persistent. Uh, it sure-- it, uh, feels like putting this thing together is going to be a monumental effort, so you wanna do it once or at least very rarely. Uh, and of course, all your agents in the company need to have access to this thing. The second thing is that, uh, you would argue that this knowledge layer needs to be, uh, highly specialized. If your agent, uh, is in charge of HR practices or optimizing your, uh, your kernels, uh, in your code base or,

uh, resolving customer issues, that they would care about different things. They would wanna, you know, address-- They would wanna have different information available to them. Um, and I would argue irrelevant of, of specifically how it's created, I would argue that it's a good practice to have the domain experts in the company own those and have b-being able to interact with them and make sure that they actually, uh, are aligned with what the company wants to do. And finally, which is the hardest thing, is they

need to stay up to date, and they need to stay concurrent constantly because companies change, and you need to make information, uh, uh, deprecated when it becomes deprecated. If the company replaces its CEO, the very next day, everybody in the company knows who the CEO is, okay? Even though nine thousand documents going from, you know, a day before yesterday to, uh, the last ten years say it's somebody else. This has become

common tribal knowledge immediately and everything else, even ev-- you know, every RAG, uh, search will tell you otherwise, this is now the overriding fact. Okay? Um, I will argue before we move on that I'm not suggesting that this knowledge layer should replace everything else we're doing. Our agents are already very capable and should keep being very capable. Uh, they have access to LLMs, of course, uh,

tooling, local files, search, of course, like RAG, vector search, text search, and so on. And I would argue that, uh, the knowledge layer needs to be a separate entity that all these agents have access to, as, as I explained, uh, why this is sort of a fundamentally different kind of data source. So I wanna tell you more about Nexus. Um,

um, Nexus is a product that we're, uh, uh... Of course, it's a knowledge layer, but I need to-- I wanna explain to you what it does. Uh, of course, uh, so the, the four parts are roughly connectors, what I would call context, uh, uh, tasks, and queries. I'll explain each one of them, uh, now. So connectors, of course, do the obvious things. There's nothing much to talk about them. Um, uh, they bring

data into the platform, uh, re-respect access, uh, rights and, uh, and control, and so on. Um, very dull as a topic of discussion, but incredibly important in enterprises, incredibly important to do right, uh, and everybody who's built that knows how painstaking it is to get this thing to work well. Um, the interesting-- Sort of the first, uh, properly interesting part is, um, the

context. So the context, think about that as the, um, the sum of assets that the, that Nexus or this knowledge layer saves about a specific topic. Think about this as a context for HR, for operations, for engineering, and so on. Okay? Of course, it has the sources, the connections to the data, and so on. But then it has the manifest. Okay? The fe-- manifest is the first truly novel

idea and, and a, uh, important, uh, uh, entity. Okay?

The way that we today give context to our agents is with skills and plugins and, uh, commands and sort of markdown files they write ourselves. In a knowledge layer, you don't do that anymore. Okay? You tell the n-- you tell the knowledge layer, you tell Nexus what you care about in a manifest. You say what kind of tasks you're, you wanna complete, what entities you wanna track, what kind of information you care about. And it's, it's the knowledge layer's

work to keep track of those files, okay? And, and organize its own data. Um, uh, and then it has the knowledge files themselves. So think about the manifest as the meta knowledge, uh, file or set of files. Um, and then there's the knowledge. The knowledge itself, uh, think about this as five types of contents. One of them-- The, the first kind is a semantic map. Okay? This is a file that

organizes the catalog of where everything else is, the key schemas, the glossaries, all the information that you need to be able to operate this context. Then there are, of course, a bunch of markdown files that contain information, decisions, memories, uh, all sorts of unstructured information. If you've, uh... Who here has, uh, heard about, uh, uh, LLM Wiki? Okay. So LLM Wiki is this, uh,

idea that Andrej Karpathy suggested to have agents basically, uh, manage some stash of markdown files to sort of keep their own notes of stuff. This is a very similar idea, okay? Sort of on steroids, but similar concept. But that's not enough. The context contains also a bunch of SQL tables for tabular data, for aggregation, for facts that they need to memorize. Uh, a vector database, uh, to be able to semantically search and text search and filter a bunch of

information and all the information that it itself decides is necessary to keep and, and remember. And graph entities and graph indexes to remember, uh, again, entities, relationships, uh, causal chains, and so on. Okay? And all of that is managed and created for you. You don't touch any of this. You don't have to, uh, know how any of this is created. It's all maintained through the manifest. So these are-- this is

the context. Now what are the tasks? What are the actions? These are the nouns. What are, what are the verbs? What can we do with those, uh, objects? Uh, first of all is, of course, import, which is bring new information in the system, new files, new events, uh, new, uh, concepts. Um, curate is probably the most and most elaborate, uh, so one of the two most elaborate functions. Its, uh, job is to take, um, to take the- Uh, then

say call it a file. It doesn't have to be a file, but let's say it's a new PDF or a new meeting transcription or a new, uh, presentation for the board and say, "Now I need to update my view of the world." Okay? So it takes that, it takes the manifest, which tells, okay, now what, what things do I care about? What concepts do I need to track? What, uh, needs to be, uh, uh-- What do I need to memorize here? And then it takes the semantic map that I told you contains all the in-- like, organizes where information is

and understands, okay, fine, I need to go update these tables, these relationships to my graph, uh, this, uh, these text indexes and so on. Um, and when it's done, it has essentially said, "Okay, fine. I have assimilated the new information." Uh, it's already-- It's, it's sort of persisted in your context, and we can move on. Okay? Search is sort of a internal function, very important. It basically abstracts with simple

functions, all the access to that information so that the agent, that I'll tell you in a second, doesn't have to know, uh, how to sort of access the internal parts of the context. Now comes, uh, the second, I think most, uh,

most, uh, interesting, uh, part of, of the system, which is NoQL. NoQL is our way to s- to, uh, specify how we want information out of the context. It's not enough to just ask questions. If you're an agent, it's really important for you to give us the budget in tokens or in dollars, how much time you want us to invest in this. Sort of like, do you want us to-- just a quick off-the-cuff, uh, kind of immediate answer, or do you want us to go full tilt and

scour everything we know to make sure we give you the most accurate answer, uh, and so on. So the structure and, and so on. It's, you know, it's, it's an agentic tool. This is not like a chat interface. Okay?

Um, and of course, what you get back is like something that's agent-friendly. This is, uh, you know, grounded text and, and kind of structured in the right way. Format. This, again, is not designed to be like a chat interface. This is really what your agents expect. Okay? Um, so this is, this is a, uh, um, this is a, uh... Trying to figure out how much time I have.

Okay, I need to go a little bit faster. So,

uh, this is a, uh, sophisticated crowd. I wanna sort of, uh, pop the hood and show you a little bit about how things work. So, uh, if anything, uh, this is the interface. I wanna--

There are many, many cool ideas, uh, that make this work, but I wanna tell you a little bit about just one of them. Okay? What you're seeing is the interface for Nexus. Okay? You will see at the top a question, uh, that, uh, the, uh, the agent, uh, issued, and you see, you see on the left, uh, the different steps that it went through. Okay? What you see on the bottom right is probably the most interesting. You see generated code on the bottom right. What Nexus

does, which is very different than other systems, it's what we call a runtime coding agent. Okay? Unlike other coding systems in, in-- that, that you're used to, uh, where the task is, uh, take a large code base and then help me edit it, right? But when the edits are done, what I have is a piece of software, and that is deployed. That is the artifact. Okay? What is running does not contain the model anymore. It's just

the code. Okay? This is not what's happening here. The-- In query time, you should think about the engine essentially building something like a Jupyter Notebook, something like, uh, like a Python REPL, right? And they literally write code and execute it and write code and execute, and if they get the answer, they know what to do with it. And if it's, you know, if it's not what they expected, they rewrite that piece of the code. And what you get in the end is a, is a piece of code

that completes the task, that gets the information that you wanna get. Okay? That is savable, that's re-runnable. Okay? But it's also incredibly flexible because now the, the answer is got not by, by a query to a database. It's wr- it's written by code, which is in, of course, incredibly flexible. Okay? That code is, by the way, interestingly enough, if you replace-- if you write software this

way, the, the amount, the, the amount of prompting that you need is significantly reduced. We went down from having something like a hundred and fifty thousand tokens to give our agents all the tooling they need to less than a thousand tokens to specify all the interfaces they need to get information out of Nexus. Okay? And so the ans-- the, the... So what it looks like when you run it is like this. Okay? So on the left, of course, you'll see Nexus with, uh, uh,

NoQL answering the question. And then on the right is sort of-- is the same, uh, task given to an agent with all the tooling to complete the task. Okay? But remember this on the right, it's, uh, severely handicapped. It's like the employee on the first day on the job. They have to read a bunch. They have to figure out where the data is. They have to figure out what the task is. They have to write a bunch of code. Uh, it's just, it's just severely handicapped. And there's, uh... It's just, I don't know if you... Okay, it stopped. But,

um, needless to say, this is by far slower. It's a lot more expensive in terms of token consumption. Uh, but interestingly enough, it's also a hell of a lot less accurate because, again, they don't have the right context, the right objectives and so on in your company. Um, I'll just say that we work, uh, obviously with a bunch of, uh, early access customers already. Uh, you see some results here, but I'll just sort of fly through them and say

that across different domains, ac-across different enterprises, across different kinds of tasks, you will universally see the same thing. That if you shift to this paradigm, you get significant cost savings. Uh, seventy-seven, ninety, you know, percent token cons-- token reduction is pretty common. Um, uh, it gets a hell of a lot faster, anywhere from twenty, thirty percent to sometimes, uh,

uh, uh, seventy-seven percent faster. Okay? But-- And the most important thing is it actually becomes a lot more accurate as well in the same time. So this is really, uh, it's a slam dunk. I'll just wrap up by saying that, uh, um, the, that Nexus is coming out of early access and into public preview literally tomorrow. So go, uh, try it out.