AI Engineer World's Fair 2025
GraphRAG methods to create optimized LLM context windows for Retrieval — Jonathan Larson, Microsoft
Read the talk
GraphRAG for Code: From Understanding a Game to Changing Doom
A small terminal game exposes the gap between retrieving code and understanding a repository, before Doom and LazyGraphRAG show what structured context can enable.
From a talk by Jonathan Larson
Before you start: Familiarity with retrieval-augmented generation, LLM context windows and multi-file software repositories will help you follow the examples.
What does this game actually do?
What would an LLM need to explain a game: its source files, or an understanding of how those files work together? Jonathan Larson, who introduces himself as leading Microsoft Research’s GraphRAG team, approaches that question through structured LLM memory. The GraphRAG paper and public repository had already inspired derivative work, including Neo4j’s card-game explanation. Here, the progression moves from repository understanding to agents that act on it, then to evaluating retrieval quality with BenchmarkQED and LazyGraphRAG.
The first example is a terminal game: jump over an obstacle and earn points; collide with one and lose points. Larson describes it as roughly 200 lines across seven files, with code the LLM has never seen. That makes it small enough for a person to understand completely, yet the team found that placing all its code directly into a context window still produced poor understanding. Context capacity alone did not solve the problem.
The team asks an ordinary RAG system, “Describe what this application is and how it works.” Its answer identifies a game initialized through a main function, a configuration file, and separate classes and functions for the screen and logic. Those details describe code organization, but they leave the reader unable to picture the game.
With the same repository and question, GraphRAG for Code gives a more behavioral description: a curses-based terminal game, a player that jumps vertically, obstacles moving horizontally, a static background, and the Spacebar as the jump control.
| Approach | What the answer explains |
|---|---|
| Ordinary RAG | Entry point, configuration, classes and functions |
| GraphRAG for Code | Player movement, obstacle movement, background and controls |
This is a global query: answering it requires connecting behavior across the repository. A local query can focus on a narrower piece of code; explaining the application requires assembling those pieces into a coherent account. The demonstration makes the distinction visible in the answer itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generating Rust is not the same as translating a game
Once the system can explain the game, the next test is whether it can preserve its behavior in another language. The team takes the same Python repository—four main source files within seven files total—and asks for a Rust translation. The demonstration first runs the Python game, establishing the behavior the translation must retain.
The baseline puts all the source into an LLM. Larson says the team tried a variety of prompts aimed at translating the application as a whole. The model produces Rust, but attempting to compile that output reveals problems throughout it. Producing plausible source text has not produced a working application.
The GraphRAG for Code path instead uses the repository’s graph structures during translation and generates a collection of Rust files. Larson then shows the translated game running natively in Rust. The useful comparison is between two end states: generated code that fails to compile, and a translated application that runs. The talk does not identify the model, enumerate trials, or establish a translation success rate beyond this demonstration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Understanding Doom across files
The next repository is Doom, which Larson describes as approximately 100,000 lines across 231 files and about 30 years old. Its public history creates a different challenge from the unseen terminal game. Larson initially expects models to know the code from training; his team’s tests instead suggest broad familiarity without the implementation detail needed to make meaningful modifications. Recognizing a repository is not enough to change it reliably.
The team starts by generating documentation. At the global level, that means explaining entire modules rather than summarizing files independently. A description of Doom’s sound system, for example, must connect responsibilities spread across roughly 20–30 files. Local queries then let the reader drill into individual files beneath that module-level account. Global understanding and local detail serve different parts of the same task: first establish how a subsystem works, then inspect the code that implements it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adding a feature that crosses file boundaries
Documentation, Q&A and translation lead to the next experiment: feature development. Original Doom does not let the player jump, so the team chooses to add that capability. It requires coordinated changes across files—the kind of task where an agent can make a convincing edit in one place while breaking another. Repeatedly repairing those breakages can leave the repository in a cycle of incompatible changes because the system lacks a sufficient understanding of how its parts fit together.
After showing the original game, the team connects the GitHub Copilot coding agent, introduced at Build, to GraphRAG for Code. The workflow is short:
- Create an issue against the Doom repository asking to add player jump capability.
- Add a couple of sentences describing the request and assign the issue to the coding agent.
- Let the agent consult GraphRAG for Code to develop a holistic, top-down plan before making the changes.
GraphRAG supplies repository context to the agent; the agent turns that context into a plan and edits.
Larson reports that the agent changed many files and the feature worked out of the box, whereas the other agents his team tried failed on this task. The resulting demonstration shows jumping in Doom. This is the concrete payoff of pairing structured memory with an agent: repository understanding becomes a resource for coordinating action. The comparison remains a reported demonstration, without a controlled agent benchmark or detailed trial accounting.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluating both retrieval and synthesis
The coding demonstrations raise a measurement problem: how should a system be evaluated when some questions target a specific fact and others require understanding the whole dataset? Larson introduces BenchmarkQED, announced as newly open source during the talk, to address both local and global answer quality.
BenchmarkQED separates three responsibilities:
| Component | Responsibility |
|---|---|
AutoQ | Generate queries for a target dataset |
AutoE | Evaluate answers using an LLM as judge |
AutoD | Summarize and sample datasets |
This separates the questions a benchmark asks from how it judges responses and how it prepares the underlying data.
AutoQ organizes questions along two axes: local versus global scope, and data-driven versus persona- or activity-driven intent. Data-driven questions start from information in the dataset. Persona- and activity-driven questions adopt the perspective of someone working in the domain and ask what that person needs to know.
The examples come from an AP News dataset focused on medical events:
- Data-local: “Why are junior doctors in South Korea striking in February 2024?” The country, profession, event and date give retrieval specific targets. Larson expects ordinary RAG to perform well here.
- Activity-global: “What are the main public health initiatives mentioned that target underserved communities?” This asks for a synthesis of initiatives across the dataset. A relevant passage may help, but retrieving one passage does not establish the main patterns across the corpus.
AutoQ generates questions across both dimensions so that evaluation does not quietly reduce overall quality to performance on precise lookup questions.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
What larger context windows change—and what they do not
Once AutoQ supplies the questions, AutoE judges the answers. Its composite combines comprehensiveness, diversity, empowerment and relevance. The first three carry forward the criteria used in the original GraphRAG work; relevance is the added dimension. These are judgments of answer quality, not a factual-accuracy score.
Larson uses this framework to compare LazyGraphRAG with vector RAG using 8K, 120K and one-million-token context windows. A bar above 50% means the judge favors LazyGraphRAG over the corresponding baseline.
The companion evaluation article describes 1,397 health articles, 200 queries, six trials per query per metric, and a GPT-4.1 judge, with ties counting as half a win. Its GPT-4o comparisons precede a separate GPT-4.1 million-token experiment; these should not be read as one fixed-model context-length sweep.
The tie rule can be expressed directly in Python when aggregating pairwise verdicts:
python
from collections.abc import Sequence
from typing import Literal
Verdict = Literal["win", "tie", "loss"]
def preference_rate(verdicts: Sequence[Verdict]) -> float:
if not verdicts:
raise ValueError("At least one verdict is required")
scores = {"win": 1.0, "tie": 0.5, "loss": 0.0}
return sum(scores[verdict] for verdict in verdicts) / len(verdicts)
This computes preference for a chosen system from supplied verdicts; the four judging criteria determine what those verdicts reward.
For data-local questions, Larson reports the following LazyGraphRAG win rates under the composite judging criteria:
| Vector-RAG context window | Reported LazyGraphRAG win rate |
|---|---|
| 8K tokens | 92% |
| 120K tokens | 90% |
| One million tokens | 91% |
The local results surprise him because these are the questions expected to suit ordinary retrieval. He reports strong performance across the question categories, with a somewhat larger advantage on global questions.
Increasing the vector-RAG context window did not materially close the judged-quality gap in these tests. Larson had expected longer contexts to help with global questions, but the presented comparison still favored LazyGraphRAG. He also reports that, in a separate off-slide test, LazyGraphRAG cost one-tenth as much as the one-million-token-context comparison. The talk does not specify that cost test’s model configuration, pricing or whether indexing costs were included. The result points toward the value of selecting and organizing context, rather than assuming that a larger window alone supplies the needed understanding.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From repository knowledge to scientific reasoning
LazyGraphRAG had first been discussed in a November blog post the previous year. At the time of this talk, Larson describes Azure Local integration as forthcoming following a Build announcement, and integration into Microsoft Discovery as underway. Discovery extends the application from code to scientific work: hypothesis leads to experiment, experiment produces learning, and learning becomes knowledge for the next cycle.
The closing demonstration shows a copilot reasoning over scientific knowledge, with GraphRAG and LazyGraphRAG powering the underlying graphs and answers. The same relationship seen in the Doom example reappears in another domain: structured memory gives an agent connected knowledge to reason with, and the agent uses that knowledge to carry work forward. Larson ends on that pairing—memory with structure, amplified by agents—and invites further questions outside the session.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Original GraphRAG research on entity graphs and community summaries for corpus-wide questions.
Research implementation with documentation and a command-line quickstart. The repository currently describes itself as largely in maintenance mode.
Tools for query generation, answer evaluation and dataset preparation, with AP News and podcast datasets.
Launch article explaining the query categories, judging procedure and LazyGraphRAG evaluation conditions.
Further reading
November 2024 explanation of deferred LLM processing, relevance-test budgets and the original evaluation.
Historical Build 2025 announcement of GitHub's asynchronous coding agent.
Microsoft's initial announcement of its agentic platform for scientific research and development.
- GraphRAG – the card gameArticle
An imaginary music quiz illustrates how following graph relationships supplies additional evidence for an answer.
Read the complete timestamped transcript
- 0:00
[electronic music] Um, well hello everyone.
- 0:16
Um, thanks for coming to the session here today. My name's Jonathan Larson, and, uh, I run the GraphRAG, uh, team at Microsoft Research. Um, some of you may have seen our paper that we actually, uh, released last year, uh, the GraphRAG paper, or you m- uh, might have seen our, uh, GitHub repo, uh, that was out there
- 0:33
and, uh, has a lot of stars on it. Um, when we released this last year, to our surprise, it got a lot of attention, and also it inspired many other offerings that we saw that were out there as well too.
- 0:43
Um, uh, and my favorites, of course, were the ones that Neo4j did with the card game, because I'm a huge bo- uh, board game fanatic as well too. Uh, but, uh, there's been lots of derivative work from it.
- 0:51
And today, though, I'm not here to talk to you about some of the things that have been. I'm here to talk to you about some of the new horizons.
- 0:58
And the th- two things I really want to leave you with here today are that, uh, LLM memory with structure is just an absolutely key enabler for, uh, building, uh, effective AI applications.
- 1:09
The second, as we'll talk about a little bit later in the talk as well too, are that, uh, agents, of course, paired with these structures can provide something even more powerful.
- 1:17
And let me go ahead and go back to this slide here. So today I'm gonna tell you about a couple things. The first is actually showing you GraphRAG as applied to a specific domain.
- 1:25
In this case, we're gonna apply it to the coding domain to look at enterprise productivity, uh, in, in the coding space. The second is we're gonna take a look at a new release that we are announcing today.
- 1:35
Uh, we're actually having a blog post on this tomorrow. Uh, we have BenchmarkQED just went open source, uh, as well. And then I'll talk about some, uh, new results that we've had on, uh, some GraphRAG evolutions on a new, uh, technology we've been working on called LazyGraphRAG.
- 1:48
I won't be going into the specifics of how any it works. I'll just be showing some benchmarks and talking about some of the ways you c- be able to access it soon.
- 1:55
So with that, let me go ahead and jump into GraphRAG for code and tell you about how we've been using GraphRAG to actually help, uh, drive repository-level understanding. And I wanna start first with a little demonstration.
- 2:06
Uh, everyone likes to watch little videos, so I'm gonna go ahead and hit play. And yes, it's showing on the screen there. So this is a little, uh, terminal-based video game that one of my engineers have put together.
- 2:15
It's a one where you jump over obstacles, and then you get points when you jump over them. When you run into them, you lose points. And two important things, though.
- 2:22
First, the LLM has never seen this code before, and it's small enough for the human to know the ground truth holistically 'cause it's only about 200 lines of code across seven files.
- 2:31
But it's complex enough that the LLMs had a lot of troubles understanding it if you just provide all the code directly into the context window. So with that as a background, let's take a look at what happens if you use typical regular RAG over the top of this.
- 2:43
So if you're using, uh, one of the tools that help analyze your code, this is the type of answer you might get back. And this is... We ran this, uh, through with a regular RAG system, and we asked it, "Describe what this application is and how it works."
- 2:56
And just to read you, 'cause the, what it says, 'cause it's too small to read, "The application is designed as a game that is configured and initiated through a main function.
- 3:03
The game leverages a configuration file and has its main components, such as the game screen, game logic, encapsulated in separate classes and functions." Totally useless. It just says it's a game.
- 3:13
That's... There's nothing more [chuckles] to it than that. Um, and it just put a bunch of other, uh, cruft in there. If you use GraphRAG for code over the exact same code base with the exact same question, you get a much, much better description.
- 3:26
This is just a very stark contrast you get between the two. So in the GraphRAG for code, it says, "The application is a terminal-based interactive game designed to run with a, in a curses-based terminal environment."
- 3:37
So far, not much better. But this next line just kills it. "It features a player character that can jump vertically, obstacles that move horizontally across the screen, and a static background layer.
- 3:48
The game controls via keyboard input, specifically using the Spacebar to trigger the player's jump action." So what this is showing you is semantic understanding. So if you've, um, read our paper or used any of the, um, GraphRAG code base that we've, uh, put out there, you've, you'll know about the concept of what we call local and global
- 4:05
queries. And so this is what we would call a global query over top of the repository. It requires understanding the whole repository to answer that question correctly. And we can see that it excels at that.
- 4:16
So one of the next things we decided to do is, well, if it can answer questions pretty well, can it maybe do code translation? And so that was the next thing that we, uh, aimed it at.
- 4:25
So taking the same code base here, we took on the challenge of taking this Python code and then asking to translate it directly into Rust. And let me show you just how that actually works.
- 4:34
I'm gonna play the video here. So what we're gonna do is start off with, um, in VS Code here. We're gonna look at the four source code file. Well, there's four main ones.
- 4:41
There's seven files total for this game. These are the source Python code, uh, files that we're working with here. We're gonna go ahead and run those just to show you again.
- 4:49
This is the same game. So this is again the same side-scroller game here. And then what we're gonna do is we're gonna actually take those source code files and just put them straight into an LLM.
- 4:57
We're first not gonna use GraphRAG for code. We're just gonna take all that source code, 'cause it's only 200 lines of code, and put in some nice prompts. We tried a wide variety of these prompts and then try and see if the LLM could actually translate that code holistically in all of its pieces into Rust and actually
- 5:12
just work out of the box. And so this is actually the translation happening there. Um, we're gonna go ahead and copy the Rust code that it generated. So it did generate Rust code.
- 5:19
I got to gr- give it credit for that. But after it generated the code and we reviewed it there, we went and we tried to compile it, and there's problems everywhere.
- 5:27
Doesn't work out of the box. Uh, so then we used GraphRAG for code, again bringing structure and these, uh, these graph structures to the code base there. We're gonna just go ahead and run the translate function there on GraphRAG for code, and it generates, on its side, all these s- all these new Rust files that you see
- 5:44
are on the side there. I'm just gonna skip past here 'cause I don't y- wanna bore you with clicking through a bunch of Rust files. But once we generated all these Rust files, we can go ahead and run those Rust files on GraphRAG for code side of things, and presto, we have a full video game completely translated
- 5:59
from Python, uh, now working in Rust natively. But we didn't wanna stop there. The next thing that we wanted to do was, well... That game is kind of a toy example.
- 6:09
It's like 200 lines of code. Let's go to a code base with 100,000 lines of code, and that's what we did next. So we went to the Doom code base.
- 6:16
It's about 30 years old. Now, we did run a bunch of tests over the top of this here first, uh, because I figured, well, all the LLMs are trained on the Doom code base.
- 6:24
I mean, it's, it's gonna just know these things on, um, natively and automatically. So we did a bunch of tests with that. We actually figured out, while it knew the Doom co- Doom code base, it didn't know any of the specifics.
- 6:34
So if you ask it to actually modify the Doom code base in any sort of meaningful way, the LLM models just fail again completely. Um, so then what we did next was, well, if we can, uh, reason and understand over top of this code base, 100,000 lines of code, 231 files, um, maybe we can generate some new
- 6:50
outputs that you otherwise couldn't generate. So with that, we then immediately had it start- used it to start generating documentation. So this is actually showing a high-level repository-level documentation.
- 7:01
Again, going back to the concepts of GraphRAG, we have local and we have global queries. This is showing you the global query results. And so it's not looking at the understanding of a single file.
- 7:10
This is looking at modules in their entirety, like across 20, 30 files, and being able to actually give you a sense for, like, what's the sound system inside of the video game, and how does that work?
- 7:20
But then you can still drill down into what we would call the local-style queries and actually see the individual files as well too. So then we thought, "Well, this is kind of neat.
- 7:28
We can look at the documentation. But if we can look at the documentation, we can do Q&A, and we can do code translation, can we maybe take this one step further, do feature development?"
- 7:39
So that's the next thing I'm gonna show you. So, um, we had a lot of, uh, video game players in, uh, our office, and we were kind of thinking about, like, what would be a cool thing we put over top of this Doom video game?
- 7:48
And somebody to me said, "Well, doesn't... You can't jump in the original video game. What if we w- added the ability for a player to jump?" And that's a complex thing to add into Doom because it requires multi-file modification.
- 7:59
And if you tried to use AI systems to do multi-file modification, what it'll often do is it will do a great job editing one of the files and then completely break a bunch of the other files in the process of doing so.
- 8:09
And then it, that just pr- rinses and repeats and continually [laughs] kind of, uh... You end up with something that doesn't work is what, what you oftentimes run into, and it's because of this lack of understanding how everything fits together.
- 8:20
But again, this is where GraphRAG can really help out. So with that, we went ahead and this is... If you haven't seen the Doom video game, this is actually a little video of the original Doom video game, uh, being played, just to sh- give you some context for what it was.
- 8:34
Then we used, uh, the new, um, GitHub Copilot coding agent that was announced at Build, and we wired it up directly to a GraphRAG for Code. So we're gonna go ahead and s- create a new issue.
- 8:43
This is pointing to the Doom code base. And we're gonna tell it, "Add jump capability to the player." We're just gonna add a couple sentences of description here and then tell it to go.
- 8:52
And then we're gonna assign it to that GitHub Copilot coding agent.
- 8:58
And then the next thing we're gonna do is we're gonna then go ahead and look at what's happening underneath the hood. So this is actually the GitHub Copilot coding agent reaching out to GraphRAG for Code on the back end, coming up with a plan that's holistic, and it's approaching it from the top down.
- 9:12
And then this is the really the mo- the aha, aha moment we had for us. It changed a whole lot of files, and it worked out of the box, where all the other agents that we had tried completely failed on this task.
- 9:22
And because it worked because of those GraphRAG structures that we were bringing to bear and actually using those, uh, with that. So then the end result was we were elated with joy and jumping in Doom specifically.
- 9:34
So, uh, that's [laughs] kind of the story of how we got to with that. So with that, um, I do, uh, have another part of the talk I'm gonna talk about today, and that's also, um, uh, five minutes to talk about that one, and that's BenchmarkQED.
- 9:47
Uh, just shifting topics a little bit, I just showed you GraphRAG as applied to one vertical. Uh, next I wanna spe- specifically talk about, um, how do we measure and evaluate systems like GraphRAG?
- 9:56
How do we build systems to evaluate those local and global quality metrics? And so for that, today I'm announcing BenchmarkQED. This is available now open source on GitHub, so you can go ahead and, uh, check it out at the link.
- 10:09
We just, I think, got it live last night. Um, and there's three components to this. The first is what we're calling AutoQ, and AutoQ is focused on doing query generation for target datasets.
- 10:21
So it allows you to take a dataset and then generate queries for it. AutoE is the evaluation using LLM as a judge to actually then evaluate how those queries performed on that said dataset.
- 10:32
AutoD is the third component, and that really focuses on dataset summarization and dataset sampling. Let's jump into just a couple of these in a little more detail. So if you take a look at AutoQ, AutoQ is taking a look at the local and global aspects.
- 10:46
You can see that there on the X-axis. On the Y-axis, it's then combining that against data-driven basic questions that are generated based on the data itself or persona or activity-driven, which is the second type that we have there.
- 10:59
Those are more complex, like if you take on the role of a person in that domain field and then using that to generate questions. So to show you a few of these sample questions, I'm just gonna choose a couple here.
- 11:09
This was built on a AP News dataset that was focused on, like, medical type of events. And so a data local question might be, "Why are junior doctors in South Korea striking in February 2024?"
- 11:20
There's a lot of highly specific information in this, and I would expect regular RAG to perform very well on this type of question. In contrast, take an activity global question here.
- 11:30
It might be, "What are the main public health initiatives mentioned that target underserved communities?" There is nothing in that question that you can really pivot on in terms of, like, embedding or, uh, indexing that would really work on it.
- 11:41
You really have to holistically know the entire dataset. And so this, uh, AutoQ will help generate questions across that whole spectrum, from local to global, from data-driven to activity-driven, and give you these categories of questions that you can use.
- 11:55
Now, once you generate those questions, we can then start measuring them using AutoE, which is the evaluation platform that we released along with, um, this as well too. So just to get to show you how this in, in practice and how it works, it c- use...
- 12:07
It gives you a composite score of four metrics. The first is comprehensiveness Diversity and empowerment, which if you're familiar with our paper, those are the three original ones that we used back in that one, and a new one that we also call relevance.
- 12:19
And so we actually did some comparisons, uh, just to show you how this works on Lazy GraphRAG, which is one of the newer technologies that we've been working on, and we compared it to Vector RAG on 8K, 120K, and million-token context windows, and we had a few takeaways from that.
- 12:33
The first is if you take a look at these charts, any bar that is above the 50% mark, uh, gives, uh... means that Lazy GraphRAG is winning in that benchmark against those specific Vector RAGs.
- 12:43
So Vector RAG blue here is 8K, 120K, and a million-token context windows, and it's winning at, correspondingly at 92%, 90 and 91% of the time against data local questions, which is kind of a surprise because one of the first things that we started seeing is that Lazy GraphRAG was actually prov- providing dominant performance, um, across the entire
- 13:01
span of questions, whether they were local or global. And we do expect RAG to be performing better on local questions, and you can see that from the fact that data local and the data global, there's a little bit of a lift between these bars between the global and the, and the local ones.
- 13:15
Second thing we noticed is that the long context window didn't really make much of a difference. We were expecting that maybe the long context windows might give you a little bit of a, um...
- 13:23
'cause you have a better understanding towards those global types of questions. But it actually turns out that Lazy GraphRAG in this case was still able to dominate those metrics, and in fact when it, when we ran the test, it's not on the slide here, we actually found that Lazy GraphRAG was a tenth of the cost of what
- 13:37
we saw in the one-million-token context windows as well too. So those are a couple things to show there. Now, uh, with the last minute that I have, I did want to address another thing too with Lazy GraphRAG.
- 13:48
Um, you may have read the blog posts about it in November last year when we first discussed it. Um, it is now, uh, officially being lined up for launch in a couple products.
- 13:56
Uh, the first is Azure Local, and so, uh, it will... they just announced at Build that, uh, Lazy GraphRAG will soon be incorporated into their platform for that, so you can actually try it out for yourselves there.
- 14:06
And it's also being incorporated into the recently announced at Build Microsoft Discovery, uh, platform tool as well. So if you're not familiar with Microsoft Discovery, it does graph-based scientific co-reasoning, and just to give you a quick, uh, summary of it, it goes from hypothesis to experiment, learning, and knowledge.
- 14:23
I'll just play a quick like five-second video here, and the questions... the, the answers to the questions that you start coming back to with on this are powered underneath the hood by GraphRAG and Lazy GraphRAG as well too.
- 14:32
So as you can see, the copilot here is generating the deep reasoning over graph-based scientific knowledge. Those graphs are being powered by, uh, um, GraphRAG and Lazy GraphRAG. So with that, I just wanna leave you with a couple takeaways, and the first is just that LLM memory structure, uh, [laughs] LLM memory with structure is just a really, really
- 14:51
powerful tool, uh, to keep in your tool belt. Um, and the agents, of course, can massively amplify this power. And if you have any questions, I'll be outside if you, uh, want to talk about any of this further.
- 15:01
Thank you so much for your time. [outro music]