AI Engineer Europe 2026
How We Solved Context Management in Agents — Sally-Ann DeLucia
Read the talk
Keeping an Agent’s Context Small Without Losing Its Memory
Alyx’s evolution from blunt truncation to retrievable context and isolated search agents shows how context management shapes reasoning, evaluation, and product behavior.
From a talk by Sally-Ann DeLucia
Before you start: Familiarity with LLM context windows, tool calls, and observability traces will help; a span records an individual operation within a trace.
What should an agent remember?
What should an agent remember, and what can it safely forget? For Alyx, an AI harness for building AI applications, that question became a practical constraint on the product. Sally-Ann DeLucia introduces herself as Arize’s head of product and a hands-on Alyx contributor, describing nearly a year of work on the agent. At the time of the talk, she describes Alyx as having advanced planning and more than forty skills, spanning prompt optimization, data generation, augmentation, and annotation.
The shift from prompt engineering toward context engineering, illustrated by an Andrej Karpathy post in the presentation, changes the engineering question. Writing good instructions is only part of the job; the system must also choose which information accompanies those instructions. Context management means selecting what the model sees, not merely filling its token window. A useful strategy preserves what the agent needs and removes what it does not.
Alyx operates on Arize’s observability data. A single trace already contains user input, prompts, and metadata; the user’s interaction with Alyx adds still more context. Asking for patterns across traces multiplies that burden. The product therefore needs both selective visibility and a way to work over data too large to show at once. Missing the right evidence produces bad answers, making context selection a product and UX problem as well as an engineering problem.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
When debugging adds to the failure
The team built Alyx using Alyx: if the agent could help them develop their own application, they would have evidence that it could help their users. But analyzing its own trace and span data created a feedback loop:
- Alyx analyzed a trace containing its previous activity.
- Growing spans pushed the analysis beyond the context limit.
- Alyx failed, and the failure became additional recorded data.
- A retry encountered the larger record and failed again.
The system meant to explain the data was constrained by the size of that same data. Retrying did not remove the cause; it added to it.
Escaping the loop required three related changes: control the active context, separate that context from memory while designing both together, and move heavy work out of the main agent. The first experiments addressed what to keep in the window.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Truncation and summarization lose different things
The first approach kept the first 100 characters and discarded the rest. It appeared adequate for simple interactions, but follow-ups exposed what had disappeared. Alyx might answer a question about the most common inputs, then fail to understand a request for more detail about input B. The follow-up depended on a reference established earlier; without that information, it looked like a new conversation. The problem was not simply a shorter answer or less detail. Over-truncation had removed information needed for reasoning across turns.
Next came LLM summarization: compress the context into fewer tokens and pass the summary forward. That sounded like the natural solution, but the team found the retention of important information inconsistent. The summarizer decided what mattered without enough control from the surrounding system.
| Approach | What remains | Failure observed in Alyx |
|---|---|---|
| Keep only the beginning | An initial excerpt | Follow-up references lose their meaning |
| Summarize the context | Information selected by an LLM | Important details survive inconsistently |
These were failures of the team’s particular strategies. Their summarization experiment did not establish that summarization is unsuitable for every agent; it showed that uncontrolled compression was not reliable enough for this workload.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Remove information from view without destroying it
The deployed approach combined truncation with retrievable memory. In DeLucia’s description, Alyx retains the first 100 and last 100 characters, removes the middle from active context, and stores the omitted material. These are the excerpt boundaries described in the talk, not a universal input budget. The crucial change is that removal from the model’s current view no longer means permanent loss.
A small TypeScript function makes that boundary operation concrete. It returns the visible excerpt separately from the middle that must be stored; short inputs remain intact.
typescript
type ContextExcerpt = {
visible: string;
omitted: string | null;
};
function splitContext(text: string): ContextExcerpt {
const characters = Array.from(text);
const boundary = 100;
if (characters.length <= boundary * 2) {
return { visible: text, omitted: null };
}
const head = characters.slice(0, boundary).join("");
const middle = characters.slice(boundary, -boundary).join("");
const tail = characters.slice(-boundary).join("");
return {
visible: `${head}\n[Middle stored for retrieval]\n${tail}`,
omitted: middle,
};
}
This separation expresses the core contract: visible is the material for active context, while omitted is material to preserve outside it. Retrieval must remain possible before the full text can be removed from the active conversation.
The surrounding policy matters as much as the slicing operation. Alyx handles duplicate messages and lengthy tool output by retaining the latest result, preserves the system prompt, and can retrieve earlier tool results or conversation messages when they become relevant. DeLucia reports that this combination had worked well enough to remain unchanged for several months, although the team was beginning to revisit it. Her distinction is precise: “context decides what the model sees, memory decides what survives.” The model does not need every retained fact in every request, but it needs a path back to the facts it later discovers it requires.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Test the turn where forgetting appears
Users often keep one chat open while moving between pages in the application. That behavior exposed a weakness that short interactions had hidden: smart truncation could appear successful early in a session, yet Alyx would forget something much later. The team initially discovered these failures through user reports or manual inspection of conversation data.
Their long-session evaluation procedure loads ten conversation turns and tests the eleventh. This turns late-session forgetting into something the team can deliberately exercise instead of waiting for a complaint. The preload establishes the conversation state; the next turn probes whether the agent can still use the context it needs. It is an evaluation procedure, not a claim that eleven turns is the system’s limit or that every such test succeeds. Evaluation checks the consequences of context selection; it does not replace the selection mechanism.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep search’s working data outside the main conversation
Even with truncation, memory, and evaluations, some tasks still produced too much data for one agent. Search over Arize data made the problem especially clear. A trace can contain hundreds of spans, and locating the relevant evidence may require multiple queries, large results, and intermediate reasoning. Those steps help perform the search, but they do not all need to remain in the user’s main conversation.
The team moved that heavy work into sub-agents. Previously, chat history and data search occupied one agent’s context. After the split, the main agent kept the conversation and light context, delegated the search, and received its result. The search sub-agent held the heavy task data.
| Context owner | Before | After |
|---|---|---|
| Main agent | Chat history and heavy search data | Chat, light context, returned result |
| Search sub-agent | No separate context | Queries, heavy data, search work |
The result handoff is the boundary: the main conversation receives what it needs to continue without inheriting the entire search process.
Memory retrieval remains available when more detail is needed. This makes delegation compatible with later follow-ups: the result can be concise without making supporting context permanently inaccessible. DeLucia describes the split as a major improvement and reports that the team subsequently rolled out many sub-agents for data-intensive operations.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The remaining limits: size, memory, and selection
Truncation plus memory worked better for this team than its summarization attempt, but very large prompts and inputs could still hit provider limits. Alyx has an unusual source of pressure: other agents’ system prompts, user messages, and conversation histories are themselves the data its users want analyzed. As those applications accumulate more context, Alyx inherits a larger analysis problem. The team keeps returning to sub-agent decomposition, while continuing to investigate how far that strategy can take them.
Longer workflows also raised a separate memory problem. DeLucia observed conversations growing from fewer than ten turns to twenty-plus as users moved across application pages and asked more questions. This is a usage observation, not a measured success rate. At talk time, Alyx’s memory store preserved retrievable conversation context, but it did not provide the long-term memory needed to recall previously discussed issues in a new chat. That capability was still being developed.
Selection within a session remained heuristic too. Keeping the first and last hundred characters does not establish that those are the most useful characters. The team did not yet have a principled context budget or clear metrics for context quality; it used task evaluations as an indirect signal and was researching more sophisticated selection. DeLucia also reports seeing a similar truncation and compression strategy in publicly exposed Claude Code code. That comparison was her observation, not a demonstrated equivalence between the systems.
The resulting engineering practice is iterative: choose context, preserve recoverable memory, and evaluate whether the agent can still do its work. DeLucia closes by emphasizing all three and describing the team’s shift from concentrating on prompts to concentrating on context. Her claim that agents fail because of context captures what repeatedly broke Alyx; it should be read as the lesson of these failures, not an exclusive explanation for every agent failure.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Database retrieval does not settle cache preservation
The audience’s final question introduces another constraint: can context management preserve the prompt cache? The questioner points to effort reportedly visible in the exposed Claude Code implementation to avoid invalidating cached context. DeLucia does not claim Alyx has solved that problem. The team had done relatively little cache work and was prioritizing long-term memory.
Her answer does make the existing retrieval mechanism more concrete. Stored material lives in a database with IDs. Alyx has a tool that exposes those IDs, where the material sits in the conversation, message-count information, and a preview. These give the agent enough information to locate and request earlier context without carrying all of it in the active window. They explain retrieval, not a cache-preservation strategy.
More sophisticated handling remained on the roadmap, but current retrieval was working. Long-term memory was receiving priority because that was where DeLucia was hearing the most complaints. The next investment therefore followed the product’s observed failure: users wanted continuity across conversations, beyond the ability to recover omitted material within one.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
The Alyx team's account of context limits, stored data references, conversation evaluations and sub-agent isolation.
Introduces Alyx's planning approach and workflows for prompts, datasets, evaluations and trace debugging.
Concrete examples of searching dense traces, aggregating failures and using Alyx in its own development.
Anthropic explains context selection, conversation compaction, external notes and isolated sub-agent contexts.
Updates since the talk
Current instructions for opening Alyx, adding context, configuring settings and using its Arize AX workflows.
- How Claude Code uses prompt cachingDocumentation
Explains prompt-prefix reuse and how changing earlier request content affects cache hits.
Read the complete timestamped transcript
- 0:00
[upbeat music] All right, welcome.
- 0:15
Thanks so much for coming today. Um, I'm here to talk a little bit about context windows, and I'm really excited because I get to talk about something that my team and I have been building for honestly close to a year now, uh, which is RAIA Agent Alyx.
- 0:28
Um, so I'm gonna talk a little bit about some of the lessons we learned about context management and, uh, escaping the context window. So who am I? I'm Sally-Ann.
- 0:36
I am the head of product at Arize. I have a technical background. I started out in data science, and now I build products for teams. Um, I'm hands-on. I'm a core contributor of Alyx.
- 0:45
I'm not only a PM, but I also function a little bit as a part-time AI engineer as well. So I know the pain of building these products firsthand, and it is not easy to build a successful agent.
- 0:54
Uh, my job today really is to turn those pains into tools that may actually help AI eng and AI PMs.
- 1:01
I'm gonna talk a little bit about Alyx. I don't wanna spend a lot of time on Alyx. If you wanna know more about what we built, come find me in the booth downstairs.
- 1:07
I'll give you a demo, but basically what Alyx is is an AI harness. It's here to help you build your AI applications. We have advanced planning, forty-plus skills built into it, core workflows across prompt engineering, like prompt optimization, data gen, data augmentation, annotations, et cetera.
- 1:24
Uh, that's just a screenshot from our product, but yeah, come find me if you'd like a demo.
- 1:29
For today's talk, I'm gonna talk a little bit about the problem of context engineering, context management, tell you a little bit about a vicious loop that we got stuck in, how we escaped that loop, and then how long conversations can break agents, a little bit about what we learned about sub-agents, and then I'll tell you a little
- 1:43
bit about what we're still working on 'cause we certainly haven't figured everything out. So the problem, I think like mid last year, this term context engineering started to become more and more popular.
- 1:52
This is an X from Andrej Karpathy about plus one-ing context engineering over prompt engineering. I think very early on everybody was really, really focused on the prompts, but we started to realize that the context is what really made an agent fail or succeed.
- 2:06
Um, and so the stack has really changed. We're no longer really focused just on the prompts. We're focused on the new engineering problem, which is context. So my little perspective is the best context strategy is one that lets your agents remember what it needs to, um, and forget what it doesn't.
- 2:21
Hope we get a little bit of that. Thank you. And so we're gonna talk a little bit about how you do that, but first, let's talk about why context management even matters.
- 2:28
So I think a lot of folks think, um, like context management is just like what fits in the window, but context engineering is really choosing strategically what the model sees.
- 2:37
It's really important that you think about what the data is that is most important and not just think about, "Oh, I only have X amount of tokens. Let's shove as much as I can in there and see how it does."
- 2:45
So it's not just saying under that token limit, it's being strategic about it, and that's why it really matters. All these different applications, a lot of the times it's running on top of your context, and so what you choose to let the model see really matters.
- 2:58
It can make or break the experience there. And so our reality with Alyx is Alyx is built on, on top of Arize, which is our observability platform, so we have to deal with all of the traces that come with AI agents.
- 3:09
And so when we have one trace, we are getting, you know, the input from the user, there's prompts, there's all of this metadata. Then the user is interacting with Alyx, and so it becomes really large, and that's just when we're talking about one trace.
- 3:19
But what happens when they wanna see patterns across all of their traces? Well, this just continues to multiply and multiply and multiply. Um, so being strategic about context was a non-negotiable for us.
- 3:29
We really had to figure out, okay, what was most important for Alyx to see, and how do we handle when it needs to kind of see everything? Um, and so this is the problem that we really aim to solve.
- 3:39
And so I, as a product person, like to say that context management is, is a product and a UX problem, not just an engineering one. It's certainly one that the engineers are going to try to solve.
- 3:47
There's gonna be a lot of different strategies that people try, but ultimately it comes back to the product and the UX because, uh, if an agent doesn't have the right data, it doesn't have the right context, it's going to give bad answers, and if you give bad answers, nobody's gonna wanna use your product, right?
- 4:00
Um, and so that's why it really becomes a, a product problem and not just an engineering one.
- 4:06
And this is the vicious loop that we got stuck in. So when we were building Alyx, basically what we decided to do is like if we can build an agent that makes our lives easier in building our application, we'll know we'll have something that our users really want to use.
- 4:17
And so we built Alyx using Alyx, and this is the vicious loop that we kept getting stuck in, where Alyx would run on our tran-- our trace and span data.
- 4:25
The spans would grow. There would be too much data. We'd hit a context limit. Alyx would fail. Then the span has that data, so it would try again. We'd add more data to it.
- 4:32
It would run, and then it would fail. So we kept getting stuck in this loop where our context was growing and growing. We couldn't get Alyx to actually perform on it, and so we knew that we needed to come up with some kind of strategy.
- 4:43
So the system analyzing the data was constrained by the data, and that was a major problem for us. Alyx was never going to be able to succeed unless it could understand and take in all of this data.
- 4:52
So how did we solve that? Well, it's kind of a three-part thing, three things that we really learned here to escape this loop. So how to actually control context.
- 5:01
Uh, separating the context from memory, I think there's something that's really important about building them together, but they are kind of separate. And then moving heavy work out of one agent into another.
- 5:09
That was another lesson we learned. So I'm gonna walk through each of these and, and tell you a little bit about how we approached them.
- 5:16
So I think the, the very first thing that came to mind was some very na-naive truncation, where it's like, okay, we have this long, long context blob. Uh, can we just take the beginning of it?
- 5:25
Is just the beginning important? Is that enough information to give Alyx for it to actually perform the analysis that's needed? So we started off just taking the first hundred characters, and then we just dropped the rest.
- 5:34
Pretty, pretty naive. Um, and it worked until it didn't. [laughs] So, um, in the beginning, it seemed like for really simple things that this would work out, but the agent ultimately just forgot everything.
- 5:45
Uh, follow-ups looked like new conversations. If I st- asked one question, Alyx would respond, and then I said, you know, ask a follow-up like, you know, "What are the most common inputs?"
- 5:54
"Okay, these are the most common inputs." "Okay, can you tell me a little bit more about input B?" It didn't understand what I was talking about. So we learned pretty quickly that this was not going to be successful, so we needed to start considering some other options.
- 6:07
Um, so the, the The, the main takeaway from this was that over-truncation had broke the reasoning. It couldn't remember.
- 6:14
We then thought, okay, well, summarization, we have all these LLMs, they're pretty good at summarizing. Can we just summarize all the context into a shorter, um, amount of tokens so that we can send that to the LLM and have, um, a better result?
- 6:26
And that really sounded like the obvious solution, but it was too inconsistent, where there was no control over what was important. You know, we're just leaving it to the LLM to look at the data, decide what to do with it, um, and that was pretty unreliable.
- 6:37
So we learned pretty quickly that summarization was not gonna work either. This was the second thing we, we tried. Um, and so next solution was the smart truncation memory.
- 6:46
This is what we actually use in Alex today. Um, it is kind of this combination of truncation with a little bit, I guess, of, of compression here and storing in memory.
- 6:56
Um, so we take the beginning, still a hundred characters. We also take a hundred off of the tail, um, and then we, we take out the middle, and we basically store that.
- 7:03
So the agent still has access to this. So if there's any duplicate messages, um, tool calls can be really, really long in Alex. It's making a lot, a lot of tool calls.
- 7:12
Um, and so we're keeping the latest result. We don't reset the system prompt, and we truncate the middle, uh, keep the head and the tail, and then at any point, if Alex feels like there's a tool call that was important or a message from the, the previous conversation that's important, it can always go back and grab that
- 7:25
context. And so it gives Alex a little bit more control, uh, over what context is actually important. Um, and we found this to be really, really successful. Um, we haven't had to, to touch this in, in a few months.
- 7:36
We are getting to the point, I'll talk about a little bit later, that we are revisiting our strategy. Uh, but we've, we've found that this combination of truncation and memory has been really, really, uh, successful.
- 7:47
So context decides what the model sees, memory decides what survives, and so this is kind of the system we've built with the, um, smart truncation there. Um, and again, this is working quite well for us.
- 7:57
But we had another problem as we were kind of deciding how to, to handle context management, which is long sessions, and I think that this is something that a lot of people run into, which is users don't usually restart their chats.
- 8:09
Um, uh, you know, I, I think there are different approaches to this. Some people I know, like if you're using Claude or Cursor, you know, you put everything in one chat.
- 8:15
Some people like to have other ones. But we've really learned with Alex, everybody kinda wants to stay in one chat as they're traveling t- pages to pages. So our conversations grow, and our failures appear late.
- 8:25
So, uh, when we first did this munch-- smart truncation, it seemed like it was working, um, but then as we saw these longer and longer conversations, there were failures happening, and we didn't know about them till late, until like a user reported it or I was looking at the data and I realized that Alex kind of started
- 8:39
to forget things way late into the conversation. And so the solution that we came up here is long session evals, and I wanted to include this because, you know, it's maybe not rel- um, related exactly to how you handle context management, but it's a really helpful signal in understanding how your context management is doing, uh, because I
- 8:55
think long sessions are something that naturally happen with these applications. Um, and so what we end up doing is we load ten turns, and then we test the 11th to understand how the context is doing.
- 9:04
And so these bugs really become testable. I don't have to wait till I find it or a user reports it. So, uh, wanted to share a little bit, um, about that.
- 9:12
And even still, with the testability and, um, the, uh, truncation here, um, there is still, you know, too much data sometimes for, for one agent. So, uh, one big realization that we also had is that not all context belongs in the same agent.
- 9:28
Um, so I'm gonna give an example here, our search task. So this is where Alex is trying to search over data in Arize. Um, this happens within like our main chase or even when we're just looking at, you know, one trace stack.
- 9:38
You know, there can be hundreds of spans within it, and Alex needs to figure out what data it should look at. So there's multiple queries happening, tons of data, lots of intermediate reasoning happening step to step.
- 9:48
And we really came to the conclusion that not all this needs to live in the main conversation. So at once, we had one kind of main agent, uh, for our trace's, uh, skills, and we decided that that was not really necessary.
- 10:00
So the solution that we had was sub-agents, and I think this is also something really important when we were talking about context and how we manage across, uh, these agents that need to have a lot of data, uh, which is offload the heavy task.
- 10:11
The main conversation can stay small. Um, so before we had the main conversation, we had chat history, heavy data, searched all in one context. This was all handled in one agent.
- 10:19
And then after, basically what we have is these-- this main agent plus a sub-agent. So we have the main conversation with the chat and light context only. We keep it pretty light.
- 10:27
What it can do is it can delegate to the sub-agents, um, and that's where the heavy data st-stays. So we can keep all the heavy data context in our sub-agent, and then once it gets a result, we can kinda pass that over to the main cha-- uh, main agent again, and then the user can kind of share,
- 10:42
um, or keep the conversation going. And of course, it can always retrieve from the memory store as well if it ever feels like it needs more context there. So I think this is something that was a game changer.
- 10:51
We've, uh, rolled out a lot of sub-agents now that we kinda figured out this is the right way to h-handle all the really, you know, data-intensive operations.
- 11:01
So that's a lot of what we figured out in terms of context management. It's working really well. I, I think I was surprised by-- the most by the fact that summarization didn't work.
- 11:08
I think that was, again, like the obvious choice for us. But, uh, the combination of truncation along with being able to store it in memory, uh, is something that we found to be really successful.
- 11:18
So what are we still figuring out? 'Cause there's quite a lot. Uh, huge context can still break things. I think it's still a problem for us. Very large prompts or inputs still hit provider limits.
- 11:26
So because we're operating-- we have an agent operating on agent data, you can imagine all the system prompts, the user message, the conversation history, that is all what our customers are trying to use Alex to understand.
- 11:36
And so as their context is growing, we have a bigger context issue because we have to figure out how, um, to, to handle that. And the, the pattern we keep returning to is sub-agents, how we just keep breaking things up and having context handled by different parts.
- 11:50
Um, and that is something that we are still, um, kind of learning about and, and seeing if there's anything that we need to do to evolve that strategy even more.
- 12:00
Long memory is still hard. This is something actually that my engineers are working on right now. So long sessions are tricky. We are seeing conversations grow more and more.
- 12:07
I think when we started, I was seeing like less than ten turns per conversation with Alyx, and now I'm seeing folks really go, you know, push the limits up to like twenty plus.
- 12:16
Uh, what's really happening there is they're traveling across our application using Alyx, and so as they're trying to do these longer workflows, they're obviously asking more questions, and they're finding Alyx to be helpful.
- 12:25
But that's a, a problem for us because we need to figure out how, um, to handle this. So I think what we're really focused on right now is like real long-term memory.
- 12:32
It's not something that Alyx has. The memory is really just that c- um, kind of context with the memory store, uh, that Alyx can leverage, so we don't really have long-term memory.
- 12:41
Um, I also think it's important because people want to reference issues that they've previously discussed with Alyx, and so if they're-- they do decide to start a new chat, Alyx really doesn't have context for that.
- 12:50
So, uh, we're in the process of adding long-term memory, and I think that's something that'll be a real game changer for us.
- 12:57
Context selection is also still a heuristic for us. Deciding what context stays in, um, is just that basic, like I said, like first hundred, last hundred. Um, but we do keep like asking ourselves, "Do we keep the right things?"
- 13:10
Uh, we don't really have a principled context budget or clear metrics for context quality yet. We use our evals a lot of the time to measure whether our context was right or not.
- 13:18
Um, but I think there's something a little bit more sophisticated that we're researching to figure out if this is the right heuristic. Um, something that was a little bit surprising to us is, you know, I don't know how many people read here, but the, the [chuckles] the Claude Code code was kind of released for all of us to
- 13:30
read a little bit about. We were surprised that they're using kind of a similar truncation and compression strategy as we are. Uh, and we were kind of hoping to get a little bit of a, a secret from, [chuckles] from them, but I guess we'll all just have to keep, you know, doing our own research there.
- 13:42
Um, and just some takeaways 'cause I want to leave some time for questions. Um, context management is iterative. Uh, we are still learning. I think everybody should continue to learn and lean in and try to optimize their context management.
- 13:56
Um, the few things that I think are really c-clear is that context engineering really does matter, memory matters, and evaluation matters. If you're gonna take anything away from me or us at Arize, it's those three things, and that's, um, from our experience as well as our experience with all of our user base.
- 14:12
And finally, uh, agents don't fail because of prompts, they fail b- uh, because of context. I think that's something that, uh, we've learned firsthand and that we're seeing more and more.
- 14:20
In the early days, prompts were everything. Everybody was focused on prompt engineering. But ourselves and all of our users are really focused now on context engineering, um, and there's a lot of strategies, uh, that can go into that.
- 14:33
If you wanna try out Arize, just wanna give a QR code to try it out. You can try out Alyx, um, our agent. Um, we're downstairs at the booth if you'd like to see a, a demo, especially of Alyx.
- 14:42
I'd be happy to give it. But I wanted to just leave some time to honestly be able to have, you know, some Q&A and answer some questions. [audience applauding]
- 14:58
Yeah, question.
- 14:59
Uh, what were the big things that, uh, caused, uh, that came out-- Hi. Thank you. It was a great talk. Um, one of the things that came out with the Claude Code leak was how much effort they've put into not invalidating the cache with their context management.
- 15:14
Are, are you doing work on that as well, or how are you thinking about that?
- 15:18
Yeah. I think right now we're really trying to focus on the long-term memory stuff. We haven't gone too much into the cache. Basically, we have it saved off in a database with IDs.
- 15:26
And so what Alyx can do is it has a tool where it has all the i-IDs and like where in the conversation it needs to access. So was it early on?
- 15:33
How many messages? And it gets a little bit of a preview. Um, so that's how we've done it right now. I absolutely think we're going to have to get a little bit more sophisticated and invest in that.
- 15:40
Uh, but right now it's working, so we're kind of focused more on the long-term memory 'cause I feel like that's where I'm getting the most complaints.
- 15:47
Yeah. Any other questions? All right. Well, come find me downstairs if you have any questions. Thanks so much for the, the time. [audience applauding] [upbeat music]