Memory Masterclass: Make Your AI Agents Remember What They Do! — Mark Bain, AIUS

How will AI systems communicate with one another—and what will they remember from those interactions? Mark Bain’s workshop approaches that question through a theoretical introduction, four demonstrations, and a GraphRAG chat arena. Vasilije Markovic brings Cognee, Alex Gilmore demonstrates Neo4j memory over MCP, and Daniel Chalef presents Graphiti and Zep. The recording directs participants to a Slack walkthrough for setup.

Bain traces the question back to mathematics competitions, where he knew Wojciech Zaremba, and a September 2014 encounter with Zaremba and Ilya Sutskever during his Stanford studies. He recalls being skeptical about the advantage GPUs would give deep learning. At the end of their conversation about an approaching AI revolution, he asked how those systems would communicate. In Bain’s recollection, Sutskever answered: “I don't know. I think they will invent their own language.” Years later, studying attention, diffusion models, and variational autoencoders brought Bain back to what he considered a missing component: memory.

Bain begins with an unusually expansive definition: memory includes data in any format, including code, algorithms, hardware, and the causal changes affecting them. This puts memory beyond a conversation transcript or vector index. In his framing, both a representation and the changes that produced it belong to the memory system.

He develops that framing through biological, cosmological, and quantum analogies, proposing that mathematics, physics, neuroscience, and LLMs share underlying principles. His mathematical reference point is Grigori Perelman’s work on the Poincaré conjecture, for which Perelman declined the million-dollar prize. Bain connects its treatment of geometry and entropy to attention and diffusion, then proposes an identification of curvature with attention and gravity. These are his theoretical connections, presented ahead of planned papers with collaborators he describes as contributors to transformer and VAE methodologies; the workshop does not derive those equivalences.

The next step is a proposed balance between memory and compute, expressed verbally as their product aspiring to a squared imaginary unit circle. Bain associates perfect symmetry with a world in which the asymmetries needed for existence disappear. He describes learned weights as a matrix structure capturing relationships in raw data, and biases as small adaptive shifts. From there he argues that scaling data and compute improves the representation of reality, eventually imagining infinite data eliminating biases. That last step is part of his speculative model, rather than an established consequence of scaling laws.

Bain then imagines the universe as a temporal network database: a graph whose state changes according to rules. Complete predictability would mean knowing every past and future state; he instead invokes fuzziness, heat diffusion, and entropy, while proposing that causal relationships survive those changes. This supplies the bridge to the practical topic: a memory system can retain explicit relationships as well as individual facts.

For Bain, GraphRAG’s distinguishing opportunity is preserving those relationships. He connects that opportunity to reducing hallucinations and improving hypothesis generation and testing in biology and chemistry. To illustrate information’s contextual value, he points to the audience choosing this particular workshop: the same information matters differently to different recipients. He then recalls Zaremba describing hallucination as a form of imagination when memory and compute are insufficient to explore a problem’s combinatorial space. Bain characterizes reasoning as reading asymmetries and causal links, and proposes graph databases as a way to recover structure from fuzzy correlations. A graph can explicitly represent a claimed causal relationship, but storing an edge does not by itself establish causation or eliminate hallucinations.

Markovic’s Cognee demonstration turns that broad framing into a concrete workflow: compare two developers using their GitHub activity and produce a hiring recommendation. Cognee combines graph and vector storage. Its Cognify process converts ingested data into a semantic graph, while CrewAI supplies the agents that search it and make decisions. The pipeline proceeds through GitHub API ingestion, semantic structure creation, and agent analysis. As generation begins, an activity log exposes the graph’s ongoing updates.

The agents read, write, and analyze using preconfigured weights and benchmarks. Markovic reports support for more than 30 data sources at the time of the workshop. He describes custom graphs built from relational or semi-structured data, with memory association layers inspired by cognitive science. The important state transition is that analysis does not end with a response: reports are stored back in the graph, where later searches and other agents can access them. Inspectable nodes retain developer details, commits, pull requests, and repository contributions. The output of one agent becomes input available to another.

The live run reaches preparation of the final hiring report, but Markovic switches to a recording rather than wait. He explains that the demo was finished that morning and CrewAI problems prevented the planned hosted version. In the recorded finish, a green decision node recommends Laszlo, whom Markovic describes as a developer with a PhD in graphs. The node includes reasons, numbers, and benchmarks, but their values are not spoken. This demonstrates a stored decision and its supporting explanation, not measured hiring accuracy. Markovic closes by identifying Cognee as open source; Bain calls the ingestion-and-Cognify transformation semantic memory.

Gilmore’s Neo4j memory MCP server makes graph memory accessible through conversational tools. The demonstrated server focuses on semantic memory; additional memory types are described as under development. Its setup has three parts:

1. Create a Neo4j database, either in Aura or locally through Neo4j Desktop.
2. Configure Claude Desktop’s MCP connection to that instance. In the demonstrated setup, a background process obtains the memory server from PyPI and exposes its tools to the client.
3. Supply a system prompt that directs the assistant to recall relevant memories before an interaction and log new memories afterward.

The database supplies persistence, MCP supplies the tool interface, and the prompt supplies the policy for when to use it.

The example conversation plans an agentic AI memory company. Initially, the store is empty. As the conversation progresses, Claude attempts prompt-related recall, then creates entities and relationships from new information. Each entity has a name, a type, and an updatable list of observations—facts learned about that entity. Relationships connect the entities, retaining context that isolated observations would leave implicit. Further turns add observations and entities rather than merely extending a transcript.

At the review stage, the assistant reads the entire knowledge graph as context and summarizes the proposed company. The selected frame shows the expanded read_graph result beside the resulting business and technical summary. Gilmore then opens Neo4j Browser to inspect entities including Neo4j, MCP, and LangGraph, with their stored observations. The graph was created in one conversation, but it can support subsequent conversations and other clients, including Cursor and Windsurf. Persistence belongs to the shared graph, so it need not end when a particular chat ends.

Bain introduces the next demonstration after describing his own experiments with the Neo4j, Graphiti, Cognee, and Mem0 MCP integrations. He reports approximately 10,000 GitHub stars for Graphiti at the time. Chalef introduces Graphiti as Zep’s open-source temporal graph framework and emphasizes custom entities and edges drawn from the application’s business domain. The Zep implementation he demonstrates supports Python, TypeScript, and Go.

The failure case is a media assistant that remembers jazz, NPR podcasts, and The Daily, but also captures incidental conversation: the user wakes at 7 AM and has a dog named Melody. When asked for favorite tunes, semantic retrieval can return the dog’s name alongside useful listening preferences. Melody is related to music in embedding space, but the dog is not a listening preference.

Chalef’s diagnosis is semantic similarity is not business relevance. Generating arbitrary facts and placing them in a vector database or Redis leaves retrieval without an explicit distinction between the objects the application needs and incidental facts it has collected. Embedding proximity alone does not supply the missing relationship or domain meaning.

Using a recording because of poor Wi-Fi, Chalef demonstrates a finance coach. The user wants to buy a house; the assistant asks about annual income and student-loan debt. Its memory view contains explicit business objects such as financial goals, debts, and income sources. Developers define these types through Pydantic, a Zod-like TypeScript schema interface, or Go structs, including descriptions, fields, and business rules for allowed values.

The useful TypeScript contract is an explicit distinction among the objects a financial snapshot may retrieve. A small application-side model can express that distinction without mixing every remembered fact into the same result type:

type FinancialGoal = {
  type: "financial_goal";
  description: string;
};

type DebtAccount = {
  type: "debt_account";
  description: string;
};

type IncomeSource = {
  type: "income_source";
  description: string;
};

type FinancialMemory = FinancialGoal | DebtAccount | IncomeSource;

type SearchByType = <T extends FinancialMemory["type"]>(
  type: T
) => Promise<Array<Extract<FinancialMemory, { type: T }>>>;

async function financialSnapshot(searchByType: SearchByType) {
  const [goals, debts, income] = await Promise.all([
    searchByType("financial_goal"),
    searchByType("debt_account"),
    searchByType("income_source"),
  ]);

  return { goals, debts, income };
}

This captures the retrieval structure Chalef demonstrates: concurrent searches filtered by node type. His application registers its entity definitions with Zep at startup so the graph can classify objects using that ontology. The workshop demonstrates typed extraction and filtered retrieval; it does not establish that every irrelevant fact is rejected during ingestion. Current strict-ontology controls are a separate capability and should not be read back into this demonstration.

Chalef then enters a $5,000 monthly expense, which he tentatively calls rent. He reports that Zep captures the amount within a few seconds. He opens the user’s graph and points to a debt-account entity with developer-defined fields. The presentation does not make the exact relationship between that expense and the debt-account classification clear. What it does show is the operational value of typed objects: the application can inspect their fields and constrain subsequent retrieval by type. Chalef closes by pointing viewers to a supporting paper.

As workshop time runs short, Bain redirects questions to Slack and turns to a use case from his cybersecurity work. He recalls a defense-client project requiring roughly 27–29 terminals and shells, spanning Linux distributions, proprietary network-device interfaces, and PowerShell. LLMs can translate between those command languages and offer a human-language shell. Bain’s additional requirement is episodic memory: a temporal history of what happened, who did it, and on which machine. That history could inform controls over subsequent code execution, including agent actions. He calls the proposed category an agentic firewall.

The demonstration starts with familiar commands such as pwd and docker ps. After screen sharing is restored, Bain requests container information, narrows it to image and status, asks whether Apache is running, and finally asks for the command from three commands earlier. The last request changes the problem: translating a task into a command is insufficient; the shell must retrieve an event at the right position in its history. Bain acknowledges that this recall appears to be wrong.

His proposed security context spans machines, users, and terminal sessions, including PTYs and TTYs. Temporal logs would connect a proposed execution to prior activity instead of judging it in isolation. Bain predicts that agents executing commands through enterprise gateways will pass through agentic firewalls, narrowing his initial suggestion that all terminal agents would do so. The demonstration motivates that design, but does not establish a functioning security enforcement layer. He then moves to the workshop repository’s GraphRAG arena setup.

Bain recalls Jerry Liu of LlamaIndex advising him to focus on data abstractions and evaluations. He says the abstractions came together in about two months, while evaluation became the harder question: if agents continuously change their memory, how can a fixed benchmark anticipate what that memory will become? Bain argues for a simulation arena, using WebArena as the analogy. His claim that simulation is the only appropriate evaluation is stronger than the prototype establishes; the concrete motivation is to observe memory as it evolves through interaction.

The forked prototype includes Mem0, Graphiti, and Cognee, with two integration approaches to compare:

Neither approach has been declared better. The arena uses ADK for a chat interface that lets the user switch between agents. A Neo agent answers questions about Neo4j; a Cypher agent specializes in graph queries.

Bain switches to Cypher and asks it to add that Mark is passionate about memory architectures. He describes memory layers supplied by the participating frameworks and the Neo4j MCP server, then queries and refreshes the graph to inspect the stored information. The proposed comparison point is one Neo4j backend: different memory technologies can be observed creating and retrieving graph state against that shared store.

Next, Cypher adds information that Vasilije will demonstrate Cognee integration. Bain switches to Graphiti and describes searching the information just added through the other path. This is the arena’s distinctive experiment: agent handoffs can involve persistent shared state, not merely a transfer of chat text. Bain explicitly calls the prototype early stage. It shows the beginnings of cross-tool memory interaction, without reporting comparative evaluation results; the planned audience practice is deferred for lack of time.

The first audience question asks how a system decides that a memory has become bad: a belief that made sense years ago may no longer apply. Bain suggests noise and weak connectivity as warning signals, with redundancy and relationships providing ways to reduce noise. He acknowledges that other validation methods are needed. A poorly connected node may merit investigation, but connectivity is a heuristic rather than a truth test.

Markovic gives a more configurable answer. Cognee lets developers model data with Pydantic and attach weights to nodes and edges. Temporal weighting and custom logic can then change relevance as information ages. The framework supplies the modeling and algorithmic hooks; the application must decide what aging means for its use case. Bain returns to missing causal links as another potential indicator of uncertainty. Memory maintenance requires a policy, not just a place to store facts.

The next question concerns corporate secrets and personal graphs: how can someone share part of a memory without exposing all of it? Bain starts with organizational context—the intentions and authority of decision-makers—and proposes enterprise ontologies as guardrails. He also describes this as a difficult security problem, rather than a solved consequence of graph storage.

The panel then offers two concrete boundaries:

• Configured privileges: Gilmore points to Neo4j’s role-based access controls for preventing access to unauthorized data. These require configured roles and privileges in a supporting edition or service; the workshop does not demonstrate those policies being enforced.
• Separate graphs: Markovic reports good practical results from physically separating graphs per user. Bain endorses independent personal graphs as an isolation option.

These mechanisms answer different parts of the sharing problem: privileges govern access within a deployment, while separation provides an explicit boundary between users’ memory stores.

The final question returns to Bain’s opening equations. Asked to show and explain them again, he explicitly restates:

He says a proper explanation will require a series of papers, then describes attention, diffusion, and VAEs as smoothing processes that preserve asymmetries. He proposes two further equivalences:

Bain says the definitions still need to be aligned, including identifying differences where the proposed equivalences do not hold. The recording supplies neither operational definitions for these equations nor a derivation.

He illustrates the proposal by imagining a sphere being stretched and bent, associating that deformation with increasing curvature and entropy. He connects the analogy to Perelman’s resolution of the three-dimensional case after higher-dimensional cases. The mathematical result concerns the Poincaré conjecture and three-manifold geometrization; it is not a proof that arbitrary sphere deformation increases entropy or constrains AI architectures. Bain nevertheless closes with a prediction that future LLM architectures will remain within attention, diffusion, and VAE-like smoothing, allowing alternatives to VAEs that leave room for biases. That is the workshop’s unresolved research horizon: a proposed unifying theory beyond the demonstrated engineering of persistent graphs, typed retrieval, and shared agent memory.