Stop Using RAG as Memory

What should a media assistant remember about its user? The answer depends on what the assistant is supposed to do. Memory needs a business domain: a useful representation for a media player will differ from one for a financial coach.

Graphiti, Zep’s open-source temporal graph framework, supports custom entities and edges that represent a domain’s business objects. Daniel Chalef demonstrates how Zep exposes that approach to developers working in Python, TypeScript or Go. The developer supplies the kinds of things the application needs to remember, rather than treating every extracted fact as equally useful memory.

Chalef starts with a relevance problem he has observed in ChatGPT: memory can surface arbitrary personal facts, contributing to inaccurate responses or hallucinations. His media-assistant example makes the problem concrete. Jazz, NPR podcasts and The Daily are useful listening preferences. But conversation—and potentially incidental speech picked up by a voice agent—also supplies facts such as waking up at 7 AM and having a dog named Melody. Those facts describe the same person without necessarily helping the media player perform its job.

The failure follows from a straightforward memory pipeline: extract facts, store them in a vector database or Redis, and retrieve facts that are semantically similar to the next request. That pipeline gives retrieval a similarity criterion, but no explicit account of which facts belong to the application’s domain.

In Chalef’s illustration, asking for favorite tunes retrieves useful listening preferences alongside the dog named Melody. The word has a musical association; the fact concerns a pet. A plausible semantic match has crossed the application’s relevance boundary.

Semantic similarity is not business relevance. Chalef describes vectors as projections into an embedding space: proximity does not itself establish an explicit causal or relational connection between the represented facts. Improving similarity ranking alone does not supply the missing domain model. The proposed remedy is domain-aware memory, with structure that lets the application distinguish the kinds of information it needs. Semantic search can still serve retrieval within those boundaries.

The recorded demonstration switches to a finance coach. The user wants to buy a house, so the assistant asks about annual earnings and student-loan debt. Beside the conversation, Zep’s memory contains explicit business objects: financial goals, debts and income sources. These are developer-defined categories, giving the application a vocabulary for the information its advice depends on.

Developers can express these schemas using Pydantic, Zod or Go structs. Chalef opens a TypeScript financial-goal schema built with Zep’s SDK, then shows how entity descriptions and field descriptions express what should be extracted and what values fields should contain. These descriptions guide extraction; describing a business rule is distinct from enforcing it through application validation.

For a house-purchase goal, a small Zod schema illustrates that separation. Descriptions explain the fields’ meaning, while .positive() supplies a concrete validation constraint when the application parses a value:

import { z } from "zod";

const FinancialGoal = z.object({
  description: z.string().describe(
    "The financial objective, such as buying a house"
  ),
  targetAmount: z.number().positive().optional().describe(
    "The amount needed for the goal, when stated by the user"
  ),
}).describe("A financial objective the user wants to achieve");

type FinancialGoal = z.infer<typeof FinancialGoal>;

const goal: FinancialGoal = FinancialGoal.parse({
  description: "Buy a house",
});

The optional amount allows the goal to exist before the user supplies a target. The demonstrated schema view extends this descriptive approach to assets and income sources, including income type, monthly amount and reliability.

With typed objects available, the agent’s financial-snapshot tool runs multiple Zep searches concurrently and filters each search by a specific node type. The displayed code searches for financial goals, expense categories and debt accounts. Each search has a defined category of eligible results instead of relying on one broad similarity query to separate every kind of fact.

Concurrency and filtering solve different problems. Concurrency lets the independent searches proceed together; node-type filters narrow what each search can return. An agent can then assemble the separate results into a financial snapshot while preserving their meanings as goals, expenses and debts.

The next code view supplies the setup that makes those types available: at application startup, the developer registers the business objects with Zep so it knows which ontology to build in the graph. Schema definitions establish the vocabulary; registration makes that vocabulary available to the memory system; typed retrieval uses it to select context for the agent.

Chalef adds a monthly amount of $5,000, describing it as rent and then qualifying that with “I think it’s rent.” In the prerecorded demonstration, Chalef reports that Zep parses the new message and captures the $5,000 amount within a few seconds. The visible result is a memory panel whose Expenses section shows a monthly amount of $5,000 beside the conversation. That timing describes this demonstration, with no measured workload or independent latency test supplied.

He then opens Zep’s front end to inspect the user’s knowledge graph. It contains a debt-account entity with developer-defined fields. This graph view demonstrates the structured representation; it does not establish that the debt account is the newly entered rent expense.

The practical endpoint is control over retrieval: once memory has explicit types and fields, the application can filter tightly for the objects a task needs. Chalef closes by pointing viewers to a paper explaining the system. The demonstration ends with the full connection in place—from a conversational message, to structured memory, to selective retrieval for the next agent action.