AI Engineer Code 2025
Stop Renting Your AI's Memory — Dylan Couzon, Qdrant
Loading slides…
Read the talk
Stop Renting Your AI’s Memory
Dylan Couzon explains how local, searchable memory can give an assistant continuity across sessions—and demonstrates the mechanism with an offline drone-memory application built on Qdrant Edge.
From a talk by Dylan Couzon
At a glance
Ideas worth remembering
Owning inference supplies autonomy; owning persistent memory supplies continuity across conversations.
Memory needs write, retrieve and forget operations. Retrieval makes topic selection and changing relevance explicit instead of placing the entire record in every prompt.
The offline drone example turns detected pictures and labels into stored embeddings, then retrieves coffee-table sightings with images, timestamps and counts.
The proposed memory portability allows changing the reasoning model or device while retaining the same embedding model.
Shared memory should be a deliberate extension of separate local stores, with synchronization opt-in and users choosing who receives their memories.
A capable model still starts as a stranger
What happens to every preference, correction and dead end you teach an assistant when the conversation ends? Dylan Couzon, a developer relations engineer at Qdrant, opens with that practical frustration. An assistant can reason well during a session yet fail to carry the experience into the next one. His phrase for this combination is “geniuses with no long-term memory.” The missing capability is a persistent record that the assistant can use again.
Local inference makes that omission more conspicuous. Couzon describes a machine costing under $2,500 as capable of running roughly the previous year’s frontier, while explicitly stopping short of claiming the absolute frontier. This is his characterization of capability, rather than a benchmark comparison. Even accepting it, putting a capable model on a desk does not give it knowledge of its owner. It remains a “brilliant stranger” until preferences and experience survive between conversations.
That creates a second ownership question beyond where inference runs. Compute, model access, the software that runs an agent and the accumulated personal record can all depend on a provider. The last dependency grows with use: years of corrections and context make an assistant useful in ways that a fresh copy of its model cannot reproduce. Couzon frames the prospect of superintelligence around who will control that accumulated record.
Inference gives autonomy; memory gives continuity
Rented inference introduces several dependencies that become especially visible when software runs long agent loops:
- Access: a provider can withdraw access to a hosted model. Locally held weights remove that particular remote access dependency.
- Version stability: providers can retire versions or throttle service. A local copy of the weights stays fixed until its owner changes it.
- Economics: an agent working for hours can consume far more tokens than a short chat. Couzon cites developers using over $5,000 a month of compute through a $200 plan and predicts that such subsidies will disappear; that forecast is an argument for controlling compute costs, rather than a demonstrated pricing outcome.
Owning compute and weights addresses those inference dependencies. Memory solves a different problem: continuity. The next session needs access to what earlier sessions learned. A memory feature supplies that connection without requiring the underlying model to become more capable. An assistant can feel increasingly personal because it retrieves your corrections and preferences, even while its weights stay unchanged.
Write, retrieve and forget
Memory has three operations: write, retrieve and forget. Writing preserves something worth carrying forward. Retrieval selects what matters for the present task. Forgetting prevents every old observation from retaining equal importance forever. This changes the design question from how much text will fit in a prompt to which stored information should reach the model now.
A folder of Markdown files can hold a record, but dumping its contents into the prompt leaves selection to the model. An indexed retrieval system makes selection explicit. Topic filters can restrict the search; recency and frequency can influence relevance; older information can lose weight as circumstances change. Couzon names these as controls a memory system should support. He does not develop a particular deletion or decay algorithm in this recording.
The CPU, RAM and disk analogy separates the responsibilities. The model performs the computation. The context window holds the information available during the current interaction. Persistent memory keeps a record across interactions. A larger context window expands the temporary workspace; it does not by itself decide what to save, how to find it later or who controls the saved record.
The talk leaves the choice of the best memory architecture open. Its narrower design position is that the persistent record should belong to its user. Local embeddings and vector-search infrastructure provide one way to make that record searchable without placing it in a provider’s data center.
Put the searchable store inside the application
Qdrant Edge is the implementation used here: a vector-search engine embedded directly in an application. It opens a local store inside the application’s process and writes embeddings alongside payloads. The embeddings support similarity search; the payloads retain the information attached to each memory. Queries can run offline, while each application keeps its own isolated store.
The store persists after the application closes. Its folder becomes the object that can follow the user to another device. Couzon also says quantization can fit a million memories into less than a gigabyte, making local storage plausible on a phone or Raspberry Pi. The recording does not specify the vector dimensions, payload sizes or quantization settings behind that capacity claim, so it should be read as a configuration-dependent estimate.
From a first sighting to a searchable coffee table
The demonstration starts with an empty memory and recorded drone footage of a house. The memory application runs live on Couzon’s laptop, fully offline. YOLO, an object-detection model, identifies objects and produces labels. The application turns the pictures and labels into embeddings, then stores those representations as observations accumulate. The drone example therefore begins with no remembered objects and gains a searchable history as the footage plays.
At the reported point in the demo, the application has recognized 92 different objects and holds about 300 vectors. Multiple representations can belong to the same object; vector count is not object count. A two-dimensional view places similar memories near one another, giving a spatial view of the stored representations. The reported footprint for the memory and Qdrant engine together is 15 MB.
Selecting “coffee table” changes the interface from an accumulating memory to a set of retrieved sightings. Couzon reports that the query returns every coffee table previously seen in less than one millisecond. Each result carries an image, first-seen and last-seen timestamps, a definition and a sighting count. Those fields make the result useful: the application can show both what it remembered and when it encountered it. The timing describes this demo’s retrieval, not the latency of the complete detection-and-embedding pipeline or an arbitrary larger collection.
How does a passing image become something the application can recall? The flow below follows the coffee-table example from footage through detection and storage to retrieval. The important separation is between creating memories and querying existing ones: detection and embedding build the record, while search brings selected observations back with their associated details. The whole demonstrated path runs locally without a network connection.
How the application remembers a coffee table
Read the diagram as text
- Recorded drone footage. Write — The application starts with an empty memory and processes footage locally.
- YOLO detection. Write — Identify objects and produce labels.
- Pictures and labels become embeddings. Write — Represent observations for similarity search.
- Local memory store. Keep vectors with associated observation details.
- Select coffee table. Retrieve — Search the memories already collected.
- Remembered sightings. Retrieve — Return images, first/last-seen timestamps, definitions and sighting counts.
- Recorded drone footage → YOLO detection: Process the footage.
- YOLO detection → Pictures and labels become embeddings: Use the picture and label.
- Pictures and labels become embeddings → Local memory store: Accumulate memories.
- Select coffee table → Local memory store: Semantic search.
- Local memory store → Remembered sightings: Retrieve matching observations.
Carry experience between agents, models and devices
Cloud synchronization adds a shared-memory option to the local store. A swarm of drones or robots could contribute observations to a “hive mind,” allowing other systems to benefit when one learns something. Local retrieval provides each device’s own memory; synchronization makes selected experience available beyond that device.
For a chatbot, agent or code assistant, the remembered observations become corrections, preferences and unsuccessful approaches. Retrieving them in a later task can save the user from repeating a correction or the agent from exploring the same dead end. This is the compounding mechanism behind Couzon’s claim that a merely good model can feel increasingly capable: useful experience remains available even when the reasoning model itself has not changed.
Portability has an important condition. The reasoning model and hardware can change while the memory folder moves with the user, but Couzon explicitly requires keeping the same embedding model. The stored representations depend on that model. His portability claim therefore covers changing the reasoner or device while preserving the embedding setup; it does not establish automatic portability across different embedding models.
Extend retrieval from objects to a day
The next step extends the drone’s visual memory into everyday questions:
- Finding an object: “Where did I leave my badge?” asks for a remembered observation associated with a place.
- Recalling a detail: a daughter’s name or whiteboard contents asks for information encountered earlier.
- Recovering an association: the song playing when two people met connects an event with something heard.
Smart glasses supply the proposed route from occasional application memory to a record of what someone sees and hears. The drone demo establishes the object-memory example; the whole-day assistant is an extension of that idea, rather than another demonstrated system. Its usefulness would come from preserving and retrieving experiences that a person would otherwise forget.
This extension raises the stakes of ownership. A record that includes relationships, habits and encounters becomes an index of a person’s life. Couzon warns that AI providers are already building increasingly personal records. His proposed response is to let the user keep and control that index, rather than depend on continued access to a provider’s account.
Shared memory should begin with a choice
Owning memory still allows sharing it. Couzon describes Edge synchronizing selected parts of a local store to Qdrant Cloud when the user chooses. His family example starts with separate records of each person’s day, then makes chosen memories searchable together. The same pattern could serve a team: individual experience contributes to a shared record without making every local memory shared by default.
The same transfer mechanism can support consent or extraction without consent. That is why the ending insists on local storage first and opt-in sharing. The substantive decision is who gets access to someone’s continuity: which experiences leave the local store, and which remain private. Cloud synchronization is a capability; permission to synchronize is a separate choice.
Couzon closes by asking the audience to give an assistant memory and imagine that record five years later. His prediction that superintelligence will arrive within the next few years remains a forecast. The immediate architectural choice is concrete: preserve experience in a store the user controls, keep it usable as models and devices change, and let the user decide what to share. The enduring value is the accumulated relationship between the assistant and the life it remembers.
Resources
Related talks
- Beating RL With Reflection: GEPA and Optimize Anything
Explores a complementary way to carry experience forward: converting agent trajectories into reusable repository skills that reduce repeated discovery and can transfer between models.
Full transcript
Loading transcript…