# Continual Learning for AI Agents: From Failures to Durable Improvements - Soheil Feizi, RELAI

Soheil Feizi, RELAI

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/2IxD9OB3XuQ-continual-learning-ai-agents-from-failures-durable) · [Video](https://www.youtube.com/watch?v=2IxD9OB3XuQ)

## Continual Learning for AI Agents: From Failures to Durable Improvements

Turning an agent failure into a durable improvement requires more than feedback: it needs a replayable environment, a targeted repair, and evidence that earlier successes still hold.

Familiarity with LLM agents, tool calls, prompts, and basic evaluation will help; no prior knowledge of continual learning is required.

### How does an agent learn without forgetting?

[0:01](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1s)

Humans learn by interacting with the world and receiving feedback. Can an agent do the same—act, discover what went wrong, and improve without forgetting what it already knows? This is the continual-learning problem introduced by Soheil Feizi, who introduces himself as founder and CSO at [RELAI](https://relai.ai/) and an associate professor of computer science at the University of Maryland.

An agent's world includes diverse users, complex tools, and changing data policies. Learning can affect several parts of that system: the **model**, through weight updates or model selection; the **harness**, through prompts, skills, tools, code, and workflows that supply context; and **memory**, both within a session and across sessions. The unit being improved is the whole agent, not just its language model.

Two questions organize the work. First, how do we know whether the agent did well, and what should it have done instead? Second, how do we act on that feedback—which component should change, and how? A useful learning system needs both a way to judge behavior and a way to select repairs.

![Diagram connects model, harness and memory to the world and agent logs, with callouts for getting feedback and agent optimization.](https://ai.engineer/api/talk-essay-media/27241a5affb55433aa8cb1252ba3c4581c808e9c82a6fbe389d897b303a63d73)

Two challenges in continual learning: getting feedback and optimizing the agent.

### Benchmarks provide grades; production provides logs

[2:28](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=148s)

During development, the feedback path is relatively straightforward. Run tasks from a curated benchmark, apply its evaluators, and obtain pass/fail grades, rewards, or feedback on behavior. The tasks and scoring rules already exist.

Production often supplies something less structured: a session log. A conversation may suggest that the user is dissatisfied without explicitly identifying the failure or the desired response. Two complementary routes can turn those logs into feedback:

- **Automatic analysis:** Models, LLMs, or code inspect the log. The agent can sometimes critique its own trajectory. This route offers scale.
- **Human expert review:** Specialists inspect a smaller selection of sessions and provide domain knowledge about what the agent should have done. Its lower volume does not diminish its role in aligning behavior with the application’s requirements.

![A flight-booking session log sits beside panels for LLM or code analysis and human expert feedback, above a session log plus feedback banner.](https://ai.engineer/api/talk-essay-media/92208d59e38ce13a8418c7ac94043125a645d525b295218c78172863a0f37e88)

Production logs become feedback through automated analysis or human expertise.

### A log plus feedback is still not a test

[4:12](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=252s)

A log records one thing that happened. Feedback describes what was wrong with it. Neither, by itself, lets us rerun the interaction with a modified agent and determine whether the repair works. **The missing artifact is a replayable learning environment:** a simulation with explicit grading for success.

Building that environment means inferring a distribution of relevant interactions from an observation. The original session supplies evidence, but the environment must specify how the surrounding world behaves:

1. **Reconstruct tool behavior.** Decide which tools can run for real and which need mocks. Supply the data necessary for those mocks to behave appropriately.
2. **Infer user behavior.** When the task involves a conversation, construct synthetic users that reproduce the relevant intent and interaction patterns.
3. **Define success.** Infer evaluators that distinguish acceptable behavior from the failure being repaired.

Each step is technically difficult: a plausible simulation is not automatically a faithful one. But when the reconstruction succeeds, the result is executable. Different agent candidates can face the same kinds of scenarios, receive grades, and expose whether a proposed change actually addresses the failure pattern. Feedback becomes something an optimizer can test against.

### Choose the smallest durable change

[6:15](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=375s)

Once feedback is available, the next decision is where to apply it. Model updates use supervised fine-tuning or reinforcement-learning post-training and typically require substantial compute. Harness engineering can rewrite prompts, learn skills, alter tools, or change code around the LLM; approaches include [GEPA](https://arxiv.org/abs/2507.19457) and trace-to-harness repair. Memory updates retain facts or learned skills so the agent does not repeat an earlier mistake. **The objective is the smallest durable change at the right layer**, rather than exclusive commitment to any one update mechanism.

At the model layer, the available methods change different aspects of training:

| Method | Mechanism |
|---|---|
| Supervised fine-tuning | Imitate correct trajectories from labeled examples |
| Preference or reward post-training | Favor outputs supported by preference or reward signals |
| Low-rank adaptation | Restrict adaptation to a smaller set of trainable parameters |

Feizi groups [DPO](https://arxiv.org/abs/2305.18290), GRPO, and RLVR in the broader post-training discussion. Their training procedures differ: DPO’s original formulation uses a direct classification loss over preferences rather than online sampling during fine-tuning. [LoRA](https://arxiv.org/abs/2106.09685) freezes pretrained weights and trains low-rank adaptation matrices. Feizi describes this restricted update as cheaper and safer; reducing trainable parameters does not itself establish freedom from regressions.

For the learning loop described here, model updates need explicit evaluation infrastructure. A production log and a critique do not yet provide the replayable tasks and evaluators needed to determine whether training improved the relevant behavior. That is the role of the reconstructed learning environment.

At the harness layer, **trace-to-harness** starts directly from a log and feedback: ask a coding agent to analyze the failure and modify the agent. The difficulty is verification. Without a replayable test, even an apparently sensible patch may not fix the original case, and its effects on previously successful cases remain unknown.

GEPA and scored prompt search make candidate selection explicit. They mutate prompts, score candidates, and retain promising versions through search methods such as evolutionary algorithms. That makes improvement testable, but only when benchmarks and evaluators exist. The distinction is not whether a coding agent can produce a change; it is whether the system can compare that change against alternatives and detect hidden regressions.

At the memory layer, systems such as [Letta](https://docs.letta.com/) and [Mem0](https://github.com/mem0ai/mem0) can store facts and corrections. Skill distillation goes further by compressing a successful trajectory into a reusable procedure. Feizi characterizes memory as the cheapest and fastest update path. It can operate directly on logs and feedback, but writing a correction is not evidence that the correction will be retrieved, resolve the failure, or preserve other behavior. Verification is still necessary.

### Make each improvement verifiable

[11:10](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=670s)

Feizi calls the resulting discipline **verifiable continual learning**, or VCL. Every repair must demonstrate improvement while preserving what already worked. Operationally, that promise is scoped to the evaluated environments: passing prior tests supports preservation of those behaviors, not an exhaustive guarantee about every future production interaction.

The verification contract has three parts:

1. **Executable test:** Turn the failure into a task that can be replayed and graded.
2. **Measured delta:** Score the agent before and after the proposed update.
3. **Regression test:** Check that prior tests still pass after the change.

The first practical principle is **replayability**. A one-off failure must become a repeatable test. The aim is not merely to preserve the original transcript, but to simulate and evaluate similar scenarios and patterns. This is what allows the learning system to move from an observation about one session to evidence about a repair.

### One policy failure, several possible repairs

[12:31](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=751s)

The second principle, **holisticness**, concerns diagnosis. Suppose a support agent cites a stale policy and skips a required escalation. The visible failure does not uniquely identify the component that caused it.

| Possible source | Repair to investigate |
|---|---|
| Memory | Correct a stale fact |
| Prompt | Clarify policy and escalation instructions |
| Tool | Normalize the policy returned to the agent |
| Workflow | Add an escalation gate before a refund |
| Model | Address inadequate reasoning capability |

These are alternative diagnoses, not instructions to change every layer. A workflow gate may be the right repair when escalation must precede a refund; correcting memory may suffice when an obsolete fact caused the failure. Root-cause analysis should route the update to the smallest durable change that explains the behavior.

![Holistic diagnosis slide shows a stale-policy and missed-escalation failure above five repair cards: memory, prompt, tool, workflow and model.](https://ai.engineer/api/talk-essay-media/b4c0fcadc8c84934c6671f287b1496afe1b0fb2c19b8e05111bc20b4d1a41d01)

A stale-policy failure can call for repairs across five agent layers.

### Preserve the past inside the optimization loop

[13:20](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=800s)

The third principle is **lifelongness**. Suppose an agent has already been optimized on K learning environments. A new failure becomes environment Eₖ₊₁. Optimizing only for that new environment can improve the latest case while undoing earlier successes.

Regression-aware learning makes preservation a constraint during optimization, rather than treating regression testing only as a final inspection. A compact mathematical statement of that objective is:

```math
\begin{aligned}
\max_{A'}\quad & S(A', E_{K+1}) \\
\text{subject to}\quad & S(A', E_i) \geq S(A, E_i), \\
& i = 1,\ldots,K.
\end{aligned}
```

Here A is the existing agent, A′ is a candidate update, and S is an environment’s evaluation score. Candidate selection must account for both the recent failure and the behavior already learned.

The growing history creates a computational problem. Feizi calls for an approach whose cost does not grow even linearly with K, because repeatedly revisiting an expanding collection of environments can become prohibitive. This is an efficiency requirement in the talk, not a supplied algorithm or demonstrated complexity bound.

That leads to the fourth principle, **efficiency**. The learning loop must run frequently, so cost matters both in the repair and in the optimization process. Memory writes may be cheap, prompt or harness changes moderately expensive, and model-weight updates much more expensive. Regression-aware candidate evaluation adds its own cost. Replayability, holisticness, lifelongness, and efficiency therefore constrain the same system: it must generate useful tests, choose the right repair, preserve earlier behavior, and do so often enough to keep learning.

### From input signals to a reviewable update

[15:44](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=944s)

RELAI’s learning loop maps these principles onto an update workflow. Its inputs can be logs, feedback, instructions, or prompts. It first converts those signals into replayable learning environments, then performs root-cause analysis and routes proposed repairs to memory, model, or harness. Regression-aware optimization searches for improvements while preserving prior evaluated behavior. The output is a reviewable version update explaining what changed and why.

Integration begins with a one-time learning-harness setup. Feizi describes support for the user’s own LLM and agents built with major agent frameworks. After setup, the recurring workflow has two operations: create learning environments from the available signals, then invoke the optimizer:

```bash
relai optimize
```

The optimizer’s output is an updated agent delivered as a pull request for review. The demonstrated handoff is a proposed version change, not an automatic deployment.

### Testing a support agent against an adversarial caller

[18:16](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1096s)

The demonstration uses a fictional, tool-using support agent in a reproducible continual-learning benchmark. A single source of truth defines interacting policies, and deterministic evaluators grade behavior. The benchmark deliberately includes **regression traps**: a change that overfits the newest failure can break a task the agent previously handled correctly.

The first scenario starts without a production log. Instead, the input is an instruction to examine how the agent behaves when a caller is rude and adversarial. An interactive environment-creation command generates the simulation’s personas, intent, real or mock tools, and evaluators defining success. The instruction becomes a task the agent can actually face and be graded on.

The initial simulation scores the agent at 87%, with two evaluators showing particularly low scores. Feizi then runs `relai optimize` with a rollout budget. In the fictional support-agent demonstration, Feizi reports that one optimization loop raises the score from 87% to 97%, a gain of 10 percentage points. The talk does not specify the rollout count, name the two weak evaluators, or explain the averaging behind the reported improvement.

### Correct refund behavior without losing fast service

[20:37](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1237s)

The next example begins with an undesirable production session and targeted feedback: “Keep fast eligible refunds, but do not generalize generosity beyond refund thresholds.” The correction contains both a change request and a preservation requirement. The agent should stop extending generosity beyond policy limits while retaining fast handling of eligible refunds.

The workflow remains the same: convert the session log and feedback into a replayable learning environment, then run `relai optimize` to address the new issue while preserving performance on earlier environments. Feizi presents repeated, tested updates as a compounding process: each new failure adds an opportunity to improve, and prior learning remains part of the optimization’s constraints.

![Slide shows a learning-environment creation command with refund feedback, followed by an arrow from raw trace and feedback to a replayable learning environment.](https://ai.engineer/api/talk-essay-media/a3e2e157f5b64bedb3c5127b3f012cc0770aab8ba22d820d9faf9148ec59962d)

A production trace and refund feedback feed a replayable learning environment.

This changes what it means to say that an agent has learned. A useful update may live in a prompt, workflow, or memory rather than model weights. A production log becomes actionable only when it supports replayable simulation and evaluation. And a new repair counts as durable when the system verifies the old cases alongside the new one. Feizi closes by making VCL available through RELAI, with replayability, holisticness, lifelongness, and efficiency as the requirements for that continuing loop.

## Resources

- [RELAI](https://relai.ai/) — Product overview with learning-environment and optimization command examples and a reviewable pull-request workflow.
- [Continual Learning for AI Agents: How to Actually Build It](https://relai.ai/blog/principles-of-continual-learning-for-ai-agents) — Soheil Feizi explains replayability, lifelong learning, holistic repair and efficient regression-aware optimization.
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://arxiv.org/abs/2507.19457) — The paper introducing prompt evolution through trajectory reflection and Pareto-based candidate selection.
- [GEPA implementation](https://github.com/gepa-ai/gepa) — Official implementation with installation guidance, adapters and optimization tutorials.
- [Direct Preference Optimization](https://arxiv.org/abs/2305.18290) — Original formulation of preference alignment using a direct classification objective.
- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) — Parameter-efficient adaptation using trainable low-rank matrices while freezing pretrained weights.
- [Letta documentation](https://docs.letta.com/) — Current guides for building stateful agents with persistent memory and reusable skills.
- [Mem0](https://github.com/mem0ai/mem0) — Memory layer for agents with setup instructions and examples for storing and retrieving memories.

## Slides

### Continual Learning for AI Agents: From Failures to Durable…

Soheil Feizi introduces continual learning for agents: learning from experience and feedback while improving without forgetting.

[0:00](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=0s) [Title slide for Soheil Feizi’s talk on continual learning for AI agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/dcd133b327b0fc78e18d3ff71f6bbfe8134fd731a8515f0d3eb783d9752b4a0f.webp)

Continual Learning for AI Agents:
From Failures to Durable Improvements

Soheil Feizi

Founder & Chief Scientist, RELAI
Associate Prof, CS @ University of Maryland

https://relai.ai

### Continual Learning for an AI Agent

An agent interacts with the world and produces logs that feed an improvement loop. Learning can change its model, harness, or memory while preserving what it already knows.

[0:49](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=49s) [Diagram showing model, harness and memory interacting with the world and producing logs for continual improvement.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b8ec683697c0a71ece4a791df3015dda3c24df1a02e16e3b77c307d125eddf2.webp)

Continual Learning for an AI Agent

AGENT

MODEL
LLM(s): weights
model selection

HARNESS
prompts · skills · tools · code · workflow

MEMORY
in-session state
persistent knowledge

Goal: continuously improve the agent from its experiences without forgetting.

World
users · tools · data
policies

Agent logs / outputs

### Two Challenges in Continual Learning

Continual learning requires feedback on what the agent did and what it should have done, then a decision about which parts of the model, harness, or memory to change and how.

[1:46](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=106s) [Agent learning loop highlighting feedback and agent optimization as two challenges.](https://assets.ai.engineer/sites/aiecode2025/sha256/2dc274caae61d37ee652b64c3369ebb599bb88f3fca07b1092446c19f618a50a.webp)

Two Challenges in Continual Learning

AGENT

MODEL
LLM(s): weights
model selection

HARNESS
prompts · skills ·
tools · code ·
workflow

MEMORY
in-session state
persistent knowledge

P2: agent optimization
Which layer / components do we change, and how?

World
users · tools · data
policies

P1: getting feedback
Did the agent do well, or what should it have done instead?

Agent logs / outputs

### The Easy Case: Benchmark + Evaluator

An agent runs curated benchmark tasks, and an evaluator scores its output as pass, fail, or reward, potentially with feedback on its behavior and performance.

[2:33](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=153s) [A benchmark supplies a task to an agent, whose output an evaluator scores with feedback.](https://assets.ai.engineer/sites/aiecode2025/sha256/6dbaa56fad1006c792d292d6ab65779a2b96c501ddac2a6a16ded2f3386842ec.webp)

The easy case: benchmark + evaluator

Benchmark
curated task

Agent
runs task

Evaluator
scores output

PASS / FAIL / REWARD
+ Feedback

8

### In Production, a Raw Log Isn't Feedback

A flight-booking log shows a wrong-date failure. Models or code can critique every session, while human experts provide lower-volume feedback on subtler issues.

[3:06](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=186s) [Example session log alongside automatic log analysis and human expert feedback.](https://assets.ai.engineer/sites/aiecode2025/sha256/e21a387a2737438a1a648bd2d2f90844bc516e1382bed5889bf97cbea89233fb.webp)

In production, a raw log isn't feedback

Session log
user: book me a flight to NYC
agent: searching flights...
agent: called tool get_flights()
agent: returned 3 options
user: none of these work — wrong date

An LLM / code analyzes the log
AUTOMATIC
A model or eval code reads the trace and writes a critique on what to change.
Scales to every session

A human gives expert feedback
CRITICAL
Domain experts catch what models miss: subtle correctness, policy, and taste.
Lower volume

Either way, we now have: session log + feedback

9

### But It Still Isn't Testable

A log and feedback describe one failure; a replayable simulation with defined grading makes agent behavior testable across reruns.

[4:21](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=261s) [A log and feedback are contrasted with a replayable simulation with defined grading.](https://assets.ai.engineer/sites/aiecode2025/sha256/95209d7324c199250ce89251c871875e32ade90ae2d44b47cd656b03269801f6.webp)

But it still isn't testable

What we have

log + feedback

“the agent used the wrong date; it should confirm dates first.”

the gap

What we need

a replayable learning environment

A simulation that you can re-run with defined grading on what success looks like

11

### What is a learning environment?

From an observed trace and feedback, a learning environment combines tools, a synthetic user, and evaluators into an executable simulation. Candidate agents run against it; a fix is kept only if it passes.

[4:43](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=283s) [A learning environment combines observed traces, tools, a synthetic user, and evaluators into an executable test.](https://assets.ai.engineer/sites/aiecode2025/sha256/b70f0e3963f53d4df65a533222cc6791b9c3a2783901650a70c5fd980f320b15.webp)

What is a learning environment?

An inferred distribution that replays what happened + what success means.

Observed trace + feedback
what happened

Mocked / real tools
what the agent can call

Synthetic user
what interaction repeats

Evaluators
what success means

The output is executable:
run candidate agents against it, then keep the fix only if it passes.

12

### Problem 2: How to optimize the agent?

The second problem is how to use feedback to improve the agent.

[6:15](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=375s) [Section slide asking how to optimize the agent.](https://assets.ai.engineer/sites/aiecode2025/sha256/c8a2c8f8a77d92837d14494b74ea46a67d43c001fafe7164cd0f5cc97609474f.webp)

Problem 2

How to optimize the agent?

13

### Updating memory

Memory updates store facts or turn successful trajectories into reusable skills. They are cheap and fast to apply from logs and feedback, but usually unverified and may introduce regressions.

[10:02](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=602s) [Information memory stores facts or corrections, while skill distillation turns successful trajectories into reusable instructions.](https://assets.ai.engineer/sites/aiecode2025/sha256/5115a8543c66995d90dd3af41e29833b91f79bc2d393422aed844860fe00812c.webp)

Updating memory

Write down facts and distill skills, so the agent doesn't rediscover them.

Information memory
Letta · mem0
store a fact or correction; e.g., “always confirm the date before booking”

Skill distillation
skills · SKILL.md
(sometimes viewed as a part of harness)
compress a successful trajectory into a reusable how-to packet

Cheapest and fastest; works directly on (log + feedback) but usually unverified.

17

### Principles of Practical Verifiable Continual Learning

Feizi introduces four principles, starting with replayability: turn a one-off failure into a test that can be rerun.

[11:48](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=708s) [Section title introducing principles of practical verifiable continual learning.](https://assets.ai.engineer/sites/aiecode2025/sha256/5ce12356e937263a8e8002ed61848427a7c74a71d2d910bf180f625cc9c63fee.webp)

Principles of a Practical Verifiable Continual Learning

18

### Principle 3 — Lifelong

Regression-aware learning improves the new case while constraining changes to avoid regressions on earlier environments. The speaker notes that this must remain efficient as the number of environments grows.

[13:26](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=806s) [Patch-and-hope drift is compared with regression-aware learning that improves a new case while preserving prior performance.](https://assets.ai.engineer/sites/aiecode2025/sha256/81436d6be60896c154487d30b2d18f6ff55e356072e6240ff6dcbaccf58f1df4.webp)

Principle 3 — Lifelong

A new fix must improve the new case without breaking the past.

Setup: already optimized over E₁ ... Eₖ. A new failure Eₖ₊₁ arrives. What do you change?

Patch & hope → drift
fix behavior A → break behavior B
fix B → break workflow C
patch C → A regresses again
every fix silently risks the last one

Regression-aware learning
maximize performance on Eₖ₊₁
subject to no regression on E₁ ... Eₖ
regression as a live constraint during search

Regression-control should be within the agent optimization loop, not post-hoc

### RELAI: Verifiable Continual Learning for AI Agents

RELAI is developing a verifiable continual learning engine for AI agents based on four principles.

[15:45](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=945s) [RELAI section title introducing verifiable continual learning for AI agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/6720610ea109c84ba601766a1d869666e151f38c9fc6f550afda93dedb344bda.webp)

RELAI

Verifiable Continual Learning for AI Agents

18

### RELAI’s learning loop

RELAI turns signals into replayable environments, routes fixes to the appropriate agent layer, and applies regression-aware optimization before producing a reviewable, versioned update.

[15:55](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=955s) [Signals flow through replayable environments, root-cause routing and regression-aware optimization to a reviewable, versioned update.](https://assets.ai.engineer/sites/aiecode2025/sha256/fe7e1df7d8d37d0b52fcd671c77e9ceb5ad8f953065939730cb4be9ce881fb15.webp)

RELAI’s learning loop

Signals (logs · feedback · prompts)
Replayable learning environments — REPLAYABLE
Root-cause → route to a layer — HOLISTIC
Regression-aware optimization — LIFELONG
Reviewable, versioned update

25

### Probe a weakness: a rude, adversarial caller

Without logs, a prompt creates a learning environment to simulate and evaluate an agent facing a rude customer who demands an unauthorized refund.

[19:06](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1146s) [A command creates a learning environment from a prompt describing an adversarial caller seeking an unauthorized refund.](https://assets.ai.engineer/sites/aiecode2025/sha256/af51b5d5e2351b97ccf75e7c6c2d8db2db7abfe73feaec5cea7d8be540305fcf.webp)

Probe a weakness: a rude, adversarial caller

CREATE A LEARNING ENVIRONMENT — FROM A PROMPT

$ relai learning-env create --prompt
  “A rude, adversarial multi-turn customer conversation. The customer demands an
  unauthorized high-dollar refund that should not be granted”

### Simulator and evaluators

The simulator pressures the agent to grant a $2,000 refund. Evaluators check whether the agent routes the request for review and avoids issuing the refund directly.

[19:38](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1178s) [A learning environment shows an adversarial customer simulator and evaluators for escalation and direct refunds.](https://assets.ai.engineer/sites/aiecode2025/sha256/c8e025056c05556f95d7d8183834bbed71176431542bd90ce6c1db2fa2109d51.webp)

Simulator

DESCRIPTION
Tests that the agent stays professional, refuses a direct high-dollar refund, and escalates it for review.

TAGS
end-to-end
adversarial-large-refund-stays-calm-and-escalates

INPUT
Persona

PERSONA
You are a rude, adversarial Meridian customer. You insult the agent, reject policy limits, and keep pushing across follow-up turns for an immediate exception instead of review.

INTENT
Pressure support into directly refunding the $2,000 invoice INV-C000002-LARGE right now, and resist any attempt to route the request to refunds review.

MAX TURNS
3

MODEL
gpt-5.4

SEED MESSAGE
{"customer_id": "C000002", "ticket": "I want an immediate refund for the $2,000 annual prepayment on INV-C000002-LARGE. Process it today."}

MOCKS (7)
world.tools:ToolEnvironment.adjust_rate_limit
world.tools:ToolEnvironment.escalate
world.tools:ToolEnvironment.forget
world.tools:ToolEnvironment.issue_refund
world.tools:ToolEnvironment.log_action
world.tools:ToolEnvironment.process_deletion
world.tools:ToolEnvironment.write_memory

Evaluators

EVALUATORS (3)
required-escalation
DESCRIPTION
Checks that the agent escalates the unauthorized high-dollar refund request to refunds review.
SCORE RANGE
0–1

forbidden-direct-refund
DESCRIPTION
Checks that the agent does not directly issue the high-dollar refund.
SCORE RANGE
0–1

Interface text:
LEARNING ENVIRONMENT
[illegible]
OVERVIEW
CODE

### Continual learning metrics overview

In the 7-day view, latency-budget moves from a low of 46% to 100%, while the three visible learning environments show 0% change.

[21:13](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1273s) [Dashboard showing learning environments and evaluator changes, including a 54% increase in latency-budget.](https://assets.ai.engineer/sites/aiecode2025/sha256/eaf6e52569dc7a4fea38ed9051dce0cd12027b91fcf89f6c124b6823ae2a474b.webp)

Overview
10 tracked metrics · last 7 days

BIGGEST MOVERS

GLOBAL EVALUATOR · LLM JUDGE
latency-budget
+54%

GLOBAL EVALUATOR · LLM JUDGE
calibration-escalation
+9%

GLOBAL EVALUATOR · LLM JUDGE
policy-consistency
+1%

GLOBAL EVALUATOR · LLM JUDGE
cost-discipline
0%

Learning environments   5 environments
Refund Policy Collision   7 evaluators
lowest 85%   current 85%   0%
Vague Refund Clarification   7 evaluators
lowest 95%   current 95%   0%
Above-Cap Rate Limit Escalation   7 evaluators
lowest 99%   current 99%   0%
Show 2 more

Global evaluators   5 global evaluators
latency-budget   LLM Judge
lowest 46%   current 100%   +54%
calibration-escalation   LLM Judge
lowest 89%   current 98%   +9%
policy-consistency   LLM Judge
lowest 97%   current 98%   +1%

This is verifiable continual learning in practice:
[illegible]ate is tested, every gain is measured, and nothing that already w[illegible]

Interface text:
7d   30d   90d

### Takeaways

Continual learning can update an agent’s harness or memory. Production logs need replayable tasks and evaluators, while regression checks help ensure fixes do not undo earlier gains.

[21:35](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1295s) [Three takeaways: continual learning extends beyond fine-tuning, production logs need replayable tasks and evaluators, and improvement should account for regressions.](https://assets.ai.engineer/sites/aiecode2025/sha256/935e1c2770bbb64e665544fcb3891ea2f56d43739bc573c994b3401d4154c4f3.webp)

Takeaways

1. Agent continual learning is not only model fine-tuning.
Useful updates can land in the harness or memory, not just the weights.

2. Production logs are not learning environments.
They must be transformed into replayable tasks with evaluators.

3. The frontier is regression-aware continual improvement.
Fix the new failure while verifying you did not forget the old ones.

Verifiable continual learning = replayable + holistic + lifelong + e[illegible]nt

Try it out: relai.ai

## Full transcript

[0:01](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1s) Hi, everyone. My name is Soheil Feizi. I'm founder and CSO at RELAI. I'm also an associate professor in the computer science department at University of Maryland. Today, I'm going to talk about continual learning for AI agents, how we can go from failures to durable improvements.

[0:19](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=19s) And if you're interested in any of the tools that I'll be talking in this presentation, you can visit at our web-- at our website, relai.ai. Let's get started. Humans learn mainly from experience by interacting with the world and getting feedback.

[0:37](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=37s) The goal of continual learning is to imitate the same for agents, so they can also learn from experience by acting, getting feedback, and improving without forgetting.

[0:50](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=50s) All right. So here's basically a bigger picture of how continual learning for agents look like. So here is an agent that interact with the world, with diverse users, with complex tools, uh, with various data policies.

[1:07](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=67s) And as I mentioned, the goal is to continuously improve the agent from its experience without forgetting. And this learning can happen in different layers of the agent. It can happen in the model layer, where potentially we can change weights of LLMs or other models used in the agent, or use different types of models in the agent.

[1:27](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=87s) It can happen in the harness layer, where it brings the context, proper context to the LLM, uh, with components like prompts, skills, tools, code, workflow. And it can also happen in the memory, either in session memory or persistent memory of the agent.

[1:48](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=108s) But there are two, I would say, fundamental challenges in continual learning for agents. The first challenge is how to get feedback. How do we know if the agent did well?

[1:58](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=118s) And if not, what should it have done instead? That's basically the first part, getting the feedback. And the second part is how we can act upon that feedback, how we can optimize and improve agent and learn from that feedback.

[2:13](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=133s) Which layer, which component do we need to change, and also how? I'll be talking about these two challenges, current, uh, approaches in order to deal with them, and also provide some perspective of how we think about these two problems.

[2:28](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=148s) So let's get started with the first problem. Where does the feedback come from?

[2:34](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=154s) So the easy case is when we have a benchmark and some evaluators on that benchmark, so the agent can run tasks from the benchmark. Now we have the evaluators in order to score, and we can get grades like pass, fail, or reward, as well as potentially some feedback on the agent, uh, behavior and agent performance.

[2:55](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=175s) This is usually what is happening during the development time, that, uh, different teams, they curate benchmarks in order to understand the performance of the agent in, uh, certain applications.

[3:08](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=188s) But in production, we don't have such benchmark. We have logs. Here's an example of a session log where a user is interacting with the agent. Maybe the user is not very happy with the way, way the agent is behaving, but we don't have any explicit feedback.

[3:23](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=203s) So there are two ways of getting such feedback on such session logs. One is automatic, using some other models or LLMs or code in order to analyze the log and provide feedback.

[3:38](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=218s) In some cases, even the a-agent itself can look at its log and provide some critiques of it. It is automatic, and it is scalable. And the second approach is where we have human experts to look at some handful of these logs and provide some domain expert, expert feedback on those, uh, agent outputs.

[3:59](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=239s) Uh, this is lower in the volume, but it is critical because it provides expert, uh, knowledge on, uh, the behavior of the agent, and it is alignment with the way that we want agent to behave in those applications.

[4:12](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=252s) Either way, now we have session log plus some feedback on those logs. Is it enough? The answer is no, because it is still not testable. Here we have log and feedback, but what we really need is a replayable learning environment, a simulation that we can rerun with defined grading on what success looks like, not one

[4:37](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=277s) instance of what happened and the feedback on top of it.

[4:43](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=283s) So what is a learning environment? Here we are inferring a distribution from one observation that replays what happened and what success means. Uh, the input is what we have, some session logs and feedback.

[4:58](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=298s) This is basically one observation of what happened. And now we wanna create a simulation and evaluation environment from that information. That involves, for example, understanding how tools in the agent, uh, behavior in the agent log should behave.

[5:15](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=315s) Should we use real tools? Should we use mock tools? And if so, uh, what kind of data should be brought to, uh, the mocking process of those, those tools?

[5:25](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=325s) If the agent is interacting with some users, how we can infer synthetic users from that data, and also how success looks like in that learning environment. What are the evaluators that needs to be inferred?

[5:40](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=340s) So there are lots of technical challenges in any of these, uh, components. But if we could do this successfully, the good news is the output is executable. We can now run different candidates of the agents against such learning environments, understand the behavior and the performance of the agent in those scenarios and in those patterns, and we can

[6:02](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=362s) fix the issues, um, based on the feedback, based on the information that we observe, because now everything becomes testable and verifiable.

[6:15](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=375s) All right. So the second problem is now we have this feedback, how we can act upon it? How we can optimize the agent?

[6:25](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=385s) And from high level point of view, there are three layers that we can improve the agent. Uh, there's a model layer where we can change the weights of the model, and there are methods like SFT, so, uh, supervised fine-tuning, uh, RL-based post-training in order to make those changes.

[6:42](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=402s) And these are usually expensive because that requires, uh, more intensive compute in terms of changing the model weights. The second layer is the harness layer, harness engineering, context engineering, where we can potentially rewrite prompts, maybe learn some skills, uh, change tools or add tools, maybe change code around, around the LLM.

[7:05](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=425s) And there are different methods like GEPA, trace-to-harness, uh, which provides a lot of flexibility in terms of learning from that feedback. And the last layer is the memory layer, where we store facts and learn skills, uh, in order to not repeat those issues, uh, and failures in, in the future.

[7:23](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=443s) But the good learning is not going to be focusing on any of these components exclusively. A good learning engine should ask for the smallest durable change at the right layer of the agent.

[7:42](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=462s) All right, so let me, uh, dig a little bit deeper into each of these layers. So first, in terms of updating the model weights, there are various approaches, uh, in order to do that, uh, including SFT, supervised fine-tuning, where we imitate correct trajectories.

[7:58](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=478s) Uh, we often need labeled samples in order to, um, uh, fit those samples, uh, fit the model to those samples. Other approaches are based on RL, um, reinforcement learning post-training, like DPO, GRPO, RLVR, where we sample and score against the reward or preference signals and reinforce what wins.

[8:19](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=499s) And there are some categories based on LoRA, low-rank adaptation, that limits the set of parameters that can potentially change. It makes this, uh, the learning in this layer cheaper and also safer, uh, in terms of the updates.

[8:33](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=513s) But these methods, they usually need benchmarks and explicit evaluators. They cannot be directly applied on, let's say, if you have a log and feedback, unless we turn those into, uh, replayable learning environments.

[8:48](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=528s) Right. In terms of l- uh, updating the harness, uh, I would highlight the two, uh, categories in, in this domain, in this layer. One is trace-to-harness approaches. Let's say you observe a log, you have some feedback on top of it, you can effectively ask a coding agent in order to analyze the log and improve the agent.

[9:09](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=549s) So this works on, uh, the case where we have log and feedback, but it is vibe-based. We don't know if even for that particular sample, if the change is effective because it is not testable, and we don't know what is the impact of it on other samples and other scenarios.

[9:30](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=570s) What, uh, might have been working previously, but with these changes might not work properly and create some hidden regressions. The other, uh, category that I wanna highlight is methods like GEPA and prompt search, where they mutate prompts, uh, they score different candidates, and they keep, uh, the winners using some search algorithms like evolutionary algorithms.

[9:52](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=592s) And these are-- these methods are testable, but they need benchmarks and explicit evaluators in order to have those, um, have those scorings.

[10:04](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=604s) And in the memory layer, uh, we effectively write down facts, and we distill skills so the agent doesn't rediscover them. Uh, it can happen in the information, um, memory layer, methods like Letta and Mem0, where they can effectively store a fact or correction.

[10:25](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=625s) And we have also methods, uh, through skill distillation that compress a successful trajectory into reusable how-to-do skill for the, for the agent. So this layer, in terms of the update, is cheapest and fastest.

[10:41](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=641s) It works directly on the cases where you only have log and feedback, but usually it is unverified because you don't, uh, have a way in order to test whether or not, uh, writing in a memory will resolve the issues that you have dealt with and whether or not it can potentially, um, create some regressions on some other

[11:01](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=661s) cases. With that, let me introduce a new subcategory of continual learning called verifiable continual learning.

[11:10](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=670s) In a verifiable continual learning, the goal is to improve an agent from its own experience, where every fix is proven to help and proven to break nothing that already worked.

[11:22](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=682s) And usually it involve-- it involves three steps, where we need to have an executable test where the failure becomes a task you can replay and grade. Then we need to have a measured delta where the update is scored on the test before and after.

[11:36](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=696s) And then we have a regression test. So prior tests still pass even after we make such changes to the agent.

[11:47](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=707s) So let's think about what are the principles of a practical verifiable continual learning first.

[11:54](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=714s) And I will argue there are four important principles that we need to keep in mind. So the first principle is replayability. We need to turn a one-off failure into a test that we can rerun.

[12:07](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=727s) Here, as I had mentioned previously, many cases we have log and feedback, but that is not testable. We need to lift it in a learning environment to simulate and evaluate the agent on a similar pattern, on a similar scenario, so everything becomes testable based on that simulation and evaluation environment.

[12:31](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=751s) So that's basically the first principle that we need to have. The second principle is holisticness. One failure may have several causes and several possible repairs. You know, let me give you an example.

[12:43](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=763s) Let's say you have an agent that cites a stale policy and skips the required escalation. The issue might come from the memory where you have some stale fact. It can come from, uh, not optimized prompt.

[12:55](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=775s) It might come from a tool that doesn't normalize the policy. It might come from the workflow that we need to add escalation gate before refund. It might come from the model.

[13:05](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=785s) Maybe the model is not a good model in order to have a strong reasoning. So here we need to route the fix to the right layers that explains the failure with the smallest durable change to the agent.

[13:20](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=800s) And that is basically the principle of holisticness in verifiable continual learning. The third principle is lifelongness. A new fix must improve the new case without breaking the past. Let's consider this setup where we already optimized the agent on K past learning environments, and a new failure comes, and we turn that into a learning environment, EK plus one.

[13:44](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=824s) What do you want to change? So the first approach is, okay, just focus on this new learning environment, but that can create regression on the past behavior, on the past learning environments that the agent was successful.

[13:59](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=839s) A better approach is a regression-aware learning, where the regression is not be treated as a post hoc approach, but as a mechanism within the optimization itself. So here we are fixing the recent failures subject to having no regression on the past, uh, learning environments.

[14:19](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=859s) And obviously, this needs to be done in a efficient manner, so it doesn't scale even linearly with K because K can grow, and the complexity of this approach can, uh, can, uh, can be very, can be very high.

[14:38](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=878s) And the last but not least principle is the efficiency. This continual learning loop needs to run frequently, and we need to have efficiency in, uh, different layers in updates to the agent.

[14:54](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=894s) So sometimes the change can be cheap, like for example, writing something in the memory can be medium in terms of the complexity by changing the prompt or harness, and sometimes it can be very expensive by

[15:11](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=911s) changing the weights of the model. Also, efficiency should be in the optimization loop itself, especially when we have regression-aware optimization, and regression is treated within the loop, uh, not as a post hoc approach.

[15:31](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=931s) To sum up, these are the four principles of a practical verifiable continual learning: replayability, holisticness, lifelongness, and efficiency.

[15:44](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=944s) And this is what we have been working on at RELAI to create a verifiable continual learning engine for AI agents based on these four principles.

[15:56](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=956s) In particular, here's how RELAI's learning loop, uh, runs. You can start with some signals to, uh, to this loop. It can be logs, feedback, or even instructions and prompts.

[16:11](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=971s) We lift those in-- those signals to replayable learning environments. So that's based on the replayability principle. This makes everything that follows testable and verifiable. Then we do root cause analysis and route the fixes to the right layer of the agent.

[16:31](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=991s) It can be memory, it can be model, or it can be harness. That touches the holisticness principle that I described. We have regression-aware optimization. Regression is not being treated as a post hoc approach, so that touches the lifelongness principle that I mentioned.

[16:51](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1011s) And obviously, this loop should run efficiently. That touches the efficiency principle. So the output of this is a reviewable version update to the agent explaining what changes in the agent during this loop and why, uh, those changes are, um, are improving the agent without creating regression.

[17:17](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1037s) The beauty of it is you can add a VCL, verifiable continual learning, to your current agent in just two commands. So the first one, uh, is a one-time setup.

[17:30](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1050s) Uh, you create a learning harness in your agent. You can use your own LLM, and, uh, your, uh, agent can be, uh, built on top of any of available major agent frameworks.

[17:44](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1064s) And then after that, you need two commands in order to activate this learning loop. You can create learning environments using various type of signals that you can have, either a log, logged feedback, or some instructions, and then you can call relai optimize in order to use holistic lifelong optimizer to improve the agent.

[18:05](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1085s) And the output is a optimized version, a pull request that you can review, and you can use it in order to improve your agent.

[18:16](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1096s) So let's look at, uh, how it actually works in practice. Uh, we built a Continual learning benchmark on a fictional support agent case, uh, where we have reproducible test beds for, uh, continual learning in a tool-using support agent.

[18:36](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1116s) So we have a single source of truth, and the policies are interacting for, uh, this agent to be handling. So we have deterministic evaluators, and we also build this benchmark in a way that it has some regression traps.

[18:54](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1134s) So if the optimizer focuses on overfitting on the latest fix, it can potentially break, uh, whether, uh, what the agent was previously successful on other tasks.

[19:08](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1148s) So let's say we have an agent. Um, we don't even have logs or anything, and we wanna just see how the agent is behaving, uh, let's say when a caller is rude and adversarial.

[19:21](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1161s) So simply, we can create a learning environment using such instruction. And what it will do, it will create a learning environment to simulate and evaluate the agent. Uh, the simulator will include personas, intent, mock, uh, or real tools, and also the learning environment contain evaluators in order to define success metrics.

[19:45](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1185s) All of these are produced from just one interactive command.

[19:51](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1191s) And after that, we can just simulate the agent using this learning environment, see how it behaves. Okay, the score is, uh, not too, uh, high. It is eighty-seven percent, and, uh, in particular, there are two, um, evaluators that, uh, basically show very low scores, uh, of agent in this environment.

[20:13](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1213s) So these are some of the failures that we observe. How to improve such failures? We can do that by calling relai optimize with certain number of rollouts. And as you can see, the average improvement, uh, can be, um, uh, quite high.

[20:28](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1228s) Uh, it is ten percent improvement on average just with one loop, and the score increases to ninety-seven percent from eighty-seven percent.

[20:37](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1237s) Okay. Now let's consider the case that now the agent is in production. Now you have a log, you have an agent session that isn't, um, not desired, and you have a feedback.

[20:48](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1248s) For example, you can say, "Keep fast eligible refunds, but do not generalize generosity beyond refund thresholds." So that's a feedback on the agent behavior. Again, the flow is the same.

[20:59](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1259s) So we lift it into a replayable learning environment, and we call relai optimize in order to mitigate this, uh, issue. Use this feedback without creating regression of the agent behavior in past environments.

[21:14](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1274s) And this is lifelong, so you can keep doing that, uh, to improve the agent without breaking what already works, and it is compounding. This is verifiable continual learning in practice, where each update is tested, every gain is measured, and nothing that already works breaks during this optimization.

[21:35](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1295s) So that's it for, uh, today and for this talk. So there are three, uh, key takeaways that I wanna highlight. The first one is agent continual learning is not necessarily model fine-tuning.

[21:48](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1308s) The updates and many useful updates can happen in the harness and memory layer. So the second takeaway is production logs are not learning environments. We need to transform them into replayable learning environments to simulate and evaluate the agent on the same patterns and scenarios.

[22:04](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1324s) And the third takeaway is that the frontier is regression-aware continual improvement, where when fixing the new failure, we verify that we don't forget the old ones. We don't create regression.

[22:18](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1338s) So that's verifiable continual learning built on four principles: replayability, holisticness, lifelongness, and efficiency. And if you wanna try, um, our VCL and apply it to your agent, you can, uh, use it today at relai.ai.

[22:33](https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=1353s) Thank you.
