An Agent's Change Is Not Safe Until Its Recovery Has Been Tested

4:26:20Today joining me is Dolly and we will be

4:26:23presenting the missing undo button for

4:26:26coding agents.

4:26:31So imagine a coding agent working on a

4:26:35repository issue. It applies patch run

4:26:39the tests and everything passes. It even

4:26:43creates a release tag but then a

4:26:46downstream integration fails. Now here

4:26:49is the question. Can the agent actually

4:26:53get everything back to the state it was

4:26:56in before?

4:26:59So git can tell us what changed in the

4:27:02repository. But what about changes

4:27:05outside git such as external API calls

4:27:09or runtime configurations or

4:27:11notifications.

4:27:13And simply retrying an operation does

4:27:16not necessarily undo what already

4:27:19happened. So we wanted to understand not

4:27:22just whether agents can complete task

4:27:26but whether they can recover when

4:27:28something goes wrong.

4:27:37This brings us to our first research

4:27:40work undo bench. We wanted to

4:27:43investigate whether an agent's ability

4:27:46to complete a task also translates into

4:27:49its ability to recover from failure. We

4:27:53evaluated agents across 2,880

4:27:57paired trials comparing normal execution

4:28:01with execution under an injected

4:28:04failure. So under normal conditions, the

4:28:07agent achieved an 84% success rate. But

4:28:11when we introduced failures only 47%

4:28:15of the normally successful trials

4:28:18recovered successfully. Notice that

4:28:21these two percentage though have

4:28:23different denominators.

4:28:25But yeah so even when an agent

4:28:29uh can complete a task successfully

4:28:32recovery remains a significant

4:28:34challenge.

4:28:38Now let's look at a specific failure

4:28:41scenario. Imagine a coding agent sending

4:28:45a request to an external system to

4:28:48create a ticket. The request has been

4:28:51sent and the agent is waiting for a

4:28:54response.

4:28:57Next, the server receives the request

4:28:59and successfully creates ticket number

4:29:02781.

4:29:04But the acknowledgement gets lost on its

4:29:07way to the agent.

4:29:10From the agent's perspective, the

4:29:12request has timed out, but the agent

4:29:15does not know that the ticket has

4:29:17already been created. A timeout tells us

4:29:20that the response didn't arrive, not

4:29:23that the operation failed.

4:29:28So the agent tries the request without

4:29:31checking whether the oper whether the

4:29:33first operation committed

4:29:37and now we have two tickets even though

4:29:40the agent intended to create only one.

4:29:43So in undo bench knives

4:29:47produced duplicate effects in 54% of

4:29:51these kind of trials. That's why we need

4:29:54to distinguish a failed response from a

4:29:58failed operations.

4:30:05Now

4:30:07the failure phase also changes the

4:30:10answer. What do I mean is that recovery

4:30:14also depends on when the failure

4:30:17happens. Before the mutation retrying is

4:30:21generally safe because nothing has

4:30:23changed but during the mutation the

4:30:26system had may be partially modified. So

4:30:30we need to inspect its state and after

4:30:33the mutation the operation may have

4:30:36committed even if the acknowledgement

4:30:38was lost. As we just saw, retrying could

4:30:42create duplicates. So, we need different

4:30:46recovery strategies depending on the

4:30:49failure phase.

4:30:57Now,

4:31:00why can't we just use get reward? So,

4:31:04Git can restore repository changes, but

4:31:08coding agents also modify their runtime

4:31:12state including tool registries,

4:31:14middleware and configurations. They can

4:31:17also trigger external effects such as

4:31:20deployment or database rights. So, get

4:31:24rever does not reverse all these

4:31:27changes. The recovery mechanism has to

4:31:31match the type of effect the agent

4:31:33created.

4:31:39And this becomes particularly important

4:31:43when coding agents start modifying their

4:31:46own execution environment or what we

4:31:49call the agent hardness. This includes

4:31:53prompts, tools, middleware, event

4:31:56listeners, resources, and routing

4:31:58configurations.

4:32:00Such changes may improve the agent's

4:32:02performance, but they can also make

4:32:05recovery more difficult. In other words,

4:32:08an agent can become more capable while

4:32:12becoming less recoverable.

4:32:20Uh that's why uh that's why undo needs

4:32:25more than an inverse command. Uh now

4:32:28like undoing a change is not always as

4:32:31simple as reversing a command. Suppose

4:32:35an agent changes a timeout from 30 to

4:32:3960. To undo that change, it needs to

4:32:42know that the original value was 30.

4:32:46That's why EVO undo captures a pre-state

4:32:49witness before applying the mutation and

4:32:53that witness provides the information to

4:32:56restore the original state and later

4:32:59verify that uh verify that recovery

4:33:04actually worked.

4:33:06Now

4:33:08question comes

4:33:10how often uh do improved hardnesses

4:33:13fail? So in Evo Undo, we evaluated 600

4:33:18self- evvolution tasks. Out of those 478

4:33:23produced capability improvements, but

4:33:26197 of those improvements had recovery

4:33:30defects. That is around 42% of the

4:33:33capability positive cases. So here key

4:33:37takeaway is that a mutation can improve

4:33:40an agent's capability and still leave

4:33:44the system unable to recover reliably.

4:33:51Now now we can say that why please fix

4:33:55the undo did not work. So naturally we

4:33:59wanted to use prompting and wanted to

4:34:02tell agent that hey could you simply

4:34:04repair all these recovery failures. So

4:34:08we tested four verifier guided repair

4:34:12strategies but under the original

4:34:14recovery language none recovered the 197

4:34:18naturally occurring failures. The

4:34:21problem was not necessarily

4:34:24the agent's reasoning. In some cases,

4:34:27the agent lacked precise information

4:34:30about the original state. In uh in

4:34:34others, the recovery language could not

4:34:38express the operation needed to reverse

4:34:41the change. So, we investigated these

4:34:44two limitations separately.

4:34:50So, two bottlenecks.

4:34:53First is state grounding and another one

4:34:55is recovery expressivity.

4:34:57The when we provided like precise

4:35:01information about what had changed and

4:35:04where then recovery improved from 0 to

4:35:0738 out of 48 cases. And the second is

4:35:12recovery expressivity. So when we

4:35:14introduced richer undo operation

4:35:17recovery improved significantly from

4:35:20zero to 142 out of 143 cases. So these

4:35:24results come from the two different

4:35:27experimental tubros but together they

4:35:30show why recovery needs both accurate

4:35:34state information and ability to perform

4:35:37the required end to operation.

4:35:46Now question comes to our mind that can

4:35:49better undo mechanism rescue failed

4:35:52mutations.

4:35:53So what happens like when we give the

4:35:57agent better recovery capabilities. So

4:36:00with the original recovery language none

4:36:03of the 197 recovery defective mutations

4:36:06could be restored. But after extending

4:36:10the recovery language, we successfully

4:36:12recovered 180 out of 197 cases. That is

4:36:16around 92%. As a result, the number of

4:36:20admissible improvement increased from

4:36:23281 to 461

4:36:26out of 600 task. That is a 30 percentage

4:36:30point improvement. And

4:36:33that brings us to the next question. How

4:36:36do we build these recovery checks into

4:36:39the agent's evolution process? I will

4:36:43now hand it over to Dolly to walk

4:36:46through the EVO undo algorithm.

4:36:51All right, thanks San. Uh so let's get

4:36:54into the working on how Evo and actually

4:36:57works. So first we will start with the

4:37:00agent pro. Uh first

4:37:02the agent will start the agent pro

4:37:05processes the mutation on which state it

4:37:08is currently in along with the effect

4:37:09contract and the recovery plan. And

4:37:12before applying the mutation

4:37:15we captured the state witness

4:37:18representing the original state

4:37:21and even before that the then a testing

4:37:25in the testing environment agent will

4:37:26apply the mutation and execute the

4:37:28recovery plant uh an independent

4:37:31verifier will also verify the original

4:37:34if uh verify to check if the original

4:37:36state has been restored or not. If the

4:37:38mutation improves uh the capability

4:37:41based on the userdefined criteria and

4:37:44passes the recovery um check the agents

4:37:48admit the changes otherwise well

4:37:53uh the agent attempt to repair or the

4:37:55verifier itself rejects the mutation

4:37:57altogether. This way the recoveryability

4:38:00becomes a part of the admission decision

4:38:02rather than something that we strategize

4:38:04or even plan once we are uh deployed the

4:38:08deployed the AI agent.

4:38:12Okay. So with next

4:38:16we will walk through as as Tan have

4:38:19already explained in given an example

4:38:21let let me walk through how the ex uh

4:38:24how that example of a timer changing

4:38:26from 30 to 60 would look like in action.

4:38:29So let's take let's start with the

4:38:32original state the simple example that

4:38:34uh simple example of um our agent having

4:38:37the time value of 30 before changing

4:38:40anything the evoando would first read

4:38:43the original value

4:38:46and then it will save the it will

4:38:48capture the original value as a state

4:38:50witness. We know exact we so we know

4:38:53exactly what need to be restored when

4:38:56when and when if we have to restore it

4:38:58we would know what what need to be

4:39:00restored or which state would be the

4:39:02true state to return back to. Now the

4:39:06agent so next come the mutation. Now the

4:39:08agent changes the timeout from 30 to 60

4:39:11as we want it to do. And if recovery is

4:39:14required uh if recovery is required the

4:39:18undo uses the saved witness to restore

4:39:20the timeout to 60.

4:39:22Now independently the verifier finally

4:39:25an independent verifier would also

4:39:26confirm that the uh recovered value

4:39:29matches the original value. So the

4:39:31recovery isn't considered successful

4:39:33simply because an undo command was run

4:39:36but also the verifier would verify the

4:39:39resulting state to make sure that we are

4:39:42in fact doing the uh recovery those in

4:39:46the state we want it to be.

4:39:50Well now comes the bigger question. So

4:39:53so we have this capability of recovery

4:39:55in our arsenal. Should we be using it

4:39:57for uniformly and applying the recovery

4:39:59mechanism to all the failures? Well, of

4:40:02course not. Um, not every failures can

4:40:05be recovered the same way. If nothing

4:40:07has changed, then the then simply if if

4:40:11nothing has changed in the in the

4:40:12system, we would simply want it to um

4:40:16recover uh we we would simply want it to

4:40:19safely retry. we would want the agent to

4:40:22simply retry if uh and try to achieve

4:40:25the state that we want to be. However,

4:40:27if we have the original state uh once we

4:40:30have the original state, we can restore

4:40:32it. If the external action has already

4:40:34been committed and we need uh

4:40:39it's then when we would need a

4:40:41compensation mechanism, compensation

4:40:43action to comp compensating action to

4:40:46reverse its effect. And if we cannot

4:40:48recover the recovery uh and we cannot if

4:40:52if and if we cannot the cannot verify

4:40:55that recovery is safe or the verifier

4:40:57have rejected the recovery then the

4:41:00agent should stop and escalate. The

4:41:03right response definitely depends on

4:41:04what actually happened and what we what

4:41:07can be safely reversed or not.

4:41:13That brings me

4:41:16to the almost the last slide which is

4:41:18about bringing everything together on

4:41:20our two paper undo bench and ebo undo.

4:41:22The so the key takeaways here are first

4:41:26the undo bench shows the successful task

4:41:29execution.

4:41:31Uh the undo bench shows that the

4:41:34successful task execution doesn't

4:41:36guarantee doesn't always guarantee the

4:41:38successful recovery of it. Uh secondly,

4:41:41the EVO undo algorithm provides a

4:41:43structured process. Capture the original

4:41:45state, apply the mutation, recover

4:41:47independently, verify and restoration.

4:41:50So the EVO undo algorithm does work in

4:41:53the in the fash in the way we intended

4:41:55it to. And third, we should admit

4:41:58self-evolution changes only when the

4:42:00when both the capability and the

4:42:02recoverability requirements are met. So

4:42:06ultimately as we give agent more

4:42:08autonomy, we should also give them more

4:42:11uh reliable ways to recover from their

4:42:12actions.

4:42:16All right. So and finally I hope you

4:42:19remember the agent from the beginning.

4:42:21It was looking for an undo button. Well,

4:42:24now it has one. We have started building

4:42:26this capability through our research

4:42:27project Evo undo. And you can explore

4:42:30the work through the in in the GitHub

4:42:32repository. can scan the QR code on the

4:42:36screen to get to the page. Our goal is

4:42:38to make the recovery a fundamental

4:42:40capability of AI agent so that we not

4:42:43only have the the agent does not even

4:42:46execute

4:42:48the action but also recover when

4:42:50something goes wrong as well. Thank you

4:42:52so much.