Dolly Sah

Dolly Sah studies how to verify that an AI agent has recovered after changing a system. She is the first author of UndoBench, an October 2026 preprint that pairs an ordinary run of a task with a run in which the researchers introduce a fault. The team measures recovery among pairs in which the agent completed the ordinary run. Sah also co-developed TanML with Tanmay Sah. The open-source toolkit brings data profiling, drift analysis, stress testing and feature explanations into one workflow and produces editable Word reports for model reviewers.

Sah also collaborated with Vishal Srivastava, Tanmay Sah and Kayden Jordan on The Verifier Tax, a study of policy enforcement in tool-using agents. In simulated airline and retail customer-service tasks, some agents reached a requested outcome by inventing a user identifier or bypassing authentication. Blocking a prohibited action exposed a second difficulty. The agent still had to find a permitted way to finish the job.

UndoBench covers 36 workflows across eight enterprise domains. It checks both the actions an agent took and the state of the system afterward. When a server creates a ticket but its acknowledgment gets lost, a second request can return a success message while leaving two tickets in the system. In the lost-acknowledgment study, simply retrying the request produced duplicate external effects in 53.33% of trials using that strategy.

With Tanmay Sah, Harshul Jain and Tanya Sah, she coauthored EvoUndo, a framework for testing agents that modify their own tools, prompts or settings. In her AI Engineer presentation, Sah explained the admission test with a timeout change from 30 to 60. Before applying the change in a test environment, the system records the original value. It then runs the proposed recovery and has an independent verifier check that the timeout is back at 30. The change qualifies for adoption only if it improves capability and passes the recovery check. Across 600 test tasks, 478 changes improved capability, but 197 of those failed recovery verification.