Respect The Process

Hi, everyone. My name is Andrew Dumit, and I work on AI engineering at Watershed, the sustainability AI platform. At Watershed, I work on AI for building product carbon footprints, and in turn, measuring the emissions associated with all the things that a company buys and sells.

Sustainability, the vertical that Watershed is in, is one with a ton of expert judgment calls spread throughout it, and this has made building and deploying agents both exciting and challenging.

In this talk, I'll go deep on one of the tasks within sustainability that we spent quite a bit of time working on and share learnings from our work deploying coding agents on that task, and why do-- how-- why doing so has required that we respect the process.

So just to give a bit of context on sustainability as a vertical, we need to answer questions like: What are the emissions attributable to one bottle of wine? Or which method should be used to allocate the emissions to various co-products when there are multiple sellable co-products that come from the same industrial process?

In these cases, there are many ways to get the right answer the wrong way, and there are also many right answers that experts will disagree on. And so you have to verify the process in addition to the answer, because the answer is really only justified insofar as it, the process that produced that answer is correct.

One example of this comes from a study, uh, in twenty twenty, where six experts were given the exact same data on the exact same bottle of wine, and despite having all-- access to the exact same things, they came to answers that varied by up to fifty percent.

Their expert judgment were all correct in a sense, but pilit-purely validating the answer here is not sufficient to know that the answer produced by a system trying to mimic those experts is itself correct.

So with that context on sustainability broadly and what kind of things that we need to answer, let's zoom into a specific task that we'll be talking about for the remainder of the talk.

That task is to help users edit complex graphs. Each of these graphs represents the supply chain of a single product. For example, on the right here, you can see the graph represents dark wash jeans.

Upstream of that is the assembly of that jeans from the denim thread labels, uh, and zipper, along with the energy, transportation, and packaging that's required to move, uh, move that through the supply chain and create, uh, and run those industrial processes.

Each graph here represents the entire flow of all of those things moving through the supply chain and is comprised of thousands of nodes, each with rich metadata describing the materials and processing at each step.

And so when we first tried to solve this problem a little over a year ago, it worked decently well on one graph. It did have some problems, but we gave our React agent highly specified tools for exploring and interacting with the graph via function calls, and th-- even while it worked, a few problems were already apparent.

Um, it lacked consistency, where it could struggle to explore sufficiently, and those tool calls on a single graph took a lot of context as it read deeply across these nodes with lots of metadata.

But then when we tried to scale it up to many graphs, or frankly, even just a few graphs, it absolutely broke. The lack of consistency was greatly magnified with the agent taking one approach in one graph, a different approach in a second graph, and completely forgetting about the third graph or even to handle it at all.

And exploration itself also became a huge bottleneck because it took many tool calls to explore and operate across these graphs at scale. And it also worsened the context problem because these tool calls really gobbled up context, both on the edit side, where it actually needed to, like, make some changes, and on the exploration side, where it needed

to figure out what changes to make. And worse, the agent then really started to hallucinate different parts of the schema as those contexts got eaten, and despite those specialized tools, this led to retries and ultimately errors.

In data terms, the task now comprised tens or hundreds of graphs and tens to hundreds of thousands of nodes that the agent had to operate over. So time passed, and coding agents got way better, and we thought, "These will work great."

So naturally, we swapped them in. And swapping in a coding agent gave us, like, three really important outcomes that we were missing before. First, it gave the agent the ability to delight users.

The agent could start to find clever ways to solve these underspecified and kind of not fully formed problems. It could also create fancy visualizations on the fly by writing code to do that, and it could even answer related questions because it had kind of the full power of the coding agent in this environment.

And what from our end, we saw the agent could explore and now edit way more efficiently than before. It could write loops over graphs and nodes. It could write scripts to unpack and summarize the node content underneath it all, basically following the same pattern of data exploration that goes on in kind of agentic data science workflows.

And beyond that, it also gave it the flexibility and power to do stuff outside what we even designed it for, which was really exciting, and we're, to this day, consistently finding entirely new use cases via new questions users ask and new things that they want to try with this agent.

We were really excited. We put it out into the world. We wr-- started to write a bunch of evals for it, and we quickly learned that unconstrained code is quite scary.

If you're watching this, you've probably had that experience with Claude Code where it's gone a bit haywire towards achieving whatever goal you set it out to do. It reached for something you didn't think it could or should reach for, or even had access or found a way to access something you didn't think it could.

Um, for us, this looks like the agent will find creative ways to pretty much solve any problem. In our case, we saw it write Python when we expected TypeScript and, and instructed it to write TypeScript because it found Python on the virtual machine that we had given it.

Or it would directly edit these graph artifacts underlying the data without leaving any lineage behind by just, like, directly modifying, um, parameters or data rather than actually, like, writing code to effectively do that.

Second, the agent actually started to gaslight users sometimes, saying it had made edits when it hadn't. Um, it would write code that did s-- that it thought was gonna do-- ma-- like, have the effect that it wanted to, and then it said, "I'm done.

Everything's, everything's working. It's exactly what you said," and the edits were not actually made, and so it, it kind of lied to users or at least led them to believe that the edits were done as they expected it to be done.

And then lastly, once the agent had completed its task, it was now much harder to review its work. Um, manual review of code is not something that our, is in our user's wheelhouse.

They are not software engineers or may or may not have had experience with code before. And when the agent is right for the wrong reasons, which happens, uh, or can happen frequently in our, in sustainability broadly, it's really hard to check its work without going and reading that code directly.

And so this process is not the answer. Story is really not new. Um, a recent paper, uh, in from twenty twenty-six, the Open Proof Corpus, showed that there can be quite a big gap between the correct final answer and, and the correct proof, though that is, you know, potentially closing.

But-- And this is in math where there is like a fully verifiable answer. And so this-- we view this problem as even worse where you can't fully verify the answer, and this has been found in the agents will reward hack.

Um, even as something as complex as the Irish problems, um, you're gonna get many, many pages of, of errors in your proof which have, which lead nowhere, um, to impossible bench, to, um, other kind of areas in math where po-- this can, where they can-- the authors of this paper Beyond Correctness conclude that this can pose significant

risk for critical applications. Now, all this is not to say that we shouldn't use coding agents. We don't want to constrain how the agent reasons. We get so many benefits from these powerful models, but we also can't perfectly verify the answer in our case, so we don't even know when the final answer is definitely correct, uh, because

j-- these expert judgment calls are ever present in our space. So the main lever left is validating the process and making sure that we are following it in a way we expect.

And to do that, we frame it as constraining the effects, not the expression.

So our task, uh, begins with a user request, and from this, the agent can then go off and freely write code to execute on that user request. However, we require that all the critical code, really the stuff that edits the graph, goes through a filter of this typed SDK that we've put together where we can lint and

check for errors, and then we own the final execution step to guarantee that we create these typed objects that we can commit as the actual edits to the graph.

And these are then faithful to the process. And all of this together, um, means that the entire process is valid, traceable, and replayable. And if it's not, we can reject and retry and send it back to the agent, um, to let it know that something has gone awry.

So let's go a little deeper on the typed SDK and the deterministic execution. So first, our SDK is the only door. What we give to the agent, in our case, is a TypeScript SDK with all the edit primitives the agent needs to make changes to graphs, as well as explore and interact with the kind of graph objects.

This, for us, enforces which fields are editable versus which are derived from other fields, so the agent can't create conflicts with itself, where it edits just the target thing that it cares about, but not the thing that creates that target thing.

And it also guarantees that it emits objects that we expect it to so that we can actually like make use of the outputs of this in a deterministic way.

Now, of course, this does require teaching the agent how to actually make use of this SDK, but this is the same, uh, kind of pattern that we see with teaching an agent to like write code in your code base where you tell it how it works.

You can, you know, do the prompt and all of that kind of stuff. The agent also has full access to the docs and the code underlying the SDK if it really needs to go and read that as well.

Just for a brief example on the right here, you can see an example with a few pieces of the SDK on display. We import, um, from our API. We-- The agent has to then define kind of a top-level edit function with a well-defined name.

It then can use, uh, find nodes by exact name to find specific nodes that it's looking for in the graph. It can write assertions that we give it to make sure that, um, its code fails early, um, as well.

And then it can use, uh, specific mutator functions like set rate and edit node to actually interact with the graph in a way that will produce those kind of typed objects that we care about.

So that's the SDK, but the actual guarantee here is when we own the deterministic execution. So even with the typed SDK as our entry point, that really only guides the agent towards the desired end state.

And the real guarantee comes from the final script that we orchestrate on agent completion. This run executor script, um, i-is called when the agent has kind of completed running, and we think that we're in a like almost completed state or a ready to be completed state.

And what that script does is it starts by linting the agent code where we can immediately send it back to the agent if something is wrong. Better to kind of fail early rather than fail later.

We can also detect conflicts. This is where sometimes the agent in one part of the code edited something in another part, edited the same thing or something that depended on the first one and didn't realize that that happened.

Because we have these guarantees by giving the agent this well-defined structure, we can detect those conflicts for it so that we can send it back to the agent. We then run the agent edited code, and that means that we can then validate the output artifacts.

Of course, if the code itself fails to run, we can send that back to the agent too. And then finally, once we have the validated output artifacts, we can create a well-structured review artifact that makes it easy to review what the agent did without having to act-- ever go read the underlying code that produced it.

To give you a flavor of what that looks like, here's a little snippet from an emissions report where the graph edit function impact analysis ran on fifty graphs. Um, there were two functions that it applied that produced seven hundred and forty-nine edit actions, and it ultimately, in this case, reduced the overall emissions by forty-five point six percent.

And we can even go to, go to see there are two code... There are two edits here. Um, one kind of did a big chunk, and the other one made a very small change.

Maybe you'd go look at what those actually did. And then you can even go at a graph level and say, "Okay, for this graph, like, what was the option one, and which nodes did it edit within that graph?"

To get a sense of, um, what actually happened under the hood. So you can see all of this without actually having to go read the low-level code.

So to return to those three problems that, uh, we started with, now that we have these kind of two broad concepts that are part of the harness, um, when the agent will find creative ways to solve every problem, this is now a great thing, but we've limited it a little bit.

These invalid actions that the agent will take on its route to solve whatever problem we've presented are actually prevented with the Typed SDK as the only way to properly edit the graph, and then we can guarantee that the graph was only edited through the SDK by owning the deterministic execution and the validation of that written code.

The second problem about the agent saying that it made an edit when actually that the edit didn't go through in the way that it or the user expected, these false reports are caught via that deterministic execution so that we can surface those back to the agent, um, to make sure that it's doing what it claimed.

And finally, our users never have to look at the code because the deterministic execution system produces artifacts in an expected form, and we can build verification against those to catch both the kind of good outcomes and the errors that are now surfaced clearly to both the user, um, and the AI.

So even when it hasn't maybe gotten exactly the right answer or the one that the user expected, it's easy to follow its logic and kind of get to that, um, and loop back on itself.

Now, the harness is all well and good as it kind of adds constraints, but I also want to emphasize that even with those great guarantees, you still need to hill climb to get better at the task itself.

Since we started tracking how well the agent did at these complex edits, where each task here, um, in our eval set is a collection of graphs and a goal, um, we've been able to improve our outcomes from about forty-three percent to ninety-two percent on our set of internal evals.

And to achieve that, all the typical approaches were still effective. We did a ton of prompt improvements, like rewriting the system prompt and all the kind of skills that the agent had access to.

We've added few-shot examples to both teach and coach the agent how to make use of the SDK and work with these different classes of tasks that we expect.

We also had to make the tools, the SDK itself, and the tools the agent had access to better fit for use by improving their kind of ergonomics and how obvious they were for the agent to use.

We also broke the task down through doing this, breaking up the, um, overall problem to a plan and execute kind of standard loop, as well as teaching the agent some of that expert judgment that is endemic in our space, making that easier for the agent to operate on it and to ex-elicit that expert judgment from the user.

But the important thing here is, even though we've kind of improved the accuracy here, which is amazing, and it's great that we'll like now kind of get better at that first time a user goes through the full workflow and the task, um, even when the agent errors or even our ground truth is, you know, one of those

points on the possible range of expert judgments, we can still guarantee and validate the, um, process itself.

So to bring it all together, in a domain full of expert judgments, you must respect the process too, and for that, you really need guarantees about the process. We all know coding agents are necessary for complex tasks, and they will continue to be necessary and extremely useful, but those power-- that power comes with many risks that are,

like, good to go and clear-eyed about. For tasks with complex data and judgment calls, we now think that the harness should do a few important things. First, it should give the agent well-scoped primitives to interact with the kind of external system that it's actually supposed to work with.

Um, free, freely operating code can, you know, lead to a lot of the things that we talked about earlier, and this really allows the builders of the system to allow and disallow specific things and exercise the full power of these coding agents while still trusting the process that they're working in.

I think the harness should also maintain full control of the final execution. These very smart agents may declare victory in an unexpected way from what you or your user really want them to declare, and so it's important to be able to constrain and validate that the agent has actually done what it said it's done.

And finally, you should use that deterministic final outcome to produce outputs that are easy to validate, even for non-coders. The code is kind of just the means to an end.

And then, of course, the kind of standard context and prompt engineering will of course remain critical to better scoping and defining that task and making it clear what the agent needs to do.

That's it for me. Again, I'm Andrew Dumit at Watershed, the sustainability AI platform. Thank you again for your time and attention.