AI Engineer World's Fair 2026
Agents in Production: How OpenGov Built and Scaled OG Assist
Read the talk
Building OG Assist: An Agent Loop for Government Software
OpenGov’s assistant connects product data and page actions to an Effect-based agent loop, with shared contracts, human approvals, isolated execution and production feedback.
From a talk by Gabe De Mesa
Before you start: Basic familiarity with TypeScript, language-model tool calls and frontend/backend application boundaries will help you follow the implementation discussion.
A rate-code question inside utility billing
How should an assistant answer a question about rate codes when the user is already working inside utility billing? For Gabe De Mesa, a software engineer on OpenGov’s AI agents team, the starting point is the application itself. OpenGov builds ERP software for budgeting, procurement, asset management and permitting, with a mission of supporting more effective and accountable government. De Mesa describes a company founded roughly fourteen years before the talk, now adding an agent interface across its products.
OG Assist is a shared entry point to product-specific capabilities. Its button sits in the navigation bar, while individual product teams supply the tools and skills behind it. In the utility billing demonstration, the user opens the assistant and asks about rate codes. The agent calls tools against that suite’s data, then presents a rate-code table and quick takeaways alongside the workspace. The chat interface is useful because it can reach the information in the product the user is actually using.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From answering questions to acting on the page
OG Assist began when a principal engineer formed an AI agents team and invited De Mesa to join. As the assistant spread across OpenGov’s products, the integrations grew to include both backend capabilities and frontend interactions. The agent could inspect what was on the screen and take action on the page.
The next demonstration asks the assistant to identify what is on screen and highlight possible next steps. The agent considers its available tools, selects a page interaction and highlights a clickable element while explaining it. That adds a different kind of help to the rate-code lookup: the assistant can guide someone through the interface in which the work happens.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Owning the agent loop with Effect
The team chose Effect, an open-source TypeScript library, as its foundation. Its schema facilities serve a role comparable to Zod, while error handling, logging and tracing provide common infrastructure for services. De Mesa credits that combination with helping the team structure its architecture and build the core agent loop.
The first loop used LangGraph, which initially met the team’s needs. As the team grew and its use cases evolved, it moved to an Effect-native implementation so it could directly change the loop for complex features. Owning the loop gave the team control over orchestration while keeping tracing, structured concurrency and logging in the same programming model. The motivation was control over evolving behavior, rather than a reported performance comparison between frameworks.
The demonstrated Effect AI building blocks are a chat abstraction and a language-model abstraction. The application instantiates a chat, passes a prompt to streamText and receives streamed output. Dependency injection supplies the underlying language model, allowing the team to swap it without replacing the surrounding orchestration. The linked Effect v3 AI documentation labels its integrations experimental; the talk’s Chat/streamText demonstration should therefore be understood in its package-version context, rather than as a pinned current API recipe.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A shared contract for frontend and backend
For the boundary around that loop, the team adopted Agent2Agent, or A2A, the open protocol introduced by Google for communication between agents. OpenGov also found it useful as an application contract: backend routes, models and schemas follow the specification. An Agent Card, for example, describes an agent through fields such as its name and description.
That shared specification gives frontend and backend developers the same structures to consume and produce. Instead of independently designing both sides of each interaction, they can align on a common contract. De Mesa also points to A2A extensions, including metadata, and mentions A2UI as a related possibility. Those extension possibilities are separate from the runtime-form demonstration later in the talk; he does not identify that form as an A2UI implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Production feedback and CI evaluations
“Shipping is the start, not the finish.” OG Assist collects feedback through calls and email, but its main in-product mechanism is a thumbs-up or thumbs-down response rating. That gives users a direct way to flag a useful answer or a poor one, and gives the team a signal to investigate and improve future responses.
Automated evaluations complement that feedback. In CI, the team runs prompts against real completions and checks behavior: did the agent call the expected tools, and did it perform the intended task? These checks and user reports feed changes to the tools, skills and harness—the system that runs the agent. De Mesa reports that the combination helps the team improve responses and iterate quickly, without supplying an accuracy score or an evaluation pass rate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Interrupt before an approval-required action
When a tool call requires human approval, OG Assist deterministically interrupts the agent loop and displays an accept-or-reject interface. The model’s decision to request a tool call does not itself authorize execution. The human explicitly accepts or rejects the proposed action, a boundary De Mesa emphasizes for mutating operations. This applies to calls designated as requiring approval, rather than necessarily to every tool invocation.
A small TypeScript example makes that boundary explicit. The approval function represents the UI decision; while it is pending, execute has not been called. A rejection returns without executing the proposed operation.
typescript
type ProposedCall<T> = {
name: string;
requiresApproval: boolean;
execute: () => Promise<T>;
};
type CallResult<T> =
| { status: "rejected" }
| { status: "executed"; value: T };
async function runTool<T>(
call: ProposedCall<T>,
requestApproval: (name: string) => Promise<"accept" | "reject">
): Promise<CallResult<T>> {
if (call.requiresApproval) {
const decision = await requestApproval(call.name);
if (decision === "reject") {
return { status: "rejected" };
}
}
const value = await call.execute();
return { status: "executed", value };
}
The key property is the placement of the gate in application control flow: approval is a prerequisite for execution, not an instruction the agent is merely asked to remember.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An isolated place to execute code
Code execution and file creation have another boundary: an ephemeral sandbox. OG Assist provisions these environments on demand so the agent can write code, execute it and create files in an isolated space. The environments are torn down afterward. De Mesa presents this isolation as protection for production systems; the talk does not specify the sandbox provider or its precise isolation guarantees.
The demonstration asks the agent to create a PDF for attendees of AI Engineer Conference 2026 and make it downloadable for sharing. The agent creates the PDF inside the sandbox, and the resulting conference greeting appears on screen. This gives the execution environment a concrete purpose: the assistant can deliver a file, extending its output beyond a chat response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep a running summary, not just recent messages
As conversations grew, OG Assist encountered token limits and overloaded context, particularly with older models. The team found rolling summarization more effective than continually filling the prompt with the latest messages. A running summary preserves earlier context while a recent-message window retains the immediate conversation.
De Mesa describes summarizing after N messages and retaining a recent window such as N − 5 or N − 10 messages. These are illustrative choices, not a disclosed production configuration. The reason to preserve a summary becomes clear when a user returns to a topic from 100 messages earlier: a recent-only window may have dropped it, while the summary can still retain a useful detail.
Recall then operates over what the summary preserved. When the user asks whether the agent remembers an earlier topic, it can use that retained tidbit to continue the thread. This is memory within the conversation, with the summary carrying context forward; it is not a promise that every detail of the original history remains available.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Generate choices with a registered form
The next request asks for a long essay, along with suggestions for what it could cover. OG Assist has a registered form primitive, which the agent uses to build a form at runtime and offer selectable topic options. The application supplies a UI capability, and the agent fills it with choices relevant to the current request. That lets the conversation collect a decision through a form instead of requiring another free-text exchange.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Follow work across service boundaries
“You can’t scale what you can’t see” introduces the observability layer. De Mesa describes how Effect spans feed traces and let the team drill into function calls. Effect provides built-in tracing support, but arbitrary functions do not all become traced automatically: the Effect tracing guide explicitly instruments effects with Effect.withSpan and configures trace export.
The illustrated trace comes from the Effect team. It follows a request through an API, an endpoint and a handler, exposing the nested work behind the request. Timing those spans helps locate bottlenecks; tracing failures across services helps connect an agent error to another team’s API or a platform capability. For an assistant assembled from many integrations, that visibility supports debugging and maintenance across the whole request path.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build capabilities from tools and toolkits
Alongside Effect, the team’s other major commitment is to tools and skills. The concrete example is GetDadJoke, drawn from the Effect AI tool-use tutorial. It illustrates how a callable capability becomes available to a language model.
The construction proceeds in three steps:
- Define a tool for the operation the agent can request.
- Add it to a toolkit, a collection of available tools.
- Supply the toolkit to the language model so it can request those operations while answering a prompt.
With a prompt asking for dad jokes about pirates, the model has a relevant tool available to help produce its response. De Mesa presents this pattern as the foundation for OpenGov’s tools and, eventually, skills, though the walkthrough does not introduce a separate skill implementation. The recommendation is to start with these Effect AI building blocks and compose capabilities around them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use agents to build the product
The same approach extends to OpenGov’s own development work. The team uses Claude and Cursor, and builds internal tools and skills alongside those exposed to customers. De Mesa also mentions Claude Agents as part of that workflow. These agents help engineers read, write and review code, then ship it. He describes a substantial acceleration qualitatively, without a productivity benchmark: the team developing the customer-facing harness is also creating capabilities that help it develop that harness.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Effect v3 documentation for provider-independent language-model services and composable AI workflows.
A walkthrough defining a dad-joke tool, implementing its handler and supplying a toolkit to a language model.
The original introduction to Agent2Agent and its purpose of enabling communication across agent frameworks and vendors.
How agents advertise extensions and clients activate additional protocol capabilities.
The legacy v0.8 specification for transporting interactive UI messages and declaring UI capabilities through A2A.
Further reading
OpenGov describes OG Assist's contextual guidance, skills-based architecture and integration into government workflows.
Examples of span creation, nested traces and OpenTelemetry integration in Effect v3.
Read the complete timestamped transcript
- 0:01
Hi, everyone. My name is Gabe De Mesa. I'm an engineer here at OpenGov, and today we're gonna be talking about agents in production, specifically how OpenGov built and scaled OG Assist.
- 0:13
Uh, so, um, this presentation is going to be jam-packed with just so much good stuff. Uh, we're gonna talk about, uh, AI agents. We're going to talk about our harness.
- 0:25
We're going to talk about, um, evals, observability, traces. We're going to talk about, um, tools and skills. Um, it's-- there's gonna be a lot of good stuff in here.
- 0:38
We're gonna talk to you guys about, uh, what we do at OpenGov and how we operate at the scale that, uh, we operate at, um, in production. So you'll be able to see a real use case and workload, uh, with AI agents.
- 0:52
Um, so without further ado, let's get started.
- 0:56
Okay, agenda. So just really quickly going to go through, uh, high level, what we're going to talk about today. Uh, I'm gonna tell you guys a little bit about OG Assist and what, uh, OpenGov is.
- 1:07
I'm gonna tell you guys the origin story of how this all kinda came to be. Uh, we're going to talk about OG Assist's, uh, big bet on Effect, uh, a little bit into our core agent loop.
- 1:20
Uh, we're gonna talk about the A2A protocol, evals, and sandboxing. We're gonna talk about how we manage long context. We're gonna talk about, um, monitoring, observability, how we collect feedback, uh, and how we iterate on that feedback.
- 1:35
We're gonna lastly, uh, also talk about tools and skills and how at OpenGov, uh, we use, um, AI not only externally, uh, that we, uh, serve to customers, but also internally to improve our development workflows.
- 1:51
Just a little bit about me before we go any further. My name is Gabe. I'm a software engineer here at OpenGov. I work on the AI agents team, and, uh, I'm one of the folks that helped build, uh, OG Assist and some of the systems that you guys will be seeing today.
- 2:07
So a little bit about OpenGov. OpenGov is a software company, uh, on a mission to power more effective and accountable government. Um, so OpenGov sells ERP software. That's things like budgeting, procurement, asset management, and permitting.
- 2:22
And, um, we were founded about fourteen years ago, and what's cool is, um, we have this thing called OG Assist, and OG Assist is this little button on the top of all of our products in the, in the navigation bar.
- 2:38
And what's cool is, um, all of our product suites and product teams, um, have built sc-- tools and skills in order to power this button. So for example, if I open up, uh, this, this, um...
- 2:54
If I click this button and I open up OG Assist, it says, "Hey, um, I'm gonna ask about rate codes," which is very specific to utility billing, the current product that I'm in.
- 3:03
And you can see that inside of this kinda chat interface, I'm able to speak to an agent, and the agent is able to make tool calls in order to, um, look up information against data inside of that suite.
- 3:15
So it's really cool, um, to be able to kinda first party create these experiences, uh, through the capability that we've built called OG Assist.
- 3:27
Okay, so just a quick story about how this all came to be. So, um, a little while back, we, we, we saw that AI was really starting to take off, and a principal, uh, spun up this new team called the AI agents team and asked me to join.
- 3:42
And, um, instantly I said yes, and OG Assist started to, to grow, and we started to integrate, uh, OG Assist into all our products, and, uh, not only our back-end capabilities, but also our front-end capabilities as well.
- 3:56
So you'll see that one of the capabilities that we give the agent is it's able to, um, see what's on the screen and, and see and, and, and take action, uh, on what's on the page.
- 4:08
So you could see that, um, I'm asking the agent here, "Hey, hey, what's on the screen? Can you maybe highlight, uh, some of the next steps that I could take?"
- 4:16
So you can see that the agent here is thinking. It's saying, "Okay, what tools do I have available to use?" And, "Hey, let me go and highlight something that you could actually click on and, and tell you more about it."
- 4:27
So just another capability of OG Assist and just a little sh-short story about how this all came to be.
- 4:35
So the big bet on Effect. Um, so I really wanted to include this slide because, um, here on the agents team, we made a huge bet to, um, to, to bet on Effect.
- 4:48
And suffice to say, it's paid off in dividends. Um, we write Effect. So Effect is this library for TypeScript. It's open source, and it helps you write better, um, TypeScript code.
- 5:01
Uh, you know, it's got a lot of, uh, stuff baked in it, like a sc- a, a schema similar to like Zod, if you've ever used that. It's also got, um, things for error handling, uh, for logging, for traces, for, uh...
- 5:16
It's just got so much in there. It really helps write better code and structure your code better and, uh, helps with architecture, spinning up new services for, uh, and, and for us on the agents team, really helping, uh, design and build the, the, the core agent loop.
- 5:31
So you'll see throughout this presentation sprinkled in, um, how Effect on our team, uh, has paid off in dividends. So we, we really love Effect here at OpenGov, and we encourage other folks to try it out and, um, yeah.
- 5:47
Let's keep going. The Effect native loop. So originally, we were on LangGraph, and that was fine until the team really started to scale, uh, and our use cases started to evolve.
- 6:02
So we decided to move over to our own kind of Effect native agent loop to have full regency over this, uh, agent loop, such that if we have complex use cases or features that we need to build, we could kind of get in-- We, we had full control of the, of the agent loop.
- 6:23
And not only that, but now we're fully on Effect. So all the cool things you get with Effect is now p-propagated throughout the entire agent loop, like the tracing, structured con-concurrency, the logging.
- 6:34
Everything is more fine-grained control, and it, it really allows us to really unlock the full potential, uh, having our own agent loop from the ground up. Um, so another thing I wanted to mention is on the left side, you'll see a code example.
- 6:51
This is really the basics of the Effect loop that we're using. Uh, we're using this thing called the Effect AI package, and in that package, there's this thing called, um, there's a chat and a language model.
- 7:05
So with the chat, you can instantiate like an, a chat, for example, and then you could stream text using, um, that, that kinda stream text function. You could pass in a prompt, and what's cool is, uh, with a language model under the hood of...
- 7:22
Since we're kinda doing dependency injection, we could pass in a different language model if we were to, uh, s- hot swap to another one, for example. So really just having full control of our own agent loop just kinda gives us all the levers, and it really just unlocks the full capabilities of the model and, uh, for the
- 7:41
team as well to have full agency over this, this loop. Another thing I wanted to mention is the Agent2Agent protocol. So here on the agents team, we've had a lot of success with this protocol.
- 7:54
So this protocol being the protocol that Google created, um, kind of an open protocol for agents to intercommunicate. But, um, we found this very useful for, uh, defining our agent routes, like for example, i-in the back end and our model and our schema to follow this kind of, uh, agent protocol.
- 8:15
So we modeled-- So for example, there's this thing called an agent card, which you see here, and it's got the name of the agent, a description, et cetera, right?
- 8:24
And having this kind of rigorous protocol, this rigorous spec, really helped drive our development and drive alignment because, you know, all we had to do was, um, align with this spec and follow this spec, and we knew that this was kind of the contract that our front end and back end would both consume and, and produce.
- 8:48
So, um, this, uh, I would say also has been, uh, very helpful for us. And, and what's really cool is A2A has a lot of extensions, right? So you could extend the protocol, uh, add in like metadata.
- 9:02
Uh, there's also A2UI. Um, so lots of fun stuff, uh, with A2A protocol, but, uh, this is kind of what's worked for us. So just sharing that with, with you folks. [lip smack]
- 9:16
Feedback and evals. So here the quote is, "Shipping is the start, not the finish." So what we do here, uh, on the agents team is we have kinda multiple ways we do evals and collect feedback.
- 9:29
Um, obviously, you know, we'll have folks, uh, call in or, or email us or, or just let us know and tell us. But the main way is we have this thumbs up and thumbs down mechanism, and here, uh, someone is able to tell us, "Hey, this, this worked really well.
- 9:44
This was a great response," or, "That wasn't a great response." And that signal we take and we're able to iterate on, uh, and we can take it back and help improve, uh, you know, the response in the future.
- 9:55
Um, we also have automated evals. So in, in the, in RCI, we, we have evals that run against real completions, so we could test a prompt against, "Hey, did it hit some tools?
- 10:07
Did it do what it's supposed to do?" And that also helps with our accuracy. So, uh, those automated evals in conjunction with collecting feedback really help us, um, improve our, uh, our, our tools, our skills, um, our harness, and, and that's really how, how we're able to iterate so fast and so quickly.
- 10:31
Humans in the loop. So this is a really cool feature we built where we deterministically interrupt the agent loop if there is a tool call approval required. So if an agent tries to make a tool call that it needs human approval for, it'll show this UI, and the human, uh, can click accept or reject, so explicitly rejecting
- 10:51
or explicitly accepting, uh, the action that the agent is trying to make. And this ensures that, uh, you know, we're building trust and also ensuring that, uh, you know, we're being safe, especially when the agent's trying to do a mutating operation and always, always, always making sure that, um, humans are in the driver's seat.
- 11:13
Sandboxing. So another thing that we, uh, worked on, um, kinda similar to the safety slide we just saw was, um, whenever an agent tries to execute code or tries to create files, it does so in a sandbox.
- 11:29
So we gave our agents sandboxes such that it could spin up these sandboxes on demand, and it could use those sandboxes to honestly write code, execute code, create files, and it's kinda this safe, ephemeral, isolated space such that the agent can, can, can take action in there and not-- and we don't have to worry about, um, any
- 11:52
risk to, you know, our, our production systems. Um, and it's, it's really cool 'cause they also get, uh, tied, uh, teared down at the end. So, um-
- 12:03
In this example, I said, "Hey, create a PDF, uh, for the folks of the AI Engineer Conference 2026, and, um, allow me to download it so I can share it with them."
- 12:12
And you can see that the agent created this really cool PDF inside of that sandbox. So just really wanted to cover this sandbox feature and just give you a brief kind of overview of, of sandboxing.
- 12:27
Long context. So, um, inside of OG Assist, we have hit many hurdles, like with, especially with legacy models now, like with, uh, token limits or just way too much, just completely overloaded with context, especially as conversations get longer.
- 12:49
So we found that having some sort of, um, rolling summarization was more effective than, you know, always stuffing in the latest and most recent m- uh, messages. Uh, rather just, you know, give like a running summary after N number of messages and, uh, maybe you only want the, like N minus five
- 13:14
most recent messages or N minus 10 most recent messages, right? Um, and it may be that you're only talking about a specific topic now, but you may want to refer to, uh, context earlier, like 100 messages above, then, um, it, uh, that's where kind of the, the memory component comes in, 'cause when you have this rolling summary
- 13:35
of a, a really long conversation, then you could do recall over that s- uh, summarization and, um, you know, if you ask the agent, "Hey, remember that thing that we talked about?"
- 13:46
Then the agent within the thread will, uh, be like, "Yeah, I do know what you were talking about. I have kind of this short little tidbit," and it can, you know, follow up and, and, and do more kind of with that, uh, uh, summary, rolling summary in mind.
- 13:58
So that's kinda how we handled long context and memory, and it's worked pretty well for us. So, uh, just wanted to share a little bit about that and how we've, uh, solved the long context problem.
- 14:11
UI on the fly. So, um, in this example, I said to the agent, "Hey, generate me a long essay, but give me some examples about what the essay could be about."
- 14:20
So what's really cool is the agent had this primitive registered of, uh, this form, and it was able to build out this form for me at runtime and give me some options of what I could choose from.
- 14:32
So it feels very personal, and it feels very kinda in the moment that it's able to, to give me these options just at runtime. So, um, this is kinda just a short little, um, thing I wanted to include here about generative UI and how we, uh, are able to render UIs on the fly.
- 14:53
You can't scale what you can't see. So, um, this, uh, this kind of section is about tracing, uh, and observability. Really, what's cool about Effect is you kinda get tracing out of the box.
- 15:10
Um, you know, when you use these Effect functions, they all get kinda tagged automatically with like these spans, and kinda the span gets picked up and feeds into these traces so that you can kinda get these kinda drill downs of these function calls.
- 15:26
So here is an, an example of a trace from the Effect team. Um, I have it linked. Uh, you could see that like, hey, when you hit this API, it goes to this endpoint, to this handler, and, uh, you know, et cetera, and it takes...
- 15:41
And what, what's really cool is you could profile all your traces, right? So this takes a total of this many seconds, and you can see where the bottleneck is here.
- 15:49
If there's a failure, you can cross-reference it across services. So really, really important, especially working in agentic systems where we're, we're integrating with other teams and other APIs and other, um, other platform capabilities.
- 16:06
So, uh, what's cool with, um, Effect is you get all this tracing out of the box, and it really makes building this agentic experience, debugging it, and maintaining it just a breeze.
- 16:19
Tools and skills. So, uh, not only did we make a big bet on Effect, but we also made a big bet on tools and skills. So we believe that tools and skills are really all you need.
- 16:31
And in this case, you could see on the left, we have this tool called Get Dad Joke, and this is kind of the Effect way and kind of the building blocks of how we do things here at OpenGov.
- 16:43
But, you know, this is pulled from the O-Effect website, but you could see, hey, this is how you make a tool, and then you add it to a toolkit, which is a collection of tools, and then you can register this, this toolkit with the, with the, uh, language model.
- 16:58
So for example, if you had a prompt that said, "Hey, generate some dad jokes about pirates." Well, guess what? The agent has a tool, uh, that can help get, uh, a dad joke.
- 17:11
So, um, really this is the, the building blocks of how we did tools, uh, and eventually skills, and really just is, has paid off wonderfully for, for our organization.
- 17:25
So, um, really recommend trying out this Effect AI package from, uh, Effect and, um, just trying out building out your own tools and skills.
- 17:38
Developer velocity. So not only do we build agents for our customers, but we also use agents, um, here internally, uh, in OpenGov. So we, uh, use a lot of Claude and Cursor, uh, and it's just really been a game changer for our team.
- 17:55
Um, it's funny 'cause we're building tools and skills for customer-facing, uh, agents and, and that has been great, but we're also building them internally as well to help accelerate our development workflows.
- 18:05
So things like Claude, Cursor, Claude Agents, they really help accelerate how we read, write, review code, and, and ship. Um, so it's, it's just been such an accelerant, so definitely just wanted to mention that, uh, before we wrap up.
- 18:23
That's it. Thanks so much for watching. You've made it to the end. Let's build agents that ship to production.