The Rise of the Agent Workforce: Rethinking Agent Infrastructure

2:46:35All right. Uh well, super excited to be

2:46:38here. My name is Ankit. I'm from Data

2:46:40Bricks. Uh I work on AI infrastructure

2:46:42there. And today I'm going to talk about

2:46:44>> [music]

2:46:44>> uh the rise of the agent workforce and

2:46:46how we're building our agent

2:46:48infrastructure to support what we think

2:46:50is the next generation of agents.

2:46:52I wanted to start by taking a step back

2:46:54and and thinking about how we help

2:46:57computers become good at other things uh

2:47:00better even than humans. And one of the

2:47:02games I grew up playing a lot was chess.

2:47:04Uh it's one of the great examples of

2:47:06something where computers became way

2:47:07better than humans a long time ago. But

2:47:10I think it's important to think about

2:47:12why that happened and how we can apply

2:47:14those lessons today. One of the biggest,

2:47:16you know, to summarize at a high level,

2:47:18a lot of work went into it. The biggest

2:47:20thing that agents were able to do, that

2:47:22computers were able to do was explore

2:47:24the space of decisions much more

2:47:26effectively than humans. For most

2:47:29complex problems, there's actually a

2:47:30bunch of different ways you can solve

2:47:32the problem. And at each turn, you can

2:47:34kind of think of three more decisions

2:47:37that you can make. And really, it's the

2:47:38exploration of that space that all of us

2:47:40are engaged in. All of us who solve hard

2:47:42problems are engaged in the search space

2:47:44for what's an amazing product, what's

2:47:46the right solution to this math problem,

2:47:48and etc., right? uh and I think we've

2:47:50seen some really exciting progress from

2:47:53the labs like Enthropic, OpenAI and

2:47:54Google on solving problems that are

2:47:57wellformed. So, you know, if you think

2:47:59about chess, it can be modeled

2:48:00effectively. You can evaluate your

2:48:02position. Uh you can actually play

2:48:05simulate it. You can try different moves

2:48:07and safely think about all the different

2:48:09alternatives. Similarly, with math

2:48:11problems, often the ones that agents are

2:48:13really good at solving, they you can

2:48:15actually verify if you got the result

2:48:17right. uh and how do we then take that

2:48:20mindset and apply it to the problems

2:48:22that we actually face day-to-day.

2:48:25I think like let's a lot everyone here

2:48:27is an engineer. So let's take it to

2:48:29engineering for a bit. Uh I think you

2:48:31know if you take it to the preai days uh

2:48:34most people think about solving a single

2:48:37problem and they're kind of

2:48:38singlethreaded on that problem for the

2:48:39time that they're writing code on it. uh

2:48:41and you know as a company if or a

2:48:43product if you wanted to solve more

2:48:44problems you brought in more people and

2:48:46the very basic version of agents just

2:48:49helps create leverage for that engineer.

2:48:51Now you can be singlethreaded on three

2:48:53things at the same time and ultimately

2:48:55that helps you explore new product

2:48:57ideas, ship faster, iterate faster and

2:49:00that's obviously very exciting. But it's

2:49:02it's not actually close to what we

2:49:04needed to do to win at chess or any of

2:49:06these other problems. What we're seeing

2:49:08at the absolute frontier are these teams

2:49:11of agents that are able to make

2:49:12significant progress by exploring the

2:49:14decision space more effectively. So how

2:49:16do we move towards that world where we

2:49:18actually have these teams of agents that

2:49:20are coherently attempting to solve

2:49:22problems? It's actually quite complex

2:49:24and we think it requires a little bit

2:49:26different type of infrastructure.

2:49:28Specifically, I think an important

2:49:30question to ask is how do we give agents

2:49:32a realistic world to explore? Because in

2:49:36other problems, we're able to help

2:49:37define that world very effectively. But

2:49:39even in software engineering, we

2:49:41actually really struggle to do that.

2:49:43Like just consider this movement towards

2:49:45software factories, right? Where you're

2:49:47away from the from the agents and you're

2:49:49actually operating a bunch of agents

2:49:50that are working on a problem. How do

2:49:53you validate that those agents are

2:49:54effective? Like before you had a branch

2:49:58uh you had main, you branched your PR

2:50:00off of that. Maybe you have an agent

2:50:01working on that PR and you have some

2:50:04reasonable assertion that this is kind

2:50:06of a sim, you know, you have some ass

2:50:08like some tests or something like that

2:50:10that help you guide your development.

2:50:12But if you're actually merging tens of

2:50:15PRs a minute or even more, right? Like

2:50:17if you play out the future of factories

2:50:20of agents building code and software,

2:50:23your PR is just out of date the moment

2:50:24you branched. The the moment you

2:50:26started, you're already out of date. So

2:50:28it demands a different mindset of

2:50:30thinking of how do we verify these

2:50:32things? You actually might want to move

2:50:33most of your verification to postmerge.

2:50:36And so that kind of brings me to this a

2:50:38little bit radical idea. I I don't think

2:50:41we should try to write hundreds or

2:50:43thousands of unit tests. Instead, I want

2:50:45to explore this idea of simulating

2:50:48reality. Can I give my agent a faithful

2:50:51reproduction of what's actually there in

2:50:53the real world in my production code in

2:50:55my production agent and let it work on

2:50:58that? And with when you can do that, you

2:51:01can actually do that at scale. If you

2:51:03find a way to simulate reality, then you

2:51:05can actually do that across lots of

2:51:07different explorations.

2:51:09At data bricks, one of the big things

2:51:10that people do on our platform is build

2:51:12data pipelines. We've got tens of

2:51:14thousands of data pipelines on data

2:51:16bricks. Uh and the biggest problem that

2:51:19happens with pipelines is that they

2:51:20break. Um and in this era of agents,

2:51:23what people want is they want to fix

2:51:25those broken pipelines with agents

2:51:28because these are serious business

2:51:29problems that they that are, you know,

2:51:31uh caused by a broken pipeline. If a

2:51:33pipeline breaks, that business can't

2:51:35operate that day. Uh and so this becomes

2:51:37a major emergency.

2:51:40But like how do you actually validate

2:51:41this with an agent? Do you trust it on

2:51:43your production data pipeline? It's

2:51:45pretty risky because writes might not be

2:51:47identy

2:51:54broken. It's just partially broken. One

2:51:56of the things we started doing was

2:51:58building on top of branchable

2:51:59infrastructure. And so what we did was

2:52:02we branched the entire data platform. We

2:52:05branched the tables, the compute, the

2:52:07governance and we gave the agents a

2:52:10place where they can actually play with

2:52:12the data platform and explore different

2:52:14hypotheses. And these hypotheses can be

2:52:16ambitious like can I investigate that

2:52:19the upstream pipeline is healthy? Maybe

2:52:21I can do that by changing the pipeline

2:52:22code and running it again. In any

2:52:24reasonable data platform, you wouldn't

2:52:27do that with an upstream pipeline unless

2:52:28you had a bunch of engineers take a look

2:52:30at it and make sure it was super safe.

2:52:32with our branched data platform. You can

2:52:34actually do that now. So it can

2:52:35investigate a bunch of different things.

2:52:37So we can investigate 10 ideas in

2:52:38parallel with our agents actually get to

2:52:41a faithful conclusion about whether it

2:52:43fixed the real data pipeline and then

2:52:46make that assumption made that assertion

2:52:48back either to a main agent or to a

2:52:50human. And what we saw before was

2:52:52happening was that people would just

2:52:53like LLM would give us text back some

2:52:55human would take a look at it and be

2:52:57like okay this might be an interesting

2:52:58theory. then they would go apply that

2:53:00theory in some way. And ultimately that

2:53:02became more of this like AI assisted

2:53:05version of fixing the data pipeline. But

2:53:07really where we want to go is is is

2:53:09hands-off full auto for this kind of

2:53:13problem. And we were we were struggling

2:53:15to do it without this branchable

2:53:16infrastructure. So how do we do it? I'm

2:53:19going to talk a little bit about that in

2:53:20this talk. Uh we built this on top of

2:53:23our Postgress native agent platform

2:53:25which started with the Neon database

2:53:27which is a branchable database that

2:53:29gives us a single place that we can

2:53:31actually use for our services and for

2:53:32our agents. And then we added in a bunch

2:53:35of really core agent primitives that

2:53:37allow us to build sophisticated

2:53:38different types of agents. Uh and those

2:53:40are ones that are probably familiar to

2:53:42all of you. Things like memory, things

2:53:43like sandboxes, file systems, and ways

2:53:46to host those agents. And these

2:53:48low-level primitives we can build we can

2:53:50build with them. But the key is that we

2:53:51made just all of them branchable. So you

2:53:53can branch the entire reality of stuff

2:53:56that's built on this platform for us for

2:53:57ourselves. Uh so how do we actually do

2:54:00that? Well, let me walk it through like

2:54:02kind of case by case. The first thing is

2:54:05just the database, right? And you know

2:54:06we can take an example of application

2:54:08services. But here I'm going to talk

2:54:09about just even storing the conversation

2:54:11state for for an agent. When you fork or

2:54:14branch an agent, you actually might want

2:54:16to inherit the history. And you might

2:54:18even if you have a team of agents give

2:54:20access to a given member of that team to

2:54:22what the other members of the team are

2:54:23doing. Uh and we found the database to

2:54:25be a convenient abstraction for this. Uh

2:54:28the key was that hey like when a new

2:54:30reply comes in on the on the branched

2:54:32agent it shouldn't necessarily change

2:54:34anything about the other one. Uh and

2:54:36this is possible because Neon has a

2:54:38branched database. But then we actually

2:54:40found that agents need more than just

2:54:42structured databases. Of course they

2:54:44need structured databases but they also

2:54:46like to do this you know write these

2:54:48things called memories and memories are

2:54:50you know at a very high level they

2:54:52compile down to an unstructured database

2:54:54for agents. There are two ways that

2:54:56memories get generated. The first is

2:54:58just by synthesizing the context and the

2:55:00conversations that someone's already had

2:55:02with them and the second by agents

2:55:04actually explicitly saving content to an

2:55:07unstructured place. The key thing here

2:55:10is that you actually want to consolidate

2:55:12and distill that information. there's a

2:55:13ton of information coming in through

2:55:15there and if you just give that you know

2:55:17if you just let it kind of go on its own

2:55:19you have problems of staleness you have

2:55:22conflicts in the information and all of

2:55:24those can be extremely damaging to teams

2:55:26so we have this thing that basically

2:55:28dreams overnight we use offline compute

2:55:32uh and it takes the conversations a

2:55:34different agent essentially highlights

2:55:35things that are conflicts and

2:55:38consolidates the ad hoc notes that were

2:55:40written by various members of the team

2:55:42and then all members of the team can

2:55:44actually recall from that memory

2:55:45database. And you can use this in a

2:55:47bunch of different ways. We use this

2:55:49within our Slack agents, but we also use

2:55:51this when we're launching a parallel set

2:55:53of agents to investigate a deep issue,

2:55:56for example, in the data platform. Now,

2:55:58how does branching come into this? Well,

2:56:01some of the agents like in many of the

2:56:02use cases, they propose memories and

2:56:05it's not clear whether those memories

2:56:06generalize to the actual broader set of

2:56:09agents or if that's a specific thing for

2:56:11just that agent. And so with the

2:56:13branchable database for memory which is

2:56:15built on top of this neon database, we

2:56:17can actually just propose that memory

2:56:18somewhere in there and that branch can

2:56:21continue to iterate and at some point we

2:56:23can find a way to merge it back in if

2:56:24it's necessary. uh we found this to be a

2:56:27flex an important layer of flexibility

2:56:29because without the ability to

2:56:31completely branch the memory we found it

2:56:33was very hard to maintain the memory for

2:56:35a team of agents especially as we

2:56:38increase parallelism you can actually

2:56:39experience like significantly reduced

2:56:41quality if all the agents are pursuing

2:56:43different goals uh I I I like to think a

2:56:46little bit of it as as like kind of

2:56:48similar to dynamic programming right if

2:56:49you've explored a branch of exploration

2:56:51and then you have a different agent that

2:56:53happens upon that same pathway It's very

2:56:56damaging to like always revisit that but

2:56:58sometimes you want to and so the ability

2:57:00to propose a memory is extremely

2:57:02important to us.

2:57:04How is all of this kind of branchable

2:57:06database possible? Well, the way that

2:57:08Neon does is it is it separates compute

2:57:10from storage. Compute uh for Postgress

2:57:13is running in some separate place and

2:57:15then we actually have the shared history

2:57:17and the storage in a different layer and

2:57:19what that lets us do is copy on write

2:57:21for branching. So when we have a

2:57:23branched version of the database, you

2:57:25can actually have A's uh you know

2:57:26compute A's rights separated from

2:57:28compute B and that's the kind of the

2:57:30core layer of infrastructure that allows

2:57:32us to build these applications on top of

2:57:34that database.

2:57:37But then we found you know obviously

2:57:38agents are far more than storage and and

2:57:41branching storage wasn't enough.

2:57:43We also found that as they were running

2:57:45in sandboxes, agents like to take notes

2:57:47on stuff and write down scripts and use

2:57:50those to actually iterate on various

2:57:52kinds of problems. It it it's one of the

2:57:54most exciting things about agents that

2:57:56I've found that they are able to

2:57:58generalize this capability that's needed

2:58:00maybe for testing code and and things

2:58:02like that and solve other more broader

2:58:04general problems using that same con

2:58:06construct.

2:58:08But then when you branch it, you

2:58:10actually need to give the future agent a

2:58:12similar context. If you we found that

2:58:13when we started the branched agents

2:58:15without the same files that that agent

2:58:17had to begin with it just wasn't as

2:58:19effective. So when you're investigating

2:58:21an outage for example with an agent you

2:58:23might have notes and plans and a future

2:58:26branch of that agent might update those

2:58:27notes. It might find new things but we

2:58:30don't want that to go back because we're

2:58:32actually exploring 10 different pathways

2:58:34at the same time. Uh so building a

2:58:36branchable file system was extremely

2:58:37important and allowed us to build what

2:58:39we call a branchable sandbox because

2:58:42then what we can do is we can launch

2:58:44that sandbox compute and and that's a

2:58:46hard problem in and of itself. It's not

2:58:47one we're going to talk about today but

2:58:50that compute can be attached to storage

2:58:52and critically the agents actually

2:58:54really aggressively leverage files very

2:58:57often. Uh and so it can be very painful

2:58:59to make all your storage fully remote.

2:59:01Uh and so the really core piece of what

2:59:03we've been building here is a syncing

2:59:05process between what we call a local

2:59:07cache and the remote cache. Uh and then

2:59:10what that allows us to do is as we're

2:59:12able to make progress on that when we

2:59:14want to branch we can actually branch a

2:59:16second sandbox use that similar file

2:59:19system and it reduces our latency for

2:59:21for branching while also keeping

2:59:23performance of the regular agent

2:59:25reasonable.

2:59:27So what this allows us to do at a high

2:59:29level is branch the entire working

2:59:31state.

2:59:33And this gives us the platform layer to

2:59:35actually go do that. Now this isn't

2:59:38obviously sufficient. You're going to

2:59:39want to branch even more. If you're

2:59:41building on top of software, you want to

2:59:42find a way to branch that too. And so

2:59:44this is along the lines of how we're

2:59:46thinking, right? Like we the data

2:59:47platform was built on top of some of

2:59:49these things, but there's a lot more

2:59:50that we've been able to branch there as

2:59:52well. Uh but we're really excited about

2:59:54this layer for just even building

2:59:56agents. And just to like kind of

2:59:58separate a little bit even from software

3:00:00development like another thing a lot of

3:00:02us are spending a lot of time thinking

3:00:04about is security.

3:00:06Uh, I mean just taking a step back, I

3:00:09think this is probably the single most

3:00:11important thing for any product that

3:00:12anyone sells at this point because trust

3:00:15is kind of the most important thing that

3:00:16we're selling when we work with other

3:00:18people. And uh, with agents being the

3:00:21ones who generate the vulnerabilities,

3:00:24uh, it's kind of impossible to find a

3:00:27different solution than having agents be

3:00:28the ones who are monitoring for those

3:00:30exact vulnerabilities in your code. So

3:00:33you should be thinking about how you can

3:00:34build an agent factory that's detecting

3:00:36security issues in your codebase. Um and

3:00:39so this is something we spending a lot

3:00:40of time out spending a lot of time on.

3:00:43So typically what happens in a security

3:00:44incident? Well, let's say you suspect

3:00:47something. An agent might be always

3:00:48looking for it. Why why is there this

3:00:50like network traffic that's erressing

3:00:52all of a sudden from my control plane?

3:00:54Well, I can assemble an incident

3:00:56timeline and that's obviously the first

3:00:57step in an investigation. But anyone

3:00:59who's been a part of a serious security

3:01:01investig investigation knows that time

3:01:03is of the essence. It's a perfect

3:01:05problem for parallelism. And that's

3:01:06exactly what you do when a security

3:01:08incident happens. You bring everyone

3:01:09online so you can explore 10, 15, 20

3:01:12different theories at once because an

3:01:14hour earlier that you can you can detect

3:01:16the real issue is an hour that you're

3:01:18actually more safe. So okay, what if we

3:01:22did that with a branchable agent? Let's

3:01:24say there's two paths to investigate. Uh

3:01:27so you know the agents assembled the

3:01:28incident timeline and now you can ask

3:01:31two questions. Hey did a recent

3:01:32deployment in introduce some kind of new

3:01:34integration like maybe I can go look at

3:01:36the code or I can explore a totally

3:01:39different theory. Is a compromised

3:01:40identity accessing this data and the

3:01:43core thing that we are able to do here

3:01:44is we branch that whole original agent

3:01:46that was starting the investigation

3:01:50and then the new one is able to work in

3:01:51a completely isolated copy of the world.

3:01:54And in some cases when we're able to, we

3:01:56can even branch the actual service that

3:01:57we're investigating. Uh, and so when you

3:02:00have this isolated copy of conversation,

3:02:02memory, and storage, you can fully

3:02:04explore that security risk and and let

3:02:06the agent loop on its own. And that's

3:02:08kind of the key thing here. The way that

3:02:10these agents are good is that they're

3:02:12able to loop, verify, try new theories.

3:02:15I think Andre Karpathy said it really

3:02:17well. They're far more tenacious than

3:02:20any human probably in the history of

3:02:23software. And so they're able to just

3:02:25but like we keep getting in the way by

3:02:27inserting ourselves as an approval

3:02:29layer. Uh and so this is a solution that

3:02:31we think helps with that. Just like take

3:02:33away the consequences of making a

3:02:35mistake a little bit. Branch the entire

3:02:37state so that you're not actually you

3:02:40know worrying about that same risk. And

3:02:43then you we you we've been able to find

3:02:45this to actually uh come to a reasonable

3:02:47conclusion as well. So what does this

3:02:50all add up to for us? We think that your

3:02:53infrastructure needs to be branchable

3:02:55and that's what we're building with Neon

3:02:57with data bricks. Uh and we're super

3:02:59excited about that branchable future

3:03:01because we think that teams are going to

3:03:02be the next generation of agents both

3:03:05for software and the way that we

3:03:07actually build products. Thank you.