Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic

In 1969, just about 30 miles from here, Ken Thompson, uh, was sitting at Bell Labs, and his wife had just taken their [REDACTED], uh, son out to California to visit his parents for a month. And so he sat down and he said, "I have a month, and I have this project that I wanna do. I'm gonna carve it up, and in one week I'm gonna build a kernel, in one week I'm gonna build an editor, in one week I'm gonna build an a- assembler, and then, uh, you

know, one week I'm gonna build, uh, uh, one other thing." And he had... At the end of that month, he had, uh, Unix. Okay. So what- what's beautiful about this process that Ken Thompson went through is that he had what's called conceptual integrity, and that means that one person can hold the entire project in their head and execute the entire project end to end. Now, there's another operating system, uh, that, uh, was, uh, you know, a huge effort, and that happened

also in the '60s and the early '70s at IBM when they were trying to build System/360 and OS/360. And this was an undertaking where previously IBM computers, if you wanted to upgrade your computer, every time you upgraded the actual hardware, you had to rewrite all of the software that came along with it. And so OS/360 was this massive effort where they really bet the company and they hired tons and tons of engineers, and they- they sat down and did some back of the map, back- back of napkin math and said, "Okay, this is gonna take about

5,000 sort of man years, so let's hire and throw as many as 1,000 engineers at this project and see if we can get it done in, uh, in a, a short window." And there was a lot of learnings that came out of that project. Fred Brooks was one of the, the managers of the project, and he came away and said like, "Okay, we learned a whole bunch of stuff. Let's write it down." So he wrote it in, in this book called The Mythical Man-Month. So it's like a very popular book, uh, and this is a bunch of organizational learnings and management learnings from how to build and, uh,

ship this really massive project in OS/360. One of the big takeaways was that, uh, a really effective way to organize a team is kind of like a surgical team where you have one sort of leader or one chief, and they're surrounded by 10 experts. And so you might have, uh, you know, the surgeon, and then they have, uh, you know, a language lawyer, and they have a sidekick, and they have a co-pilot. And then all of those people have their specialties, but the surgeon ultimately has everything in their head, and they can maintain that conceptual integrity. And that's great, and you can perform

miracles, uh, surgeons do it all the time with this small team, but we also wanna take that same sort of structure and scale it up much further. If we fast-forward to just a, a month ago, a couple months ago, we saw Linux 7.2 was released, so this is another operating system project. This took nine weeks, and we had contributions from 2,600 something different developers across 300 different companies, and, uh, it didn't start out that way, like the actual contribution to Linux didn't

start out that way, but we sort of evolved the way that we can contribute to Linux into this structure. And so if you look back at the kernel mailing list from 1998, you'll see that Linus was actually like kinda having a bit of a breakdown. He was receiving patches from people that were kind of like emailing him and then, uh, you know, saying like, "I've emailed you three times with my patch. Why haven't you, uh, submitted this to the, you know, to the kernel?" Or whatever, and he's like, "That's it. I'm going on vacation. I don't wanna hear it. Get out of my inbox."

And, uh, sort of, you know, he- he- he hit his breaking point, and I think a lot of us who are orchestrating and managing lots of agents now might start to feel this feeling of the, being the bottleneck and being the breaking point as we have lots of agents contributing tons and tons of PRs, and we're sitting there, you know, in our, in our GitHub trying to like approve PRs and make sure that everything works as expected. And so, uh, it wasn't until a couple years later that we actually changed. But actually the day after Linus

posts this like kind of, "Ah, you know, I'm done. I'm going on vacation," uh, another person posts and they s- uh, Larry McVoy, and he says, "The- the problem is that Linus doesn't scale." And, uh, Larry also did a little bit of self-promotion and he's like, "I have an idea. How about we use version control?" And was kinda like pushing this commercial product. Uh, and so it actually wasn't until maybe four years later, around 2002, that Linus was like, "Okay. You know what? It's true.

I don't scale. And instead of, uh, operating as a star structure where everyone is sort of contributing directly to me, we need to work more as a network where we have these layers of trust, and people can submit contributions to trusted maintainers who are gonna review that code. And then only those exceptional cases will be surfaced up," uh, but Linus can still kinda like ultimately be the person that is, uh, committing to main. On the left you'll see this star structure has like 36 different actors that are all

kinda contributing in. On the right, we end up in this like nice beautiful structure that only has like maybe at most four people at each level sort of reporting those changes up.

Uh, in San Francisco, uh, if you were at AI Engineer, you heard from Tariq and Gary, and the message was, "We have Fable. It's really exciting, and it's time to be unreasonable. It's time to be more ambitious and build bigger things." And Gary told you it's time to boil the ocean. And so we have these very, very capable models, and we're excited to talk a little bit today about how we can orchestrate these models into, uh, systems that can

collaborate with each other to build really, really massive, uh, projects and, and solve even bigger problems. So I'm CJ Avilla. I am a member of technical staff at Anthropic, and this is

Harrison, and I'm also a member of technical staff at Anthropic. And good morning, everybody. How we doing? We excited? Nice. Lot of energy. Lot of energy. Woo.

Uh, all right. Well, as we get going here, um, just wanna give you guys a quick lay of the land of how this session's gonna run. Um, it's gonna be some part, like, telling you what the heck even is Claude Managed Agents, and what are we about to talk about, and, like, this whole multi-agent, uh, orchestration engine that we're, uh, gonna spend a little while explaining. Um, it's also going to be part hands-on workshop, so, um, laptops out. We'll have, uh, developer credits, uh, on, on slide later on, so you can actually get your hands dirty, uh, trying out some of these features that we're super excited to ship. Um, but before that, we, uh, we,

uh, we'll give you a, a little bit of over- of an overview on what the heck is going on here. So, on Friday, um, we are so excited to announce within a product surface called Claude Managed Agents that Anthropic ships or shipped back in, uh, April, that we support a new feature called Flows. Flows are essentially, like, the next version of multi-agent orchestration, where you can let Claude fully orchestrate the entirety of its own multi-agent topology. So, like CJ was just

talking about these really complicated interconnected webs, uh, that map to organizations in the real world, we envision that there's a future where Claude is able to basically orchestrate itself, right? And, uh, Flows are the way that we do that on our API surfaces. So before we get too deep into, uh, the API surface and the experience specifically, um, a lot of you are probably familiar with Dynamic Workflows and Claude Code. I mean, you type in, "Hey, Claude," like, "Go and do X, Y, and Z. Use UltraCode." It spins up this,

like, pretty complicated web of agents. It's fully orchestrated itself. It uses a really interesting JavaScript engine under the hood to basically support, like, MapReduce-esque like, uh, activities in, deep within Claude Code. Um, and we know that this has become a really, really popular product, uh, for, like, typing into a terminal and, like, using it live. Um, so there's a bunch of examples on slide, but, um, the idea is that instead of having to, like, literally draw out this classifying act or fan-out-and-synthesize adversarial verification style, um, like topology, where you say,

like, "This is agent one, and it does this, and this is agent two, and, like, wait until agent one takes some action," and yada, yada, um, you can literally just tell Claude to use UltraCode to, uh, achieve its goal, right? And Claude will just know, based on, uh, model training that we've done internally and generally just, uh, its own capabilities, how to effectively orchestrate itself in order to accomplish harder and harder work. Um, so we, we think this is, like, we're getting a taste of how we're going to solve some of what

we think are humanity's hardest problems using, like, fully agentic automation. And so this is, like, I don't know. I, I definitely nerd out about this stuff, but I'm, like, so, so excited about the direction that this is even headed. Um, what's really cool about Dynamic Workflows and just generally, like, this u- using multi, uh, multi-agent topologies rather than just a single agent, is that a single agent keels over much, much earlier with much less ambitious tasks. Um, so, uh, you know, the hand-waved numbers here, right? Uh, a single agent runs for an

hour. Uh, a coordinator and roster, which is, like, uh, our first pass at multi-agent, where we had, like, kinda this, like, single coordinator that, like, has just one level of agents underneath it that can go off and accomplish very specific tasks, um, can run a little bit longer. But then Dynamic Workflows, we think this is, like, the key to solving actually hard problems, which enables arbitrarily complex nested sets of, uh, uh, of agents operating all in parallel and interacting with each other. Um, so we spend a lot

of time talking to our users, um, and I'm also, like, super excited to talk to all of you guys today because I imagine some of y'all have some hot takes, um, on the latest in the industry as well. And, um, we, we've, we've been hearing a lot of common, uh, uh, common themes throughout. Um, so first of all, you guys are doing some crazy stuff, uh, building out your own multi-agent harnesses, right? You've got incredibly complicated systems that you're building in order to basically solve for some of those problems that we were talking about just a second ago with the single agent.

Um, so you're, you're hand-rolling multi-agent orchestration. You're having to handle container restarts, right? Like, what if one of your agents out of 100 dies, uh, you know, like two hours into the task? How you actually pick back up where that agent left off, make sure you understand generally the state of the whole topology, et cetera. Um, and it's often, like, very non-trivial to do so. Um, additionally, we, uh, uh... This, this bottom left, uh, point in the slide is basically like you're having a single agent have to keep track of, like, 60 as an, as an

example, um, agents underneath it. And like CJ was talking about, uh, Linus doesn't scale like that. You know, he's, uh, an incredible coordinator, but, uh, unfortunately, there's just, like, some cap to how much the single agent, um, is able to keep in their head. Um, and then last but not least, you have to figure out how do you even inspect the graph of agents that, um, you've, you've built out. Um, do you have to go and, like, poke at each of the nodes in the graph and, like, ask where it's at in the, in the process? Do you run everything through the coordinator? Generally speaking, like, how, how do you

understand the state of the graph at any point in the process? Um, so we've already heard this just from people using Claude Code, right? Um, and, uh, our first renditions of our own APIs as well. Um, y'all might remember, there, there was this meme that when, uh, I think it was probably, like, post Opus 4.5, when, when the models really took off and the, uh, the capability just made autonomous coding so, so, so powerful, where people were, like, walking around with their laptops slightly open so that their Claude Code wouldn't die and, like, making sure

that, uh, their agents were still running. It's like people in the park, like, pressing two, like approve, approve, approve, approve, approve, um, just to keep the, uh, uh, the thing running. Um, and we think that that's, like- Hugely limiting, right? Like, that is, uh, if we wanna solve truly hard, like, society-level problems, that's just not going to scale, uh, for our ambitions. Um, so fast-forward to Claude Managed Agents and how we've embedded this kind of complex multi-agent topology within our own ecosystem. Um, so I recognize that a lot of you in this room probably ... Actually, let

me ask. Um, how many of you have heard of Claude Managed Agents? Oh, wow. That's actually, uh, pre- pretty solid, so about 50% of the room. Um, for those of you who are not inducted, you should check it out. Uh, it's also, like, all I think about every day of my life- ... uh, basically for, uh, the past year or so. Um, but it's essentially an API surface that lets you, uh, have Anthropic manage the, uh, the agent life cycle. So you define an agent. It's basically, like, a list of skills, tools, MCP servers, et cetera. Um, you instantiate that

agent within a session, which is like an, a request essentially to, to get real work done, and then that agent just goes off and g- goes off and cooks, uh, based on whatever it is that it's trying to accomplish. Um, so we have a whole API surface that makes it, uh, possible to store these, like, stateful API resources and then monitor everything in real time via durable event log. Um, so yeah, check it out, Claude Managed Agents. Uh, and w- we're here of course to, uh, to talk you, talk to you all about it. Um, but back in, I think it was May, we shipped what we called, like,

multi-agent V1, right? And this is where you have to define each of the sub-agents that your primary agent is able to spin up, um, and then you have to listen to basically all of what we call the threads that each of those agents are operating on, right? So, um, it, it's a lot like, you know, an operating system, right? Where you have the, the overall process has different threads. Each of those threads have their own context. Um, there is some sh- uh, state that's shared between them, but typically in order to share a state, you have to, like, manage message, message sends, uh, back and forth between all the threads.

Um, my point in saying all this is that, um, it, it wasn't the most, uh, trivial thing to set up, even though our API surface supports it. Um, but fast-forward to last Friday, um, where we shipped this new thing called Dynamic Workflows within Claude Managed Agents, and we think that this is, like, way easier and way more possible to get power out of the, these increasingly capable models in a very trivial way. So the way that it works is that you have a lead agent that is essentially responsible for creating a plan for how it wants to accomplish a

task. So, you know, migrate my million line monorepo from Python to Rust, as, as an example. Um, that agent is able to then spin up ... Uh, one, once that agent spins up it pl- its plan, it's able to automatically fan-out to its own set of agents that it spins up and down based on, uh, its own, uh, uh, I, I guess rationality or, um, capability. And across multiple different phases of the multi-agent orchestration, it's able to accomplish, like, discrete

verifiable units of work without you having to, like, manage any of the interconnections between these things. Um, so very basically it might be like phase one, like read the files, and it spins up a bunch of explore agents to go off and do so. Phase two, maybe I, I don't know, identify something about those files. Obviously, the, the agents are way smarter than me, uh, these days and would, would come up with a better plan than, than I could on the spot. Um, but then at the end, um, y- we think that this is essentially, like,

the path to getting really, really, really hard work done on platform via an API surface. So if you wanna build, like, this into your products, this is a really trivial way for you to do so without you having to somehow, like, have Claude Code open again and, I don't know, like, connect that to your, to your own systems. Um, so while these dynamic workflows are running, um, it's, it's very trivial to, to first of all watch what's happening in real time. Um, there's a single event stream that basically gives you everything that the agents are doing in real time. You can check agent by agent what the events, what events are happening in real time.

Um, on top of that, we know that token hunger is, like, a real problem, um, and you can set budgets on your sessions before kicking them off so you don't accidentally burn, uh, substantial amounts of tokens. And then if you ... Let's say that you have, like, some system that's listening to generally what the agents are doing in real time, you feel not so excited about the direction that they might be going or you, you wanna steer them, um, in real time, then you can do so by just, like, stopping them, um, by interrupting via our API surface. So in comparison

to Claude Code, um, I, I recognize that, like, this can be a little bit confusing because the, uh, they're, they're both workflows, right? But we're talking today about these workflows within Claude Managed Agents. So we're an API surface that you should be able to integrate into your own application surfaces or your own internal processes or your own, you know, whatever it is that you might want to, uh, uh, wherever you want to, like, embed Claude within your systems. Um, you don't have to be, like, typing in a terminal or, um, interacting with the

thing in real time. Maybe you wanna hook it up to CI, right? Like, this is, like, the, the trivial way to do so. Um, so, um, in our comparison, uh, chart here, I think the thing that I would highlight, um, is that we, we see a lot of users using ... like, running these things on schedules, um, listening to webhooks. The GitHub example is a pre- pretty classic one of, like, maybe you wanna build out a, uh, a, a bug hunter workflow, right? Um, that we'll actually talk about and, and show, show off a little bit later. Um, and yeah, I think that that's probably about what I

wanna share here. Um, so with that said, let's go ahead and start one of these things. Um, enough of me yapping. Let's, uh, let's check it out.

Is that ... Yeah. Yeah. Seamless transition. Let's see. Can you see that? Yeah. There we go. All right, sweet. All right, we all good? Yeah. Okay, I'm gonna make it a little bit bigger. Um, so I also recognize that some of y'all might not have ever seen the Claude console before, so just I'll spend a minute here, uh, talking a little bit about it. Uh, uh, so, so, so hang with me here. Um, the Claude console is where you as an API integrator can manage

essentially all of your resources within Anthropic's API surface. So in the console, I'm in the default workspace, and I'll see this nice drop-down. I'll zoom out a second just so you can see on the left side. This n- nice drop-down, Claude Managed Agents. This is where all of the API resources live for the Managed Agents product. It's also where we're going to be spending our time today. Um, if you click into the quick start, you can see that there's a whole bunch of basically one click to get you going kinds of, uh, uh, workflows that you can

run. But if you scroll to the bottom, you'll see this nice little tag here called Dynamic Workflow, and that's what I've spent the last few minutes talking about. So because I recognize there is a, um, a bunch of finance-oriented folks in the room, we're going to run the Stock Watchlist Scanner Dynamic Workflow. Um, so you can see what this config is doing is essentially creating a agent that's using Claude Opus 5.5, passing in this

type called Multi-Agent 202610-01, and then providing a system prompt that is basically like what, what the agent itself is, is, uh, uh, what the persona of the agent is. Um, I'm not gonna belabor this too much, but, um, basically the, the, what... For all of your edification, this is creating the agent resource under the hood when we click use this template. So let's go ahead and get it rolling and see, see how this goes. Um, so the first thing that your agent is going to need in order to accomplish anything in the real world is an environment,

right? This is a, uh, a computer essentially for, for Claude to operate on. Um, for the last, oh my gosh, like maybe it's like 10 or 12 months at this point, we, we've really been bought in that giving Claude access to a computer makes it way, way, way more capable just across the board. And configuring your environment is like the, the way, uh, that you give, uh, Claude access to this computer. I've already created an environment, um, so I'm not going to recreate it, but if I was creating a new one from scratch, I would just hit this four button. But the, um, Stock Watchlist Scanner

env is the environment that I used when testing things out this morning. And you can see that I can just click this button that says start a session. Um, and so I am interested in Apple, Nvidia, Amazon, JPM, Exom, Cost. I don't know. Anything else I should add in here? Um, hmm, hmm. I don't know. I'll leave it because I'm not feeling very creative right now. Um, so let's go ahead and k- uh, hit enter. And what this is doing is kicking off a session that's where the

actual multi-agent orchestration is going to happen. Um, I'll jump into that in just a second. We'll like actually take a look at what's going on. But broadly speaking, this is just using the existing primitives that our API supports today in Managed Agents and going off and getting real work done. So let's jump into the Sessions pane. This is where you can see my other stuff I've been doing on the platform. Um, and, sorry, the Zoom is a little funky here, um, but I'm gonna do a, a whirlwind tour of this because, again, I'm not sure how many folks have seen this

live in the real world. Um, but in this case, we've got our agent that's our Stock Watchlist Scanner agent. This is like the, uh, I guess in the analogy, um, I forget the, the, the name of the final person who's like the, overseeing the web of everything, but this is like the person at the top of the pyramid, the, the CEO, if you will, who's, uh, managing everybody else underneath them. Um, then we've got the environment. That's the, again, the computer that I was talking about where Claude is operating, and then we've got a bunch of information about what's happening in real

time. Um, so you can see the agent is off and doing something. Um, so here's, here's the actual run happening. Um, and we'll explain more in detail about like what some of these abstractions are in just a second. But basically, like Claude is going step by step based on its own self-orchestration using its own self-defined, essentially, set of agents to go off and step by step achieve each of these things. So you can see like the, the session thread thing I was talking about earlier, which is like the, the single context window, the single instantiation of an agent, um,

is, is being spun up in parallel. Um, so yeah, pretty cool. I, I, I don't, I don't know. I, I think it's pretty, pretty sweet that all this just happens basically by, by prompting Claude. And, uh, we, we can jump into, uh, what, what, uh, what's happening in our prior session because it's a little bit farther along than this one. Um,

so yeah. The, this is basically just showing the, the same thing I was just showing, but these are all done now. And I don't know, uh, our output file that's, I'm not going to go into too much detail on, but basically the, the, the main point here is that, um, we've, we've had, I think in this case it was only like six concurrent agents or so at each step. Um, but Claude can, can spin up and down the, the number of agents based on the complexity of the task.

Um, yeah. I'm trying to think if there's anything else I want to show there. Um, I think that that mostly covers just like the, the, what this surface is. Um, but we can of course, uh, take questions in a little bit to, uh, uh, go a little bit deeper. Do you mind jumping back?

Yeah, no, let's do it.

Let's do it. Um, yeah.

Another seamless transition.

Yeah.

Yeah. So in the, in the previous demo, thank you, Harrison. Yeah, good walkthrough.

Whoo.

Yeah. Um, so that was all like, uh, an example of how you might prototype in the console and going through and setting up your, your, your, uh, your flow, testing it out in the console. This is a really common way to go through and experiment and see if the flow is behaving as you expected. And then once you're happy with the way that your flow works for whatever your use case is, typically you'll go and then integrate this into some part of your,

your own application or your own workflow. So whether that's like, oh, I actually wanna trigger this, uh, watchlist scanner on a schedule so that it runs every day or every weekday, you know, before market open, so I can like schedule this to run five minutes before or something so that I get my report right before market open. Uh, or you might wanna do something where you, you wanna kick off these flows or trigger these tasks from Slack or some, some other like interface. And so this is, uh, the code for how you would, uh, create the agent. So it's just like passing

in this multi-agent config to enable multi-agent. And then when you've create your first event and send that, uh, that first event to the lead agent, you're gonna just say run a workflow. And that's all you need to do to get Flows up and running a- and working, is just to, like, tell tr- tell Claude, "Yes, use multi-agent," and, "Yes, run a workflow." Now, by default, this is kinda like the easiest way to get started, but, um, in this example, Claude is going to, like, on the

fly, create whatever subagents are required to, uh, to perform that work. And so, um, it's, it's also possible in the, uh, in the agent config to pass in pre-configured a- uh, subagent. So if you have, like, a set of subagents that know how to, like, look at, uh, whatever Yahoo Finance or certain news sources, or, you know, the, the SEC's EDGAR website or whatever. Like, if, if you have subagents that specialize in certain activities that are important to your

research process, you can also configure those, uh, as part of setting up your agent. So you can set up, uh, individual agents that specialize in something. Another thing that's kind of like, um, really, uh, I think is, is nice to know at this point, is that you can also give these agents access to custom skills, MCP servers. You can give them access to, uh, run your, your own, um, uh, yeah, like, custom tools that run inside of the environment. So, uh, they're really, really powerful, and then all that works in

concert with Flows. Um, and we talked a little bit about the environment, and you can also run that environment on your own, um... Like, you can bring your own sandboxes. You can run this self-hosted environment in your own infra if you want to. So, um, just running through again kind of like what that watchlist scanner is doing. So, uh, typically if you were to create a watchlist scanner with a single agent, you might go into Claude Code and say, "Here's the tickers. Please go research them and come back and tell me."

And, um, Claude or any, any, like, single agent would try to do its best to look at each of those tickers. But there's a couple different things that agents are gonna struggle with when it's just a single agent, and we'll talk about those struggles later and how, uh, Flows addresses those. Um, but what, uh, what with Flows, what we're gonna do is, like, split up that list of tickers, and for each ticker, we're gonna pass it to a scan agent. So we're gonna talk about, like, the scan phase. So we take one single ticker and pass it

to a scan agent, and that is an isolated context where all it cares about is its one ticker. And it can operate without being, like, influenced by any of the other tickers or any of the other search results from the other contexts. Once that scan is done, it can pass it off to the next phase. And now, like, the review step is also its own isolated context, which may or may not need to, like, pass back to the scanner or surface those report details that need to be written out. Um, and we went through and we

ran kinda like this watchlist scanner as a single agent and also as Flows. And, um, in, in most cases, when you think about using Flows, you wanna think about, like, "Okay, I want to make sure that I have a really, really comprehensive look at this problem that I'm trying to solve. And I wanna go really deep and make sure that I'm leaving no stone unturned, and I'm willing to spend the money to do so." And so what you'll see on the bottom right here is that, like, it costs,

uh, much more to run a Flow. But it also, in the middle pane you'll see that it, like, is much more effective at finding all of the details. And, uh, if you've used, uh, you know, Dynamic Workflows, you've probably experienced something similar with Claude Code. Um, so yeah, it's, i- this is, uh, also an opportunity to talk about, like, your, um... When you're prompting Claude to build a Flow, you can ask it to throw different resources at different phases. So you might wanna say, "Okay, I actually wanna allocate more test-time

compute to the scan phase." So you can prompt Claude and say, "Please use, uh, um, you know..." Or actually, f- let's say for the scan phase, we can use Haiku, a really fast, efficient model that's all it's gonna do is just go out and scan the internet, right? And so we can say, you know, "Spin up, uh, maybe two Haiku agents per ticker to go and, like, do a really comprehensive scan." And that might be, like, a very affordable phase. And then you say, "Okay, actually, once I get all the web pages back and I'm going through the details, sometimes I'm missing

those details. So actually I wanna, like, invest more in the review phase. So I wanna have Fable review all the details that came back in depth." So I'm gonna say, "Claude, please use a Fable agent at the review phase, and maybe, like, do two or three agents per ticker at the review phase." And so you can kind of, like, imagine each of these Flows being something that you can kinda, like, expand and contract based on how much you wanna invest in that point in the process. Um, and what's... Yeah, so this is, like, a, a really effective way to kinda, like, go through, play

around with your investment at each of these phases in order to get the exact result that you want. And so, um, definitely recommend going through, like, a, uh, like, a comprehensive eval set to try and make sure that the Flows are doing what you want when it's, like, a very, uh, you know, big, important task.

The watchlist scanner that we showed you from the console is, uh, one of these templates that you can start. You can kind of just go click through. Um, there's also some of the templates are built directly and bundled with Claude Code in the Claude API skill. So if you have Claude Code, um, and you installed it starting on Sunday night, uh , uh, then you should be able to run Claude API Managed Agents onboard with watchlist scanner. And that can spin up a template, um, and then allow you to kinda, like, manage your agents directly from Claude Code. I think this is

actually, like, a really effective way to drive Claude Managed Agents. And generally to use the Claude API is from inside of Claude Code. You can direct Claude Code to say, you know, "Spin up this resource, spin up that resource," and it will use another command line interface called Ant CLI to drive and, and interact with the Claude API. And so, um, oftentimes instead of using the console, I'll use Claude Code and say, you know, "Spin up a managed agent that does X." Um, so, uh, today when you're going through and experimenting, uh, this is one of the ways

that I would recommend kinda jumping in if you don't wanna use the console Um, and then there's-- we, we experimented quite a bit with, like, what kind of initial system prompt or, you know, initial message is best suited for Flows. And we found that, like, the, the more ambiguous and kinda like freeform you let, uh, you are about letting Claude do its own thing in the beginning, the more effective it can be. So just tell Claude, "Hey, I want you to, um, use a workflow, one ticker per agent, and, uh, you know, go out and do this

scan. This is what I'm kind of looking for." And let Claude take control over, like, building out the initial Flows, and then you can kinda like, uh, fine-tune that afterwards.

Um, so let's, uh, let's play around. So on screen, you'll see a QR code. This is for, um, uh, some API credits, so you can get up and running. If you don't already have a Claude platform account, another thing we just announced this week is that, uh, with Max and Team subscriptions, you, uh, you get some, like, API credits that are part of your subscription. So, um, you can also, like, go link your, your subs to make sure that you have those API credits. So if you have Max, I think it's like, uh, Max five X,

you'll get like $100 a month. Um, and then Teams is up to five hundred, and then this is like another 50 bucks to go and, uh, experiment with, uh, this Claude Managed Agents Flows. And, uh, yeah, I think at this point we can pause. We have, like, a bit more content that we wanna go through, but w- let's pause. Please try it out. And then, um, yeah, we can, we can open up the floor for questions, uh, if anybody has any questions off the bat. Okay. So, uh, good question. So the question was, when you're in Claude Code and you run the /Claude API

skill in Claude Code, is that running on my machine or is that running as part of Claude Managed Agents, right? So, um, the Claude API skill and Claude Code are gonna run i- on your machine or in the cloud, depending on how you have Claude Code set up, and that is like a completely separate system from Claude Managed Agents. So you ha- you'll have, like, your Claude Code is gonna be running either locally or in the cloud, depending on how you have Claude Code set up. And then when you kick that off, that's gonna make API calls from the Claude Code environment to create the

agent, create the session, create all of these different resources in the API. Uh, is there a roadmap for how it merges with plugins?

So plugins right now encompass a bunch of different, like, types of resources, right? One of those is, is skills, right? Um, and so when you take a look at the API surface, you'll notice that there's, there's a field called Resources associated with it, and then there's also, like, some other API fields that are related to things like skills that show up in plugins. And so the, the short answer is that, like, the individual components of a plugin should just drop into the API surface already today. As far as, like, merging it into-- Sorry? Oh, no worries. Uh, so the other direction being, like,

your workflow generates a skill or something, and you can pull it back into your Claude Code? Or do you mind just saying a little bit more about that? I see. Okay, so the question is basically, given a dynamic workflow that you've created for, in this case, um, research on potential acquisitions, can you distribute effectively the workflow, uh, uh, as a plugin? Um, the answer is not today, and I don't have a good answer for if that will be, like, fully supported out of the box at some point in the future.

But you could take the prompt basically that's generated as part of your workflow runs and just drop that into, like, Claude Code or wherever. Um, check it in. It's a text file. It'll, uh, uh, pro- probably be pretty close. Yeah, yeah. So the question is, what about running, like, some agents in, like, a Mac environment or other environments that aren't just, like, presumably the hosted surface where, uh, Claude Managed Agents are running? Um, we have this abstraction called custom tools that's basically like a way for the model to kick out to you to ask you to take some action. So the way that this would work is you

would define a custom tool, uh, for your agent as part of the workflow run. And then anytime you want to run something, like on, let's say, a local macOS machine or, like, some, you know, arbitrary other computer on the internet or something, um, you would basically just, like, take that custom tool call, do whatever you wanna do on that machine, and then re- uh, return the result back to the model. Um, did that answer your question? Okay, sweet. Yeah, totally. So the question is, uh, we all care about token spend at some point, um, and we need to be able to

optimize, um, our flows over time to not basically spend, uh, uh, tons of tokens. Um, so there, there are a bunch of different ways to do this. This is kind of a generalizable problem that's like hill climbing on some prompt based on whatever criteria it is that you internally care about, whether it's token, whether it's, uh, capability, uh, etc. And so, um, there is-- I, I forget if we published the Hill Climb skill-

Yeah

... in Claude Code. So, so there's a Hill Climb skill within Claude Code that can probably help you hill climb at least the prompt itself. Um, as far as the tuning of, like, the, the, the, the flow operations, um, this is kind of a cop-out answer, but you effect-- You'll have the full transcripts of everything that Claude has done as part of the, uh, the flow runs. And so you can just pull those transcripts, give them to Claude, ask Claude to identify any, like, places where the flow is maybe doing

too much or not doing enough, um, and it's able to essentially optimize itself. Um, and that way you can ask Claude to optimize based on whatever criteria you, you, you want. Um, but I will say, like, uh, we shipped this on Friday, and I suspect that this type of, like, native integration is something that is, uh, gonna be top of mind for a lot of folks.

Yeah. One thing I would add to that too is that, uh, there's another product called Dreams and Dreaming where you can, um, within Claude Managed Agents, give a set of sessions to, uh, go through this dreaming process where it can kinda like crystallize and, um, like, uh, basically like improve the memory store that is related to those sessions. So you can attach a memory store, run your flows, and then after you've done it, you know, ten times or whatever, you can have those sessions

run through Dreams and have those dreams kinda like crystallize and improve the memory from, from like session to session. So I don't know if you have- Uh, more dreams, uh, thoughts or ideas, but-

Yeah, I think, uh, I basically described hand-rolling your own dreaming. Uh, but I forget we have our own product that, uh, does this for you.

Yeah.

So-

Right.

Um-

So, so you can u- you can either use the product Dreaming or you can use Flows to implement Dreaming.

Yeah.

So-

How concentrated have you been- Yeah, totally. So you can, you can visualize it directly in the console if you just want, like... Or, or sorry, stepping back. The question is, can you, uh, can you visualize the, the multi-agent topology as the flow is running? Um, within the console, you, you can. Um, and we'll, we'll, uh, give you guys a chance to check it out in just a second. Um, but if you're, if you wanna build like a custom UI around this thing as well, you totally can. Because each of these flows spins up subagents that have their own session threads, that's like the context in which it's operating,

and all of that information on a per thread basis is a- a- accessible via our API surfaces. So, you can spin up arbitrarily interesting, uh, UIs around it for, for whatever it is that you want. Yeah, CJ is probably about to, uh-

I guess you can like-

Yeah

... look at all the threads. You can drill into individual subagents. You can see what they were doing. You can also look at, um, if you're, you're accessing this programmatically, there is a huge, like really comprehensive set of events that are emitted, like, as the session is streaming. And so you'll see every single, like tool call and result, and like when it's handing off between, um, from, from the subagent back to the main, uh, you know, the lead agent, et cetera. So like, um, yeah, within the flow right

now, the UI in the dashboard is definitely something that we'll continue to improve and iterate on. Um, but yeah, you can, you can absolutely dig in and see like lots and lots of detail here. Um, so you can see like, okay, yeah, this is the thread message received event. And so you can tell like, okay, what did the lead agent actually pass to the subagent? And you can look at that payload here to deeply understand kinda like what is actually happening for like every transition in this entire graph.

And I'll also say, this is built on the public API, like the dashboard itself. So anything that you see here, you could totally hand roll yourself. The question is, is it self-aware of its own budget? So right now, the model does not know its budget, like as it's, as it's running. The budget is primarily like guardrails to make sure that it doesn't go off and, uh, run away token spend. Um, we've found that at least in today's latest models, um, if you tell it the budget, um, Claude can get a little bit anxious, and we don't, uh, we, we don't,

we don't want that to happen. Um, but I would highly suspect that over time, that, that'll improve. That's-- So the way that it works with Flows is if you hit $5, you get an event basically being like budget reached. And then if the, if you wanna like take a look at what happened and potentially up the budget, you can as the session is running. And if you wanna add more money, then you can, and the flow will pick off like where it left off. So there's this really nice pausing component where any work that you've already accomplished is not thrown away if you up the budget. Um,

but that's totally up to you based on, um, your own... Yeah. So the question is, what is the plan for integrating with Claude Tag? Um, so Claude Tag is essentially like a Slack bot that connects all of your information, uh, uh, uh, together within the Slack surface. And so it's a slightly orthogonal product surface, whereas this is an API surface where if you want to like build an inter- integration with Claude directly, you would use something like this. Um, I doubt, at least for now, that there would be any direct connection.

Other- So we roll it ourselves. You what? We would roll it ourselves. Yeah, yeah. If you wanted to like build something like Tag yourself, you could use Flows. So really Tag, we would roll an integration with- Yes. If you're using- We would kick off this $800 workflow. Exactly. Exactly. So like, yeah. Yeah.

Yeah. I, I would say the other thing too is that like, um, this is, this is something like we want to improve on a lot, is like making all of these product surfaces work better together. And so I would say expect, you know, over the next, you know, several months, whatever, to continue to see these seams start to like disappear. You may have noticed that like, you know, Cowork and Chat, kinda like we merged some of those. You can see now you can use Claude Code from lots and lots of different surfaces. So over time, we'll kinda like, uh, close the gap

there. And yeah, um, Claude Tag is definitely like a really effective way to, to use a lot of these tools. And in like we use that a surprising amount. Claude Tag is like a huge daily driver for us for, for a lot of implementation, so.

So the, the question is, is there a way for the agents to communicate with each other, um, like the subagents within the flow? Um, there's not like a direct like, uh, uh, way for you to like via the API say, subagent A, like message subagent B, and like send payload X. Um, but the a- the subagents all share a file system. And so if there is a need f- uh, within your workflow for them to communicate with each other, they can effectively just use the file system as a way

to like send messages back and forth. Um, but there's not like a, a direct abstraction that, uh, other than these agent message sent and received where you can like see the message sends back and forth, there's not a direct way for you to like force one of the subagents to message another one. Yeah, so the question is, is there a way for the flows to essentially modify themselves and hill climb their own prompts based on issues that they might find? Um, so the best way to do this is probably similar to like the Hill Climb skill that we were talking about just a second ago, as well as

potentially the, the Dreaming product surface, so that the, the agent can see like what it did in the past and learn from its mistakes. There's not like such a thing as, um... So actually, here, I'll, I'll just show really quickly. We, we have this, um, concept of deployment, and basically deployments are, um, a way to run these sessions on a schedule. And you-- There, there's not a way for you to like on a schedule, like self-improve the thing, but I could totally imagine a world where you like you could build that using these components, and that'd be pretty cool.

Also, like, for the, the Hill Climb skill that we keep mentioning, this is another sub-command of the Claude API skill. Um, so if you're in Claude Code, you can do, like, /claudeapi hill climb, and then tell it what thing you wanna, like, iteratively improve on. This is kind of like, uh, uh, taking, um, Andrej Karpathy's auto research and sort of baking it into a skill that everybody can use to kinda, like, iteratively improve on their own prompts. And so if you start with a prompt that, you know, the one off the shelf to do the watchlist scan,

then you can, uh, take that prompt and, and go into Claude Code and say, you know, "Claude API Hill Climb this prompt," and then it will work with you to iteratively, like, figure out, okay, what are we optimizing for, and then it'll just kinda, like, grind until it gets, like, a really nice prompt for you to then, like, use that, uh, to, to, yeah, meet your needs. Someti- so depending on, uh, what happens during the Hill Climb, sometimes it will give you... I- it really depends on exactly the project or, like, the problem you're trying to solve. So I, I, f- when Haiku

was announced a couple weeks ago, or everything happened so fast, I think that might have been last week. I don't know. Uh, when Haiku was announced, I built this tool for playing charades with my kids. So it just has, like, a webcam, and it'll give us a card, and we try to act it out, and then Haiku will guess, like, you know, what we're doing. And so I used this, the Hill Climb, um, command, and Claude generated an artifact for me that let me record sessions of myself doing, like, these hilarious things, rowing a boat and, like, playing tennis

and all this, so that it could, like, improve and iterate on the prompt and say, oh, we need this many frames per second, and we need to, like, uh, update the prompt to do this. And so depending on what the problem is, like, y- when you run Hill Climb, Claude may or may not give you an, a human interface to kinda, like, iterate on that, and you could also, like, request Claude to do that for you. So yeah, it's actually, like, yeah, quite powerful.

Yeah, so the, the question is about, um, Navier-Stokes and, like, these scientific discoveries that are all of a sudden happening kind of all at once, and if something like this is, like, what we would use to, uh, uh, to, to, to do, to do that. Um, I think maybe in our, our demos, our, uh, our, our stock, stock ticker is not ambitious enough, uh, because you pretty, you pretty much hit the nail on the head, right? That, like, the way that you pull forward substantial amounts of scientific progress is basically shorting the- shortening the timeline that it takes to do this kind of work. Um, and

Flows are basically a way excuse me, um, to throw test-time compute at hard problems and get more capability faster out of the models. And so, um, I can't speak to, like, specifics of, like, how internally we, like, r- we, what our harnessing looks, looks like for, uh, running these types of research workloads, but, like, we think, especially as an API surface, that unlocking the power of this is, like, the first step for us making these types of discoveries or, um, uh,

scientific progress, like, broadly doable by the general public. So, um, yeah, I'm pretty jazzed about that.

Yeah, one other thing that I'll add to that, too, is that, like, in our previous multi-agent setup for Claude Managed Agents, if y- where you have, like, a coordinator and then a roster of agents, we had some limits around, like, how many subagents you can spin up. With the Flow, with Flows right now, I think it's 1,000 agents. And so, like, uh, the other, the other, like, interesting piece of this is that your lead agent could spin up multiple Flows at a time or, like, after one Flow finishes, start another Flow. And so you might have,

like, a really massive project that you want to, like, give to, give to a set of agents. I actually, like, was really hoping it would be done by now. It is not. Uh, but I, I on Friday kicked off a Flow with 1,000 agents to build an operating system to show to you, and it has already been working for, like, 85 hours. And so, like, it's not done yet, and it's f- like, three or four Flows in and has been working with, like, tons and tons of agents on this, like, m- bigger project. And so these are things you can absolutely throw at

it, um, and obviously because they take a little bit longer, uh, this is something that you ki- I kinda, like, just started on Friday and then walked away and was hoping that maybe by 9:00 AM on Monday it would be ready. But, um, definitely, like, uh, the type of project that you could, um, you could throw at, uh, yeah, Claude Managed Agent Flows. Yeah. Let's save that for an after. But yeah, it was, like, it was actually, like, very lightweight. I was like, "Hey, Ken Thompson made Unix in a week. Can you try and make an operating system that's Unix compatible with a Python interpreter? Um, use

Flows, you know, feel free to be brave and use up to 1,000 agents," and that was basically it. And then, like, let it run. And, uh, g- I also gave it a GitHub repo so that it could work in that GitHub repo. The last time I checked, there was, like, 1,000 commits in the GitHub repo. So it's, uh, yeah, the agents are still grinding on it. Yeah.

Uh, the, the question is, uh, if we've noticed diminishing returns from, like, spinning up more subagents in the same way that, like, an organization, like, on an incremental basis for every new employee you add at a certain size, um, there's, like, coordination costs and such, uh, that, that increase over time. That's actually a really good question that I don't have a perfect number or answer for you on. Um, I think that, like, this whole research area is pretty new, and I would be very unsurprised if, like, these, these types of exact, like, uh, numerical

assignments to, like... Or, or basically, like, the, it's almost like the capability curve per subagent or, like, the incremental, you know, the slope of the line as you're adding a new, an, an extra agent, um, is more calculable in the future. Um, but I, I don't have a good number right now. Um, yeah, it's a good question.

Yeah. One thing I will say is, like, with Flows you get the benefit of, like, throwing them at a problem in an organized way versus an unorganized way. So, like, if you take the same 100 agents and throw them at the problem, like, it's gonna do better with Flows because there's some organization and isolation between those contexts with gating in these different phases. So...

So the question is, is there a way to prevent infinite loops or cascading errors? And the answer is yes, sort of. Um, the, there's mechanisms to make sure that, like, you don't end up with runaway sessions. So for that, I would throw on a budget, right, based on what- whatever it is that you, uh, your, your budget might be. But then otherwise, Claude is actually pretty good at detecting subagents that are- Misbehaving essentially, and managing them accordingly. Um, and so we found that, at least in our own evaluations, like Claude is, Claude is pretty good at like

detecting these types of errors and, and cutting them out, um, uh, automatically. The other thing is, we were talking about like dreaming and hill climbing and stuff, and like learning from past runs, and we think that that's a really powerful way to, if you like very specifically want to optimize parts of your flow or parts of what Claude is trying to accomplish, um, that's like a really, really nice way to do so. Let me state your question back to you and tell me if it's, if it's correct. Um, so basically you're, you're trying to understand if there's a way to like, based on some criteria, like a smaller set, may-

maybe like a signal, like a SHA change, you should only execute potentially like some parts of a flow or some, some parts of a, a, like, like basically you don't wanna burn tokens because like s- nothing happened, right? Um, there's not really anything like that on platform today other than just like telling Claude to conditionally do stuff with a single flow. Like if I, to be totally honest with you, I would probably like layer something else on top of Flows yourself within your application layer to make that

detection. Um, so you could like size basically the, uh, the s- the, the work that you're trying to do. I don't think that Flows would be a good fit for something that's like, you know, a single agent can accomplish in 30 seconds anyway. Um, it's more for like these ambitious, complex tasks. So, yeah. Yeah, for sure. So the question is, uh, if I can talk a little bit about the context guardrails between the agents, and effectively like the worldview that each of the agents have. Um, so each of the agents is

spawned into its own session thread that's like totally, like physically in the database, like separate context windows. And so there's, there's no leakage of those context windows via like file systems or anything like that, um, uh, by default. Um, beyond that, each of the agents has their own system prompt. So whether you wanna bring your own system prompts or just have Claude spin up the system prompts, um, you can do so, and that is like a really good way to enforce the totally separate personas. Um, on top of that, um,

the, I mean, the thread message sends and thread message receives are a good way to see the inter-inter-agent communication if you wanna make sure that like, you know, middle management agent isn't like doing something with, uh, uh, its underlings that, that you don't want it to do. Um, but generally speaking, those are like, that's how it works. Did that answer your question? I see. So the question is about the shared file system that I alluded to as like a methodology for sub-agents interacting with each other, and if there's a way for you to like physically separate those out. Um, the way to do that today would basically just

be prompting, um, and telling, like if you're, if you want your agents to only focus on like a very specific set, uh, like a subset of directory paths or something for, uh, wh- where it does its work, um, you would just prompt that. There's no like permissioning layer for the directory paths. Um, but honestly, like that seems pretty useful. Um, maybe we'll, uh, maybe we'll take that back and, uh, come back, come back with something better. So, so the, the, the follow-up was, what about using something like hooks to, uh, control access to different parts of the file system? Um, that's

... The, the way that this would work in Managed Agents is not quite like hooks, but you're actually kind of answering the question a bit for me, because, um, one way that you could do this is we have an abstraction called tool permissions that basically like anytime an agent wants to read a file, it, depending on the configuration that you provide, might need to ask you for permission, or you being like your system for, for permission. And in this case, if you wanted to like lock down, you know, /mount/foobar, um, then you could just read the tool, basically the request that Claude is making to

read that file, check the agent that's actually like running that, uh, read request, and just deny it if, uh, if you don't want the agent to have access. Wow, thank you. Appreciate that. Oh man, you're, you're teeing me up. Um, we're gonna-

Yeah.

So the question is about, uh, credential management. Um, so like the, the, the punchline here, right, is like, oh my gosh, like 1,000 agents. That's crazy. Like, that doesn't seem like safe to just like let, let run wild on the internet. Um, and we're gonna spend like basically the entire second half of this presentation talking about agentic safety. The question is if there's a way to allow human feedback as part of a flow. The way that you would implement this today is using something called custom tools. Basically like give Claude a tool that's like, request human feedback, and then Claude would request that human feedback from

you whenever it needs it, and you could provide whatever response you wanted in that, from that tool call. So the question is, um, in the example of building an operating system, uh, what if, like w- what if the thing dies or something? And like, like how do you see the intermediate states? How do you interact with it as it's going? Do you wanna take that since-

Yeah, sure. I, so, um, part of the prompt I asked it to have like a, a team of docs writers, and I was like, use Sonnet 5.5, I love the, like the documentation that, it comes out of that. And, uh, at the end of every single flow, a team of docs writers goes back to write documentation about what that phase implemented. And so I can just go to the GitHub repo and like look at the documentation as it's going. That's one, one way to do it. Another way, uh, so you kinda asked about like is there a way to like pause it or

stop it? Um, and so there is, uh, so w- another thing that we'll talk about later is, um, interrupting a session. So you can interrupt the flow and say like, "Actually, like you're going the wrong direction. Please stop right here." Um, you can also w- prompt Claude to give you reports or like to update a certain report as you go, or update, you know, like a... Typically it'll like-

Oh

... um, give you these markdown files that you can open.

Yeah, and one thing that's pretty cool about the, um, the, the product surface is that you, like no matter what type, whether you're using Flows or not, um, if, if you have Claude write to the out- its outputs directory, you have like a set of downloadable files that are like totally, uh, durable with a, another API surface we have called V1 Files. And so, or, or maybe you're like writing commits to GitHub, like, um, CJ's has been. And so there's just like a lot of like outputs, um, as part of these runs that, uh, you can inspect yourself. Yeah, so the question

is like halfway through some crazy ambitious problem, you find some information like externally or something that can potentially change the trajectory of the, the overall run. Uh, do you just wanna burn two days of tokens and like start from scratch? Um, so the way this would work today is you would interrupt the session and like provide that more information. You know, Lemma X is actually probably, probably useful for you, Claude. Take a look at that. Um, and Cla- like the, the primary agent that you're interacting with has this like full state of h- where the flow is in its

process, and it can inspect where, where it basically left off in order to, uh, uh, kick off essentially a new flow that takes into account that information as well as the past history. Um, what's not supported today is like, you can't like take the first flow, like the first plan, and like inject a single step like halfway through, um, and like rerun later steps or anything like that. Um, but yeah, that's like how you would in-ingest more information. Uh, so the question is like as a game developer, let's say you wanna put in like in-game telemetry,

um, uh, via something like Flows. Um, I, I, I wanna make sure that I, I'm ... Tell me if I, I don't answer your question, because I might, I might be going in the wrong direction. Um, so when I think of telemetry in this case, I think of like information about what's happening as the agent's operating, right? So you could, you could drop an agent into whatever product experience you want, whether it's a game, whether it's, you know, your website, whatever. Um, but the, the telemetry aspect is more like, what's happening right now? Do I need to take actions based on like what the,

the, the session is doing? Am, am I going in the right direction here? Okay. Sweet. Um, so we have, as part of our API surface, we have this extremely rich, durable event stream where you can stream in all the events as they're happening in real time, or you can like list them via a, a static API, um, endpoint. And then we, um ... Yeah, I'm trying to think. I don't know if we've-

Is it multi-modal like could you-

So like if you wanna throw in like, basically in Claude Managed Agents, if you wanna take like a game screenshot and like throw it in and like take some action, um, I think today that would be a pretty bad user experience because it would just take too long, depending on what it is you're trying to build in your game. But I will say that Claude Managed Agents does support image inputs, and Claude is like really good or getting really, really good at inspecting images. And so you could ... There's totally a world where you like just drop in the screenshot and like Claude can, uh, take some action de- depending on whatever it is that you wanna do. Um,

it's not gonna feel like real time right now, but, um-

Yeah.

Yeah.

More for developers.

Oh, yeah.

Tests.

Yeah, yeah. Totally, totally. Yeah. That's actually a, a, a great point, which is like maybe telemetry, like debugging a crash or something, you like take a screenshot of what's happening in the game and like throw it into an managed agent. Totally, totally feasible. Um, so the question is about sharing these things, right? Like, you wanna share these resources. Maybe you're like an organization that like you have like a set of templated agents, right, that are useful for you, and you wanna make sure it's not just you who has access to them. Um, the way that this would work today, um, number one, we, we have a Terraform provider that's like you can use Terraform to like create

agents, uh, uh, for you. But also, um, we have a bunch of like export style functionality in, in the console that lets you export agents and drop them into other places. But there's not like a one-click share, share button where you like share an agent with other, um, people. Um, basically we've just tried to break down the barriers to make like taking the body of an agent or, you know, whatever, uh, API resource. Um, we've tried to make it as easy as possible to take

that and just like upload it somewhere else. Um, but there's not like a first class like sharing mechanism within the API right now.

I would say like, uh, the most common way that you might give this tool to a customer is to build a product interface for it. So like you would build your own like website or something, or app, and they come to that and interact with it through something that you've built that is just like a layer on top of your managed agent. And so like a lot of, a lot of Claude Managed Agent interaction is just through the API directly, and many, many customers are offering that to their end customers as like, uh, built, like just built into the product. So

like if you're, you're in very popular project management tool that has a Kanban board and you drag one card from like a column to the next column, that might trigger a Claude Managed Agent session to go like, you know, using the API, spin up a flow, or spin up some session that does something, um, in the background. And so instead of having it like be a like front end component that is directly available that you can kinda like share out across cloud services, it's something that you would build into your own product, um,

that's ki- more behind the scenes. Yeah.

By the what which is being given to them. So the question is like, is there a way to tweak the behavior of agents rather than like in the system prompt of the agent, instead to be keyed based on the type of work that you give the agent? Um, little bit of a cop-out answer, but I would probably just system prompt that based on the, like, like, like if you want the agent to think about ... Let, let's say that you know rough sizes of work, like the agent should like take different types of action based on that size of work. There's not like a knob,

um ... Okay. Well, actually, I, I take that back. One way that you might be able to do this, um, depending on what exactly you're trying to accomplish, um, is that we have effort knobs on the, um, uh, on a per session basis, and we also have the ability to like override some subset of fields on a per session basis. And so if you like, given a task and an agent, you can like set those effort knobs or like change different parts of the agent based on what it is you're trying to accomplish. Is probably how I would do that. Yeah. So the

question is how does Claude even know, right? Like how does Claude know what scale, uh, to, to operate? Um, my ... I know it's probably not the most like satisfying open answer, um, but the answer really is like evals and lower defaults, where you ... Like if you're, if you're asking like the stock ticker thing to run, like we shouldn't be spending up 1,000 agents that are all burning a ton of tokens, right? But if you're asking about Navier-Stokes, right, like maybe- You would have to be pushing a little bit harder on the flow and, like, asking it

specifically to do that, uh, like, like spin up that l- uh, level of agent or, yeah, that level of parallelization. So, yeah. All right, I think we probably should keep rolling.

Yeah.

Um, we got one more.

We can al- oh, yeah.

We'll still- we'll sneak in one more.

One last question and then-

One more

... yeah, we can also take more questions at the end, too.

Yeah. Yeah, totally. Gotcha. So the question is, is using basically like markdown files to both like see what the agent is doing, but also presumably like define the agent, et cetera, um, the, the right way to do it? Um, I, I think there's a lot of... It depends, uh, for, for lack of a better answer. Um, the, the reality is that our API surfaces have like a ton, ton, ton of functionality depending on what it is that you're trying to do. Like, we'll talk

about credential vaults in an example. Like, if you throw in your API key to a markdown file that you just like give to Claude, that's probably bad. Um, and sorry, I should say that's definitely bad. Uh, do not do that. Um, and so it really just depends on like the complexity of what, what you're trying to express with the API surface. I will say that if you're just trying to like literally have like a basic markdown with like a basic set of... Like, like Claude basically take that and like do something with it, um, on kind of a one-off basis, then Claude Managed Agents

probably isn't, isn't the right abstraction for that. But like a lot of what, how Claude works under the hood with, you know, all of this stuff is basically like text, right? So it's, it's not too far off.

Yeah.

Okay, let's show... Oh, you wanna?

Yeah, I'll, I'll just add one more thing and then we can keep going. Um, so one, uh, con- like thing to consider is, uh, like how relevant certain context is to each sub-agent. And so one technique is to gradually reveal that context by having like a markdown file that links to other markdown files. So like if you are, uh, part of the engineering team that's working on the kernel, maybe you have like the main document will say, "Okay, here's the separate docs," like an, an index of like kernel is here, and assembler is here, and editor is here. And

so like it can then read the details that are relevant to that sub-agent, and that can kinda like save some tokens. So.

We gonna roll?

Let's do it. Yeah.

All right. I'm sorry, we gotta keep moving. We'll, we'll have questions at the end.

Yeah.

So, so save your questions and we'll, uh, we'll also probably hang out for a little bit after-

Yes. Yeah, totally

... if I, if I had to guess. We didn't even come around and like look at people's consoles.

I know, right?

I, I, I hope it worked. Uh, if it didn't, we're happy to help you, uh, run your first flow, uh, later on.

Um, okay. So earlier we talked about, or, uh, Harrison mentioned Bug Hunter, and another common theme that fell out of the last AIE was around loops. How can we close these loops to make all of these systems kinda like work more on their own and run more autonomously? And Bug Hunter is a very, very effective flow, and a pattern that we have been using. Um, and I wanted to talk a little bit about it. It's also available, uh, as a template and also in Claude Code.

So the idea behind Bug Hunter is you can put up a PR, and then you can have a Bug Hunter flow review that PR and, uh, surface problems that might be, uh, performance issues or, uh, you know, like actual like, uh, breakages, um, and, uh, allow you to more quickly review PRs as a human. And so, uh, instead of just having a single agent review a PR, Bug Hunter is a, is, is a much more effective way to go about reviewing PRs because it

uses a flow where it takes your code, splits it up into these slices, each of those slices is handed to an individual context. You have this like, uh, an agent that has, uh, the responsibility of reviewing only that slice of code and starting to like write down, "This might be a problem. This might be a problem. This might be a problem." Then handing that to a second phase where that reviewer can say like, "Okay, given this slice of code, I agree or disagree with these different problems." And then having that all

be surfaced to one f- uh, phase that can like pick candidates that might actually be real problems that we need to go out and verify. So like if you were a human reviewer of a PR and you, uh, are looking through some code, you often will have context about different parts of the code base that aren't even mentioned as part of this PR. And so that's often why we trust humans to go and continue to review the code that's entering our code bases. Uh, but this becomes much more effective

because the agents can start to surface those problems to you. Um, one of the things that you might do as a human is say like, "I think this is gonna be a problem because of this, this other like corner of the code base that I know about. Let me pull down this PR and verify that that's actually like going to be an issue and how big of an issue it might be." And so you can run through this prover phase where you could download, have an agent download the repo, try to reproduce the bug or like verify that the bug is actually like a, a legit problem, and then have these surfaced in a

report. In this case, we just say like, you know, it's, it's a report, but in practice, these are added as comments to your, your actual PR where Bug Hunter would raise an issue like in line and say, "Okay, you're making this change to this file," and then way off on like this, this distant corner of the code base, that's gonna cause like this unexpected problem for you. Um, and it can, it can give you priorities. You know, this is like a very severe or like less severe thing. Um, so again, this is a, one of the instances where you can like throw test time compute at different

parts of the problem depending on your code base and the composition of like, you know, is it monorepo or you have like lots of different repos, et cetera. Um, Bug Hunter, really, really effective tool for, uh, helping close the loop on, um, your PR process. This is how you would spin it up with Claude Code. Claude API skill, Managed Agents onboard, Bug Hunter. Um, and, uh, yeah, this is kind of like how, uh, how that works. Another really fun flow, um, how many people follow us on Twitter, Claude

Devs? Do you follow, follow Claude Devs on Twitter? Okay. All right. Oh- A lot of you ... not enough of you. Hundred percent. Not enough of you. Yeah. Okay. So, uh, if you follow Claude Devs on Twitter, you'll see that a lot of times when we announce new products or new features, they come with, like, an animated video that explains how that product or feature works. So we actually use a flow to generate many of those videos. And so, uh, you can... I can just say, you know, kick... I can kick off a, a flow and say, "Explain how dynamic workflows works," and then it can run through each of these phases

or even, uh, as subflows or, like... I'm sorry, not subflows, just individual separate flows if we wanted. Uh, so it's gonna have, like, a research phase that's gonna go out and find what are all the actual details about how this works, and plan and scout and fact-check, and then it will build a brief, and then it can take that design brief and hand it off to production that will go and make individual scenes and animate those scenes. And then, uh, in practice, we actually have, like, another set of phases that will do, like, sound effects and voiceovers and scripting and, like, all this other stuff, and at the end, you get a

video. Um, so Explainer Studio, this is, like, actually, like, a quite, quite a fun flow to use. Um, and, uh, as someone who helps a little bit with our Twitter profile, I think it, like, saves me a ton of time. I think, like, you know, four years ago, making the videos that we publish, uh, on Twitter would've taken, like, I don't know, 20 hours or something to animate and, like, make beautiful, and now it's just, you know, one very small prompt off to, uh, off to a flow. Um, you too can do this in Claude Code, uh, with the Claude API

skill Managed Agents onboard, Explainer Video Maker. Um, and yeah, just ask how... What is DNS? How does DNS even work? I don't know. Let's see. Uh, and it'll generate a flow. So we've got... Yeah. We've got a couple, uh, of videos that it made. So let's, let's see. All right, so what if the task has too many pieces for one agent? All right, so we have 300 contracts. We want it to go through all those contracts and find something out. So with Dynamic Workflows, the agent is going to

write the workflow

and then find these change of control clauses. So the server in the background's gonna run that workflow.

This is... Well, this one is just my voice, yeah. 'Cause we were, uh... Yeah.

Sounds so good, right? No. The results come back. They're all combined. So these are, like, the different events that we talked about. So you can kinda, like, see it's actually, like, pretty well animated, explaining, you know, how session threads work, how to enable it through the API by passing this multi-agent, uh, argument, and then telling you, "Oh, yeah, it's probably important. Set up a budget, and if the budget changes, you can... " Or, like, if, if you, if you, uh, end early without spending it, you can extend it, et cetera. Um, yeah. Does it auto deploy to YouTube

or how does that- It does not deploy to YouTube, no. Uh, and um, that's a good question. It's a, um... We should, though. We should. Yeah, or Instagram and IG- Yeah. Right. So if you, if you, uh... There you go. A good follow on YouTube is the Claude AI channel on YouTube, where we have done some, like, really fun things where, like, it'll stream music, and, uh, Claude FM did, like, a very kinda, like, nice, uh, autumnal kinda, like, v- vibe music thing, uh, live streamed recently. Who wants to see how a bicycle is made?

Okay. How many people wanna see how a toilet is made or how a toilet works? Yeah, not, not how they're made, how they work. So, okay. All right, let's see. Yeah, toilet, I think, was the winner, just, like, not by a ton, but- Yeah. I think we know how bicycles work, so yeah. Also, love that it's, like, Claude branded. The videos, like we ask them, it'll give us, like, a lockup that has, you know, sort of the, the, the same colors and everything, um, for the backgrounds, and you can kinda, like, give it your

own brand style. Another, like, plug would be to use Claude Design to give it your brand style and go through and create your own, like, design system. So you can go in, like, Claude AI/design or use Design directly from the, the Claude, uh, desktop app, and then given your Claude Design style, you can, uh, give that to your flow and say, like, you know, "Follow my brand, uh, my, my brand settings." And it even... Uh, like, the font at the bottom is even, like, the Anthropic fonts and everything, which is pretty cool.

So really, really powerful tool, especially if you are responsible for animating videos to post on the internet. Uh, and yeah. Uh, another really fun example of Flows. But there's lots and lots of different use cases, whether you're trying to solve some, um, you know, im- really important millennium problem, or you're trying to generate video, or you're trying to, like, improve your stock picks, or, uh, you know, your dev flow. There's lots and lots of different things that you can build with Flows. Um, so one thing that we talked about a little bit earlier

was, uh, a handful of things that single agents struggle with. And this is a, a big takeaway that we learned as part of Dynamic Workflows, and I wanted to just talk a little bit about them and how they apply to, uh, sort of from, from Claude Code and how they apply to Claude Managed Agents. So one is agentic laziness, another is self-pre-preferential bias, and the third is goal drift. So, um, it turns out that, like, if our prompt is something like, "We're having an outage right now. Please go look at

these 50 different services and tell me if, uh, you know, the... Tell me the root cause of, uh, of the outage, but don't touch production. Like, don't actually, like, change anything in production. Go research is there an outage. Don't touch production," a single agent might have this problem where after it sees 35 services are healthy, then it starts to, like, convince itself that, "Oh, it's, like, everything looks good. It's probably fine. Like, I don't really actually need to look at the rest of the, the services." So, um,

we have seen that agentic laziness can be, like, one of the things that a single agent will run into, and after it's seen so many that are good, it can kinda just be like, "Okay, I think we're good. Let me just not look at the rest, and let's save some tokens and tell, tell our human that, like, we're good to go." Coordinator runs into a sort of a similar problem where the, you might have three sub-agents reporting back to the reporter, or to, to the coordinator, and the coordinator still has to, like, keep track of, like, "Okay, how many have we seen?" And, "Are all of the services healthy?" And

those sub-agents might also themselves get lazy at a certain point. And so, um, you don't really, like, improve too much over having the single agent. When we have flows, we have this sort of... It's, it's not necessarily deterministic, but we have the ability to split up and dedicate a, an individual agent context to each of those services, so they're only responsible for their small scope of work, and we can get a much better answer back because we can tell from the flow, you know, fan out across all 50 services and dedicate

an agent to each of those services to go and see if that service is healthy. And by the end, we know for sure that we went and checked every single service. So this is like the fan-out and loop until done kinda like combined. Self-preferential bias. Um, if you ask an agent to write a poem, and then you ask the agent, "Is that poem good?" It's gonna say, "Yeah." Um, this is, like, a pretty common thing. If you ask an agent to write some code, and then you say, like, "Is the code good?" It's gonna say, "Yeah." That's because it, like, generally was the one who did the work and, like, is

happy with the work that it did. And so if you can, um, split that up, you get better results. We know this from the advisor pattern where you might have, like, one agent that is the generator and one agent that is, like, judging the results of that, like, kind of, uh, taking advantage of that generator discriminator gap. But when you have, like, a really massive problem, you're still gonna, like, run into this pro- this... You'll, you'll run into a situation where, like, you, uh, your, your roster is not gonna, like, necessarily scale up to a

point that you need, uh, without having, like, these separate and distinct kind of, like, checkpoints. And so with a flow, we can spread out across lots of these different agents and have, like, adversarial review that has its own isolated context that isn't being, like, um, influenced at all by previous things that it checked. The last one is goal drift. So if you remember, like, the, the initial prompt was something like, "Go out and scan across all these services and tell me if we're healthy." But the

last thing that I said was like, "Don't touch prod." Right? And so if we are filling the context window and compacting, and we're filling the context window and we're compacting, we're going through all these services, not only might be, be lazy and we're, we'll, like, prefer our own answers, but also that piece of don't touch prod could potentially, like, get lost in that process. And so one of those things of, like, uh, i- it's important to just remember that, like, as the contexts grow and grow and grow, we kinda, like, might lose track of exactly what our

goal was. Whereas with a flow, we can get, like, a very specific goal at the beginning, and because we have all these i- you know, individual contexts, we can make sure that that, that goal is copied in, like, a very consistent way into each of the sub-agents for all the different phases.

So I kind of, uh, prefaced this a little bit earlier, but I think one of the most... Or, or frankly, the most important thing when you're running these pretty, pretty hefty agentic workloads is making sure that you're not doing bad things or your agents are not doing bad things when they're running. And now we're going from a single agent to potentially hundreds, right? So, so how do we scale, uh, our safety and security posture in order to make sure, um, that, that basically we keep, we keep track of, of what's going on and make sure that, um, uh, nothing bad happens on platform? Um, and this is gonna get

pretty technical, um, because, uh, A, I think this is really important, but B, we've put a lot of thought into our API abstractions to make it, uh, much, much, much easier to create safe agents by default. And I think, um, if I had to say, like, o- one, one thing to get right, uh, when you build out your agents is this. Um, so, uh, bear with me as I , um, uh, talk to you a little bit about something that I, uh, I personally care a, a ton about. Um, so I'm gonna walk through each of these safety risks. Excuse me. Um,

I know safety can be, like, a pretty overloaded term. Um, these are just six very concrete things that, um, everybody should be aware of if they're doing, excuse me, agentic development. Um, I think the first is probably a, a generally pretty well-known one. Um, if your agents are scanning the open internet, they can pick up instructions by accident from people or entities that are not yourself. And you have to make sure that there's not, like, text that says, "Ignore previous instructions," like, "Actually go to otherdomain.com," and, like, "Exfiltrate all my information," right? Um, and

this is, like, fundamentally something that we've been working to solve both at, like, the model layer, like the, all the way down the stack, but also we, um, we've spent a ton of research energy into building out classifiers that make, uh, tool call checks, uh, and like Auto Mode and Claude Code basically, um, uh, like a, a default thing that can work on platform. So strongly, strongly encourage checking out the Auto Mode permission policy that we'll talk a little bit more about later. Th- the second thing I talked a little bit about earlier when I was talking about the markdown

files and, like, don't just throw API keys in markdown files and give them to Claude. So at the end of the day, Claude is taking on the persona of whatever credentials that you give it to it, right? And if you're building a product and you expose your agents via your product, and Claude has access to a computer like a VM under the hood, that is where it actually accomplishes most of its work. If you have a file on the disk that is basically like my apikeys.md or my apikeys.txt or whatever, even if you make it like a hidden

file, even if you put it in like a secret directory, um, there's always a chance that you, uh, uh, like that leaks somehow, right? Whether it's a user figuring out a way to prompt Claude or prompt your product in order to get it out, um, there, there's just always a risk, right? That that plain text somewhere, um, could, could, uh, make it out. And so we, we've created an entire suite of abstractions to ensure basically that credentials never have to touch the sandbox. And this is an incredibly important decoupling, and I'm gonna talk about it extensively later. The other, or the third one is

data exfiltration. If your agent has access to sensitive information, you wanna make sure that it's not sharing that sensitive information with third parties who are not yourself. Additionally, you wanna make sure your agent is not overreaching what you've asked it to do. I talked a little bit about budgets earlier. There's other limits that you can set on the platform. Um, on top of that, you wanna make sure your agents are not operating outside of, like, their, uh, their wheelhouse by establishing permission policies on the agent so that only tools that you give the agent access to can actually run. And then last but not

least, like, it's pretty table stakes to have some visibility into what the agent is doing. And so we have a durable event stream that has essentially the full trajectory of everything, uh, your agent is doing across the entirety of its life cycle. So your job as a developer is three different things. We like this nice, like, very catchy declare, contain, detect. Um, but that's, that's the reality of it, right? You have to declare what your agent is, the permissions it has, what it can do, what you-- what persona it should take on, skills, tools, et cetera.

Um, and then, like, the contain piece is literally, like, the, the physical limitations of the agent. So how do you contain the agent so that even in the absolute worst of case, the amount of harm that it could do on the open internet is, is minimal? And so this is where we have very explicit configurations in your environment policy where you can say, "You can only access these domains," or, "You can only access, um, a web search, but not web fetch," or vice versa, depending on, on, on your needs. Um, and last but not least, that durable event stream

that I keep talking about is, like, a very crucial way for you to det- to detect things that are happening, um, in real time. And if, if the agent is doing something that you are, are not happy with, depending on whatever, like, your definition of happy is, um, you can always listen to these event stream in real time and immediately interrupt any sessions that are, um, um, going in a different direction than you'd like. Um, let's see. Cool. So when we shipped Claude Managed Agents back in, uh, April, there was almost, like, this meme that Anthropic, like, settled the debate,

right? The, the brain and hands, uh, should, should be separate when you're building out an agentic infrastructure. Um, and, like, jokes aside, we think this is a very, very critical component of our infrastructure that makes safety kind of just default possible. Um, the idea here is that the brain, which is, like, the loop that's actually running inference in the just general agentic harness, um, as well as the session state, so, like, what's happened across all of the life cycle of every agent in a session, is fully

separate from what we call the hands of Claude, which is the place that the agent is actually able to take action in the real world. Um, you can think of hands as, like, running a grep on a computer that Claude has access to. You can think of it like a custom tool that, uh, or rather a custom tool that might run on a, um, uh, you know, your macOS machine, um, that we, we talked about earlier. Um, broadly speaking, there should be, like, no way for you to give any hand access and totally steer the

brains in any way that could be considered harmful. Um, and this is, like, the, the, the key thing of, like, how we separate all of the secret information that can-- that needs to be accessible in the brain and the harness from the hands themself. And so the way, um-- There's this really interesting way that we have made sure that, uh, credential injection works, and I think I talk about it in a little bit more detail later, if I remember correctly. But, um, the, the, the idea is that in every container that has access to any secret uses

totally fake secrets, right? Like, th-those values don't actually work if you try to call, like, myapi.com with that secret. And only on the way out, um, at this egress proxy layer here on the right side, do those credentials actually get substituted for what, like, the actual credential is. So there's no way, like, physically to prompt Claude or to, um, uh, like, given access to a sandbox, get that credential off of the sandbox or off of, off of Clau-Claude's brain. Um, so this is-- this, like, totally decouples the, the agentic

stuff from the, um, uh, the actual, like, uh, credentials themselves. So, uh, yeah, I kinda prefaced this already, but basically, like, you-- by default, uh, like, you know, you should not be able to foot gun yourself, um, in using Claude Managed Agents or building any agentic workloads, uh, yourselves, um, into leaking your credentials. So, like, definitely, definitely, definitely take this seriously. I think this is very, very crucial. Um,

so I'm gonna hand it over to CJ to talk about-

Yeah. So the, uh, we'll go through each of these different things and how you can control it from, uh, from, uh, the API. So again, one of the important ways that you can make sure to contain, uh, Claude Managed Agents, and we've talked about this a few times, is setting a budget. And so we can say, "Okay, run this flow. You have $1.25." Uh, and then if it hits $1.25 and the flow is not finished, then you can decide whether or not you wanna up the budget and let it keep running, and it won't have to, like, redo any of the work that

it had already done. Um, or you can just say, like, "Okay, that, that's what we had budgeted for that specific session." Um, so every time that you kick off any session, whether it's Flows or not, you can use this budget. And this pairs really nicely with, um, this other part of Claude Managed Agents called Outcomes. And so the combination of these two things is really beautiful because now you can say, "I would like this result. Here is the outcome that I want and the rubric that I expect, and here is a budget. So go accomplish this

outcome given this budget." Um, and the budget, again, is this way to just, uh, make sure that we're containing and not letting these agents just, like, uh, continue to spend tokens, um, unexpectedly. The second, uh, code example there below is how you can interrupt a flow in the middle of the process from the API. So you can just pass this event, user.interrupt, and this will stop the flow, uh, and you can, like, then re-prompt Claude or, um, do whatever you need to do to continue on. So these are two different things. So one is the

budget, the other was you can interrupt in the middle of, uh, of a session that's running. And so one thing to kinda, like, test this out is I would t-- you know, take the flow that you ran today and then set a budget that's lower than what you expect it to go over, maybe, like, you know, uh, a dollar or something, and then run the flow Increase the budget, watch it continue, so you can kind of like experiment with that a little bit. Um, the second one we talked about, the, um, th-this ability to have a reviewer agent go back through all of your sessions

and understand what happened as part of that session and getting visibility into what were the tools that were called and did anything unexpected happen in any of these sessions. This is another use case where you could spin up a Claude Managed agent with a prompt that says, "Here's the things that I want you to look for that happened." And because we have that durable log and we have every single event that happened, you can pull that transcript with the API and pass it to a managed agent and say like, "Review this entire transcript and all of these past transcripts over the

last week," whatever, "and tell me if my agents did anything that was unexpected or accessed anything that w- they weren't supposed to or called any tools that were kind of like odd or unexpected." And so, um, this is another, um, kind of like a handy thing that you could do is build a Claude Managed agent and hand it the sessions from today and just ask it like, "Was there anything odd that happened in these sessions?" The last, and I think this is actually like the easiest lowest hanging fruit, uh, if you have any Claude Managed agents integrations today, and that is,

uh, to take your permission policy from either always ask or, like, uh, a-and turn on Auto Mode. Turn on Auto Mode. This is like super, super important and also very, very low level or like low hanging fruit. Um, we did a, a bunch of research and there's some like public research about how, uh, you know, when your laptop was open and cracked a little bit and you're at the park and Claude keeps asking you like, "Do you approve?" And everyone's like, "Two, two, two, two, two. Yes, yes, yes, yes, yes. Approve, approve, approve." You know, like ninety-seven percent of the time we're gonna click approve as humans. And

so, um, we are, yeah, we easily get like lulled into just being like, "Yes, yes, yes." But Claude is like much more patient and using Auto Mode, y- it can like determine whether or not a single tool call is safe or it's gonna take some destructive action and whether or not like it should actually like do that thing. So Auto Mode, all you have to do is switch from always ask to Auto Mode or whatever your permission policy is today to Auto Mode, and it's like the easiest lowest hanging, uh, fruit thing to do. So...

Sweet. And so I wanna go into detail a little bit about the vaults thing that I alluded to earlier, um, and why we think credential management is such a crucial part of any platform that, uh, you run your agents on top of. So in this example, we are using the Twilio API via an environment variable. So in the naive situation here, we'd like take our container or our, rather our, uh, session, maybe like upload a file that has this like .env in it that includes this Twilio API key so that our agents could read it and

then actually call the Twilio API from that container if it was trying to take some, some action in, let's say, a Python script. Um, but the problem with that, again, is like all the plain text stuff I've, I've been talking about for, for a minute now. Um, and so instead of like that default, just like getting thrown onto the container, um, you can use credentials and credential vaults in order to manage these secrets. Um, so in this case, we're using a vault credential in order to create a, uh, or in order to store our Twilio API key as an environment variable and this

injection location thing is basically for all network egress on the way out of the container, where do I expect the secret to show up? Do I expect it to be in like a, a bearer auth, uh, token on the way out or do I expect it to be like literally in the request body? Um, uh, you can configure, um, based on this field. Then additionally, the networking hash here with this allowed host thing is basically where, given a call to domain X, uh, like what, what, where-- or sorry, given our credential X,

what domain should we substitute on the way out? Because if we just naively were able to like guess these secrets, um, and, or, or these like placeholder secrets that exist on the container, we don't, we don't want like our Twilio API key to ever go out to, I don't know, like mydomain.com. Um, so really, really important to, um, uh, handle this stuff safely. And then once you've created your credentials i-inside of your vault, then you just pass in the vault and when you create your session and the, the platform will be smart enough to

know like based on what's in the vault and what secrets exist and at the egress layer what, um, like placeholder secrets exist, it can like figure out what, uh, how to inject in the correct values. We also talked about giving agents a, the smallest tool set possible. Um, so one of the like most, um, I think interesting tools that we have in our agent tool set is web fetch and web search, and maybe you wanna disable those by default because you don't want your agent to have any access to the internet, um, without you

very explicitly allowing it. In that case, you can set the enabled field to false for any of the, the, the types of tools that you wanna disable. Um, so here's just some basic examples for agent tools. MCP tools are, are real- a really interesting one because, um, you don't know the MCP tools that are gonna come back from an MCP server up front. So like let's say that you're calling out to the GitHub MCP, like yeah, we, we trust GitHub, um, but like what if something happened and they were compromised and an attacker added like, you know,

my exfiltration tool or something to it? Um, you probably don't want that to be picked up by your agent and so you, you on a tool by tool basis can, can enable, um, uh, per, per MCP tool set. So, um, additionally, you're able to limit the network access that your agent has access to, um, like physically using the environment configuration. Um, so you can think of environments again as like the computer that Claude is operating on and as such, if I only wanted api.twig.com to be accessible,

then I can like lit-literally say that in the environment configuration. And so, um, now I have like my API key on the, like, like hidden away from, um, a-any potential like theft and I can call api.twig.com physically. If I tried to call, you know, someotherwebsite.com, it would like physically fail at the, uh, firewall layer, um, on a-as, uh, like y- like when egress tries to occur.

All right, so just kind of bringing it back to the beginning, uh, we think that we're, in many cases, trying to, like, reinvent the wheel of these organizational learnings that we had, you know, in the '60s and '70s about how to build really massive teams of humans to go out and, uh, achieve these, like, engineering feats. And so we're really excited to give you access to Flows, and, uh, we're excited to see what you'll build with Flows. Um, so again, hopefully, uh, you can take these and

go achieve some really, really ambitious projects.

So again, start from a template, shape it, uh, shape it to be your own. Tell us what you find, and, uh, yeah, we would love to take some more questions. I think we're getting close to the end here, um, but we can also-- we'll also hang out outside in the hallway track. Absolutely happy to answer any questions. So I see one over here. Yeah? Yeah, so the question is, like, uh, you try to, you know, give an agent a task, and the thing that it gives you back is, like, not very verifiable. For example, with the video, that's, like, very much, you know, your human taste.

Um, so then you give it some feedback, and how do you sort of like iterate on that within Flows? Uh, I would say one place to start is to start with Claude Code and iterating on some process first. So you can either use Dynamic Workflows in Claude Code or directly try to work through the process of generating your first video and then have Claude encode, like, what you did into a skill that you can then use inside of Claude Managed Agents, uh, as part of a flow. So I think, like, usually, there is, like, quite a bit of iteration in the

beginning to, like, figure out, okay, uh, what are all the different phases, and how does this process actually work? And maybe you need to do fact-finding across the organization to understand how an existing process is built today, and then trying to, like, do your best to, uh, encode that and then use the Hill Climb skill to, like, iterate on the, the actual prompt and improve it. And that Hill Climb thing can be sort of human-driven. So you can say, you know, "Let's Hill Climb on making these videos, uh, like, really great." And so it'll generate one. You can say-- or it'll, it might generate two, and you can say which

one you prefer, and then it'll generate two more. You can say which one you prefer, and it kinda like will, uh, iterate on its own prompt that way. Yeah.

So I think the question was going back, tell me if I'm wrong on this, but going back to the operating system example, when would you use UltraCode versus when would you use Claude Managed Agents, like Flows, right? Cool, cool. Yeah, so, so UltraCode is gonna be the situation where you're, like, sitting at your laptop, like, typing into the terminal and using Claude Code directly and interactively. The Claude Managed Agents Flows tool would be if you have, like, a repeatable thing that you want to happen on some regular basis and want to integrate an AP-- against an API that can hook into, like, whatever product surface, uh, you

want, rather than, like, directly just using Claude Code. Yeah, I, I think that that way that you described it at the end there is, is, is the correct way to think about it, that UltraCode is, like, the, the single instance of one of these things, essentially. And we actually took the UltraCode, like, uh, abstraction or methodology and generalized it into what Flows is on the Claude Managed Agents API. Um, and so you can think of UltraCode as, like, the my one-off, like, me interacting with the thing, and then Claude Managed Agents is, like, the, okay, I wanna, like, level this up, uh, a layer and

reuse them and, um, run them on whatever trigger or schedule I want. That's a great question. So the question is, how do you think about liability if, like, malici- something malicious happens on platform? Um, unfortunately, I am not a lawyer, um, and I will not be able to, like, like, explicitly weigh in too much other than, um, I would think of it a lot like, you know, running any other, like, platform-y kind of business, where if you integrate against an API and, like, build a product on top of that, um, at the end of the day, like, the, the, the platform

itself is going to have the relationship with, with the user directly. And so it's gonna be kind of like a defense in depth kinda thing of, like, making-- like, each layer of the stack should do-- be doing their best to make sure nothing bad happens on platform. Yeah, so the question is about the durable event stream and use cases, um, that, that folks have, uh, have built out internally. Um, so it-- I'm trying to think of a, a, a fun one. Um, somebody internally built out, like, a, uh, a competitive, uh, Catan, uh, uh, like, I guess, simulation

where it uses, like, all the events generally just to explain, like, why each of the players is making the choices that they're making. It uses, like, custom tools, which is one of the, uh, it's, like, the way that you would expose your macOS machine to, uh, uh, to Claude Managed Agents in order to, uh, like, take action, like, uh, outside of just the hosted environment. Um, I don't know. You, you might have some fun, fun ones here.

I think, like, the easiest way to think about it is if you wanna build your own UI that has, like, shows all of the different tool calls and everything. So I, I think a lot of people are reinventing this, the wheel here, and there are some, like, front-endy tools to help you expose your agent's event streams and stream back generally, like, the messages that are coming back from agents. But being able to sh- like, build a chat interface, you know, it's kind of like, um, pretty table stakes these days, but a chat interface where you're interacting with a, you know, a conversation with an agent, and you can see the tool calls,

and then you can open those tool calls, and maybe you're, you're having a, a tool call that has, like, some generative UI, and it'll show you, like, the chart. And then for, especially for, like, the finance use case, you might say, like, "Okay, every time my talk, my, my stock ticker watchlist thing encounters something interesting, also render, like, the chart so I can do, like, my-- or, like, add technical analysis things to my chart and make it so that it's, like, really interactive." And you could do a lot of those with, like, the event stream that's coming back. Um, but yeah, h- hopefully that answers the question. I think there's a-- because there's, like, a

lot of events, it's a very generic set of tools that you can go and use to build basically any UI on top of.

So the question is if there's any compatibility difference between, like, native Anthropic and self-hosted. Um, there are some features that aren't fully supported on self-hosted sandboxes, but Flows are, like, totally supported. Like, will, will work out of the gate, um, with self-hosted sandboxes. Um, yeah, I'm trying to think what, which ones aren't supported. But generally speaking, we try and support, like, features, like, across all environments, whether it's self-hosted or not. Um, just the environment that you pass in when you create your session will change, um, types. So, so the question is in the Bug Hunter example where there's like a

multi-repo dependency graph, how do you actually or how do we actually structure like... Like, like how does Claude actually know, right, what, what, uh, what to do? Um, and the, the truth is that it's kind of up to Claude. Um, if you give Claude access to the GitHub repositories, those will all end up mounted inside of the container where Claude is operating. And so then Claude can make the decision, like if you have a really nice like Bazel build and like have a really, really crisp, clean dependency tree, Claude can use whatever tools it wants on the container in order to inspect, uh, those dependencies. Whereas

if you have, you know, plain text language ABC that doesn't have like a very explicit, uh, dependency mapping or it's like non-trivial to, uh, uh, build out that graph, then Claude would probably have to like literally search files for symbols or something like that. For every PR review that you have, you would have to remount the container, remount the, uh, repository on the container, yes. It's a lot like, um, if, how you think about like, uh, just a CI job generally, where you have to like run some checkout on the Git repository, um,

and, uh, then you can take action on it. So the question is if you g- are able to build out a dependency graph ahead of time, will that help you optimize the flow? And the answer is totally. If you, if you have some representation of the dependency graph that you know ahead of time you think would take a long time to generate, you can upload it as a file into the session. Um, like we support just general file uploads as part of the platform, and then you can be like, "Hey, Claude, like, by the way, your full dependency graph you care about is at like this file path." So the question is about how many limits of nesting are there in these multi-agent

topologies. Um, the, the real answer today is it's just one layer of nesting. We don't support like arbitrarily complex, like mega fan-outs of, uh, topologies. Um, we, we probably at some point will want to, but we need to make sure like everything is really, uh, uh, like the safety and stuff is really locked down. Um, so the first one I'm gonna, I'm gonna try and restate it and, and tell me if I'm, um, understanding it correctly. Basically, like there's some, some form of like identity or data access or something that you don't have access to on your machine, uh-

Because of VPN

... because it sits behind a VPN, and is that data accessible via any like internal MCP servers or-

Internal MCP, but only while it is on my machine.

Only on your machine. Okay, gotcha. Um, because the, the, the way that you would abstract this would be using something we have called MCP tunnels, which is basically a way for you to give a private access, like private MCP access to the agents on platform using, um, Cloudflare Tunnels today. And basically, it's like a way for you to give access such that, uh, you don't expose anything over the open internet, and I think that's like probably the closest to what you would be trying to do, um, uh, or that you would need to do in order to give

these things access. So there would be some- Yeah, yeah, yeah. It's, yeah. It's, it's like the connection is directly from, um, wherever your private MCP server is running, uh, to, to the, uh, uh, place that the MCP calls happen. Um, so yeah, I would look into that. Um, number-- The second question was on, um-

question

... just like how, how stuff works at Anthropic. Um, we're very like Claude Tag pilled basically, um, where Claude Tag does a bunch of work for us autonomously and, um, uh, like the, these workloads where like the thing is looking at Jira and all the activity in the past twenty-four hours or whatever and, uh, even like proposing proactive changes to tickets and such, we, we do, uh, a lot of that work through something Claude Tag shaped. So, um- Yeah, yeah. Uh, Claude Tag supports scheduling-

Yeah, Claude Tag, Claude Tag can do that.

You, you-- I think if you just say @Claude like, "Every, every day at 6:00 AM, like do this," I, I think it would work, um, out of the box.

Yeah. Thank you all so much for your attention. Appreciate it.

Appreciate you guys.

All right.