Claude for long-horizon tasks

Talk transcript

1. 0:00

   [upbeat music] I'm good to go. All right. Well, take a quick sip, and then let's start.

2. 0:19

   It is great to be here. Um, this is, like, my, my third year coming to this conference, and I always really enjoy it. And thank you for coming to this workshop.

3. 0:26

   I know there's many interesting talks. Let me talk a little bit about, um, our view of asynchronous agents at Anthropic and some things we've been up to lately.

4. 0:36

   So this is kind of a way I think about models and product. So you can think about Claude as a light source, and you can think about products as windows that allow the light to pass through.

5. 0:46

   And what's kind of interesting is, over time, the window that you need to actually kinda see the light of the model kinda shifts.

6. 0:54

   And we've seen this over the past few years. So I'm plotting here different Claude models and their task horizon. So how much autonomous work can they do over time?

7. 1:05

   And you might recall back in, like, the Opus three days, uh, this was kind of like twenty twenty-four, models could only do, you know, maybe ten to twenty minutes of autonomous work.

8. 1:13

   This is measured by meter. And in that regime, only certain product surfaces made sense. Things like auto-complete, things like chat, where your human is very in the loop, 'cause the model's really only doing a very short amount of work before you're steering it.

9. 1:26

   Now, over the past year, we saw the rise of synchronous coding agents like Claude Code, and this is, you know, a kind of a shift because then models could do maybe an hour of work.

10. 1:35

    So it made sense to have them run, but typically locally, where you could still steer them easily. And it's kind of interesting because during this regime, I remember efforts, and I was involved in some efforts, to build kind of asynchronous agents.

11. 1:48

    But when models can only do, like, an hour of work, async as an experience is kind of bad. The, um, the model goes off, and it, like, hits an error, and it comes back to you over a short period of time.

12. 1:59

    In order to really unlock async, we needed longer task horizons, and so we're starting to see that now.

13. 2:06

    And kinda with this shift in capability and time horizon came a shift in the API surfaces. So if you look at the lower left, Messages API came out, like, two years ago.

14. 2:17

    It's basically prompt response. It's great for building harnesses, but it's a very simple API. Again, you're just passing and sending messages. You get a response out. There's no sense of deployment with that, so you basically take Messages API, and you can roll your own harness, you can deploy that harness, and you have an agent.

15. 2:34

    Now, over the past year, we saw the rise of, you know, coding agents in particular, so we released Agent SDK. So that's basically a way to programmatically call Claude Code, and that's, like, basically us giving you a harness.

16. 2:47

    But over the past few months, since April, as we've seen longer and longer task horizons, we released a new API called Managed Agents, which basically packages both the harness as well as all the managed deployment infrastructure for you.

17. 3:01

    And I wanna talk about some of the themes that underpin this new, uh, surface, Claude Managed Agents, and, and some of the themes that kind of extend beyond just Managed Agents broadly to think about this kind of new type of asynchronous agents, uh, which can apply, of course, to Clause and other types of kind of longer-running, long-horizon

18. 3:17

    agents. So theme one is decoupling the brain from the hands. Um, so when we first set out to build Managed Agents, we started with the container. We put the harness and the sandbox in the same container.

19. 3:32

    Now, the problem here is: What happens if the harness dies or the container dies? What we saw is we actually lose the session. So basically, this architecture is kind of tricky for long-horizon agents because what can happen is your agent's running, and if that container dies, you lose everything with it.

20. 3:54

    Also, as models get more capable, putting the credentials in the same container with the agent itself can be problematic. So, for example, giving Claude access to a bunch of your secrets and letting it run for ten hours and you're not watching it can be a little bit spooky and have some security concerns, especially as models get extremely

21. 4:11

    capable. So for this reason, we kind of decouple what we call the brain, that's the harness, from the hands, the execution environments. And Managed Agents is set up like this.

22. 4:22

    So the story here is that the harness becomes a stateless process that talks to a session. The session is an append-only event log and that can reach out to hands which are just containers.

23. 4:36

    So that's just, those are sandbox where work is done. And one thing that's interesting is Claude is increasingly capable of managing many hands. So that is, you can give one harness access to many different containers to perform tool execution, and Claude can manage this very easily and, and, and, and effectively.

24. 4:55

    If the session... Sorry, if the harness dies or a sandbox dies, it's completely fine because the session is always backed up and it's append-only log, and credentials are never actually added to the sandbox.

25. 5:08

    They're stored in a separate vault. So this decoupling actually makes it quite reliable and safe, particularly for long-horizon tasks, and this is kinda one of the core ideas that underpins Managed Agents architecturally.

26. 5:20

    And I think an interesting thing that falls out of this is related to, you guys may have kind of seen or come across the recursive language models work. The session becomes an external context object that the model interrogates, and this has all sorts of benefits for context management.

27. 5:36

    So you think about it, when you're doing something like compaction, you're choosing some logic to retain some amount of context and naively, in a typical, in a kind of a typical step, you're discarding all the context that you didn't compact.

28. 5:49

    In this architecture, and also more broadly with recursive language models, the idea is that the context object is persistent and is unadulterated, so it's append-only, and the model can always go back and fetch old context.

29. 6:02

    So it basically creates a very nice architecture for context engineering because the core context object- Is immutable in the sense that it's, it's, it's non-destructive, and you can only-- and you only append to it over time.

30. 6:14

    So we've seen this be-- to be quite nice in terms of long-horizon conduct engineering as well.

31. 6:20

    So the second theme is use verifiers. And one of the problems that we've seen with Claude and other models in general is that when you ask them to do a bunch of work and then say, "Okay, grade your work," if that same context is being used to both do the work and grade, you can get lots of

32. 6:42

    odd artifacts and confabulation and, and basically odd behavior. For example, this is just an image showing... You can think of that, that context window is filled with lots of different information, and the model is grading itself.

33. 6:55

    Often, it's not properly tuned to do kind of crit-critical verification. And so what we found is it's quite effective to separate verification into a separate context window. This is a very general trend we talk about in a number of different engineering blogs.

34. 7:11

    Um, and the reason is the verifier context can be tuned very specifically for the critiquer verification task.

35. 7:18

    And so the way this works in practice is when you build loops, you can have a loop of a build context and a verifier context, and this can be a build agent, verifier agent.

36. 7:27

    And what happens is the verifier has some goal or rubric, and it's verifying the result or work of the build agent, and this continues in a loop until verification is complete.

37. 7:40

    And this is really the big idea behind this whole loops trend that you might have heard about, and we found it to be a kind of a very powerful paradigm, especially for working with k- some of the higher capacity models.

38. 7:50

    And here are some of the primitives. So in Claude Code, you have goal. In Manage Agents, you have outcomes. And the principles are really the same. You're setting up a measurable end state in both cases.

39. 8:01

    You're using an independent context model. You're using independent context to grade over the course of this loop.

40. 8:09

    The loop can run, and you only exit the loop until this independent verifier has verified that it has the outcomes or outputs that you want. That's the key idea.

41. 8:18

    Now, let me tell you a story about how I've used this. So this is a kind of a fun and interesting challenge called Parameter Golf. It's a benchmark that's set up-- that was put up by OpenAI,

42. 8:28

    and it tests model's ability to effectively do kind of ML research. So it asked the model to basically take a small, uh, kind of model and train it in-- with eight, eight-- with eight H100 GPUs in less than ten minutes.

43. 8:45

    And what you see on the Y is basically you can think about it as loss, so lower is better, okay? And what I did was I set up a kind of a verifier loop using managed agents and outcomes to test the ability for Opus-four-seven and one of our frontier models, like Mythos class models, on this task.

44. 9:04

    And what you see is basically I allow the model to continue to iterate until the outcome that I specify is, is satisfied, which is it finished on-- exactly twenty iterations and kind of met-- it kind of met all the experimental criteria as defined by the benchmark.

45. 9:21

    What you see is the frontier capability models are extremely good with this pattern of kind of loops and self-ver-- and, and kind of verification. Because what happens is,

46. 9:33

    instead of encoding steering me-- uh, into, like, me as the human, you're encoding the signal into the environment, so the model can self-correct when it receives feedback from, for example, the verifier.

47. 9:47

    And using this kind of paradigm with very high capacity models, you can get very strong results. So the main point I'm trying to make here is that this paradigm of loops, which a lot of people have been talking about today, paired with very high capacity models, is a very j- good general primitive for long-running asynchronous work.

48. 10:05

    That's really the key point here. Now let me talk another-- a-about another theme, uh, of self-learning.

49. 10:15

    So the human brain has two kind of interesting systems for memory. So one is, as you go about your day, the hippocampus is kind of writing traces of kind of short-term, very fast, kind of experiential memory.

50. 10:27

    Like, you might remember what you had for lunch today. You had lunch an hour ago. You kind of remember. That's kind of written to short-term memory. When you go to bed at night, though, an offline process or out-of-band process, dreams.

51. 10:39

    And dreaming stores certain important details to long-term memory in the cortex. So for example, tomorrow, you might not remember what you ate for lunch today. That's kind of a local trace.

52. 10:47

    But, like, if you had a very important experience today, maybe this talk, maybe not this talk, but if you had an interesting experience today, that might be written to long-term memory.

53. 10:55

    That's kind of the point. So these two subsystems in human work like this. And we've actually found memory systems with Claude actually can employ these same two principles. So this is showing Claude's capacity as an in-band memory writer.

54. 11:10

    So you basically give Claude memory tools, and when I say memory tools, I mean it's basically the ability to write to a file system that is basically a memory directory.

55. 11:17

    That's really it. Now, this is showing some work on Claude-based Pokémon with Claude Sonnet three point five. And here's the key point. When Sonnet three five is given this access to a memory directory and it can write memory, quote-unquote, "in-band," as it progresses through this game, it's not very good.

56. 11:35

    So the memories it writes are pretty crappy. It's kind of, um, uh, it's kind of, uh, tactical notes. It's, it's not very strategic, and the game progress is quite limited.

57. 11:46

    But with more recent models, like this is looking at four six, the notes are much more strategic and game progress is much further. So the key point I'm making here is that Claude has gotten much better at this in-band memory writing across model generations.

58. 12:02

    And this is another way to show that same result. So this is a benchmark that I ran called Continued Learning Bench. It's an open source benchmark. I took one of the tasks.

59. 12:10

    This is a task that basically asks the model to perform a sequential question answering with a SQL database, and it can write memory in between each step. And what you see is basically the performance improves across models

60. 12:25

    Um, and so what this is kinda showing is that models get natively better at this in-band memory writing with respect to model capability.

61. 12:36

    And some of the most interesting things I've found from this are that the main differentiation between, there we go, um, the main differentiation between, like, a lower capacity model and a high capacity model is kind of this distillation step.

62. 12:52

    And so basically, higher capacity models have a better sense of, like, what abstraction to save to memory that'll be useful later. Like, they're not just writing a specific fact.

63. 13:02

    They're writing, how do I-- how does this generalize to future sessions? That's kinda key difference that I found that higher capacity models kind of have when they're writing memory.

64. 13:11

    So this is a very important thing to keep in mind, that models are getting better, better and better at this kind of in-band memory writing across model generations.

65. 13:19

    Now, there's a little trick here which is very important. So we talked about kind of in-band memory, and we talked about dreaming. So at night, I dream and I write things to, like, long-term memory.

66. 13:29

    Dreaming is very important because when I'm writing memory in-band over the course of a day, over the course of a session, sometimes you can write incorrect memories.

67. 13:37

    And-- or you're writing things that are locally optimal, but not globally optimal. So you're writing over the course of a task to, like, kind of help you solve that task, but not necessarily looking forward to future tasks.

68. 13:51

    This is a very important nuance, and this process of dreaming is, is kind of an offline or out-of-band process that we've used to consolidate and improve memory. And I wanna show you a fun example that I've used dreaming for.

69. 14:04

    So this is, again, Pokémon. And this is-- I played a lot of games of Pokémon with Claude to find this. So this was a very hard-won lesson, so I hope you appreciate it.

70. 14:15

    Okay. Here's the point. So basically, what happened is Claude wrote an incorrect memory, okay? And what happened is this incorrect memory was related to the location of... The details don't necessarily matter.

71. 14:29

    Uh, the point is that this incorrect memory causes Claude to mislocalize itself, or Po- Pokémon to mislocalize itself, and it falls through this trapdoor, okay? That's the key point.

72. 14:42

    So it writes this incorrect memory, this incorrect memory causes it to mislocalize in the game, and it falls down this trap. And this is very consistent. So I saw this in five replicates.

73. 14:50

    Five out of five replicates with raw memory store fell down this trap. With dreaming, this error is corrected, and it's able to properly localize itself and not fall, fall down this trap.

74. 15:03

    And I'll show you kind of a fun visualization of this. This is looking at kind of memory traces or, or basically traces of game progress. So going upwards on the Y-axis is improvement.

75. 15:12

    That's like moving to the next level. Going down is backtracking. So what's interesting here, the no memory baseline, which is that gray bar, kind of doesn't make much progress at all.

76. 15:23

    It just kinda like is stuck. It's, like, a particularly hard level, okay?

77. 15:28

    The memory, which is the orange, actually keeps falling down this trapdoor and falls back, so it backtracks. The dreaming traces, though, consistently kind of fix this error in its memory and proceed to the next level.

78. 15:41

    So this is a very practical example of how dreaming can kind of work out of band on your memory store to fix corrections. Because what it does is it looks at your memory store, and it looks at all your prior traces or sessions and kinda can find and correct errors.

79. 15:54

    That's the key point, and that's why this dreaming process can be very helpful because in band, while Claude is writing to memory, it can make mistakes, and those mistakes get stuck in memory unless you have an offline process to kinda correct them.

80. 16:05

    That's the key intuition. Um, and theme four, and I'll s- open up for questions after this, is what I think, um, is kinda this trend that we're gonna see moving towards org level harnesses with async agents.

81. 16:20

    And so we released Claude Tag, and a lot of the reaction was like, "Ah, Slackbot." And like, look, I have actually created a lot of Slackbots myself. I understand not every Slackbot is particularly interesting or great.

82. 16:33

    In fact, I've created many Slackbots that are quite bad. But what's interesting about Claude Tag is not the fact that it's accessible through Slack. What's interesting about it is the fact that it has a very, very rich kind of system underneath it, which I wanna just tou- touch on briefly.

83. 16:49

    And in particular, what's interesting about it is it represents what I would consider an org level harness. So agents historically have been kinda single player. So you have an, like, an agent like Claude Code on your machine with your global context that you've tuned and configured for yourself.

84. 17:03

    What's interesting about Claude Tag is it is a harness that everyone in the organization has access to and can use. So it is a multiplayer harness. And what's nice about that is it has its own identity.

85. 17:14

    Its identity and credentials are not tied to a given user, and it has access to organizational context, not just my local context. This has many interesting and useful implications, including the ability to, like, check other's work before you do an experiment, the ability to kind of deduplicate findings, the ability to, like, do internal research, the ability to

86. 17:33

    give everyone access to kind of a, a very well-developed harness on day one. Whereas when you-- your own personal harness, often new employees, takes them weeks or maybe even months to kinda ramp up fully to configure all the right connectors and so forth.

87. 17:47

    So w- org level harnesses are a real leveler of the playing field, and I think it was... People kinda saw the Slackbot piece, but they didn't really appreciate the depth of kind of-- the, the depth of benefit you get from building out org level harnesses.

88. 18:00

    So I do think that was kind of an important thing to note. And I think we're gonna see the rise of the kind of harnesses that operate across orgs, across many different users that can operate increasingly on longer asyn- async, um, async agents that can operate on the longer timeframes.

89. 18:16

    That's kind of one clear kind of follow-up that we're, I think we're gonna see from this. And another thing where I think we're gonna see is that asynchronous agents, um- Are gonna be increasingly proactive.

90. 18:28

    Uh, so typically with, for example, like local- locally scoped agents, they tend to be reactive. They're responsive to how you steer it, versus async agents increasingly have the ability to steer proactivity, and that's one very nice thing about Claude Tag.

91. 18:43

    Um, where basically you can configure it to tell you things when, like looking at this org-level context, alert me with things I might need to know about. Um, and this is a very important kind of new kind of UX that I think is gonna be more and more common with async agents that kind of have access to

92. 18:59

    organizational context. And of course, multiplayer. So the, the ability for a single harness to be steered by many, many different people kind of concurrently is an important shift in agent UX, uh, that I think, uh, will be quite interesting going forward.

93. 19:14

    So, um, yeah, let me, let me just open up for questions, and, um, thank you for listening. [audience applauding]

94. 19:30

    Sure.

95. 19:31

    Um, one thing that we see sort of empirically and also on the benchmarks is that the frontier models perform better on these long-horizon tasks and are somewhat better at abstracting what they need to do if you're non-data source is not.

96. 19:44

    What is your view on, and like relatively has been saw on in like the lighter weight models-

97. 19:49

    Yeah

98. 19:49

    ... like the moving source models, so like if you generate all those moving stuff. Um, what's your view on why that is, and then how long the frontier will be able to maintain that gap?

99. 20:00

    I see. So the question was kind of on the, the gap between the frontier models kind of on, for example, a benchmark like Meter on like long-horizon tasks.

100. 20:09

     Yeah.

101. 20:10

     Okay. Um, so in the latest results that I saw from like, for example, Codex like five six, I think it also kind of is in that twelve-plus hour regime on Meter.

102. 20:21

     So I, I think you're right that like the frontier models like the, you know, Mythos class models, strong models from OpenAI kind of are in this like twelve-plus hour regime.

103. 20:29

     Um, why is it that non-frontier models are not kind of in that regime? I am actually not necessarily sure. I do think, I do think, um, in order to build agents that can effectively operate in this regime, it's important to note that it's not just the model capability.

104. 20:50

     Like for example, with the Claude Tag product, it's actually a combination of improvement in memory, because memory is very important. If you have agents working for, for example, twelve hours, you wanna make sure that your productivity preferences are well encoded in memory, so it knows when to reach out to you if it gets stuck, for example.

105. 21:06

     So memory is very important. Security is very important, so resistance to prompt injection. Um, also like model architecture, like kinda the agent architecture is very important, that decoupling of brain and hand, so it's secure and safe and like resistant to failure.

106. 21:18

     So actually, I think to build real agents that can operate in these long time horizons, a bunch of things need to come together in terms of like architecture, r- infrastructure, security, memory.

107. 21:31

     That might be why, and, and but Frontier Labs invested in all these areas, so that might be why you see kind of a gap, uh, in terms of like the agent products that we've released.

108. 21:37

     And so we spent a lot of time, for example, building managed agents to kind of have these kind of considerations baked in.

109. 21:44

     Yeah. Sure. Okay, uh, kind of a side anthropology. I just have a question. So like, I guess, you know, long-running agents,

110. 21:53

     you have like GPT agents, memory is playing a, an important role. So any particular reason, you know, generally, uh, the design focus has been towards bringing memory in file systems rather than bringing memory in database?

111. 22:05

     Yeah, okay. This is interesting. Um, the question was about, um, kinda like the, the best memory substrate, so like why file systems versus, for example, databases. Um,

112. 22:17

     this is kind of a subtle point that actually, um, I wanna think about carefully. So I don't necessarily think that it has to be the case that you use a file system for memory.

113. 22:31

     I think what's quite important that we've seen is that it's you want something that is highly programmable with simple primitives that the model can manipulate to like write, manage its own memory.

114. 22:42

     So for example, a database could work fine relative to the file system. But what I've seen doesn't work is when you specify the structure of memory for the model very explicitly, whether that's in a file system or database or whatever, like a memory schema.

115. 22:58

     I pre, I kind of pre-populate, "Here's the types of memories you need to save." 'Cause I th- that ends up being not very bitter lesson built in the sense that models can learn to manage their own memory much better than you can intuit these memory types for the model ahead of time.

116. 23:11

     So I think what we've seen is that very general substrates for memory, be it just your database or file system, are good because the model can manage them freely versus a very, very kind of r- like prescriptive memory schema that you're trying to pigeonhole the model into.

117. 23:27

     That's when you see performance drop. That's the key differentiation. So you're saying there could be that the more prescriptive the medium is to what- Right ... the second order forms to store.

118. 23:37

     That's the key point. Let the model structure maintain its own memory. Don't give it a prescribed memory schema, and that's like a common failure because models are getting good enough that they can manage their own memory much more effectively than you can reason about types of...

119. 23:50

     This is like very classically bitter lesson build. But like, you can re-- models can reason about their own memory and context structure much better than you can prescribe for them a way to structure their own memories.

120. 24:00

     That's the key observation. I see. Yep. Essentially, you are saying give any medium as a tool to- Yes ... and let's say this is where you store memory too from your mind.

121. 24:10

     That's right. Exactly. General substrates for, for memory management.

122. 24:16

     Yep.

123. 24:16

     Yeah. So I'm just curious. So, uh, you mentioned like dreaming.

124. 24:19

     Yes.

125. 24:20

     Um, so actually stripping off what-

126. 24:22

     Okay. So that's a good point. Basically, the question is, so you do this dreaming thing. You look at the sessions, you look at the memory store, you update the memory store.

127. 24:33

     How do you know those are correct? Um, so evaluations obviously are one way to do it. Um, this is kind of a fun anecdotal example from Pokémon showing that like you can do-- perform corrections via dreaming.

128. 24:45

     The key point is that we've actually run a lot of different evals showing that dreaming can indeed improve performance for in- very intuitive reasons as you see here. But of course, evals are important in like your own context to confirm it's actually worth the offline compute.

129. 24:58

     Yep, I guess we're done. Thank you all. [audience applauding] [outro music]