Building Durable, Production-Ready Agents with OpenAI SDK and Temporal

Talk transcript

1. 0:00

   [on-hold music] I'll introduce myself in j- in just a moment, but I'd like to get to know a little bit

2. 0:24

   about you. So you know, you can see that there's, there's two brands up on the screen here. There's OpenAI Agents SDK in particular, and there's Temporal. I work for Temporal.

3. 0:36

   I'll tell you more about myself in just a second. Um, I'm curious, how many folks are using the OpenAI Agents SDK today?

4. 0:46

   Okay, about a quarter of you. Um, y- any other f- um, a- agentic frameworks?

5. 0:56

   Okay, about the same set of you. So it's-- it looks like there's a handf- quite a number of you who are not using an agent framework just yet, so I'll teach you a little bit about that.

6. 1:06

   Okay, next question. How many folks are doing anything with Temporal?

7. 1:12

   Not very many. Awesome. I'm gonna get to teach you some stuff. Um, okay, cool. So, uh, w- we're gonna talk today about both those technologies. I'm gonna talk about them each independently, but I'm gonna spend a lot of time on them together.

8. 1:28

   Uh, spoiler, we actually have an integration between the two products that Temporal and OpenAI worked on together. Um, and so, and, and you'll see, it's, it's really quite sweet.

9. 1:40

   So let me very briefly introduce myself. My name is Cornelia Davis. I'm a developer advocate here at Temporal. Um, I have spent a lot of time, I think the bulk of my career has been spent in this distributed system space.

10. 1:53

    So I was super fortunate to be at Pivotal, um, working on Cloud Foundry from the early twenty tens, so I was really there during the kind of movement toward microservice architectures, distributed systems, those types of things.

11. 2:07

    Um, any Cloud Foundry folks in the room? Oh, just a few. Um, so for those of you who don't know Cloud Foundry, Cloud Foundry was, um, the early, the early container technology out on the market.

12. 2:21

    It was incubated as a open source project at VMware, and it used, um, uh, container images, Linux containers, container orchestration, eventual consistency, all of that stuff before Docker even existed, and b- well before Kubernetes existed.

13. 2:39

    So I was very, very fortunate that I was there at the beginning of that movement o- over toward platforms that supported this more agile distributed systems way of doing things.

14. 2:49

    And because I spent so much time in the microservices world, I also wrote this book. Okay, so what we're gonna talk about today is we're gonna talk about the Open Agent, A- uh, OpenAI Agents SDK, then I'm gonna give you a Temporal overview.

15. 3:06

    I'm gonna do lots of demos, and I'm gonna show you the repos if you wanna f- if you wanna follow along. You can go ahead and grab the repos.

16. 3:14

    Both of my demos I actually changed this morning, so they're sitting in branches instead of in the main branches, but I will make that very clear as well. Gonna do lots of demos there, and then I'm gonna do, uh, move over to the combination of the OpenAI Agents SDK and Temporal together, and we'll do more demos there

17. 3:33

    as well. And then I'm gonna talk a little bit about orchestrating agents kind of in the general sense. So this here is a notebook that we're-- I'm not going to use today.

18. 3:44

    And so I decided-- I just ran this workshop earlier this week, and I decided that for the AIE crowd, it was way too basic. That said, if you're interested, you can go there.

19. 3:57

    It will take you through, it's, it's set up with Jupyter notebooks. You can run it in Codespaces on GitHub, and you can run your first OpenAI Agents SDK agent, then you can run your first 101 Temporal a-- uh, not agent, but Temporal application.

20. 4:13

    Then you can move all the way through the agenda that way, but it's pretty basic, and I decided that for this crowd, I wanted to do something more advanced.

21. 4:22

    So we're not gonna use that today, um, and I just crafted some of these demos this morning. Okay, so without further ado, this is gonna be the shortest part of the presentation, is I'm gonna give you an intro to the OpenAI Agents SDK.

22. 4:35

    This was launched in, I think, around the May timeframe or so. Um, and I, I'm not gonna read you these slides. And, oh, just so you know where we're going, I am gonna use some slides because I'm one of those people where I think the pictures really help.

23. 4:51

    I've got lots of diagrams in here, but we are gonna spend a lot of time stepping through the code as well. I don't think I need to define what an agent is.

24. 5:00

    I will tell you that, that for me personally, the distinction that I make between GenAI applications and then when they get to agents is when we give the LLMs agency.

25. 5:11

    When the LLMs are the ones that are deciding on the flow of the application, that to me is what, what an agent is. And these frameworks, like the OpenAI Agents SDK, are designed to make it easier for you to get started with those.

26. 5:25

    And in fact, we'll see that really-- We'll see a contrast on that with the two major demos that I'm gonna show you today. It's available, um, in both Python and TypeScript.

27. 5:37

    And here is the most basic application. So what you see here is that we've defined an agent, we've given it a name, and we've given it instructions, and it's taken defaults for the rest of it.

28. 5:49

    Uh, there are thing- things that are defaulted are things like the model itself. I don't know what the default is right now. And then all you can d- all you need to do after that is you basically need to run it.

29. 6:01

    And anytime you see that runner.run, what that corresponds to is an agentic loop. And we'll talk about the agentic loop several times throughout the presentation. Every time you see one of those runner.runs, it's its own agentic loop, and when we get to the orchestration stuff later on, you'll see why I make that distinction.

30. 6:21

    It also-- as, as I said, this is really simple here, but it has a lot of other options that you can put in place into the agent, configurations that, that drive how the agentic loop works.

31. 6:33

    You can have handoffs. We will talk about those, so I'll clarify that later. But you can put guardrails in place. You can, you can add tools, and we're gonna see both of my examples are heavy duty on LLM agency and it deciding which tools to use.

32. 6:49

    So I'm gonna show you tools. So there's a lot more that you can do in here, and I'll show you examples of that as we go along. And really, this is the picture of what I'm talking about, is that every one of those runner.runs basically has a loop that is constantly going back to the LLM, and after

33. 7:06

    the LLM call, it decides to do things. And if the LLM, for example, has said, "I want you to invoke some tools," it will go ahead and invoke those tools, and then it'll take the output from the tools and route it back to the LLM and keep going.

34. 7:21

    And the LLM gets to decide when it's done following the system instructions, and we'll see that. Okay, so that is the basic, um, you know, agent framework overview, and there's lots of agen-agent frameworks o-out there.

35. 7:37

    Okay. Since very few of you know, know Temporal, I'm gonna t-slow down a little bit here and tell you more about Temporal. So Temporal is an open source project.

36. 7:48

    It's been around for about five or six years. So yes, it well predates the gen AI boom that we're in. It's designed for distributed systems, and what are AI applications if not distributed systems?

37. 8:03

    So it turns out that Temporal is beautifully suited for these-- this category of AI-- o-of use cases. Now, it's used in a lot of non-AI use cases. So, for example, every snap,

38. 8:18

    Snapchat goes through Temporal. Every Airbnb booking goes through Temporal. Pizza Hut, Taco Bell orders go through Temporal. Um, there's lots of other ones that I'm not remembering. OpenAI Codex runs on Temporal.

39. 8:33

    So now we start moving into the AI use cases. Codex runs on Temporal. OpenAI's Image Gen runs on Temporal. Those are the two I can tell you about. Those are the two that are publicly known.

40. 8:44

    Um, so we've got lots of others out there. Lovable runs on T-on Temporal. So we're definitely making, you know, inroads, lots, lots of use in the AI space. So I've told you who's using it, but let me tell you what it is.

41. 8:59

    What it is, is distributed systems as a backing service. So I think everybody's familiar with the notion of Redis as a backing service or Kafka as a backing service or a database as a backing service.

42. 9:14

    So I've got my applications that are running, and I use these back-end services to serve, you know, to play a part of my application. Temporal is a backing service.

43. 9:24

    What it delivers is distributed systems durability. And I'll make that clearer as we go through the presentation. What that means is that you as the developer get to program the happy path.

44. 9:38

    You get to program your business logic, and the business logic that we're gonna program today are AI agents. So you get to say, "You know what? What I wanna do is I wanna call an LLM, then I wanna take the output from the LLM, and I might want to invoke some other APIs, and then I wanna loop

45. 9:56

    back to the LLM." And you don't have to build the logic in there that says, "What happens if the LLM is rate limited? What happens if my downstream API is down for a moment?

46. 10:08

    What happens if my application crash-crashes?" You don't have to program any of that. We do it for you, and I'll show you a few pictures on how this works in just a moment.

47. 10:20

    So there's a, a Temporal service that is the backing service, and the way that you connect to the backing service is through an SDK. And so the SDK sits alongside your business logic, so you get to program your business logic.

48. 10:34

    And the way that you craft your business logic, you put wrappers around certain functions, and that allows the SDK to say, "Oh, hang on, you're making a, a downstream API call.

49. 10:47

    I'm gonna step in, and I'm gonna provide you some service. So I'm gonna provide you retries. If that downstream service, uh, succeeds, I'm gonna record that for you. I'm gonna record the answer for you so that in s-in the event that something happens and we need to go through the flow again, I can just get the, the

50. 11:08

    result that you called before." What that means is, for example, if you have used Temporal to lend durability to your agents, when you're on the one thousand three hundred and fifty-second turn to the LLM and your application crashes, no sweat.

51. 11:25

    We have kept track of every single LLM call and re-return, and you will not be re-burning those tokens. That's what it means. That's what durability means in this space.

52. 11:36

    We support, um, we formally support seven different l-programming languages, but Apple just a couple of weeks ago, um, released a Swift SDK as well. So there's support in just about any language.

53. 11:50

    There's also experimental stuff out there in Closure, those types of things. I said it's an open source project. The vast majority of it is MIT licensed. Um, there's a little bit of Apache too left over from i-in the Java SDK.

54. 12:04

    So very, very permissive licenses. And, um, those of you who don't know the history, uh, Temporal was a fork of a project that was n-created out of Uber called Cadence.

55. 12:16

    Anybody know Cadence? Yeah, okay. So a few people know Cadence. So Cadence, uh, pretty much every application running at Uber runs on Cadence, and it's because they can program the happy path and all the durability's just taken care of for you.

56. 12:30

    So that's kind of the overview of what Temporal is. So there's really kind of... I'm gonna talk about two foundational abstractions. There's a handful of others as well, but the two foundational abstractions that you need to know about as a developer is you need to know about an activity.

57. 12:48

    And an activity is just a chunk of work. This is work that either is going to make external calls, so it's work that might fail. It's like a lot of work that might fail.

58. 13:01

    Or if you are doing a lot of work that you don't wanna have to redo in the event that something goes wrong, you might wanna put that in an activity as well.

59. 13:10

    So it's things like withdrawing from an account, depositing into an account. We'll get to the AI use cases in just a moment. So those are activities. You wrap that...

60. 13:20

    Oh, and I didn't mention it, but the SDKs are not just thin wrappers that are sitting on top of a REST API. These are SDKs where, as you can imagine, be- li- uh, delivering durability across distributed systems means that all of those algorithms that you thought that you had to implement, like worry about concurrency and, and, and,

61. 13:43

    um, uh, quorum and all of that stuff, that's all implemented in Temporal. And so our SDKs have a lot of that logic is in the SDK. The service is mostly persistence for that.

62. 13:56

    So there's a lot of intelligence in the SDK.

63. 14:00

    So these activities, if you've said, "Look, here's my work. Here's a heavy-duty piece of work or something that's going external. Let's put an activity decorator on that," then the SDK says, "Oh, okay, I'm gonna give you some special behavior."

64. 14:15

    Then you orchestrate those activities together into your business logic. And what we call those orchestrations is workflows. Okay? So and you'll see that what happens when you put activities and workflows together, that's where the magic really happens.

65. 14:32

    There is some level of magic in the activities, and in fact we're just starting to release what we call standalone activities, so you'll be able to use activities without workflows and get some of those durability benefits there as well.

66. 14:45

    So there's all sorts of evolution that's happening. But the type of magic that I'm talking about when you, um, bring workflows and activities together is that I overlaid, um, a bunch of icons on here.

67. 15:00

    So I overlaid these little retry icons. So what you do is you specify in your workflow logic, you specify the, um, retry, uh, configuration. So you can decide, are you gonna do exponential back-offs?

68. 15:15

    Are you gonna do un- unlimited retries? Are you gonna top out at five retries? Are you gonna have a maximum window between retries? You get to configure all of that.

69. 15:25

    And as soon as you do that and you orchestrate these things together, now you get retries. And you'll see the code in just a minute. Simply by calling these activities, I don't have to implement the retry logic.

70. 15:38

    I don't have to implement any of this other logic. It just happens for me. So I get retries. I also get these little queues. So what looks to you like a single process application, I'm calling this, then I'm calling this, and I'm calling this.

71. 15:55

    Every single time you call into an activity, every time you come back from an activity to the main workflow, all of that is facilitated over queues. So that what looks like just a single monolithic application is already turns into a distributed system.

72. 16:12

    So you can go and deploy a whole bunch of instances of those, and you can basically scale by just deploying more instances of it. You don't have to manage, and I-- You don't have to manage Kafka queues, any of that stuff.

73. 16:24

    It's all built in. I spoke with somebody this week here at AI Engineers, um, who's, who's a, an open source user, actually a customer of ours. And I asked him, it's a relatively small startup, and I said, "You know, why did you pick Temporal?"

74. 16:39

    And he said, "Because we tried to build all of this with Kafka queues, and we s- ended up spending all of our time doing operations on Kafka and spending twenty-five percent of our time on the business logic.

75. 16:51

    When we switched over to Temporal, we're spending seventy-five percent of our time on business logic," and they're using Temporal Cloud. I didn't mention our business model is that we have a-- that, that s- that service, we offer that as SaaS.

76. 17:04

    So they're using Temporal Cloud. So they basically shifted from twenty-five, seventy-five to seventy-five, twenty-five by moving over here. No longer have to manage Kafka queues or Re- Redis or anything like that.

77. 17:18

    And speaking of Redis, you see in the upper right-hand corner, you see state management. And so one of the things that we do as well is we keep track of where you are in the e- execution of your application.

78. 17:29

    We do that by recording the state. Again, every time you're making calls to an activity and coming back, we record that. It's basically event sourcing. That's what we're doing.

79. 17:39

    It's not only event-driven architectures, but we're doing event sourcing as a service. So you, you get to do that. So we store all of that state so that if something goes wrong, and I'm gonna demo that, we're gonna see things going wrong, um, it will pick up where it left off because we will just run, run through

80. 17:58

    the event history and pick up where we left off.

81. 18:02

    So those little icons that I showed overlaid on the logical diagram, I'll get to your question one second. Actually, all of those services live up here in the service, so they're all durable.

82. 18:13

    So it's not that they're living in the process, but they're living here in the service. You have a question?

83. 18:22

    The agents I have, they handle streaming data. So I wanted to see if Temporal can help with that. Oh, great question. So- I don't know how to- Yeah, so the question was, a lot of the agents that I'm building are doing streaming.

84. 18:35

    Do you do streaming? And the answer right now is a very simple no, we don't. But it is one of the things, and my colleague at the back, Johan, um, is, uh, head of AI engineering.

85. 18:47

    Um, so chat with him. [laughs] Chat with either of us. It is one of the two top priority things that we're working on right now. Um, the other one is large payload storage.

86. 18:58

    If I don't have a chance to, to talk about it here during the workshop, come find one of us. We can tell you about that. You can imagine what large payload storage is, is that you're doing LLMs.

87. 19:08

    You're gonna... You're gonna... Passing big stuff around. Instead of passing it around by value, pass it by reference. That's what large payload storage is. That's Johan.

88. 19:16

    And I'll just mention, there are a bunch of people using workarounds-

89. 19:21

    True

90. 19:21

    ... to do streaming in production today at scale. So happy to talk about that, but there's gonna be a much more integrated, uh, solution coming.

91. 19:29

    Yeah. So I- J- I'm just gonna repeat what Johan said in case you, you couldn't hear it. So we do have customers that have built streaming support on top of Temporal, but what we're doing is building it in natively.

92. 19:42

    So yep. So you can do it today, it's just a little bit more work. It's not the happy path. So okay. Um, so with that, uh, I want to give you a demo.

93. 19:53

    So this is gonna be my first demo that I move over here. Let's see if I can get my screen back. Okay, so if you wanna follow along, the first thing I'm gonna do is I'm gonna come over here and I am going to...

94. 20:05

    Let me increase the font size. Um, so I'm gonna point you to two repositories. This is actually the second repository, but what I have up on the screen right now is that if you wanna get started with Temporal, super simple.

95. 20:20

    You don't have to use Temporal Cloud. You can just run a Temporal service. So the backing service, you can just run it locally on your machine. So you can do it by curling this.

96. 20:30

    You can Homebrew install it as well. Um, and then to run that local server, you can just say temporal server start dev. And now you've got a Temporal service that's running locally.

97. 20:41

    And all of my, um, applications here are just connecting to my local host. Um, and we're-- we'll see the UI in just a moment. Um, I'll come back to this, uh, repository in just a moment.

98. 20:53

    The repo that I'm gonna demo for you is this one. And, um, sorry, I don't know how to, uh, increase the font size, but you can see that the org and the repository here is-- the org is temporalio.

99. 21:08

    That's also where y- you'll find all of the open source for Temporal. Um, and then we have something called the AI Cookbook, and that's one of the examples. I, I, I actually extended the example just this morning.

100. 21:21

     But, um, and you're gonna find that, um, in the, the branch that we're gonna demo here today is called the agentic loop d- branch. So if you want to f- you know, go back and take a look at this later on, um, yourself, that's what we're gonna be looking at.

101. 21:39

     Okay. So with that, let me get to my right terminal.

102. 21:46

     And, um, so this is where I'm gonna run it, but I wanna show you the code first. Okay. So am I in the right, uh...

103. 21:59

     One OpenAI. Nope, this is the wrong one. My other cursor. Here we go. So this is the agentic loop. So I'm doing two demos throughout today.

104. 22:10

     And so what you see here on the left-hand side, let me make it just one tick bigger, is that remember that I talked about activities, and I talked about workflows.

105. 22:21

     So that's the first thing I'm gonna do, is I'm gonna show you the activities. Remember, we had withdraw and, and, and, uh, and deposit, you know, that type of thing.

106. 22:29

     Here, of course, what we're doing is an agentic loop. So my activities are going to be, um, call the OpenAI API, not the Agents SDK yet, just the OpenAI A- um, API, and invoke some tools.

107. 22:46

     So these are my two activities. So let's look at the first one, and you'll see how simple it is. I promised you the happy path. It really is that.

108. 22:55

     So here is my call to the OpenAI Responses API, okay? It is exactly what you would expect. I'm passing in the model. I'm passing in some instructions. Um, the user input is gonna come in my tools, which I'll show you in just a moment.

109. 23:11

     And then I've got some timeouts that I can configure there. That's for the OpenAI.

110. 23:17

     What I've done is I've wrapped that in a function. It takes in that request, so all of those parameters came from a request that I'm passing in, and you'll see how I invoke this in just a moment.

111. 23:27

     And here is that annotation. Now, the different SDKs have different approaches. TypeScript, for example, doesn't require a bunch of annotations. It just figures it out and knows where the activities are.

112. 23:40

     Java has, um, annotations, those types of things. But this is Python, so you can see here that we just have an activity decorator. So by just having that decorator, so you can see it's not complicated at all.

113. 23:53

     All you need to do as a developer is say, "Here's a, a bunch of work that I wanna do that I wanna kind of encapsulate into a step." And you just put an a-- You put it in a function, you put an activity decorator on that.

114. 24:07

     So I'll come back to the tool invoker in just a minute because there's something interesting that's going on here. So now if we go to the workflow, the workflow is also pretty darn straightforward.

115. 24:19

     So what I have here is I have my workflow definition. You can see it's a class. The reason it's a class is because there-- when you create a workflow, you create the application main, what I call the application main, and that's what has the workflow.run on it.

116. 24:36

     But this workflow also, I'm not gonna cover these abstractions today, but we have a handful of other abstractions like signals. So for a running workflow, you can signal into it, and we also have an abstraction called an update.

117. 24:50

     It's a special kind of a signal. And we also have the analog, which is queries. So those things are added to this workflow class as just functions that are annotated with signal, update, or query.

118. 25:05

     So that's why we've got a class for the, the, the workflow. And if we take a look at what the logic is in here, you can see that I have a while true.

119. 25:15

     So this simple application is just that same picture that I showed you earlier where I said the LLM, it-- we're just looping on the LLM, and if the LLM makes the decision to do so, we're gonna call tools.

120. 25:29

     That's the whole application. But you're gonna see that I'm doing a couple of interesting things with Temporal here. So in order to invoke the LLM, I execute that activity.

121. 25:40

     So you can see that I'm passing in my model. Um, the instructions here, I, I won't show it to you, but you can see it all in the repository.

122. 25:48

     The helpful agent system instruction basically just says, "You're a helpful agent. If, if you-- if the user says something, and you think you should be using a tool, let me know.

123. 25:58

     You know, choose a tool. Otherwise, respond in haikus." You'll see that in just a moment. Like, haikus are like the FUBAR of the AI world, right? Every-- we're all, we're all gonna write agents.

124. 26:09

     It's the hello world of, of the agentic space. So we're gonna respond in haikus, um, and that's it. So we're doing this in a while true, and I've got a couple of print statements there.

125. 26:20

     You're, you're gonna see how this runs in just a moment. [clears throat]

126. 26:24

     Simplifying assumption here, I'm making the assumption that it's only calling one tool at a time, so I'm grabbing the output of that, and then I just take a look at it and say, "Is it a function call?"

127. 26:34

     And if it, if it is a function call, then I'm gonna handle that function call. I'll show you that code in just a second. And then I'm gonna take the output from that function call, and I'm gonna add it to the conversation history.

128. 26:45

     So I'm not doing any fancy context engineering here, none of that. I'm just basically tacking on to the end of the conversation history, okay?

129. 26:55

     Now, handling the function call is really straightforward as well. So the first thing that I'm doing is I'm adding the response from the LLM. So there's-- we're gonna-- by the time we're done with this function call, we're, we're gonna have added two things to the conversation history.

130. 27:12

     We're gonna have added the response from the LLM, which says, "Please make a function call." And then we're gonna do the function call, and then we're gonna add the result, and I just showed you where we're adding the result of the function call.

131. 27:23

     So here, this is just me adding that to the, um... And this is some of the squirreliest stuff. I'm-- I have this, this application running against the, um, Gemini API as well, and the biggest pain in the butt in all of this stuff is that the formats are different.

132. 27:41

     So I have to rewrite because the JSON formats of conversation history are different between the different models. Yes, I know there's, um, Lite LLM out there, but I don't like least common denominators and al-also I like to understand what those formats look like, so.

133. 27:56

     Um, but you can see here that I'm just doing some ugly parsing. And then I'm executing, remember, I'm handling the tool call here. I'm executing the activity with that tool call.

134. 28:08

     So I've pu-pulled the tool call out of the response from the, um, LLM, and then I'm gonna invoke that activity, which is execute activity, and the item name is the tool.

135. 28:22

     Now, one of the things that I was really intent on here is that I didn't wanna build one agentic loop application that does one set of tools and then have to rebuild a whole 'nother one when I have a different set of tools.

136. 28:38

     The agent itself, and we heard, I don't remember who talked about this, but somebody talked about this on stage this week, where they said, "Look, the agentic pattern is the th-- is, is, is fairly standard, and what we're doing now is we're inserting things into this standardized agentic loop."

137. 28:55

     And that's exactly what these, um, AI frameworks are doing, these agent frameworks, and I wanted to do that here in the Temporal code as well. Well, the cool thing is that Temporal has something called the dynamic activity.

138. 29:08

     The dynamic activity allows you to call an activity by name, but that a- that activity is dynamically found at runtime. So the activity handler here, and I'm gonna show you the code in just a second, is basically gonna take in that name and say, "Oh, okay.

139. 29:27

     I..." And remember, this is event-driven. So we have an activity that's waiting for things on a queue, and so you can configure an activity, you can configure one of our workers to say, "Hey, this is a worker that will basically pick up anything off of an activity queue."

140. 29:44

     Doesn't matter what the name is. So you don't have to tightly bi-bind to a specific topic name, for example. Yes, question.

141. 29:51

     Do I need in advance to map which tools would be available for the agent based on the activity?

142. 30:00

     Um, uh, that is separate, and I'm gonna show you that module. There's a module here that you see that's called Tools, if you see the Tools directory. The way that I'm running it here, it is-- it loads that stuff at the time that I load the application, so I'm not doing any dynamic loading.

143. 30:17

     But I can swap in and out that Tools module, and the agentic code does not change at all. So I'm not going all the way to the point where I've implemented a registry and I'm doing dynamic calling of those things.

144. 30:31

     You can do that, but this simple example has basically just Put that all into a separate module, and you'll see how that module can be switched in and out because I'm loading it at, at, at, at, um, at start of runtime.

145. 30:44

     So, um, simplifying, but yes, you could do that. Okay. So I'm just gonna call an activity. And so let's take a look at what that activity looks like. It's this tool invoker.

146. 30:57

     And so you can see here that it has the activity decorator, just like I showed you before, but now it says dynamic equals true. So that means that this activity handler will pick up anything that is showing up on a queue that isn't being picked up s- by some, uh, other activity already.

147. 31:16

     So it'll pick up, it'll pick up get weather, it'll pick up get random number, it'll pick up whatever shows up in there. You have to register, um... You do have to register-- No, you don't have to register all of those.

148. 31:29

     Those things can be done dynamically. You don't have to register them into the worker.

149. 31:34

     And what you can see here is that we basically grab that, we get the tool name, and then what you can see here is that I'm effectively looking up the function.

150. 31:45

     You can see here there's no tool names in here at all. It's basically looking up the tool name from a dictionary. It, and it, it's metaphorically a dictionary. I'll show you those, those, uh, um, functions in just a second.

151. 32:00

     So I have one function, which is a get tools function, um, which by the way, let me go back to that. So in the OpenAI responses, uh, no, sorry, it's down in the workflow.

152. 32:11

     When I invoke the LLM, right here, notice that I made this get tools call. I'll show you that get tools call in just a second. It's completely outside of the scope of the workflow and the activities.

153. 32:24

     It's in its own module. I'll show you that, um, function in just a second.

154. 32:29

     Okay, so back to the tool invoker. It's basically now taking the name, and then it's doing a get handler. So somewhere here is a get handler call.

155. 32:41

     Here's the handler. You just passed it. I just passed it. Sorry.

156. 32:47

     Seventeen. Seventeen. Thank you. I appreciate it. So here's the get handler, and I'll show you that function in just a second. So great question on the like how tightly bound are these things.

157. 32:57

     Let me show you where that binding is right now. So I have a tools module here, and I have in the init is where I've got those two functions defined.

158. 33:07

     So I've got the get tools, and the get tools are basically just taking the list of functions, and I'll show you those functions. And those functions, what we're passing in here is we are passing in the JSON blobs that are passed to the LLM as the tool descriptions.

159. 33:25

     So these are the tool descriptions. So for example, let me show you the get weather one. So if we go over here to get weather, you can see that the, um, the JSON blob is right here.

160. 33:39

     And it's interesting because OpenAI, um, in, in the completions API, they had a public API that allowed you to take any function that has doc strings on it and generate the JSON for the completions API for the tools.

161. 33:58

     The responses API has no, no such public API. So there's a warning in here that says, "This API that I'm using," um, which is in this tool helper. I'll show you the tool helper.

162. 34:12

     Where is my tool helper? Uh, helpers. Here we go. I guess I could put that in tools. Is that there's a thing in there that says, "Warning, there is currently no public API to generate the JSON blob of tools for the responses API," so I'm using an internal one.

163. 34:29

     There is an open issue on this, so there's just a warning in there that I'm using an internal one. So if we go back there, I've used an internal API to just take my get weather alerts, um, request, which is the, uh, it's a Pydantic model that has the, the functions in there, and has some, you know,

164. 34:51

     additional metadata around it, and gen-generates the JSON blob. So again, that's what you see when we go into the agent, is we-- that's what you're getting with get tools, is you're getting the, the, the, the array of JSON blobs for each of the tools.

165. 35:07

     And then, as I said, the get handler basically has-- It, it's, it's basically a dictionary that I've implemented as, as a set of if-thens. So it's, uh, takes the tool name, and then it picks up what the actual function is.

166. 35:22

     Completely independent. And so this particular example has a set of-- and I'm gonna demo this for you in just a second. It has a set of tools here, um, and you can just switch those things out.

167. 35:33

     You do have to restart your Python pro-process at the moment just because of the way that I've implemented it. Okay. Make basic sense? All right. Let me show you this in action.

168. 35:46

     Um, and so what I've got here is I'm running, uh, the worker. I'm not spending a lot of time here talking about workers, but you remember that I said that this is all event-driven.

169. 35:57

     Um, and so there's something that is picking up work off of event queues and then executing the right workflows and activities based on what it's pulled off of the event queues.

170. 36:07

     The m-- the, um, the thing in Temporal that does that is what we call a worker. So a worker is a process that you run. With that worker, you register the activities and the workflows that that worker is gonna be responsible for.

171. 36:20

     So it's gonna be looking for things on the queue to pull off of those. That worker itself is multi-threaded, so it is not one worker, one process. Um, in general, people run...

172. 36:31

     It depends, you can do worker tuning, but in general, people run at several hundred threads. So you run one worker, and it's already a, a, a, you know, concurrent- Multi-threaded architecture.

173. 36:44

     Okay, so this is some solid stuff. Temporal is just the coolest stuff. It's really truly is distributed systems designed. Okay, so I'm running the worker up here, which is effectively where you're gonna see the outputs coming from the activities and the workflows, and I'm gonna go ahead and run, um,

174. 37:02

     run a workflow. And so let's say, are there any awar-- weather alerts in California? That's where I'm from, and I think a lot of you are from, and hopefully where I will be headed back to tonight.

175. 37:18

     And so we're going to start, and what you can see here is the way that this application is written is basically I, I, um, say whether or not I'm calling a tool.

176. 37:29

     And so you can see here that it said, oh, I made a call to get weather alerts, um, and that is what's happening. So there's a tool call that's happening, and in just a moment, I happen to know that as soon as we get a few drops of rain in California, you know, there's alerts all over the

177. 37:44

     place. So here we go. Here's a whole bunch of weather alerts in California. You'll see why I'm, I'm pointing that out. It's kind of fun. Um, okay. Now, let me show you what this looks like in the Temporal UI.

178. 37:59

     So over here, I have the Temporal UI, and you can see that I've run a bunch of workflows this morning. And so let me refre-refresh this, and this is the one that I just ran.

179. 38:09

     And what we see here is, yep, there's all of that dense fog advisory. That's, that's about as, as extreme as we get in California. A little bit of fog, a little bit of wind, high surf, beach, beach hazards.

180. 38:22

     But here is what happened. So you can see in the Temporal UI, you can see each one of those activities that I called. You can see that I made a call to an LLM.

181. 38:32

     That's the create line that you see on the bottom. So we're working from bottom to top. Then you can see that I did a dynamic activity, but the dynamic activity that I did in particular, it doesn't say generic dynamic activity.

182. 38:46

     It says I did a get weather alerts. And then as we do with agentic loops, we take the output of the tool, and we send it back into the LLM, and we get this.

183. 38:57

     Now, I wanna show you something here. I'm gonna show you a different example. So I'm gonna ask the question, are there any weather alerts where I'm at? So does it know where I'm at?

184. 39:12

     And I'm gonna start this, and I'm very gonna-- quickly gonna come over here, and we'll see this one running. And you can see... Oh, I'm gonna be too slow, it looks like.

185. 39:23

     But I'll, I'll come back, and I'll redo this demo. But you can see how it just, it's, you know, brought those things in. So the other-- So I have three tools registered right now.

186. 39:32

     I have a tool that takes in a state, I have a tool that takes in, um, an IP address and returns a state, and I have a tool that gives me an IP address for the currently running computer.

187. 39:43

     And so I didn't, I didn't wire that up. The, the LLM made those decisions just based on the tools. So all I did was I provided those tools. But you can get that visibility across this in Temporal.

188. 39:57

     So you can see that we started with get IP address, then we got the location info from that IP address, then we got the weather alerts. And here's the ironic thing is there are no weather alerts in New York.

189. 40:10

     So I think of New York as a place that has much more weather, but maybe today is cal... You have fog, but there's no fog advisories. So y'all are a lot more resilient than, than the rest of us Californians.

190. 40:22

     Okay, so I wanna show you one other thing, which is I'm gonna come back over here. I'm gonna run this again, and I'm gonna try to be... I, I, I can be a lot quicker.

191. 40:33

     So I'm gonna hit okay, and now I'm gonna come up here, and I'm gonna Control + C. No workers running. My agent is not running. It's not running at all.

192. 40:48

     And so if we come over here and we take a look at what this looks like in Temporal, and this is gonna give you the clearest picture of what I mean by durability and durable agents, is that I have this agent running, and you can see that it made the first LLM call, then it got the-- it

193. 41:05

     made the tool call to the IP address, and now it's, it's stuck. It started to call the LLM, but hang on. It... So-something went wrong. The agent itself is not running.

194. 41:18

     And by the way, I could have also done a demo, and I don't have the time to do all of those today, but I could have done a demo where I cut the, the network.

195. 41:27

     So I could have cut the network, and what you would have seen here in this little red bar is you would have seen it create attempt one, create attempt two, create attempt three.

196. 41:35

     I could have brought the network back, and then it would have gotten through. But for brevity here, because I still wanna-- I still have more stuff to cover, I'm just showing you one of the failure scenarios.

197. 41:45

     There's tons of failure scenarios that are covered. So I'm gonna come back over here, and I'm gonna restart the worker. And what we should see happen is, oh, sure enough, it keeps going.

198. 41:57

     So it picked up where it left off. Now, when I say it picked up where it left off, of course, I killed the process. There was nothing running in memory anymore.

199. 42:08

     So when I brought the worker back, it had to reconstitute the state of the application. It did that through event sourcing. So that's the way Temporal works fundamentally. Any questions on that?

200. 42:22

     Yes.

201. 42:23

     Can you delegate one agent to another agent?

202. 42:25

     Can you delegate one agent to another agent? Absolutely. Now, we do not yet have native support for A to A, if you're thinking about that protocol in particular. But one of the things that you can certainly do is that you can have an agent act as a tool.

203. 42:41

     And so there are a number of different ways to do that. You can either have an activity invoke another agent, or you can use some other mechanisms. We have child workflows and those types of things.

204. 42:52

     I'm not gonna cover that adv-- more advanced use case, but yeah, absolutely

205. 43:00

     It's very idempotent. There was, like, one single line essentially. Uh, how do you make sure that the developers are using idempotent functions so that the retries are possible-

206. 43:10

     Yep. Okay

207. 43:11

     ... because they're happening a lot. So, uh-

208. 43:14

     Yep

209. 43:14

     ... the second question is the latency which is added through because of the framework.

210. 43:17

     Okay. I'll... So first question is about ar- around idempotence. So you have recognized that, um, the activity, activities themselves should be idempotent. Um, we don't require it, we don't check on it because we really don't get into the inner workings of the activities.

211. 43:36

     We leave that up to the developers. Um, but that is the, the guidance, is that if they're not idempotent... Because remember that when we do retries on your behalf, we don't know w- why we never heard back from the first invocation, so we are gonna keep retrying until we get back a response.

212. 43:57

     Of course, it could be that the request never made it to the activity. It could be that it made it and it invoked the downstream function. Could have gone wrong in a number of places.

213. 44:06

     So how do you make sure that your developers are doing, uh, creating your activities to be idempotent? Education. So we don't have any silver bullets there. The second question was around latency because, yes, I am going up to the server here, um, with every one of those activity calls.

214. 44:25

     And so when we, w- when we think about agents, um, we...

215. 44:30

     The type of latency... And Johan, I don't remember the exact numbers. Do you remember what the numbers are? I mean, it's tiny. The latency to go up to the server around activities.

216. 44:40

     Uh, so it's gonna depend a bit where the server is, but it's tens of milliseconds.

217. 44:45

     Tens of milliseconds. So it's pretty small. Um, so whether... I, I, I think I have heard of several customers who are using this in quite real-time applications. But in the case of agents, especially agents where they're long running and they're running over minutes, hours, days, or they have user interaction, tens of milliseconds is, is, is tolerable.

218. 45:07

     So there might be use cases where it's not applicable because of that latency, but it's pretty small. It's applicable in most cases.

219. 45:16

     Okay. All right. So that is the Temporal overview. Now, I wanna switch back over to the Agents SDK now and show you the differences and the similarities. So let's come back over here.

220. 45:31

     I'm gonna go through a few more pictures.

221. 45:37

     Okay. So the OpenAI Agents SDK, the combination of these two things at a very high level looks like this. At the foundation, we have the OpenAI models and the OpenAI API, not the SDK, but the OpenAI API.

222. 45:55

     So you might have noticed that I've a- already been using the OpenAI API, and we're now gonna start using the Agents SDK. And then we also have Temporal as a foundational element.

223. 46:07

     And now that's what the Agents SDK has layered over the top. So those are two foundational components. So it isn't that we're making... that Temporal's sitting on the side or OpenAI models are sitting on the side.

224. 46:20

     We have actually integrated these things, and I'll make some comments when we get to the code on how we did that integration, and I'll totally invite Johan to add to that as well because he led the in- led the, the engineering for the integration here.

225. 46:34

     So now you've got the Agents SDK, and now you're gonna use the Agents SDK to build your agents, to add guardrails. We're gonna... I'm gonna show you some tracing.

226. 46:44

     So I showed you the Temporal UI, but the Agents SDK has some really cool tracing features as well, and you'll see that we've integrated those things together. You're gonna see those come together and, of course, tools.

227. 46:57

     So I've been talking about these Temporal activities, and we already saw this. I'll skip that, uh, skip that, um. And so now, uh, we're... I'm gonna take the same exact example that we just went through.

228. 47:11

     I'm gonna have three tools. One is the weather API, and the other two are those location APIs. And what are we gonna do? Well, we're gonna put activity decorators around them, and I'll show you what that looks like in the other code base in just a moment.

229. 47:26

     We are going to make sure we have docstrings in there because I showed you how the OpenAI API had the, um, the, uh, the internal helper function that allowed us to generate the JSON blobs to describe the tools.

230. 47:42

     Well, the Agents SDK actually does even more of that for us, so you'll see that some of my code went away. Um, and then, uh, yeah, and then we'll, we'll ca- continue on from that.

231. 47:53

     And then... And this is gonna be our loop here. I'm going to show you when we get to the code that the, um, the way that we create that JSON blob is part of the integration.

232. 48:05

     So you can take an activity, and we have provided for you a function called activity_as_tool that will take in the activity, the function itself, so you don't have to worry about serializing it yourself.

233. 48:18

     No internal APIs. This is a public API. It's part of the in- of the integration. And you're gonna call activity_as_tool, which is gonna generate the, um, the JSON blob.

234. 48:27

     And then you can have your timeouts as well. There's another part which is really important, which is that you have to configure the integration. So you have to... The Agents, uh, SDK alone doesn't use Temporal, and if you want to use Temporal, you need to make sure that you include a plug-in, and I'll show you the code

235. 48:49

     for that in just a second. And then, of course, we're gonna run it, and we're gonna run it the same basic way. Okay, so let's demo that. So we're gonna spend a lot more time in code and demo again.

236. 49:03

     Let me come back over to Cursor. Okay, so I've got four, four files that I wanna show you here I'm gonna start with the activities again. So here's the get_weather activity, and you'll see that it actually got a bit simpler.

237. 49:19

     It's just has the activity decorator on the get_weather alerts, and then here's the function where it actually makes the, the, um, National Weather Service API. So there was another-- there was a bit of, um, code in there that was cr- doing some formatting where it made that API call to generate the JSON blob.

238. 49:38

     That goes away because we have a, a supported function for that. The location activity's equally simple. So literally, this is the entire file. So I've got these two functions with the activity decorators and my docstrings.

239. 49:54

     So the docstrings are describing the, the arguments and so on.

240. 49:59

     Okay? So now what about the agent itself? So remember, activities were just the things that those were the tools that the agent was using. What I showed you before was an agentic loop written in Python that was orchestrating the LLM call and the tool invocations.

241. 50:19

     So now if I come over to the workflow, what does my workflow look like? This is it.

242. 50:26

     So I have... Sorry, this is it. This is the workflow, okay? And it's basically-- let me increase the font size 'cause I have plenty of space, is notice that I'm using the Agents SDK.

243. 50:41

     So I'm defining an agent right here on line 18. I'm giving it a name, You're a Helpful Agent, and I'm giving it a set of tools. Those tools were implemented as activities, and there you can see that I've made the function call that says activity_as_tool.

244. 51:00

     That generates the JSON blob. That's it. That's all there is to it. That's the way I've implemented it. Now, notice that I'm still doing it within the workflow because remember earlier I said, "We've got activities, we've got workflows.

245. 51:14

     When you put them together, that's where the magic happens"? So putting this agent inside of a workflow, that's what adds all of these durability capabilities. Now, I am not gonna demo the non-durable version of this because, again, we just have constrained time here today.

246. 51:32

     But if I had implemented this with just the Agents SDK, which is just a Python library, I would've had a single process, and if I killed that process, it-- everything would've gone away with it.

247. 51:45

     I-- There's no way for me to scale that process. And remember, I've got an, a runner.run right down here, right? So every one of those runners, I, the, it, it's just, it's just a, it's a monolithic agent, right?

248. 51:59

     It's just one Python process. By doing it this way, you can see that if I'm running multiple workers, I can just keep scaling this out by just running multiple workers who are just pulling other things off of the queue.

249. 52:13

     Yes? Do handoffs to other agents work? Do handoffs to other agents work? Yes, and I'm gonna come back to that in the final section where we're gonna talk about orchestrations.

250. 52:22

     Um, yes, they do. Absolutely. Um, okay. So now let me go to, uh, I need to-- I do need to get one other thing out of here. I need to get my worker.

251. 52:35

     Um, where is my worker? Here's my worker.

252. 52:47

     Because it's in the worker where... Where's the plugin? Johan, help me out.

253. 52:58

     I just passed it. I just passed it. Oh, here we go. OpenAI Agents plugin. Okay. So it's in the worker. Remember the worker is where all the execution is happening.

254. 53:08

     Think of it as kind of like a logical metaphorical container, and oftentimes you are running the workers in containers. And here's the configuration that you need to put in there.

255. 53:18

     It's those-- And notice that it's doing things like it's, it is, um, configuring some of the retry behavior around the LLM. You'll notice-- Did you notice that I did not give you an activity for invoking the LLM?

256. 53:33

     That's done as a part of the agent, but we still want that to be durable, and that's one of, one of the things that our implementation does here. So you're doing some retry policies for the LLM, and you're basically saying, "Hey, a- OpenAI Agents SDK, use these parts of Temporal."

257. 53:52

     So let me tell you a little bit about what we did as a part of the integration. What we did, if you go back and you look at the commit history of the OpenAI Agents SDK, you'll find a commit where it says, "Make the runner class abstract."

258. 54:09

     They did that for us because that's how we imple-- that's how we got this durability. That's how we were able to make the LLM calls durable, along with all of the tools that you just saw.

259. 54:21

     So we have our own implementation of the abstract runner class. So that is the way that this works. Okay. So we've looked at the activities, we've looked at the workflow, and then we run the tools workflow.

260. 54:38

     I don't think there's anything more to look at there. So let's go over to

261. 54:45

     this. Um, let me find my right window.

262. 54:51

     It is this window. Okay. So what I'm doing in this window, let me go ahead and increase my font size a little bit, is up here in the top window...

263. 55:02

     And I'm gonna exit out of these 'cause I only need two.

264. 55:07

     Okay. So in the top window, I'm running my worker again. You saw that before. So I'm running the worker up there. And so it's got a little bit of, you know, log messages that are c- gonna come out.

265. 55:19

     And then down here is where I'm starting the workflow. Here's where I'm interacting with the agent. So I'm going to-- I just pushed some of this stuff. Oh, I'll show you the, the repository in just a second.

266. 55:31

     So are there any weather alerts in California?

267. 55:35

     And remember the code. The code is just that agent, right? We've still implemented our activities, but the code is that agent, and we are checking... And you can see the things that are scrolling up there.

268. 55:47

     So you can see actually the, uh, API calls to the LLM. You can see the API call to the National Weather Service. And so if we come back over to the Temporal

269. 56:00

     UI, it's a different-- It's called Tools Workflow. Here you can see it looks exactly the same,

270. 56:12

     right? And that's why I wanted to spend some time showing it to you with Temporal, because Temporal, if you're building these Temporal-native applications, you get all this durability, you get this visibility.

271. 56:23

     It looks exactly the same, but you are using the Agents SDK to implement your agents, which I think is just so cool. So let's run a second one.

272. 56:34

     And, uh, I'll show you the repository in just a second. We're gonna run this second example, which are, are there any weather alerts where I am? And we're gonna come over here.

273. 56:45

     We'll watch it in progress. So here it's running, and it's running exactly the same way, and I'm gonna run it one more time when it comes back. And it says, "Nope, you're good in New York."

274. 57:00

     Let me run it one more time. I'll let it get started, get part of the way in, Control + C out of this, and we come back here.

275. 57:13

     And we see the same thing that we saw before, right? Agents SDK, durable, which is so sweet. I, I-- Sorry, I get-- I, I do this all the time, but I still get so excited about this.

276. 57:28

     Um, and there it goes. I'm gonna share with you, like, um, um, a, a way to think about this intuitively, what we're doing here. I've been writing software for over thirty years.

277. 57:41

     Yes, I have the [REDACTED:physical_attribute] to prove it, right? I have, I have written that software where I-- when I'm writing the software, I'm thinking about the processes that it runs in.

278. 57:52

     I'm thinking about the fact that, oh, I've got a process here, and what happens if something happens to that process? Or as I'm scaling things out and things might not run in the same processes, I'm always thinking about processes.

279. 58:05

     With Temporal, what you get to do is you get to write your program thinking about a process as a logical entity, and just let Temporal map it down to the actual physical processes that are there.

280. 58:19

     This is particularly... It, it's, it's so cool to look at when you do things like human-in-the-loop. So earlier this week, I did an-another-- I've done another, I-- Another one of the talks that I do is around human-in-the-loop with agents.

281. 58:33

     And one of the big pains of human-in-the-loop is that when you're thinking about building these things and you're thinking about the processes, you're like, "Okay, I need to run for a little while, and now I'm gonna wait for the human-in-the-loop," and it might take a second, it might take a minute.

282. 58:48

     Likely, it's gonna take hours or days before this human comes back. What do I do with that process that's waiting for that response in the meantime? Like, you as a developer have to figure that out.

283. 59:00

     With Temporal, you don't. You just code it as if that process... And by the way, the way that the worker architecture works is that if it's waiting on something like human in, you know, input, it, it'll keep it in memory for a little while, few seconds, then it'll take it out of active memory, but it's still sitting

284. 59:20

     in a cache, and after a little bit more time, it'll come out of the cache, and it'll be just as if you had just killed that process. So that when it does come back, after days or weeks, when the user comes back and gives you that input, it just reconstitutes memory the way that it was when it

285. 59:38

     was waiting for the user and continues on. So remember I said it's a crash or it might be other things. Operational things, all of those types of things. So it's so freeing.

286. 59:50

     Once you start to really get into this Temporal thing, it is so freeing to realize that I don't have to think about physical processes anymore. To me, the processes are just logical.

287. 1:00:00

     Temporal takes care of the rest for me.

288. 1:00:03

     Super cool stuff. All right. We have about, uh, twenty minutes left. I'm gonna go through just a couple more slides to answer one of the questions around handoffs, um, and just show you just a little bit more content.

289. 1:00:19

     And then we're, we'll have a little bit of time in the last, I don't know, maybe ten, you know, ten or fifteen minutes. I'm happy to take more questions.

290. 1:00:26

     And Johan would be, I'm sure, very happy to, to jump in as well. Okay. So let me...

291. 1:00:35

     So, um, there are-- With the OpenAI Agents SDK, this is-- What I'm talking about here is somewhat specific to the Agents SDK, but this does kind of generalize to agents in general, which is, with the OpenAI Agents SDK, the kind of paradigm that they use there is to build lots of small agents that have their own

292. 1:01:00

     independent agentic loops and then orchestrate them together. And there's two ways that you can orchestrate them together in the Agents SDK, and I'm gonna show you both of those in just a moment.

293. 1:01:11

     So what we see here on the screen are just a couple of diagrams of like, okay, I've got a triage agent, I've got a clarification agent, then I've got a human in the loop.

294. 1:01:23

     That's not an agent. Although, you might think of the human as the agent in this application, right? Then I've got an instructions agent that's gonna craft some stuff. Then I've got a planning agent.

295. 1:01:35

     Then you can see that we're doing things in parallel. I didn't talk about this, but Temporal has all those abstractions. You can-- anything that you can write in a regular programming language, you wanna do multithreaded and have a bunch of different threads and do things in parallel and then wait for them to come back together, you can

296. 1:01:53

     do that. You want to have some kind of an await that says, "You know what? As they start coming in, I'll start processing them," no problem, you can do that.

297. 1:02:01

     Anything that you can do in code, you can do with Temporal because you're just coding. So that's effectively the way that it works here with the Agent SDK as well.

298. 1:02:11

     Um, I'll show you those things again as well. So you can do parallel, you can have long waits, um, and you can have loops. So I already showed you the fact that I didn't have to craft my logic in the Python.

299. 1:02:26

     The logic, the LLM made the decisions on how the flow was gonna happen in that application, right? So I just had a loop that the loop itself is, like, fixed, but what happens in the loop is entirely determined by the LLM.

300. 1:02:40

     So that-- those are all things that you can do. So there's two ways with the Agents SDK that you can orchestrate these micro-agents. And by the way, I just have to say, I love the term micro-agent.

301. 1:02:52

     As I mentioned early on, um, I, uh, I spent a lot of time in the microservices world. And oh, my gosh, did we get a lot of mileage out of that, right?

302. 1:03:02

     That's the reason that we can deploy software multiple times a day. That's the reason why we can scale the way we can scale. Microservices have proven themselves to be very valuable.

303. 1:03:13

     We-- I think we're gonna see a very similar paradigm, uh, very similar success when it comes to building AI agents, MCP tools, and all of that kind of stuff.

304. 1:03:23

     So I love the, the notion of micro-agents that do one thing and one thing well. I've spent enough time in the land of L- Unix as well. That makes my heart sing. [laughing]

305. 1:03:32

     So just code and handoffs. The just code is really simple. It's what I described, um, before. So you can see here that I have a runner.run. I'm executing an agent.

306. 1:03:43

     I get back a result from that agent, and I pass that result into the next agent. I can parallelize, I can loop, I can do whatever I want.

307. 1:03:53

     The second way that... Oh, yeah, and I already showed all that. Um, so yeah. Yeah, yeah. I, I mentioned all of those already. Um, the second, uh, a, uh, way that OpenAI...

308. 1:04:05

     And were you talking about OpenAI specifically when you were asking about handoffs? Yeah. So for those of you who don't know, OpenAI has a second way of doing orchestrations, which are called handoffs.

309. 1:04:17

     And so what you see here is that in my definition of an agent, I can define handoffs. Those handoffs are other agents. They've been defined-- I sh- probably should have it on the slide, but I have a weather agent that is defined very similarly.

310. 1:04:31

     It's using agent, has a name, has instructions, might have tools. So these two are agents.

311. 1:04:40

     Um, and all of this works with the, the, the e- the integration with, with, um, Temporal. And but what's interesting here is that those micro-agents, when it hands off to an agent, it is not doing a separate agentic loop.

312. 1:04:57

     It effectively, and this is my w- wacky attempt at trying to describe what's happening here, is that when you do a handoff, what you're effectively doing is just changing the context of the agentic loop.

313. 1:05:12

     So the a-- there's one single agentic loop. You have a triage agent, for example. It decides that it's gonna go into delivering... I worked at Alexa for a while, so did you ask how late is Costco open, or did you ask what the current temperature is, right?

314. 1:05:28

     So those are two different agents, the weather and the local info agent. And what you're effectively doing is you are-- that, that agentic loop is taking on a different persona.

315. 1:05:37

     You're just switching the context, and we heard lots of talks this week about context engineering because that's the beautiful thing about the LLMs being forgetful, is that you can just, you can completely control what you want, what the context is that's going into them.

316. 1:05:51

     So this works exactly... So Temporal is totally handoff aware. I don't have a live demo of that, um, and so that's it. Okay, so with that, we're gonna have a few minutes left for questions.

317. 1:06:03

     I wanna leave you with a few resources. So what you see here on the, um, left-hand side is you see a QR code for the, uh, Temporal Python SDK.

318. 1:06:18

     You'll find lots of great info on our Python SDK, but you'll also find a contrib di- directory, and that's where all of the integration code to the OpenAI Agents SDK is, and you'll find lots of samples in there.

319. 1:06:32

     On the right-hand side, I didn't have a QR code for it because my marketing folks don't work on Saturday mornings, good for them, um, is you will see a URL, URL, URL here.

320. 1:06:45

     So if you go to our documentation, so docs.temporal.io, at, in the top banner, you'll find AI Cookbook. We have an AI Cookbook that implements a bunch of patterns. The one that I showed you today, the agentic loop, is in a, um, branch at the moment.

321. 1:07:00

     It is ready to be merged. It's been reviewed, but my reviewer didn't actually give it an approval review. They just reviewed it and said, "Looks good," but I need an approve, so I couldn't merge it this morning.

322. 1:07:11

     Um, but that, uh, recipe will be up there. And there's a number of others. There's a recipe in there for OpenAI Agents SDK, so you'll find that in there as well.

323. 1:07:21

     Um, so summing it up, I don't, I'm not gonna belabor that. I think we went over that as well. Two other resources that I wanna leave you with is on the left-hand side, you'll find our blog, where we describe the integration of the OpenAI Agents SDK and Temporal.

324. 1:07:37

     So that's the blog on the left-hand side. The other thing you'll notice is that, that I have a Pydantic blog there. And so this idea of in bringing durability to these otherwise non-durable age, you know, agent frameworks is a very popular one.

325. 1:07:54

     So after we did OpenAI Agents SDK, Pydantic themselves integrated Temporal into their agent framework. And Johan, do you-- I, I don't know which ones we can talk about. We have a whole bunch of other ones in progress.

326. 1:08:09

     Is that all we wanna say, or do you wanna talk about any of those in specific? Yeah, that's, that's all for today- Okay ... but there's more coming. Yeah.

327. 1:08:15

     I think we have two or three or four that are in progress right now that will be popping out either from us or from some of the other agentic frameworks.

328. 1:08:25

     So this idea of bringing durability to what is otherwise kind of just a, a proof of concept tool is, is turning out to be quite powerful. And then finally, if you would be so inclined, here's a QR code and a URL.

329. 1:08:40

     If you wanna give us some feedback on this workshop, we'd really appreciate it. Um, and include in that feedback, there's some free form in there, include in that feedback what you'd like to see more of.

330. 1:08:52

     Like, "Hey, this was cool, but it didn't go far enough," or, "You mentioned this. I'd really, I'd really like to see more about human in the loop." By the way, if you go on our YouTube channel, you will find a bunch of different presentations.

331. 1:09:05

     So the human in the loop, I did a webinar on that, like, three weeks ago. We've done some MCP module-- MCP things, even advanced. So we did an advanced one where, where, where we showed you how you can use Temporal to implement an MCP server that is durable and supports sampling and elicitation in a much more durable

332. 1:09:24

     way than it otherwise is. Um, and with that, we still have now about eight minutes. Um, so-- Oh, I'll also mention, especially if you're in the Bay Area, but if you're-- even if you're not, is that our conference, our Replay conference, the Temporal conference, um, is in May, and it's gonna be at Moscone, and so we sure

333. 1:09:47

     would love to see you there as well. And oh, I-- You know, this, this QR code here was meant for the workshop that I did on Tuesday, but heck, might as well use it here.

334. 1:09:58

     Seventy-five percent off. Yes, seventy-five percent off the registration,

335. 1:10:04

     um, for, uh, Replay. So invite you to come there. You'll find me, you'll find Johan, you'll find a whole bunch more of us. And you can see that we've got all sorts of really cool people like Samuel Co-Colvin is coming, and we've got, like, people from Replit and NVIDIA and lots more coming.

336. 1:10:22

     So I'll leave the feedback up there. So we have a few minutes left for questions.

337. 1:10:27

     Yes. Um, how is state saved in Temporal? How is state saved- If you shut down the server, it just loses all the memory of all the workers or anything?

338. 1:10:36

     Yeah. Okay, so you're talking about the Temporal server that I showed you how you can run it locally. So when you're running it locally, you can put a switch on there that says, "Hey, store it in a, like, SQLite or something like that."

339. 1:10:47

     So you can-- There is, there is state that backs it. I usually don't run that way because I wanna throw away things anyway. Um,

340. 1:10:56

     so Temporal is open source, and we have lots of users that self-host it. Because it's open source, we literally at events like this always run into people. I ran into somebody yesterday or something who was like, "Oh yeah, we're using Temporal."

341. 1:11:10

     And I'm like-- And they're just using the open source. Um, so it's, it truly is open source. Um, and so we do have people who are self-hosting it, and we support relational database in Cassandra as the backing stores.

342. 1:11:22

     On Temporal Cloud, part of the reason that, um, that people come to cloud is that, first of all, we run cloud in fifteen different A-Amazon regions, uh, I don't know, four, five, six Google regions.

343. 1:11:36

     We have multi-region namespaces, all of that durability. And then we do also have some special sauce on the persistence that allows us to do things more efficiently. Um, uh, but we have people who are hosting it quite, quite successfully using the Cassandra or the database options.

344. 1:11:52

     So yeah. Any other questions? Yes.

345. 1:11:58

     When you start and stop the instance-

346. 1:12:00

     When I start and stop the instance-- The agent, you mean?

347. 1:12:03

     I mean, how do we stop it if the user wants to stop it?

348. 1:12:08

     The work-- So-

349. 1:12:09

     The workflow.

350. 1:12:10

     Yeah. Um, so there, there are a couple of different ways that you can start a workflow. You can start a workflow expecting it to be, you know, synchronous, and it just ends, and you would have to have some kind of a kill on that.

351. 1:12:22

     But more commonly, you're gonna start it in an async mode and you're gonna get back a handle, and then you can do things against that handle to stop workflows.

352. 1:12:31

     Um, but generally, the, the most common thing is that you are going to define in your logic what it means to have completed that agentic experience or completed that workflow in some way, shape, or form.

353. 1:12:43

     And so you will decide, "Oh, I'm done now," and you'll just return. So it's really as simple as doing a return from it. It's-- That's a good question, though.

354. 1:12:52

     Since we're talking about these asynchronous things, I didn't talk about it in this session, but one of the really powerful things is that these workflows can run for hours, minutes, days, weeks, months, years, and it's super efficient.

355. 1:13:05

     And what we-- What-- A pattern that a lot of our users use is they use it, uh, they use a workflow as kind of a digital twin of something else.

356. 1:13:15

     We call it entity workflows as well. So for example, you might have a workflow that corresponds to a loyalty customer, and every time that loyalty customer scans their QR code at the checkout register, it'll send a signal into the workflow, and the workflow is otherwise not consuming any resources, and it will just pop up, take the signal,

357. 1:13:39

     process what it needs to, and go away again. So that is a very, very common pattern, is this notion of digital w-workflows as digital twins of some, um, other processor or some other entity.

358. 1:13:51

     Super, super powerful. Lots of people use that

359. 1:13:55

     I have another one.

360. 1:13:56

     Yep.

361. 1:13:57

     Um, so if work goes down, and you've got it, like, in that blocked state where it's just sitting, how... Do you guys have any kind of integration with, like, incident management, things that would, like, trigger an alert so, like, an engineer can come in and take a look at the worker?

362. 1:14:11

     Yeah.

363. 1:14:11

     Things like that.

364. 1:14:12

     So we, we... As far as I know, we do not have those integrations, but we- our customers build those integrations.

365. 1:14:19

     They build those on top.

366. 1:14:19

     Yeah. Yeah, they absolutely build those on top. Whether we have some of those in cloud, I'm not sure. But it isn't something... Like, we don't have, like, native Slack connectors or those types of things.

367. 1:14:30

     And some of that, of course, wouldn't necessarily be a Temporal thing. Like, if you're running your workers on Kubernetes, you might have, um, c- set up your Kubernetes configuration so that when the, when the, um, when the container goes down, you're gonna get alerts, or when you see something in the Kubernetes dashboard where, oh, gosh, like, my

368. 1:14:49

     autoscaler... And of course, you can have these workers and y- you can have them running in, on Kubernetes with autoscalers. So a lot of that or, you know, that, like, IT orchestration stuff-

369. 1:14:59

     Awesome, yeah

370. 1:14:59

     ... probably would come through your operational environment.

371. 1:15:02

     Yeah.

372. 1:15:03

     Um, you are-- That's actually a really good point, is that we host the server, but we do not host the workloads for you. So you're hosting the work- workloads yourself.

373. 1:15:12

     Most people love that because they want complete control over that. We are toying with the idea in some instances of maybe hosting workers in the future, but that's the-- nothing on the roadmap right now.

374. 1:15:24

     So... Yes?

375. 1:15:26

     Do you have any examples of people building, like, voice agents with Temporal, or-

376. 1:15:31

     Examples of people building voice agents with Temporal. I don't know of any offhand.

377. 1:15:37

     I, I don't know of any that are deployed. Um, people are experimenting with it. We're experimenting with voice agents. Um, and it's definitely something that, that makes sense. That's one of the places where we expect agents to go in the future.

378. 1:15:51

     Yep. Yes, question in the back.

379. 1:15:54

     Do you have templates for Claude agent SDK, not just the OpenAI SDK?

380. 1:16:01

     So I do not have Claude up in... Keep an eye on the cookbook. I do not have examples for Claude just yet. Um, I have Gemini almost done. Um, and yeah, we wanna, we wanna add Claude to the, the cookbook as well.

381. 1:16:15

     We'll also happily take PRs, so if you wanna take this example and map it over to, to Claude, we'd love to have PRs on that as well. So yeah, this, this is-- The cookbook is all open source.

382. 1:16:26

     It's all in MIT licensed in our main repository, our main org. Mm-hmm. Yes.

383. 1:16:34

     Any example agents on extracting information from Excel, PDF?

384. 1:16:38

     Example agents of extracting from Excel or PDF. I don't have any personally. Um, you know, we-- One of the other things that I'll mention is that we have a code exchange.

385. 1:16:49

     So we have, um... The cookbook is ours, and that's where we're very, we're very careful about making sure that it demonstrates best practices and we, we do rigorous reviews on those because we don't wanna mislead you at all.

386. 1:17:02

     We also have a code exchange which we have literally, I think, twenty or thirty or forty examples in there. There might be something in there. I'm, I'm honestly not sure.

387. 1:17:12

     Um, yeah.

388. 1:17:14

     Is it like a Git repo?

389. 1:17:15

     Yes, it is. So you'll find the code exchange on our website, and all in-- I believe all of the entries in the code exchange have GitHub URLs. We don't own most of them, um, because they're from the community, but they're in other people's repositories.

390. 1:17:29

     So yeah, you would find that. All right. Well, this has been... Oh, is there another question?

391. 1:17:40

     No, I, I, I just wanna comment that, uh, we've mentioned a few times that's coming or that would be really, really cool to do. And, um, you know, w- uh, my team is hiring, uh, for folks-

392. 1:17:51

     The hiring plug. [laughs]

393. 1:17:52

     ... on these AI applications of Temporal.

394. 1:17:55

     Okay, and since he said that, it-- I am a d- I'm in developer advocacy. We're looking for developer advocates too. [laughs] So if you fit the engineer profile, talk to Johan.

395. 1:18:04

     If you fit the developer advocate profile, come talk to me.

396. 1:18:08

     So okay. Well, thank you so much. [outro music]