Building durable Agents with Workflow DevKit & AI SDK

Talk transcript

1. 0:00

   [upbeat music] Thank you all for coming.

2. 0:22

   Hello, hello.

3. 0:24

   We got you. [laughs]

4. 0:28

   Uh, I don't know about you, but, um, my write agents, I like focusing on the capabilities and the features, and I like not thinking about, um, all of the extra effort that goes into getting something that works locally into production.

5. 0:46

   And something that's very useful for that is, uh, a workflow pattern, and that's why we developed the Workflow DevKit, which is what we're talking about today. Presumably, if you're here, you've had similar issues.

6. 0:59

   Um, and today, uh, we are going to, um, turn a agent, uh, coding agent, um, into a workflow support coding agent throughout this session. So

7. 1:14

   I'm gonna skip this slide. So we, um, have a open source example, um, ready to go. Uh, this is on the vercel/examples repository. Uh, so you can clone that and check out the Vibe Coding Platform app inside.

8. 1:30

   Uh, we're gonna be using this app, um, for today's demo.

9. 1:35

   Um, and after we're done, we get first-class observability built in and also durability, reliability. Um, we get a lot of extra features like resumability, um, and the Workflow DevKit makes it very easy to add human-in-the-loop workflows and similar, uh, things.

10. 1:56

    So if you think about our, uh, general agent loop, um, that we've all seen before, we mostly have calls back and forth from an LLM, um, to our tool calls and our backend, uh, code, right?

11. 2:11

    Which would include MCP servers, human approval, any kind of async tasks.

12. 2:17

    And, uh, the usual way to go about this is to, uh, wire up some queues, um, and a database, especially if you are doing long-running agents, right? That might run for hours, um, and you wanna scale, and you're running on serverless, for example.

13. 2:33

    Uh, you want something-- some kind of reliability layer in between, which usually is filled by queues. Um, and then you'll also need to add a lot of error and retry codes.

14. 2:43

    You'll need to store all of the messages that people are sending and also between states. And you probably also need to add some kind of observability layer in between.

15. 2:53

    All of those things we are going to do today, uh, using only a single library, which is the Workflow Development Kit. Um, it's open source, um, and it runs with any of your TypeScript, um, frontends or backends, and can also run on any cloud.

16. 3:13

    Uh, we're gonna be deploying to Vercel today, but this could just as easily run AWS or any of your cloud stacks, um, or Supabase or any of your custom stacks.

17. 3:26

    Um, all right. So who here has heard of the workflow pattern or has used a workflow library before? Show of hands. All right. That's less than half. Um, I'm gonna quickly explain what a workflow pattern is, um, to, uh, m-make it clear what we're doing, and then in about two minutes we're gonna go into the code.

18. 3:46

    So, um, a workflow pattern is essentially a sort of orchestration layer, um, that separates your code into steps that can run in isolation and can be retried, um, and have their, uh, data persisted, and a orchestration layer, um, that we call a workflow, and different platforms have different names for that.

19. 4:08

    And in our case here, um, the, the workflow part would be whatever the loop is that calls, um, the reason-- the LLM calls, and then goes back to the tool calls, and then back to the LLM calls.

20. 4:21

    And the steps would be our actual tool calls and our LLM calls.

21. 4:26

    Um, right. So looking at the agen-- the agenda for today, um, we're gonna be jumping, jumping into the code. We're going to add Workflow Development Kit, which is gonna be quite fast.

22. 4:37

    And then we have a lot of time to talk about, uh, cool additional features that it adds, like resumable streams out of the box, um, how to suspend and resume at any point, um, and how to add webhooks for human-in-the-loop, uh, processes.

23. 4:52

    At the end, there's going to be ample time for Q&A, but there is a reason that you're here in the workshop and not looking at this, uh, online, which is that you can ask questions.

24. 5:01

    Please do so at any point. Uh, feel free to raise your hand or just shout out the question, um, and we'll get to it.

25. 5:10

    All right. So as I said, uh, we're working off the vercel/examples repository, and we're going to be working off of the conf/1-base branch. Uh, why this branch? I've stripped a bunch of the excess code from the example to m-make sure that we can focus on the most important parts.

26. 5:29

    And every checkpoint from this workshop will have its own branch. So, uh, if you're not coding along directly, you can also check out the steps step by step, and then check the diffs and see what changed between.

27. 5:44

    All right. So, um, I have already run, uh, npm dev locally on this, uh, um, platform, just to show you what it looks like. I'm gonna run a simple query.

28. 5:57

    Um, so this is a, uh, coding agent, right? It's like a code editor, but without the code editing. And, uh, it can take a prompt, generate some files... And it'll eventually show you a iframe with the finished app that is deployed.

29. 6:13

    Um, so it's, it's mostly UI with a s-- few simple tool calls, um, that we'll look at in a second. And the file system and output, uh, runs over the cell sandbox, but you could just as easily run this locally.

30. 6:27

    Uh, looking at the code, I'm gonna go and check out our actual branch.

31. 6:36

    Looking at the code, uh, we h-have one endpoint that accepts our chat messages, right? And it does some, some regular sort of model ID checking with us to see w-whether the model is supported.

32. 6:48

    And in the end, it's going to simply create an agent. Oh, yes.

33. 6:53

    What was that branch one more time?

34. 6:55

    The branch was conf/one-base.

35. 6:59

    Okay, let me just clone that.

36. 7:00

    You just clone that. Yeah. And you can see, we'll, we'll move on to these conf/two-dash, et cetera. Uh, just look for the numbers and you'll find all the checkpoints.

37. 7:10

    Um, yes. So our main endpoint just accepts the messages and calls the AI SDK agent, which is essentially the same thing as a stream text call. Um, we'll pass some tools, and internally, we'll just loop for stream text, tool call, stream text, tool call.

38. 7:30

    Uh, and then it'll, uh, stream the-- all of the chunks generated back to the client in a format that is easy for the client to understand. This is all sort of AI SDK regular code, um, that you could replace with a different library if you want.

39. 7:47

    Um, that is mostly there to support the UI. Um, but again, all of the actual agent stuff is very simple and happens here. So, uh... Oh, let's also take a look at the tools that we have.

40. 8:00

    Uh, we have four tools: uh, create sandbox, get sandbox URL. These are very simple. They just wrap, uh, the cell sandbox.create and .geturl. And similar with run command, essentially wraps, uh, sandbox.runCommand.

41. 8:16

    And generateFiles, um, will generate, uh, a file from a simple prompt. And we're gonna take a look at one of these tool calls for as an example. Uh, we have a, we have a prompt that looks somewhat like, you know, a markdown file, sort of what to do, what not to do.

42. 8:32

    Um, and my hotkeys are not working. And

43. 8:39

    go back to the tool call. We also have an input schema that's a Zod schema, um, for, uh, what the AI is supposed to pass. This is all very standard.

44. 8:48

    And then an executeRunCommand, which wraps sandbox.runCommand with some error handling.

45. 8:56

    So that's essentially our entire, uh, agent code set up. And then in the front end, we just call useChat from AI SDK to consume the stream and display, uh, things in the UI.

46. 9:07

    So, uh, let's get started adding workflow to this. Um, any questions before I get started?

47. 9:17

    Cool. All right. Uh, so step one is we're going to run PMPM install for workflow and add workflow/AI,

48. 9:28

    which will give us the latest version. Uh, workflow is the main library, and add workflow/AI are some helpers for, uh, uh, some, some wrappers for AI that work well with the Workflow Development Kit.

49. 9:44

    So now that we have this installed, uh, we are running a Next.js app here, so we're going to, uh, extend the compiler to, uh, compiler workflow code by doing use workflow or actually with workflow, um, which c-- we can import from

50. 10:05

    workflow/next. And that'll set up Next.js. Yes.

51. 10:13

    Verifying question. You are in the example/app/live coding directory?

52. 10:19

    Yes. So adding this will let the compiler know to also compile our workflow codes, um, separately, which we'll get into more in a second. And then for convenience, we can also add a TypeScript plugin to our TS config, um, same package, and that'll give us some better auto-completion for our workflow code.

53. 10:44

    And so we talked about, uh, a workflow being, um, having an orchestration layer and having a number of steps. Uh, what we're gonna do first is we're going to write the orchestration layer.

54. 10:58

    Um, in our case, that is essentially just the agent, right? Does the loop, the calls, steps, um, back and forth. We're going to

55. 11:07

    add a new file, uh, you can call it whatever you want. And we're going to take our

56. 11:14

    agent call and move it over there. And I'm gonna call this our, uh, code workflow,

57. 11:25

    which is going to be all of our workflow code.

58. 11:32

    And then I'm gonna go and auto-complete a bunch of imports.

59. 11:42

    Thank you, AI. So we're just passing most of the, uh, a-arguments that we would otherwise get from here over there. And

60. 11:53

    this completes the refactor, essentially having done nothing but pull out some of the workflow code into our file. So this is where it gets interesting.

61. 12:02

    Um, now that this is a code, this is a separate function, we can use the use workflow directive,

62. 12:08

    which will mark this for our compiler, uh, as a workflow function. So what this does is under the hood... Yes. Question? Oh, sorry. [laughs]

63. 12:20

    Uh, under the hood, it, um, compiles all of the code related to this function into a separate bundle, and it ensures that there is no imports to anything that would have side effects.

64. 12:31

    Because the workflow or- orchestration layer needs to be deterministic, so it can be rerun, um, in a, in a, uh, de- deterministic fashion and there's no worries about state, uh, pollution.

65. 12:45

    So now that we have this, um, we would need to now mark our LLM calls as steps, and because the LLM calls are happening inside the agent, um, this is a little bit harder

66. 13:00

    to do here. And so we, uh, ended up writing a

67. 13:06

    durable agent class, which is essentially the same thing as agent with a use step marker in the actual LLM calls that it does under the hood.

68. 13:19

    So now that we have this set up, we're going to

69. 13:25

    await the actual streaming and let's see if there's anything

70. 13:32

    we need to do. Um, checking for errors. Oh, yes, we need a stream to write to. So previously, we could just write to the stream that the API handler gave us.

71. 13:45

    Uh, now we're going to have to create a new stream to write to. Um, we export a getWritable function from workflow, which is-- which gets a stream implicitly associated with the workflow to write to, and we're going to get...

72. 14:02

    get into that a little bit more in a second. But for now, we'll just pass that to our agent, and we're going to see if this is the right type.

73. 14:15

    Um, presumably not. Um, and then finally, back in our actual workflow,

74. 14:27

    we need to call our workflow in a way that the, uh, framework understands, which for us is a call to start

75. 14:37

    with the arguments being passed separately, which essentially telling it to start a new workflow

76. 14:45

    on this, on this function, and start can be imported from workflow/api.

77. 14:51

    So now we essentially have, uh, the workflow fully hooked up, and a lot of this was just pulling out some of the codes and adding a directive.

78. 15:04

    Yeah. Yeah. Um, Panay has volunteered to help anyone who's, like, following along and has some debugging questions. Um, just reach out to him.

79. 15:18

    I'm on the team as well, so let me know if you guys are following along.

80. 15:21

    I'll be around to help.

81. 15:24

    And finally, the start call returns a run instance, um, that has the stream that we just, um, end up writing to, that we can return to the UI. So this completes our workflow, um, definition.

82. 15:38

    And now, uh, we also said that we would need to mark things as steps. The durable agent class already marked the LLM calls as steps, but our tools right now are not marked as steps.

83. 15:50

    Thankfully, this is very easy. Uh, in the execute function for each of these tools, you can just write use step, and that will let the compiler know that this is a separate chunk to, uh, of code to, uh, execute in a separate instance, right?

84. 16:07

    If this is deployed to production, this would run in a separate serverless instance, and the inputs and outputs would be cached if it already ran, and it would be retried if it failed.

85. 16:18

    So I'm gonna go and go through the other tool calls and also add use step to these. Thankfully, we only have four of them.

86. 16:30

    And that should complete our transformation. So now we can go and run PMPM dev,

87. 16:41

    see if this works as expected. We're gonna reload our page.

88. 16:51

    And it seems like nothing changed. Let us actually run a query.

89. 16:57

    And we can see that it's still streaming as expected. So, uh, for us developing locally, right, all we have to do is pull out a function, um, and then add some directives.

90. 17:08

    But now, if I deploy this, um, to any adapter, again, Vercel or an AWS adapter, or maybe you have your own, um,

91. 17:18

    this will run, uh, in isolation with durability and all of those good things. And something that's really nice for local development also is that if I go and...

92. 17:31

    If I go and I'm gonna go into the same folder here, and I'm gonna run NPX workflow web, which is the CLI call to start a local web UI to inspect our runs.

93. 17:42

    Um, and you can see that our run is currently still running. Um, and every step, everything that is marked as a step will have a, uh, span here, and you can inspect the inputs, the outputs, and any associated event.

94. 17:57

    And we can see that our workflow just completed, I think.

95. 18:01

    And yeah, this gets built in. Oh, yes.

96. 18:03

    Just for clarification, every time you're prompting your vibe coder, that is, is one instance of the workflow that runs to completion. So then--

97. 18:11

    So each one is... Yeah, it's-

98. 18:13

    Exactly, yeah. And you could model this, uh, in any way you want. Um, you can also model your entire, like, an entire user- Uh, sort of, uh, session as one workflow and, uh, have the workflow sort of do a loop, wait for the next query, um, and then again, you know, uh, we can

99. 18:33

    run code for weeks if we need to essentially, um.

100. 18:37

     And I'm gonna go into, uh, some tools for that in a second.

101. 18:42

     So now that we have this, uh, set up, uh, you can see that, um, on the right side, we do not get any sort of helpful, uh, feedback. But, um, if I visit this link, you can see that our app s- has likely been created correctly.

102. 19:02

     Um, or, or it failed because some errors.

103. 19:08

     And w-- either way, we're not getting any output on the right side. So, uh, the reason this is happening is that we are streaming, um, the agent output to the client, but our tools aren't actually doing any stream calls right now.

104. 19:24

     So what we could do is similarly, in our tool calls, uh, we could get the writable,

105. 19:33

     um, which would-- which will get the same writable instance as any other, uh, as the workflow itself. Um, there is an infinite amount of streams you can, you can create and consume in a workflow.

106. 19:46

     Um, and you can also-- like, you can tag them with a certain name and then fetch them from there. Um, but this will get the default instance.

107. 19:54

     And once we have a writable, we can actually connect to the writable by getting the writer. And now we can write any kind of information, um, to the,

108. 20:08

     to the AI to be consumed. I think we want something like data create sandbox, I think is what I hooked up in the UI. And then

109. 20:18

     we'll call IDs. We want the sandbox ID. I'll create it here. So this is me just writing a data packets that our UI knows how to consume. So now that I've did this, and I-- if I reload the app and start this again, we'll see that at least the sandbox create call

110. 20:41

     presumably gets filled in correctly at the start.

111. 20:46

     Uh, Peter.

112. 20:47

     Yeah.

113. 20:48

     You said that, uh, you can-- there are infinite streams that you can create and handle. So what, what do you mean by that? Can you explain that one more?

114. 20:57

     Like, yes. So, um, a st-stream-- the workflow, sort of the adapter they use for workflows in local development, right? This would just, just be a file. In production, this might be a Redis instance, um, supports the workflow, calling it to create a new, uh, stream, for example, in Redis, right?

115. 21:16

     And then passing that stream back. And so anytime you call that writable, it'll create a stream, for example, again, in Redis with the ID of that workflow, and it'll pass that back.

116. 21:27

     So any step can attach to that, and any client can attach to that. And in local, uh, host, this would be written to a file and read from a file.

117. 21:37

     And so, you know, sorry.

118. 21:39

     Yeah.

119. 21:40

     You're setting up right now, you're building a stream to a sandbox [inaudible]

120. 21:44

     Right. So pre-previously, we had a, a API handler that, uh, took some messages, called the agent, and then streamed back messages from that API handler. Now we have an API handler that calls it, starts a workflow, and it'll pass back the stream that this workflow creates.

121. 22:03

     Um, what this allows us to do also-- I think the demo's not working correctly. I'm gonna restart the server just to see if that's the case. Um-

122. 22:11

     Anything else so far getting this set up to where, where Peter is? Need any help with

123. 22:18

     getting things set up?

124. 22:20

     Seems good. Um, to a point you made, uh, something this allows, allows us to do is that the stream is not bound to the API handler. This means that at any point, we can resume the stream.

125. 22:33

     If you lose connection to your API handler and then the user reconnects, this stream still exists, and we could reconnect to the stream, uh, to resume the session.

126. 22:42

     This is also part of the durability aspect, where everything you do in a workflow, you can resume at any point.

127. 22:49

     I'm gonna restart this query and hope that it works this time. Yeah. So now that I hooked up this data packet, you can see this special UI handling for creating a sandbox, uh, works.

128. 23:00

     But even after it's done, it's not showing up that it's done. This is because we're only writing the initial loading state packets. So I could go through all of our tools, and I could add more packets and just, you know, make the UI richer.

129. 23:12

     Um, but I'm gonna go and check out a different branch, which is the, uh, comm slash three dash sleep branch, uh, which is the next step, uh, which already has these...

130. 23:24

     Uh, actually, I'll go for the workflow one. Sorry, comm slash two dash workflow. It already has all of these writer dot write calls populated. There's no difference otherwise. Um, so now that all of the tools have these write calls, the stream would again presumably look the same as, as it did when we started out in this app.

131. 23:42

     Um, all right, so now that we have streams working again, everything is working as expected, and we have more observability, and we can deploy this with durability. Um, I talked about resumable streams before.

132. 23:56

     We're gonna see if we can get this stream to resume so we have durable sessions.

133. 24:02

     So the only thing we'd need to do to, uh, make that work is to go to our API endpoint.

134. 24:10

     Um, and what-- when we get the run instance, we're also going to return

135. 24:18

     the workflow ID As a, as additional, additional information. So I can, I can return run.run_id, for example. This is just... Again, any way you do this is fine. Um, I'm adding it as a header here, uh, because we're already returning a stream.

136. 24:36

     But any way you pass the ID to the UI, um, is something that the UI can then use to resume the stream from.

137. 24:43

     So what we do from here is, uh, the UI should be able to decide whether to-- whether it has a run ID and whether it should resume a stream.

138. 24:51

     So we're going to go and create a new endpoint. Um, let's call it

139. 24:59

     ID for chat/existing ID, and then we're gonna make a folder stream, and we're gonna add a route handler. So this is just Next.js, uh, configuration for adding an API route at /chat/ID/stream.

140. 25:17

     And we're going to autocomplete with AI. [laughs] Um, what we're essentially doing is, uh, we get the ID from the params, and then all we're gonna do is call get run in the workflow API, which gets us a, uh, the, the run instance.

141. 25:37

     And then we can return the same stream

142. 25:42

     that we return in the other endpoint, just without calling the actual agent, um, only, only doing the stream. And I think that should be good. Um, we're also taking our start index, which is very helpful from the AI.

143. 25:58

     We can get a readable stream, um, from a certain start point. I think this is why it autocompleted. Um, so if you're trying to resume a stream, like, midway, you can pass a, you know, which chunk you were on when you officially left off.

144. 26:13

     So now that this is done, I'm gonna comment out these things we don't currently need.

145. 26:19

     Um, we need the UI to spot this, uh, additional of, of whether to resume or whether to start a new chat. So I'm gonna go to our chat front end, and I'm gonna go pull in some code from a different branch, um, for simplicity, which is the-- It's under four-streams branch, which I'm gonna

146. 26:42

     just show for completion. Um, we do a use chat call already in the UI to consume the stream, and we-- all we added now is a transport layer, which is this big block here, that has some middleware for the stream that says that if I'm trying to start this call, uh, I'm going to check first whether we

147. 27:04

     have an existing run ID. And if so, um, it's not going to do a reconnect by calling this different API endpoint instead.

148. 27:11

     I'm sort of handwriting all this a little bit because it's client-side handling for, for conditionals. Um, if there's more questions about this, please feel free to...

149. 27:22

     All right. So that gives us resumable streams. Um, and I'm also going to demo, uh, what if we wanted to deploy this and see it in production.

150. 27:34

     Uh, so I'm gonna call Vercel bot, and then we can pick out a production preview example.

151. 27:40

     In the meantime, uh, the next we're going to do is we talk about

152. 27:47

     suspense and resumability. The workflows, because they run-- the, the, the way they run is that every step runs on its own serverless instance in production. The actual workflow, workflow orchestration layer is only called very briefly to facilitate the step runs.

153. 28:04

     What this allows us to do is to have a workflow suspend for any amount of time. A workflow could wait for a week and not consume any resources. Uh, this is built into the Workflow Development Kit, um,

154. 28:19

     in a way where we can-- in, inside a workflow, anything tagged with use workflow, we can simply call sleep

155. 28:26

     three days, for example. Um, and that would... I should also await this. Uh, that will pause the workflow for three days and then resume where we left off. If someone was trying to reconnect to a stream, for example, right? [notification sound]

156. 28:42

     Just was sleep an hour, the stream would just reconnect again to the same endpoint, and things would resume from there. So we don't lose anything by, uh, losing the instance that runs the code, 'cause we can always restart it, resume from where we left off.

157. 28:58

     Um, and this is useful for AI agents because we can [coughing] into a tool call. Uh, we can have the UI as the AI agent, have it be call that says sleep any amount of time, and then use it to make an agent that, that essentially uses a cron job where it says, "Every day, read my emails and

158. 29:15

     do this thing." Right? So that would be sleep one day. Yes.

159. 29:19

     Sorry. When the, when the kind of agent goes down, that means all the state goes with it down?

160. 29:26

     Yes. So when it, when it sleeps-- No. When it sleeps for three days-

161. 29:28

     No. Then, then it kind of pause. But when that would be killed because for some reason, where does the state go of that agent?

162. 29:35

     That's-- So the, the way it works is that any step call, um, is cached. So when you-- when a input goes to a step call, we register that as an event and we run the step.

163. 29:48

     And if the step completes, we cache the outputs and say, "This step has been run com-- um, to completion." Right? So if it was, if it was something like this, where we run the agent first, right?

164. 29:57

     Let's say we run the agent, and we run a bunch of steps. Um, the state of the workflow function at this point in time would be saved, and all of the outputs from all of the step calls would be saved.

165. 30:08

     And at the time where we restart the workflow from this specific line of code, it'll rehydrate the entire state, and it'll just go from here.

166. 30:18

     Um, and this happens [coughing] instantly so that, uh, again, we don't have to replay any of the code of it, uh, in a way that, that does any actual resource consumption.

167. 30:27

     Um, yeah. So we can use this to make an agent that has... It's essentially a cron job ag-again. Um, and we can use it to make agents that run for weeks or interact with any of your, um, sort of like information over, over a very long time horizon.

168. 30:44

     And while I've been talking, we have deployed, uh, our current app to Vercel. So I can check out this preview branch, for example, and you can see the app is now live online, um, and working just as it usually does.

169. 31:00

     And yes, it works perfectly. And if I then, again, I can do, uh, I can use the UI to inspect this at any point. If I call workflow inspect web or just workflow web with the dash decant Vercel and dash and preview parameters, for example, they'll just let us-- let, let it know where our deployment is to

170. 31:24

     be found, and then they'll spawn up the same UI, and now we can check on this run, run that's running in production. And you can see we're getting the same kind of information here.

171. 31:35

     Um, yeah. So th-this is sort of... Oh, I'm not gonna cancel the run. I could cancel the run. Let's cancel it. Um,

172. 31:46

     this is to show that the way it works in local, uh, locally is the exact same way that it works in production from a conceptual standpoint, um, which is the UX we are aiming for.

173. 31:58

     All right. Um, I talked a little bit about sleep and suspenders. Let us go and write this sleep tool call. It's going to be very simple. Uh, I'm gonna go and copy the...

174. 32:14

     I mean, I already actually have it typed it out here, but I'm gonna copy this and write it from scratch. We're gonna write a sleep, uh, tool call.

175. 32:27

     I'm just gonna call it sleep dot ts. And, uh, we're gonna trim down the input schema to be something like timeout milliseconds and the actual run command to be

176. 32:47

     none of this, and instead just call sleep.

177. 32:54

     Um, because sleep is already a step, um, that we export from workflow, from the workflow library, we don't need to call, uh, we don't need to mark this function as use step.

178. 33:06

     But this will now, uh... Let's see if this is... Oh, this should be a number. There we go.

179. 33:15

     Can you say that again? Why don't you need the use step?

180. 33:18

     Oh, uh, so this is already a step, um, that we export from workflow. It's going to be the... Observability will also show it as a step, which we'll see in a second.

181. 33:28

     And, um, this should just work, assuming the prompt is good, which we're gonna modify to be, say, something like... Let's see.

182. 33:40

     Most of this. Um, yeah. Do this. Um, only use this tool if the user directs you to do so.

183. 33:55

     All right. And get a double quote here. There we go. And so now that this sleep call is set up, um, that should be all that we need to, uh, do.

184. 34:07

     We'll call it run sleep command and sleep tool, and we're going to add this to our tools list.

185. 34:23

     And I think I confused our compiler a little bit, or at least TypeScript.

186. 34:33

     This seems to work. Great. Okay, now we have the tool. Um, and we also want the UI to sort of display when it's sleeping. Um, so I'm gonna add, I'm gonna add a...

187. 34:43

     Not a function to log sleep. Uh, this is-- The reason we're doing this is we cannot write to a stream directly from a workflow, because then it wouldn't be, uh, deterministic anymore, 'cause every run of the workflow would write to the stream again.

188. 35:03

     Yeah.

189. 35:03

     So in trying to run the projects, um, I had to create a Vercel API key for the AI gateway. Did that, and then it currently did that. Getting an error that says,

190. 35:12

     "Missing. We're getting... Your header is missing from the request. Do you have the OIDC option enabled in the project settings?"

191. 35:20

     You, you skipped this in the beginning, but-

192. 35:21

     Oh, yes

193. 35:22

     ... the Vercel project that links-

194. 35:24

     Yes. Our, uh... Because this co- this code uses, uh, Vercel Sandbox, uh, you would need to log into Vercel to use this. Um, my mistake. Uh, this should be running locally, um, if you don't use the sandbox, which, uh, I will, I will have a branch that doesn't use the sandbox, um, for after the talk.

195. 35:43

     Yeah. For now, you might just want to log [laughs]

196. 35:48

     Um, I mean, at this point, I'll just do it afterwards. It's fine. So here I'm just gonna add another call to writable, and we're gonna call, uh, we're gonna...

197. 35:59

     Let's see. We're gonna need tool call ID. And so now this is just going to write to the stream, and that should allow us to show it in the UI correctly.

198. 36:14

     Um, let me see if I configured the UI to correctly interpret this packet.

199. 36:24

     Um, there is no data sleep type. Which I think I'm gonna call wait. Yes.

200. 36:34

     All right. So now that I have this, um, I can go start our app again.

201. 36:41

     And... Once it loads, we can try out the second prompt here, which is sleep for thirty seconds and then return a random.

202. 36:54

     Uh, just to show that it's gonna correctly interpret the sleep call, and then

203. 37:01

     sleep. Um, it's not showing the data packet here,

204. 37:04

     sadly. But we can go to the web UI,

205. 37:10

     and we can show it has been-- it's engaging in the sleep call, and this is going to return after thirty seconds.

206. 37:20

     All right. So that's sleep. Um, there's one final thing, um, one final feature that I wanna show you, um, which is webhooks and the ability to resume from webhooks easily.

207. 37:31

     Uh, implementing webhooks is usually quite difficult or a headache. And in our case, uh, I'm gonna check out the conf slash five dash webhooks branch and show you that we can-- in the same fashion as we do sleep, we can add a new tool that I'll just, I'll just show you.

208. 37:50

     But the actual tool call is, um, just a lo-log call, and then we create a webhook, which is a function we export from the workflow, uh,

209. 38:02

     and we can then log the webhook URL

210. 38:08

     to the client or anywhere else and await webhook. And this will suspend for as long as necessary to, uh, someone to click on this URL. And then

211. 38:20

     let's see if we can-- the server's running, and I can show you this, uh, running hopefully.

212. 38:27

     Reload this and, um, mm, wait for, wait for human approval

213. 38:40

     before starting and call on a Pokemon index.

214. 38:49

     Let's see if it picks up on this correctly.

215. 38:54

     Been changing branches, so I might need to restart my server.

216. 39:05

     Um, and the way this works under the hood is that, again, we'd be creating a, a URL, and we're going to sleep the, um, the workflow until a call comes into that endpoint.

217. 39:19

     And this comes with-- I'm gonna run this query. This comes with a lot of extra features. Like, I could also do respond with if I wanted to. Uh, this is a full API, API request handler.

218. 39:31

     I could respond with a request object. Uh, I can treat this as a, uh, again, API endpoint. I could also check the body against result schema, for example, and then only, uh, resume ones that matches.

219. 39:48

     So this gives you full control. Um, but the nice thing is it does hook up the URL internally, and you can see that it's paused waiting for a human to click on this link.

220. 39:58

     And if you're running in localhost, it's a localhost link. If you're running production, it will be whatever your deployment, uh, URL is. Um, yes.

221. 40:05

     Excuse about [clears throat] both sleep and human approval. Those are-- like, a workflow is, is purely steps, and steps always run to completion, right? So, so sleep is a step. It's not like the, the suspension of, you know, like, some sort of-- like, it's not a suspension of the, of the execution.

222. 40:20

     It's like it's a-- you're-- it's a step.

223. 40:22

     No. It, it is-- So we model it as a step in terms for the observability and, and for the-- for how you call it. But it is an internal feature that completely suspends the workflow and all steps.

224. 40:32

     Nothing is running while we sleep. Um, you can also do sleep and another step, and you can promise not race them if you want. Um, it works as a step call in that sense that it's a execution that takes a certain amount of time, um, and you can use promise await syn-syntax, uh, to model that.

225. 40:51

     But again, it completely suspends unless there is anything else running at the time. And the same for the webhook. It's modeled as a s-step, um, for the observability, um, but it completely suspends unless you have other code running at the time.

226. 41:05

     So just from my understanding, if you have an agent running, uh, with a workflow, it keeps running.

227. 41:11

     Yeah.

228. 41:11

     You connect to it again, let's say through another session, and you call sleep in this session, does that, like the previous one, just, like, whatever it was doing just goes down?

229. 41:23

     So if you have two sessions, um-- So let's say we, we've had-- we have a coding session, right, and it already built an app, and then it's sleeping for a week, right?

230. 41:32

     Um, and then we reconnect to the stream. Is that the-

231. 41:34

     No. The, the thing is, uh, let's say I kick off a work, uh, workflow, and it's calculating, like, the numbers of pi. It just keeps on, right? But I connect to the same sandbox, and then I call sleep.

232. 41:46

     Will it stop calculating pi?

233. 41:49

     Um, so the, the way you would do this in a workflow is-- Again, let's, let's see how we would code this.

234. 41:55

     Do you have more sandbox there? Andrew said sleep in the sandbox.

235. 41:59

     Oh. Well, you can connect to the s-sandbox.

236. 42:02

     Oh, so the, the-

237. 42:03

     You connect again to the sandbox.

238. 42:05

     Right.

239. 42:05

     And if some thread calls sleep, does the whole sandbox go?

240. 42:08

     So the, the sandbox is, uh, it's Vercel Sandbox, which is a sort of just-- Imagine it as an EC2 instance. Um, so this is just a, a helper for, for us to spin up an instance to run this coding agent, like, run the code in order to store the files.

241. 42:23

     Um, if you met this differently, uh, you wouldn't have to use Vercel Sandbox. Um, and the sleep call doesn't happen as a, as a bash call, for example, in-

242. 42:32

     In two different stages.

243. 42:33

     Right.

244. 42:34

     Well, there's like an orchestration thing, and then when you're actually in the sandbox-- do you, do you call sleep in a sandbox here?

245. 42:39

     Okay. So there, there are two different things.

246. 42:41

     Right. So, so-

247. 42:42

     Yeah

248. 42:43

     ... there is sleep that you could call from a terminal, um, in the sandbox as a, as a, as a terminal command or the sleep from the workflow which suspends the workflow.

249. 42:54

     Uh, yeah. So we have, we have these features for webhooks, right? And we can see that after I clicked on the URL, it resumed and then coded me a Pokedex.

250. 43:05

     Um, that is all of the features we are going through in the session. And I think we have ample time for Q&A, another twenty minutes at least.

251. 43:17

     Please go for it.

252. 43:18

     How would I spin up Claude code sessions with this?

253. 43:23

     A Claude code session remotely, or are you asking-

254. 43:27

     No. Kind of run it and kick it off as an agent doing certain stuff. Um, is, is that possible? And then kind of orchestrate that as agents or...

255. 43:36

     That is possible. So Claude code, uh, is, um,

256. 43:41

     if you're talking about the, the app, like the terminal app, right, Claude code, then that doesn't use a lot of the workflow features internally. Um, so it's hard to isolate that, um, or know where the orchestration layer is.

257. 43:51

     You could write your own version of, uh, Claude code or take the Claude code source code and add workflow and use step, um, for the calls.

258. 44:02

     Okay.

259. 44:02

     And that would then run as a, as a workflow in a cloud.

260. 44:04

     But there's no way to say, like, "Okay, I have my steps, you know, spin up Claude work, uh, kind of code, uh, type this command and wait for the thing."

261. 44:14

     That would be a Vercel workflow, but how would I actually bootstrap it, like, with Claude code in and...

262. 44:21

     There's a run command tool, right, so you could... Tell me you're asking.

263. 44:23

     If you're, if you're-- so if you're calling Claude code, uh, in a, um-- So, so maybe there's a conclusion of, like, where this is running, right? For a coding agent here, if the coding agent runs make dir, right, for creating a, creating a folder, that make dir command runs in a step, but it runs against a, like,

264. 44:40

     in a sandbox there, sandbox being a VM. And so this VM state is not managed by the workflow itself. Um, so if you call Claude, Claude code on the VM, that's essentially treated like an SSH session.

265. 44:53

     But if you run any, any agents or steps within the workflow, right, those steps are going to be resumable, um, and, and observable through the workflow pattern. Yeah.

266. 45:04

     Um, and another question. How do I control, um, what my agent has access to from going out to the internet doing stuff?

267. 45:14

     This would be, uh, whatever you're, whatever you're already doing for the agent, right? If you're, if you-- In the end, you're going, you're going to be doing tool calls and stream calls to the LLM provider, right?

268. 45:27

     Um, that is, that is in your code presumably already. And whatever you're already using to control permissions there, like your tool calls, for example, right? If your tool call, uh, allows you to

269. 45:40

     delete a resource in S３, for example, then you as writing the code for that tool call, uh, can write whatever code you want, uh, in the usual way for that.

270. 45:52

     But it's my job to implement it, but it, it's not that it has some wrappers by saying-

271. 45:56

     Yeah. It is

272. 45:57

     ... all in the sandbox.

273. 45:59

     Workflows is a general orchestration layer for durable execution, um, and doesn't necessarily provide a sandbox for running code or, um, like we're running third-party code or running agent code or making files.

274. 46:10

     Uh, that's something that Vercel Sandbox is good for because every sandbox instantiation is a new VM, um, that only lasts for, for as long as your session lasts.

275. 46:21

     Yes.

276. 46:22

     Yeah. So, uh, if I'm running workflows, uh, and I'm, like, creating a lot of agent, uh, workflows through my product, uh, how does that-- do-- does that get queued up on your system?

277. 46:36

     How does that get run? Is there a rate limiting or concurrency controls that we can, uh, use to, to develop?

278. 46:44

     Yes. So this is, this is, uh, this goes into sort of, uh, some of the advanced patterns. The re-- All of this is going to be supported, um, and for the most part is supported right now, which is that if you're deploying, for example, to Vercel, right, um, as usual, if you do Next.js, every deploy is a

279. 47:02

     separate, uh, like, live URL, right? That if you call it, it spawns up a serverless instance. And so your workflows are bound to that deployment. So if you have something that-- Something very nice that you get here is if you deploy an agent and it runs for a week, but you deploy five times in-- during this week,

280. 47:21

     those new deploys are going to be isolated from the original workflow, and the original workflow is going to run to completion, and then any new workflow will run on the new deployments.

281. 47:30

     And we'll also allow upgrading between those. So if you have a, a workflow that runs for a year, right? Because it's like every month, give me a summary of so and so, right?

282. 47:38

     Uh, but you have new codes and you want the workflow to, uh, you know, take its current state and use the new code for the workflow.

283. 47:47

     Um, there's going to be an upgrade button in the UI that checks for compatibility between the old workflow and the new workflow by checking all of the step signatures and all of the existing events, and then you can upgrade the workflow.

284. 47:58

     Um, or you can currently already cancel and rerun with the, with the new workflow.

285. 48:03

     Um, is there a timeout for those workflow steps?

286. 48:07

     A time-

287. 48:08

     Timeout.

288. 48:08

     Oh, yes. So, uh, if you're doing serverless, right, and whatever platform you're on, whether it be like Lambda or something else or, or Vercel, uh, your serverless functions are gonna have timeouts.

289. 48:18

     The nice thing is that every step runs in its own serverless function, so the timeouts only apply to the steps. So if one of the individual steps you have runs the risk of running more than five minutes, maybe fifteen minutes, depending on platform, um, then you can split it into two steps.

290. 48:35

     Um, or if it's-- if it runs the timeout, right, it'll fail, it'll retry, maybe this time it'll be faster. Um, and you'll see in the UI that, oh, this step is being retried after fifteen minutes a bunch of times, right?

291. 48:45

     Presumably because it's failing, and then you can go and, uh, split it into two steps, upgrade the workflow, and it'll just continue from there.

292. 48:57

     And, uh, yeah, the other thing was, uh, around queuing workflows. Like, I triggered my agent a bunch of times.

293. 49:03

     Yeah.

294. 49:04

     Is there a queue? Like, how does that process work?

295. 49:08

     Right. You can model this in, in different ways. Uh, right now, again, we're doing this from, like, a API route where every call to this API route will create a new workflow.

296. 49:18

     Um, that is mostly-- Again, the only, the only, the only interactable output you have is a stream in this case. Um, so it'll do things, it'll write to the stream.

297. 49:27

     If nobody looks at the stream, we, we don't know the work, uh, workflow is running. Uh, you can kick off kind of these, right? Um, and they're gonna be running in the background.

298. 49:36

     Um, there is essentially no limit to how many you can create because they all run in serverless functions. So you can scale for as much compute as there is in your provider, um, which is a lot of compute.

299. 49:46

     And, um, you can also list active runs, right? There is an API here for doing crowd interfacing with your runs, look at all of the runs that are running, look at which version they're on, uh, what step they're on, um, cancel them.

300. 50:01

     I don't know if that answers your question. Oh, concurrency. Yes. Um, you can also-- So right now it's infinite concurrency, but, uh, very soon we'll add per step or per workflow concurrency, where you can say, "This workflow is only supposed to run at most ten times at the same time," and any extra queue, uh, addition gets, like--

301. 50:22

     any extra start gets queued so that it'll wait for those ten to, uh, reduce and then slot that in. You can also use that to have a free tier, for example, um, on your product.

302. 50:33

     Whereas ten instances running for your free tier at any one point, and some people that come in later will wait for the queue, but your pro tier has infinite concurrency.

303. 50:44

     Yeah. Yes.

304. 50:46

     Can I roll back steps in a way that, let's say, I have ten steps, but in step seven I think, like, okay, let's go back to step three. Would that be possible to kind of like reset the state of the workflow or?

305. 50:58

     You can technically do this, do this. We can't-- We don't, we don't currently support it, but it would be very easy to implement because, uh, we have every step, again, the inputs and outputs are cached, and we can enter the workflow at any point and sort of play from there.

306. 51:13

     So we'd need to expose this in the, in the UI to-- as a function to resume from step so and so, but yes, that would be possible.

307. 51:20

     More, more likely what you want to do is to, to keep control of the workflow [inaudible]

308. 51:25

     there and use the observer. Right. This was developed by the time. Cool. I might have to use

309. 51:34

     step and change state or do something.

310. 51:36

     Mm-hmm.

311. 51:36

     Let the event log just keep growing it. You can, you can model it like this.

312. 51:42

     Maybe my second question. So if you go through the steps, you said, okay, we're, we're passing our input and outputs kind of across, and that's kind of what get cached.

313. 51:51

     Is there, like, a way to attach, like, metadata, or does it always have to be in kind of the input outputs of the function?

314. 51:58

     You can al-also attach metadata. Uh, we'll have a tagging API soon where you can add arbitrary tags to the workflow at any point in your workflow run, and you can use those tags also to maybe decide whether to quit early or deduplicate your runs.

315. 52:13

     Um, yeah. Yes.

316. 52:17

     About the deployment, are we tied to Vercel or is it possible to, like, use-

317. 52:23

     As I mentioned before, uh, so there's, there's two aspects to this. There's the, the front end side of the framework. Uh, the docs are on useworkflow.dev. Uh, you can see for the front end sides, um,

318. 52:35

     which is also sort of the, the, uh, sort of API layer it might work with. Uh, we currently support all of these platforms, um, and more coming soon. And then there's separately the deployment target, right?

319. 52:47

     Like, for Next.js, you can deploy to anything right now, right? This would work with anything you can deploy Next.js to, for example, or any of these other frameworks. Um, and we have implementations, first party implementation for a Postgres example that uses Postgres as the durability layer.

320. 53:04

     Um, and as we'll be building this out and community comes in, we'll have support for essentially any back end. Um, because underneath, the TypeScript framework connects to any storage or queue layer.

321. 53:16

     So anything that provides a storage, a database, uh, or a queue can be used as the back end for, for this.

322. 53:26

     Yeah. A related question for the observability. You also have, like, providers to Datadog and all of this stuff.

323. 53:33

     Yeah. So, uh, we, we have a, uh, multiple things. We have an, an API that you can use to, to, uh, access data directly. Um, and we also have open source UI components that you can use to display it.

324. 53:45

     And then you can export this to Datadog, um, if you want. Yeah. We can add them in that process probably.

325. 53:58

     Yeah.

326. 53:59

     You talked about sleep a little bit and talked about how it is essentially like a cron job. Um, do you-- Is there more scheduling and cron controls within, uh, Workflow?

327. 54:10

     So, uh, because it's just TypeScript, uh, if you're in a workflow, um, you can do something like, let's say, we call sleep, for example, right? This, this would be just be resume in a-- sorry, resume in one day.

328. 54:25

     But what you can also do is, uh... This is just a promise, or you can treat it as just a promise. So you can do

329. 54:33

     while true, sleep one day, and then, uh, you know, do your code and it'll run once a day. If you want it to run- Once a day at two AM, um, you could-- you could say, you know, how much time, time, time to, you know, one AM tomorrow.

330. 54:53

     Thank you, AI. And then, you know, done. Um, and you could also wake up every hour, do some checks whether you actually want to run the rest of the code.

331. 55:05

     If not, go back to sleep. Um, anything you can do with code you can do here. And if you want, again, concurrency controlled something, um, or any kind of auto de-deterministic controls, you have full control flow in TypeScript here.

332. 55:17

     You can check, check external, um, uh, APIs, for example, um, which you have to wrap in a step, but you can make fetch calls if you want to actually check data and then determine from there.

333. 55:29

     If you, if you wanted to do a, uh, agent that runs every once in a while-- every day, you could have a scheduling wrapper or scheduling workflow that launches another agent workflow.

334. 55:40

     Is that-

335. 55:40

     Also, yeah, you can, you can, you can start workflows from workflows or you could do this, right, where you, uh, sleep a day and then call your agent. And then depending on the, the, on the stream you want to write to, right?

336. 55:53

     This is all writing the same stream. Presumably you don't want that presumably. Maybe. You can also, uh, get writable allows you to do namespace, right? Uh, one. Um, and you can get a new writable here, and then every time it runs, you can have a new stream that has a deterministic, uh, name, and you can choose which

337. 56:12

     stream to connect to. Yeah.

338. 56:17

     Is there cancellation logic? Like say I have something waiting for a long time, and then I decide to not have that be a thing. How can I just like stop an existing sleep from waiting?

339. 56:27

     Right. So you can cancel your workflows from the observability UI or from the, from the API or the CLI. Um, all of those avenues, uh, have... You can call cancel.

340. 56:37

     Or you can also say, "Well, I don't even know if I wanna sleep one day and resume." What you can do is do a, uh... Let me move this part here.

341. 56:46

     You can do something like, you know, await promise.race, and you can do the sleep one day, and you can do some other... Actually human approval. Maybe wake up earlier, if a human clicks a button, then the one day.

342. 57:04

     Um, there you go. Yes.

343. 57:09

     Um, if you have multiple agents running, uh, what would be your kind of advised way of having them communicate with each other?

344. 57:17

     Um, if, if you're... Sort of depends on what kind of communication you are looking for.

345. 57:25

     Firing things off and they're working, but I would share their follow.

346. 57:29

     So in steps, um, you have access to

347. 57:34

     all codes, uh, APIs or the Node.js APIs, fetch, et cetera. You can have a database, right? If you wanna coordinate over your own data source, you can have a database.

348. 57:44

     Um, if you wanna have multiple agents, um, you can, uh, use modeling. You can use some of the same streams, write to, um, and share a stream. Um,

349. 57:57

     yeah.

350. 58:03

     I guess it's up to us ultimately with our steps that they're like idempotent and if they have side effects when they fail halfway that it's well-behaved. Like that's, that's at, at, at your orchestration layer.

351. 58:13

     Like that's up to us.

352. 58:15

     Right. For the workflow layer, we guarantee there's no side effects. Because if you try to import some code that does side effects, I think it'll just, it'll just say like, "Can't compile."

353. 58:25

     Doesn't, you know, don't do it. Um-

354. 58:27

     That was true for workflows, but for steps.

355. 58:29

     For steps it can have side effects. That's sort of the point.

356. 58:32

     So it's up to us, like if it fails, we need to make sure that-

357. 58:35

     Exactly.

358. 58:35

     -it's transactional and-

359. 58:36

     Exactly.

360. 58:36

     -it's re-runnable and idempotent.

361. 58:38

     There, there's some, there's some, uh, there's some error controls you can add here where if a step fails, it'll usually fail, uh, with a, with an error that tells the workflow station layout that it can retry it.

362. 58:49

     You can also catch this error and say, "Well, if it's a, you know, this kind of error, don't retry it. Instead, signal to the human to do something or try this other avenue."

363. 58:59

     Yeah. See if there's anything else.

364. 59:17

     Do you have one of the branches that has like the complete code for what you just did?

365. 59:22

     Yes. So, uh, they all build on top of each other. So the conf slash five dash webhooks branch has the human approval tool called the sleep tool called resumable streams, um, and using workflow.

366. 59:40

     Is there anything to do slides?

367. 59:42

     Uh, I will see how I can post one. General access.

368. 59:48

     Can you just tweet it, tweet it out?

369. 59:50

     Yes, will do it.

370. 59:52

     Uh, the workflows are in beta, and so it's in boxes.

371. 59:57

     Yes. Yeah. Um.

372. 59:59

     Each time I look.

373. 1:00:01

     Okay. Yeah. Uh, yes, I forgot to mention this important, uh, workflow development is in beta, and it's going, uh, general access in January, I believe. And we have a, a GitHub.

374. 1:00:16

     The, uh, V zero one. You know, V zero.

375. 1:00:24

     So we're using in production, especially the durable agent stuff that...

376. 1:00:29

     Internally, we have, I think more than one million workflows have been run on, on-

377. 1:00:33

     A day.

378. 1:00:33

     A day. [laughs] Yeah.

379. 1:00:36

     So yeah, it's mostly just like getting stuff working. I guess the API to be stable and a bunch of issues. But one thing I love is that we actually have more issues.

380. 1:00:46

     But, uh, but generally [audio glitching]

381. 1:00:49

     Yeah. [audio glitching] Is there any feature that you, that you need or that you really wanna see? Uh, we have an RFC section on GitHub, uh, discussions for upcoming features, things that we'll ship by GA or, uh, shortly afterwards, and then open issues, right?

382. 1:01:06

     Um, where you can, you can add any issue, and presumably we'll be able to solve it soon, or someone in the community. Again, this is, uh, all of the adapters that help Workflow DevKit run on any kind of cloud backend, uh, or your, uh, homebrew backend.

383. 1:01:21

     All of those adapters are also open source, so you can see exactly what's happening, and you can connect it to your own backend and own data source, um, just looking at that code, and we'll be happy to help you.

384. 1:01:33

     Can you talk about, uh, [audio glitching] compatibility approach?

385. 1:01:39

     Uh, yeah. Um, what would you like to know?

386. 1:01:43

     Like a roadmap or like a theory you can ask.

387. 1:01:46

     Right. So for versioning, um, I talked a little bit about, about, uh, the ability to upgrade runs across versions, right? Uh, versioning is gonna be very simple where we have a cloud interface for all of the versions that you have created, which for most people will be a deployment, right?

388. 1:02:02

     If you deploy your code, um, you know, your C- CI deploys your code to, to a pre-deployment or production. Uh, every deployment will be one version. Um, and you can list those versions at any point, uh, using the, the workflow API.

389. 1:02:16

     And the run will know which version it's running on, and you can call run.upgrade to see if it's compatible with a new version, upgrade it to that version. And I don't know, any, any more things?

390. 1:02:29

     Workflow VC version.

391. 1:02:32

     No. Yeah. So, so every, every deployment, uh, gets its own URL, and not just in Vercel, but presumably in your, uh, if you do other AWS Lambda, for example, right?

392. 1:02:43

     Uh, every deployment has its, has its own URLs, so the webhooks would apply to its own URLs. Um, which means that you don't need to worry about versioning except for tagging a version when you first hit deploy.

393. 1:02:55

     Um, and then whatever you think is, you wanna be your main version is the one you route to via your public API.

394. 1:03:04

     Yeah. I think, um, sorry, I, like, kind of... Uh, obviously, I, I... We used to work at Temporal, so I have a lot of experience in this stuff. I think a lot of people versioning, this is like isolation of deployments, but sometimes you wanna sort of fix in place things that have been going for a while.

395. 1:03:22

     Yeah, migrations, almost like agent migrations to a new version.

396. 1:03:25

     Yeah.

397. 1:03:25

     Right.

398. 1:03:25

     I hear you do a migration.

399. 1:03:26

     Yeah. So this is the same as, as upgrading in, in that sense, right? But if you have a bunch of runs that are all on a certain version and you have shipped new code, and you want all those runs to be upgraded to the new version or migrated, right?

400. 1:03:38

     Um, in the UI, you'll be able to select how many runs you want, or through the CLI, you wanna be able to get a list and then say, "For these twenty IDs, I wanna upgrade the run to this version."

401. 1:03:49

     Uh, we'll-- it'll do an internal check of, of, can I resume these, uh, workflows from a certain point? Um, like, can I migrate them in place sort of because the step signatures overlap?

402. 1:04:02

     Or if not, it'll offer you the option to cancel all of the existing runs and rerun them on a new version with the same input. Um, if you write your code in a way that's compatible, again, um, there's, there's gonna be different options for, for, uh, in-place migrations.

403. 1:04:17

     How would it detect that? Just by code parts not being changed or?

404. 1:04:20

     So be- because we have-- b- because we are essentially a compiler plugin, uh, we can get full AST, uh, compatibility, and, and we are saving this AST, uh, the, uh, the inputs and outputs signatures, um, to a manifest that we're uploading for the versions.

405. 1:04:36

     And so for every version, we can tell what are the signatures for every, uh, step and for the workflow itself and all the other things that are happening in between.

406. 1:04:45

     Also, another thing here is the workflow function itself. So I guess we played a whole bunch of times during this panel discussion, like, what do we do anyway during your workflow execution?

407. 1:04:55

     Yeah.

408. 1:04:55

     So when you wanna upgrade and control times [audio glitching] replay or event log that, uh, it's gonna, like, ride on.

409. 1:05:03

     Yeah.

410. 1:05:03

     You can take the entire event log, take a new version of the code that you've run on the event log against. You can effectively figure out if this is kind of the version you're upgrading to.

411. 1:05:12

     Sure. Yeah. But there's a lot of variations between kind of I do a step, all the previous steps stay the same, or, you know, this one got changed. And so it's like you-- like, if everything's done automatically, it feels like, okay, I could go down immediately with all my agents or up.

412. 1:05:29

     Yeah. There's two ways you can-- there's two ways you can be versioning as you're [audio glitching] right? And I think the thing with, and that all of this stuff is you build for a platform where you assume that the code, like, your code is always going to evolve in the same place.

413. 1:05:42

     So what we've seen is you end up with, and you start your first version of your workflow that you ship. But as you start making updates to, you start adding new versions, your code now has all the cool stuff in there that have to be used.

414. 1:05:54

     Because you have no guarantees of what version of the workflow is running on the actual code that you're running. The default assumption is that my code could be running an event log from anywhere, and you end up with...

415. 1:06:06

     Starts great, and then you have to do all the mental model of that. [laughing]

416. 1:06:13

     But it was the same with serverless, like killing existing instances.

417. 1:06:16

     What's nice with Vercel is 'cause Vercel is already packed, you know, atomic environments, whatever. There's a natural step to go say, "Cool, we're just gonna assume that you, you know, you push a new version of the workflow, you pin everything [audio glitching]

418. 1:06:27

     default." And so you don't have to worry about that mental model that the code. But instead, when it's time to go upgrade or actually push a button, you might do something.

419. 1:06:38

     I can go with the entire event log replay, or you can choose, like, you know, they could theoretically choose exactly how much of that needs to get replayed and going forward.

420. 1:06:47

     There's a lot of stuff that you can do on top of this once you have the system. But what's nice is that that's, that's a hard UX and, you know-

421. 1:06:54

     ... channel for us to build. When done well, yeah.

422. 1:07:00

     Hopefully very rarely. Um, by the way, you're gonna talk a lot more.

423. 1:07:05

     Yeah.

424. 1:07:05

     You can afford it. [laughs] I, I don't know if we're, we're close to done.

425. 1:07:10

     I think we're close to done.

426. 1:07:10

     Close to done.

427. 1:07:12

     Uh, we'll be sticking around for more questions. So, uh-

428. 1:07:16

     I guess, okay, so I... Let's say I'm a third party. Like, uh, I think the other part is observability, right? Um, I don't... I, I like poked around in Vercel.

429. 1:07:26

     I don't see like much of a dashboard. I expect that obviously you're gonna build one, right? Uh, and then I also wanna import it to my Datadog, see Vercel on a Datadog, but kind of Datadog.

430. 1:07:39

     Like, what, what's the observability service?

431. 1:07:41

     Uh, OpenTelemetry spans, uh, which will, you know, be able to emit, um... Will add some context to the spans by default, presumably. Uh, so if you, if you just pipe your spans through Datadog, it'll already have a lot of information on the steps and the event log.

432. 1:07:56

     Um, and you can also emit your own, uh, telemetry obviously.

433. 1:08:00

     So is that the plan or is it, uh, you have the first party?

434. 1:08:03

     The, the plan is that we'll, we will, uh, first party support adding some of the, like, all of the sort of step and event log related context. Um, we'll, uh, presumably export a helper to add some of these, uh, this information to the spans.

435. 1:08:17

     Um, and then every-- all information you wanna tag in there is up to you.

436. 1:08:23

     Can I, uh, attach like secrets to Workflow

437. 1:08:28

     in a way that when I need to update them they kinda like, you know-

438. 1:08:31

     Yeah

439. 1:08:32

     ... pop up?

440. 1:08:32

     So, uh, for one, uh, like right now it can, it can inspect all of the input and output data, right? And it's obviously, uh, for you as a someone with access to the API, uh, which someone consuming the workflow w- or starting the workflow through an API wouldn't usually have.

441. 1:08:47

     No, no, no. But-

442. 1:08:48

     I think it's using the steps.

443. 1:08:51

     Hmm?

444. 1:08:51

     I think you use step.

445. 1:08:51

     Yeah. So the, the workflows run in the same deployment as, as it would usually do, and has access to the process, uh, environment, right? So you can inject environment variables the way you would usually do.

446. 1:09:01

     Um, and as long as you don't log them, which again, presumably you wouldn't do an- anyway, um, it's the same way as an API endpoint. And then if you want your data to be, uh, secret, right?

447. 1:09:11

     Um, right, right now we expose it in observability, um, if you have access, but we also will allow in the future to do end-to-end encryption for any data stored.

448. 1:09:21

     All right, then we'll, uh, close the session, but we'll be around a little bit more for questions if you wanna, you know, look, look over code. [upbeat music]