Make your own event-sourced agent harness using stream processors

Talk transcript

1. 0:00

   [upbeat music] Welcome to, to the workshop.

2. 0:16

   Uh, I am Jonas, um, and this is Misha. We both work at Iterate.

3. 0:20

   Hi.

4. 0:20

   And the, um, this is gonna be, uh, chaos. I think pure chaos. [laughs] First of all, because we only, we only decided, uh, last Monday to do this, and then there was, uh, uh, like, um, bunch of personal stuff, and half-term, and the kids were sick.

5. 0:35

   And then this morning we came here, and I thought this was gonna be like a, like a hour and 20 hackathon or something like that. And then we came here, and we're told, "You're gonna do it without an audience because of some fire thing."

6. 0:46

   Uh, and now there's an audience again, which is amazing. So we're gonna do, uh, gonna try and do this as like a improvised hackathon, but I've only just pushed the SDK literally one minute ago.

7. 0:56

   Uh, and Misha is gonna try and, and sort of like live fix it. But I, I hope it'll still be worth your time, because I think there's sort of some, uh, hopefully at least some interesting ideas.

8. 1:04

   Or, or maybe, like one way to put it is, I would like to find out whether this is like dumb or cool. Uh, and, and the, um... And, and you can help me with that.

9. 1:12

   So please also just like ask clarifying questions, because part of this, the reason we said last week, "Okay, let's do this workshop," is because these have been ideas that have been kind of noodling around over the last year, but we haven't really actually done anything with.

10. 1:24

    It's not really like a commercial v-value to it necessarily. But I think, um, it's, it's basically how I would like to build agent harnesses. So, you know, I've been noodling with this, uh, uh, for a while now.

11. 1:36

    And, and, you know, obviously you've got like Claude and you've got Pi now, but back in the day you didn't, and so you make your own coding harness, and there's all these different ways to do it.

12. 1:44

    And I think, um, the way I would like to do it is I would like to do it purely event-sourced. Um, and, uh, I say here, AKA debuggable, and that is because everything kind of skirts around of being event-sourced, like all these things, without quite being event-sourced.

13. 1:59

    There's always something that has a side effect that you later can't tell, and then you can only see it in the OTel traces or something. Which doesn't really make sense to me, because normally these, these, these agents, the way they work, and we'll see that in a second, is they already have a log of events, right?

14. 2:12

    So you, so if you just say everything that could possibly happen is in there, it'll be really easy to debug. Um, and as we'll see, hopefully, a little bit strange, but potentially in other ways also really easy to, to extend.

15. 2:24

    Um, and then the second thing is I want, I want this to be extensible. And, and we've seen this, I think, with Pi, which is, which is incredible, um, how valuable it is to make an agent extensible, not just by humans, but also extensible by itself.

16. 2:36

    Um, and, um, one thing that I want in the extensibility is, is I want, I want it to be composable. Like, I don't think we'll get there within one hour with all of you, but, but maybe afterwards if you say, "Oh, this is actually cool and not dumb," we'll keep working on this.

17. 2:51

    And you can say, "Oh, I made this extension," "You made that extension," and you can combine them. And, um, um, because I think we don't really yet know what is the best recipe for, for, for agents, basically, uh, for agent harnesses, and it's been too hard to experiment.

18. 3:04

    Even in Pi, like everything has to run in a single thread in, in, in, in like one process over here on this particular computer, and so on and so forth.

19. 3:11

    So I think, I think we, we can maybe do a little bit better than that. Uh, then I think it should be on the edge, publicly routable. Um,

20. 3:19

    like the way I think of it is basically an intelligent entity, like a digital one that's not a robot in the future. It's just internet connected, uh, basically server program, right?

21. 3:27

    Like it just speaks HTTP, so the slice, broadly speaking, that should be sufficient for almost anything. And then I would just put these agents on the edge speaking to the internet straight away. [laughs]

22. 3:37

    Uh, this is not very safe. None of what we're doing today is safe, by the way. There's no authentication you'll see. You'll all be able to see everybody else's event streams and agents and, and whatever else.

23. 3:45

    You should not put any secrets in the actual payloads of the events or rotate them afterwards, right? So we can, um, you know, those things can be solved. But broadly speaking, the moment an agent exists, it should have a URL.

24. 3:57

    That's just like, uh, uh, that's just my opinion. Uh, because otherwise, like you end up making all these things, and then you tie yourself backwards inventing a connector concept just so you can get Slack messages in or, or God knows what.

25. 4:08

    And that, I think that should be really easy. Um, uh, or like maybe like a human should fill in a web form, and the result of the web form should be input for the agent again or, or something like that.

26. 4:17

    That's kind of like the same thing as, as a Slack web hook. And then it should be distributed. And this is potentially, uh, like a very double-edged sword, uh, as we'll see.

27. 4:24

    Like, right? But, but by distributed, I mean you should be able to have, uh, an agent, um, running, uh, sort of on one computer here on the server and my, my plugin running over here and your plugin running over there, and yours is written in Rust and mine is written in, in, in TypeScript, and it all just

28. 4:38

    doesn't really matter. Um, and, um, the reason, uh, that is bad is because you will eventually get, uh, race conditions and loops, right? Like sort of like two plugins kind of sending messages back and forth, creating like an endless stream of events in your agent.

29. 4:52

    Um, uh, but on the flip side, uh, you have those issues normally anyway in, in, in, uh, most agent harnesses I've seen. So it's, it's quite good if you, if you can kind of preempt that a little bit.

30. 5:02

    Okay. So let's, um, let's talk about this, uh, thing that we have invented for, um... Oh my gosh. Um, what is this? This isn't the same as in my file system, so I'll, I'll use the one in my file system.

31. 5:18

    Um, basically, uh, we have created this, this, uh, service over the last week called events.iterate.com. It has a web UI, um, and, uh, we can, um, uh, we- you can use that.

32. 5:30

    I, I think you'll certainly use it. But for now, we'll just play with it with cURL because I think it's, it makes it, uh, in some sense more, more, uh, tractable.

33. 5:38

    Um, in the, in the agents', um, uh, in the events, uh, app, we have this concept of agent paths. So just like a file in a file system, every agent has a path as a hierarchy, and you just have, uh, here, let me show this here.

34. 5:51

    You just have raw events. This is YAML-formatted raw events in each of these paths. And, uh, that is basically the, the, the entire, uh, the entire primitive that we're gonna be building our agent on.

35. 6:02

    And so we can start by playing with this. We can say, okay, we, we have a path prefix. If you copy this and, uh, uh, from the repo that Misha is gonna put it in the README in a second.

36. 6:10

    I'll, I'll add workshop.md as well.

37. 6:12

    Yeah. Uh, yeah, okay. May- uh, oh yeah, maybe that's why. Um-

38. 6:15

    I, I added the README but not the workshop.

39. 6:17

    Yeah. So, uh, basically, um- Let's set ourselves a path prefix here. In my case, that is just gonna be, uh, Jonas Templestein or something. Um, they start with forward slash, by the way, which is a little bit, um, awkward potentially.

40. 6:31

    Uh, but the, uh, but the service is very tolerant of U- URL encoding your slashes or doing whatever you want. But I wa- I, I wanted good cURL economics, uh, but I also wanted them to start with a forward slash, so you have, like, a well-defined route.

41. 6:44

    Mm-hmm.

42. 6:44

    Then for the base, you-

43. 6:44

    It is now pushed to the repo, but have you sh- have you shared the repo?

44. 6:47

    Oh, yeah, because now you could actually do this on your computer. Uh, let's do that, actually. If you go to, um, [REDACTED:url], there is an, an ai-engineer-workshop. Um,

45. 7:00

    uh, and in there you should be able to ... If you click on this workshop.md file, that's what we're now going through. So you should be able to copy and paste the cURLs from there.

46. 7:10

    And again, there's no authentication, so you can just do this. And, uh, in, in fact, why, why don't a couple-

47. 7:16

    These are hit- hitting our live deploy-

48. 7:18

    Yeah, yeah, yeah. [laughs]

49. 7:18

    ... events.iterate.com.

50. 7:19

    Yeah.

51. 7:20

    So.

52. 7:20

    Uh, like, like, like this, this is entirely just sort of like a proof of concept of, um, uh,

53. 7:25

    how I think this should work. Um, what basically what we're gonna do is we're gonna turn this simple event stream into a fully-fledged agent without, like, really deploying any server software, uh, or, or anything like that.

54. 7:37

    Um, maybe, uh, yeah, I should be able to see if anybody, if anybody has managed to do it. I'll wait until two or three people have managed to successfully cURL something, um, and then I'll carry on.

55. 7:48

    Because I think it'll be, it'll be good if you, if you, if you cURL, uh, it yourself. So it's here in this AI engineer workshop, workshop.md.

56. 7:58

    If two people have the same name- [laughs] ... then that's gonna cause problems, so.

57. 8:02

    Same username. It doesn't matter. Um, it's, that's fine. Uh,

58. 8:10

    okay, there we go. Got a few people.

59. 8:12

    Nice.

60. 8:13

    Um, okay, so this is pretty good. Um,

61. 8:16

    by the way, uh, the ... you might see this here and there. There's also this thing down here called project slug. If you really start stepping on each other's toes, you can just type what- uh, like use a different project slug, and it's like a completely new database, a new namespace.

62. 8:29

    Um, you can pass that in as an HTTP header. If anybody needs that later, you can, you can ask me. Uh, okay, so now we have a few people cURLing things.

63. 8:37

    I will also for, uh, uh, start cURLing things. So, uh, now I'm just gonna say, "Hello, world," uh, here to my stream.

64. 8:46

    And it'll come back, and, and, and we straight away we notice, like, a few things. There's a basic, uh, event envelope shape. We have a, uh, type that I've just invented here.

65. 8:54

    Events don't need to have anything but a type, but most of them also have a payload. Uh, then, um, the, uh, there is the stream path, which, uh, came from the URL that we posted this to.

66. 9:03

    And there's an offset. This is just an auto-incrementing integer that starts at one. Uh, and so there's basically, like, a point of serialization at the server that just increments them, and you get a created, created at.

67. 9:13

    Uh, and then in here what we can do with cURL, which is quite nice, uh, if you do -n, you can do, uh, streaming, uh, streaming SSE. So this here is literally nothing magical.

68. 9:22

    This is literally just, you know, /jonas-templestein, uh, /hello-world. This is literally the cURL that we're doing right here. Um,

69. 9:31

    and, um, boop. There I can see the messages coming through. Very nice. And I can, um, I can add another message, and now, um,

70. 9:43

    basically, uh, all the, all the messages will show up there. And there's also a, um,

71. 9:48

    uh ... Oh, if you do ?live=true, this is, uh, inspired by the durable stream standard, but, like, ever so slightly different. But if you do, uh, live=true, the connection will stay open, and it'll just keep sending you more events as you, as you put them through.

72. 10:02

    So this is sort of already, you know, um ... Like you could say this is an AI agent if Misha was simultaneously, uh, giving responses to my Hello, World.

73. 10:11

    Uh, uh, th- this would already be an agent. And then there's, uh, my, my AI has told me you can, um, if you're gonna be doing that a lot, uh, you can pipe this, uh, through sed, get just the data elements, and then pipe it through jq dot, and then you get a little bit nicer formatting.

74. 10:25

    So this is sort of like, uh, this is so trivial that y- I actually do spend a lot of time in cURL, just curling it. Uh, then what can you do?

75. 10:34

    Um, the thing is very tolerant, just in general, of, of stuff you post to it, and this will become relevant if you wanted to, say, send webhooks from random third parties there.

76. 10:42

    Like, my theory is always that, like, the LLMs are good enough. They can basically just consume raw webhooks from anywhere, uh, and, and they'll roughly do the right thing.

77. 10:49

    So what the, what specifically the, uh, server will do is if you send it something that doesn't, um, validate, like hogwash here, it says there's, there's no, uh, type property, so there's not actually a valid event.

78. 10:59

    Then you can see here what it has appended is basically an error. But it, because we're event sourcing, the error is also just an event, and it just says here, "Okay, um, you have, uh, an invalid event appended, um, event."

79. 11:12

    Oh, another thing, uh, that we, we will notice is that the type here, uh, that our server has created, it starts with https, uh, events.iterate.com. You don't have to do that, uh, but that is something that we like to do because that literally leads to the documentation for that event type.

80. 11:27

    And if it's gonna be an opaque string, like, why not make it, make it something, uh, like that? But, uh, obviously you can, you can just do, you know, single words or two words or whatever.

81. 11:37

    Um, but you will see these URLs, and they're just, they're just types. Um, then, uh, what else is interesting that we can do? I mean, this is kind of boring, but, you know, it ...

82. 11:46

    Again, if you're gonna do webhooks, you have kind of put an idempotency key in, which just means if you have the same idempotency k- key twice, you don't get, um, you don't get the e- event appended twice.

83. 11:55

    You can pause a stream. I think in the interest of time we can skip that. You can resume a stream. The reason that pausing is important and is, we put this in even though we literally just made this a couple days ago, is because of the infinite loops.

84. 12:07

    Like it's very, very easy to accidentally put like, you know, thousands and thousands of events in a, in a infinite loop in the thing. And so if you put more than 100 events in a second or something, it is just gonna circuit break, and then the stream will be paused until somebody unpauses it.

85. 12:21

    So basically, if you try to, um... Here, actually, let's do that. Because again, like for those of you who haven't seen event sourcing, this might be a bit weird.

86. 12:27

    Like, how do you pause a stream? Well, you pause a stream by, uh... Whoa, oops, this is the wrong one. You pause a stream by appending an event that, uh, says you're, you're paused.

87. 12:37

    The, the only external interface to the agent is... Oh, um, now it's complaining. Uh, why is it complaining?

88. 12:47

    Ah, because I think there needs to be a reason or something maybe.

89. 12:53

    Uh. Yeah. Reason demo. Yeah, there you go. So now this thing is paused, so now if I, um, you know, try to send it the "Hello, World" event from earlier, it's just gonna barf and then say, "No, no more events allowed."

90. 13:15

    And that is one th- like, that is one thing that doesn't appear as an error in the stream because otherwise you would still get infinite numbers of error events.

91. 13:22

    Um, uh, then, I mean, there's other weird and wonderful things you can do. I, [laughs] I don't even know if you really have to do this, but, uh, there's like, um, the...

92. 13:33

    I have made a plugin for this already, uh, that makes it so that if I put this JSONata transformer configured event in, what it'll do is it'll just observe certain events, rewrite them a little bit, and make a new event out of it.

93. 13:44

    But I think that's a bit advanced, so we'll, we'll do it later. Um, um, it can also do weird things like it can, uh, you, you can schedule, you can schedule things.

94. 13:53

    Like some agent SDKs you'll, you'll know can do this, but, uh, here, you know, if you want like an, like a heartbeat event every five seconds. Uh, oh. Oh, God.

95. 14:03

    Now I need to resume the stream because it's paused. Uh, let's see. Stream resumed.

96. 14:14

    Okay. So now I am just creating, uh, this heartbeat event every five seconds, which can be quite handy. Or you can schedule something for 10 minutes in the future or at a certain time, and again, it's just an event that says, "Do this thing in the future."

97. 14:27

    Um. Mm, mm, mm. Um, I think I will skip over these, but, um, you can cancel the schedule. You can tell the stream by appending an event to send your server or some third-party server or your, y- you know, your Slack API endpoint or whatever an event every time an event gets added.

98. 14:48

    Uh, so you can basically, uh, subscribe to the stream. Uh, here we're subscribing in our, in our terminal. We're doing a pull subscription, right? We're doing, we're, we're connecting to, uh, an HTP server and we're getting, uh, we're pulling, uh, pulling, um, we're the HTP client and they're the HTP server, and we're getting an SSE, uh, response

99. 15:04

    stream of, of streaming events. But, uh, it can also do the other way around where it will push you the stuff that gets appended or push you a filtered stream, um,

100. 15:13

     uh, which can be quite handy. Okay. So, um, let's maybe stop there because I know also, um, it wasn't really clear from the session description how deeply technical this was. [laughs]

101. 15:24

     And so let's take any questions on what is even this thing that we're gonna be using for, for the next 45 minutes.

102. 15:30

     But questions have to be submitted as an event to the question stream. So- [laughs]

103. 15:36

     Yeah, exactly.

104. 15:36

     If you have any trouble using it-

105. 15:37

     Oh, that can be the queue

106. 15:38

     ... you're out of luck. [laughs]

107. 15:41

     Um, does... I, I assume lots of things don't make sense, so please do ask them because I think this will be extremely s- like random and strange so far.

108. 15:55

     What does Iterate do, your company?

109. 15:57

     Oh, uh, we, we're like a, we're like a, a hacker hobby club at this point to be quite honest. Like, uh, we, we, we are, uh, we have not got a launch product.

110. 16:06

     Um, but if we did, it would be, um, I think based on this architecture. It's like, uh... Anyway, I'll te- [laughs] I can talk about it afterwards.

111. 16:15

     Okay, okay, okay. [laughs]

112. 16:17

     But it's, uh, yeah. It's more, it's more homebrew hacker club than, than like commercial enterprise at the minute, but it'll get there.

113. 16:26

     Yeah.

114. 16:27

     Uh, I have a question. So, uh, I know that we'll be b- building, uh, event-based, uh, agents-

115. 16:33

     Yeah

116. 16:34

     ... now. Any, like, uh, maybe that's a, a basic question, but for like, uh, agents that are on the market right now, are, aren't they event-based? What's the, like, the different architectures?

117. 16:44

     It's just, um, I think it's just the API. Like, if you, um, have you tried to make, uh, you know, like a, like an open code extension or plugin or a Py extension or a Claude ex- uh, Claude extension?

118. 16:58

     And they all have like an API surface that is a little bit bigger than what you would have or what we're gonna have in, in a minute. It's not like by much, right?

119. 17:05

     But like, okay, like maybe the streaming chunks for partial completions are a different kind of thing for some reason, rather than just an event of type, "I've received, you know, like five letters of a streaming response."

120. 17:16

     To me, that's a perfectly valid event and, and, and the benefit then is that you can, uh, that can, you, you j- I'm, I'm kind of like a one abstraction kind of guy, right?

121. 17:23

     Like this, this should be like a, like web standards-based, there should be, there should only be one thing, which is an event. And then as you'll see, like the, the thing, um, at, uh, that was at the top of this file, I think you can build a super expressive state-of-the-art coding agent by just implementing, uh, like a

122. 17:38

     stream processor that reduces over the event stream and occasionally enact side effects. That's, that's all it does, right? Like, it, it just sort of sits there and says, "Okay."

123. 17:46

     And w- I'll, I'll show you that in a second. Like it's, it's a bit weird, um, uh, I would say, a weird way of programming. Um.

124. 17:57

     So if I understand correctly, you're sort of saying you take events that you get generated from an, uh, AI model, and then you're basically gonna pipe them through this stream processor.

125. 18:06

     Uh, is that the idea?

126. 18:08

     The stream processor is, is, or like the, the events.iterate.app is the storage. Like, you can almost think of it as a database. Like one, one, um, you know, do you know Convex?

127. 18:16

     Is like a sort of, you know, they, they kind of give you the illusion that they're running TypeScript in the database. You, you could almost think of this, this could be like a programmable stream or something, right?

128. 18:24

     Because actually one thing that you can also do, we'll, we'll see later, once you hack on your stream processors, you can also make an event that contains the source code of your stream processor, and then it'll just run on every event.

129. 18:36

     A-a-and so, like, it's basically, it is like, it is like the... Y- I think you can experiment with a huge array of different kinds of agent harness implementations,

130. 18:47

     uh, that can respond to real events on the internet and make real API requests without deploying your service or something like that. All you need is th-this sort of, um, streaming, streaming service that, um, uh, has some of, has some of these, um, things.

131. 19:01

     Like specifically, it needs to be able to wake up, you know, like other, other APIs and, and programs and so on. I don't know if that... Yeah. Um, that-- Does that answer your question at all or [chuckles]?

132. 19:13

     Yeah, I think so, yeah.

133. 19:14

     Okay. Shall we try and make, uh, now a, um, like a, like a slightly more serious version of these cURLs? Um, so does anybody here not know TypeScript?

134. 19:27

     I think, so I think the, like one of the arguments in favor of this architecture that I also put in that, um, in that, uh, in the, in the blurb, I think, is this is, in principle, polyglot, right?

135. 19:37

     And there is a, uh, like in, in any language, there is actually at events.iterate.com/uh, api[REDACTED:url], you have

136. 19:45

     just... I mean, this is an exceedingly simple API, right? Um, uh, you c- you can make yourself, there's an OpenAPI spec, you can make yourself a client, and so on.

137. 19:52

     But for the purpose of this, uh, this, um, uh, workshop here, we'll just do a TypeScript version, and we've shipped a client in a, in a, uh, NPM package already.

138. 20:01

     But I, I, I just wanted to point out, like even though we're doing everything in TypeScript, the whole point is that we wouldn't have to [chuckles] right? But we just have to do one thing, uh, uh, for this workshop.

139. 20:10

     Um, okay. So let's, uh, let's do, do something pretty simple. Um,

140. 20:16

     uh, is the, uh, AI engineer workshop, um, package, uh, now up to date and working? I think it is, right?

141. 20:23

     I think so, yeah. Uh, you might need to pull your repo 'cause there was a client-side type error.

142. 20:29

     In the Iterate repo?

143. 20:30

     Uh, no, the, in the AI engineer workshop.

144. 20:33

     Okay, that's fine. Um-

145. 20:34

     It's in example.ts.

146. 20:35

     Yeah, but, uh, I'm... Oh, maybe I will actually use that. Then, then I'll be able to experience, uh, the same, the same bugs if there are any.

147. 20:44

     Sure.

148. 20:44

     Yeah, that's probably better. Okay. Um, okay. So, uh, y- I think, um, if you wanted to just have a quick start, you can, uh, you can just pull, uh, clone the AI engineer workshop repo.

149. 20:54

     Um, there's already a teeny-tiny example file in there, and then we'll just add a couple more, um, as we go along.

150. 21:01

     And the workshop.md has runnable cURL as well.

151. 21:05

     Nice. Yeah.

152. 21:06

     That does work.

153. 21:07

     Um, if you wanted, you could put a coup- like a, a couple of the follow-on examples that are in the Iterate repo that have actual processors, um.

154. 21:17

     Yeah.

155. 21:18

     Hmm.

156. 21:18

     I can take a look.

157. 21:21

     Um, bum, bum, bum, bum. Hmm. I don't know what's going on here.

158. 21:31

     Aha. There we go. Hmm. [in background] My computer seems to have, uh, slowed to a crawl trying to open a cursor window, um [chuckles]

159. 21:48

     . This has been unfortunately like a, a somewhat regular occurrence recently. [in background]

160. 21:56

     Uh, okay. Okay. Or, oh, no, I think this might be what you mentioned about the internet.

161. 22:15

     I think it's just struggling because it can't open a terminal because the,

162. 22:19

     my, my terminal tries to connect with the internet.

163. 22:21

     It does?

164. 22:22

     Hmm. Oh, well. Then let's do the... Um...

165. 22:42

     No, it's not the internet. Or is it?

166. 22:46

     There we go. Okay. Okay. So we have here,

167. 23:00

     um, we have packaged up an SDK here to this, uh, AI engineer workshop, um, package that you should be able to just pnpm install or you just use, uh, use this example here.

168. 23:10

     And then basically, uh, what did, what did we need in our cURL? Right? Like, we needed a base URL, which is just events.iterate.com. That's where our server is. Uh, we can use a path prefix.

169. 23:19

     Um, and, um, then we can, uh, run this function here called createEventsClient. That is from our SDK. And then y- this is under the hood. This is... Oh. Hmm.

170. 23:31

     I really don't know what's going on with this terminal. It's very strange.

171. 23:40

     Hmm. Hmm. Okay, maybe, um, everything is just incredibly slow.

172. 23:53

     Do you want to try mine? Mine is-

173. 23:55

     Um.

174. 23:56

     Okay.

175. 23:57

     Maybe, but there's nothing actually hogging the CPU. It's just, I th- I do think it might be actually the network, but we'll see. Um,

176. 24:04

     so here. So here you have, um- [coughing] I don't think this has picked up the... Anyway, okay. So this is basically what we were doing just now, right? Like we, we say, "Okay, we're making ourself a client," uh, and then, um, here there is a function called stream This is basically the, um, the, the, the right-hand side of

177. 24:28

     the, of the terminal that we had earlier, and then you say live true and give them the stream path. And we can say, okay, if, if, uh, if the event is not of type ping, we will do nothing.

178. 24:37

     But if the event is of type ping, then we will... I mean, this is a completely unnecessary helper down there. Uh, then we can just do client.append. I don't understand why the, um,

179. 24:50

     the, the TypeScript language server isn't picking up any of the types for me, but I can't tell if that's just, um-

180. 24:55

     Cursor being weird again

181. 24:57

     ... your own example. Let me see.

182. 24:58

     Yeah.

183. 24:59

     Anyway, um-

184. 25:03

     Could quickly switch IDs.

185. 25:05

     No, it's fine.

186. 25:06

     It is working for me.

187. 25:07

     Okay. Yeah. I think this is unfortunately like a, a sort of cursor. [chuckles]

188. 25:13

     Okay. And then you should be able to, uh, just if you have a modern version of Node, you should be able to just run, um,

189. 25:19

     uh, or maybe not. Did I make it?

190. 25:23

     Um, uh, uh, uh. It says invalid TypeScript syntax.

191. 25:32

     You know what, I'm just gonna, I'm just gonna restart Cursor. I think that is worth it.

192. 25:39

     Is anyone, uh, able to run this example TS on their computer? There you go.

193. 25:43

     I am.

194. 25:44

     Hmm. Okay. Um, and are you able to send a ping to it and get a pong out of it? Okay, very cool. [keyboard clicking]

195. 26:15

     Okay. [keyboard clicking] Hmm. Um, yeah.

196. 26:43

     I don't... Yeah. I wonder if we used, use-- I don't know if you can see this, but basically all my commands, they have like a several second delay, which is-

197. 26:52

     Yeah, it's looking okay to me if you wanna just swap laptops.

198. 26:56

     Um, hmm. Maybe.

199. 27:12

     I had to pnpm that package, the one that you imported.

200. 27:16

     Yeah, no, I know, but it, it's just... Yeah. It's just my editor doesn't work, and I also have several non-responsive, uh, cursor sort of... [chuckles] I feel like this is, uh, unfortunately a pattern with Cursor recently.

201. 27:30

     Hmm.

202. 27:32

     I-

203. 27:32

     If you wanna swap laptops, my... It does seem to be working okay from, on mine.

204. 27:36

     I would just give everybody 60 more seconds to just get, get the basic demo working, and then-

205. 27:40

     Just let me know.

206. 27:41

     Yeah, because I think using some... You know, this is still almost an hour to go. It's not good to use somebody else's computer.

207. 28:05

     Uh. Maybe I will continue on yours, and I'll just restart my computer.

208. 28:21

     I think that's a good idea because we can... Yeah.

209. 28:27

     Dum, bum, bum.

210. 28:29

     I also have examples zero one and zero two-

211. 28:31

     Oh, really? Oh, brilliant

212. 28:32

     ... I think working with some-

213. 28:33

     What's your keyboard layout?

214. 28:35

     Uh, probably the same as yours.

215. 28:37

     One can, [chuckles] one can, one can hope. Uh, okay. And entire screen.

216. 28:43

     Looks the same.

217. 28:44

     Uh, it looks pretty good so far. Um,

218. 28:54

     what have you got a terminal? Uh, so

219. 29:01

     let's Node examples zero one hello world. Okay, perfect. Yeah. Well, this w- works flawlessly. Uh, so, um,

220. 29:18

     has everybody got that more or less working? Because then we can just make an OpenAI coding agent now. Um, I think that's all we need.

221. 29:27

     Let's maybe... I'll ask, um, do you have, uh, speech control in here?

222. 29:32

     Uh, yeah. It's Control and Option. It's Kit's one.

223. 29:36

     Yeah? And then?

224. 29:38

     That's it. It's recording.

225. 29:40

     Hello. Oh, cool.

226. 29:42

     See that red bar is-

227. 29:44

     Um, hey, I would like to make a third example, uh, called agent processor, and for that, we need to install, uh, just the normal OpenAI SDK. Please, uh, do that and just make me a new file that looks like the hello world processor.

228. 30:00

     Okay. Um, okay, so, um, I just gotta gather everybody again after I left you for about, uh, 10 minutes.

229. 30:15

     Yeah.

230. 30:17

     Here, this is a normal, uh, pretty normal spaghetti code TypeScript program, right? Like, uh, basically what we've done here is we, we sort of like, uh, doing some boilerplate work.

231. 30:25

     Um, uh, here if we want to subscribe, we, we have to make a subscription. Um, oh my gosh. This example that you put in there has like an a- abort signal and, and handles all kinds of things.

232. 30:35

     And, and, um, anyway, the, the, um... I think, um, uh, the way I think about what we're really doing is we're, we're writing a stream processor. And if you've never done stream processing, uh, this, this might be like a bit weird, and people use slightly different n- names for, for these things.

233. 30:48

     But really, um, what we're saying is, um, there, uh, exists, uh, some kind of world state that we're interested in that can be derived from the events. Uh, in this, uh, in this example here that Misha made, uh, we are, uh, trying to count, uh, the number of times an "Hello, World!"

234. 31:04

     event has, uh, been encountered in the stream. And then you can write the entire logic of your program as, uh, effectively a reduce function. This is, this is synchronous, and it just, um, it just, uh, looks at all the new events as they come in, and then it updates the state.

235. 31:20

     That's all it does. So it takes, uh, a state and event, and it, uh, returns the new state. Or you can just also return nothing. It'll, it'll be fine.

236. 31:28

     It'll just ignore it. And, um, and then, uh, you ca- you should not do side effects in this reduce function. Um, and the reason I've split the reduce function from, from after append, where you should do your side effects, is because, uh, if you think about what happens if, if your program, uh, goes to sleep, right?

237. 31:44

     Like you're, uh, you're, you're closing your laptop. At the moment, everything's running on our laptops. Uh, you know, 100 new events go into the stream. Some of these events might, in the future, need to trigger LLM requests or something like this.

238. 31:53

     And then you bring, uh, you open your computer again, you start your, your program again, and you don't want your program to go and, um, make a ton of LLM requests for all of those 100 events in the past, uh, necessarily.

239. 32:04

     Uh, instead, you want it to sort of like ca- catch up all the way, and then decide what to do, um, with the state at the end. And so we've made this, like, super lightweight, um, abstraction here of a processor that basically, um, gets hooked up into a processor runtime.

240. 32:19

     Um, and, um, what it does, uh, is, uh, it, it, it splits your work into reducing events into state. And then if you want to do some side effects, like for example, appending, um, appending to, uh, uh, you know, to the same stream again, you can do this.

241. 32:32

     By the way, I think, uh, I think you can also do this, which, which is, like, potentially, uh, this is pretty interesting. Um, like, uh, uh, you know, you, you can literally just append to, uh, you know, like a child path, or you can append to, uh, a parent path or, or, uh, or whatever, um, else you

242. 32:52

     want. Uh, it's basically like a, uh, meant to be like a file system. And so if you wanted to build sub-agents on this thing, all you would do is probably you would just append to ./ you know, Boris, and pretend that Boris is a sub-agent.

243. 33:03

     And then you would just subscribe to some special events that Boris might output, like, "I have the result." And then you would automatically be woken up again on the parent when Boris has the result.

244. 33:13

     Um, is this confirmed, Misha, that this runs? Like, how do you run this? This just exports the,

245. 33:19

     um... Like, you, you need, you need the runtime, right?

246. 33:23

     Yes. There is a CLI command that I now can't remember because my agent has been [chuckles] running it. Uh, where you can...

247. 33:32

     You should be able to do pnpmw and then path to the file, I think.

248. 33:37

     Did, did you do all that just now?

249. 33:38

     I did that about a week ago, but you may have changed it.

250. 33:40

     No, no, no. Yeah. Okay. So, okay. Okay. [chuckles]

251. 33:44

     Um, let's go to our repo. Like I promised, this is, uh,

252. 33:51

     a little bit... The, these are the, um... So you can actually also do this. If you go to iterate/iterate, ai-engineer-workshop, um,

253. 34:01

     here there are a bunch of examples, uh, of files that are, um-

254. 34:05

     Uh, workshop two. I was going from workshop one.

255. 34:07

     Ah.

256. 34:08

     Yes.

257. 34:08

     Yeah, I also-

258. 34:09

     These out-of-date ones

259. 34:09

     That was meant to be, uh, meant to be renamed.

260. 34:13

     Um, ah, here. This is, this is, this is basically the snippet we want.

261. 34:20

     Okay.

262. 34:21

     There is this, um, there is this thing, uh, that you can have. Um, so here now we've just made a processor, right? Like, this is just like a sort of inert thing.

263. 34:29

     And then, um, there's a thing called a, uh, pull subscriptions processor runtime. Um, and all that does is it takes your, um, it takes your, uh, your processor that you just made, and it, uh, runs it.

264. 34:47

     Um-

265. 34:48

     I think you probably need to assign it to a variable 'cause you'll just export default above.

266. 34:53

     Um, yes. Doot doo doo. Mm-hmm. Okay. Um,

267. 35:14

     uh, and here, let's just say, uh... Just see if we can run this.

268. 35:23

     Doop doop.

269. 35:27

     Sorry, a question. Should we, should, should I have access to these examples that you're showing?

270. 35:32

     Uh, yes, but unfortunately, they need to be modified, uh, um, in order to, uh,

271. 35:40

     um, do anything interesting.

272. 35:42

     Okay, because I don't see them on GitHub.

273. 35:44

     Um-

274. 35:45

     Yes, those ones are, those ones are, were added in the last few minutes, but, uh, usually-

275. 35:51

     Yeah. I gave slightly, um-

276. 35:54

     Version of the-

277. 35:54

     ... bad copying instructions. Um, but I think, I think-

278. 35:59

     Those aren't committed yet.

279. 36:00

     Yeah. Hmm. Uh, I think the b- we're just gonna have to sort of like slowly, slowly build it up in, in this repo and then, uh, push it once it works here, unfortunately.

280. 36:12

     Um, so here, uh, basically what I've now done is I've taken this processor and I've hooked it up, uh, uh, to this runtime, and all the runtime does is it does this, like, boilerplate for, for consuming the stream and running the reducer and run it, running, um, uh, the, uh, after append hook.

281. 36:27

     And what we should see is once, uh, you submit "Hello, World!" into this, uh, into this path here that is just jonas/example Uh, you should get a response there.

282. 36:36

     So if I go here in the, in the UI, and I go to jonas/example, and I think I said this should be event of type Hello World. Um, oh no, this, uh, is up here.

283. 36:53

     If all works, this would be sort of like the proof point that lets us...

284. 36:58

     It doesn't. Hello world seen. Um.

285. 37:05

     I don't remember if this example relies on the, uh, machine username, but if it does, then that would be mmkal, not jonas.

286. 37:12

     Um, maybe. Let's just do this. Ah. Sorry, this is like the worst workshop because now I'm also on the wrong computer. [chuckles]

287. 37:34

     Um, it does after append. And, hmm. Okay.

288. 37:52

     Is my computer back up?

289. 37:54

     It's, uh, well, yes, but I don't know your password.

290. 37:56

     Okay. Yeah. I'll, I'll do that in a second. Okay. Um,

291. 38:01

     this example here is not working. Okay. So I think we have to do plan B, but we might not actually be able to run this.

292. 38:10

     Are you getting, getting close?

293. 38:11

     I'm just gonna walk through the code as we would have written it from scratch. Sorry about that. Um,

294. 38:17

     because I think it, it sort of like illustrates the, the progression of things, and maybe if you-

295. 38:22

     Uh, it's installing dependencies, which is, is slow. Uh, so-

296. 38:25

     Yeah, but, but it's like-

297. 38:26

     ... the walkthrough-

298. 38:27

     Exactly. Eventually, eventually this will work

299. 38:28

     ... it might all be runnable by the end of the walkthrough.

300. 38:30

     Exactly. So, um, in this, in this paradigm, uh, as, as, as we, uh, we've been using it in our agent, when you want to implement some sort of feature, what you do is you say, "Okay, what is the state I actually need in, in order to do my, my, my thing?"

301. 38:45

     In this case, we want to do an LLM request whenever somebody sends a message and says hi. So, um, we have here in our, um, in our user interface, and you can access this as well, there is basically like a cheat code that makes it a bit easier to, um, to create, uh, new events of a type

302. 39:01

     called agent input added. And this is like deliberately the simplest possible thing you can imagine. This is basically just a string. And so what we're trying to achieve is that whenever somebody puts an agent input added event into an event stream, we want to, uh, react to that by making an LLM request.

303. 39:18

     And for simplicity, we're just gonna go in and do it using the OpenAI SDK. So, uh, the OpenAI SDK, um, the way it works is, um, you have to,

304. 39:27

     uh, at some point, uh, before you want to use it, you have to make yourself, um,

305. 39:33

     here a new OpenAI client, and then you can use, uh, their responses API. So you have openai.responses, uh, responses.create. And then, uh, in order to create it, um, you need some information.

306. 39:49

     Uh, specifically you need a model sta- uh, a string. And so immediately there it's saying, okay, so let's just say our state for our LLM agent, it has a model string because that is one of the things that we will need when we, when we come to actually make an LLM request and there is nothing but e-event

307. 40:03

     sourcing. So let's just say, okay, we have, um, we have a model string and, uh, that comes from, from, um, from our state. Uh, and then, you know, you could easily imagine, uh, in the future, this was one of the things that, uh, you could have done as an exercise, uh, in, in, in the hackathon is you,

308. 40:17

     you could say, let's add an event that is like LLM changed, LLM model changed, and you have the teeny tiny one-line reducer, and it just sets the model to be a different model, and then, um, off you go, you make the next LLM request, um, with a different model.

309. 40:30

     And the other thing we need is instructions. Uh, this is, I think, a bit of a misnomer. So, uh, I would call it system prompt in our state, and we say, okay, the ins- like, the system prompt for our LLM, what it's meant to be doing, we'll take that in our state, and then you need a history,

310. 40:43

     um, a state history. And, um, and, and so we've defined this, uh, and normally what you'll do is, uh, you'll just put this into, into, uh, like a little, um, into like a little, uh, TypeScript type if you're using TypeScript.

311. 40:56

     So here we're saying this is the state of our agent, right? And, and in the beginning it's really simple, but in the future, uh, you can very easily imagine how, how you quickly add things to it like state, am I currently compacting, or is a tool call currently in progress or, or whatever else, um, you might have.

312. 41:12

     Um, and, uh, the idea is to create a system in which it is trivially easy with three, four lines of code to add one of those capabilities, uh, to it.

313. 41:20

     And then you say, okay, what's my initial state? Um, because we're processing the stream initially, there's no events, so we need to know what we're, what we're initially starting with.

314. 41:27

     And so we're just saying, uh, this is-- we're starting with an empty history and this system prompt and GPT 5.4.

315. 41:34

     Um, and then the next thing you do once you've, um, uh, once you've done that, is you need to decide what events do I need to derive this, um, to derive this state?

316. 41:45

     Uh, and in this case, we will just,

317. 41:49

     um, uh, create a new event here, uh, a new... We're using Zod here for schemas. We're saying it's an agent input added event, and it just has a string.

318. 41:57

     Um, and then whenever anything comes back from OpenAI, because that is like a fact that has occurred, even if it's just a teeny tiny fragment of a string or, or something else, we store that in, um, in an event, um, and that's it.

319. 42:11

     Uh, th- these are, these are the two, the two main event, um, event types. Um, I don't think we need any of this stuff here actually.

320. 42:21

     Basically, what happened, as is often the case, I, uh, I attempted to instruct an agent to rewrite all of these examples to, uh, be simpler to execute just before, and it just, um, rewrote them quite significantly.

321. 42:35

     But this is really all you need. So, um- And then, uh, you have to write your reducer first, and the reducer is basically where you just say, "Okay, when an event is encountered, uh, how do I update my, um, how do I update my, uh, state?"

322. 42:47

     And you say, "Okay, when an agent input added event, um, is, uh, encountered..." This here is using, uh, a little, like, Zod helper library, actually one that Misha made, called Scheme Match.

323. 42:56

     Uh, it's just, uh, it, it just gives you basically type safety on this payload here. When I have an agent input added event, then I want my state to be updated as the current state plus the history, except the history also gets, uh, this little, uh, this little thing here, uh, here appended.

324. 43:13

     So we're... And, and, and, um, the, the reason we're doing that is because, uh, OpenAI's responses API, um, since we're making OpenAI processor, we're basically translating between our view of the world.

325. 43:25

     We just have this, like, agent input and, um, uh, it's just a string. Uh, but OpenAI wants you to put role user and, and then put the string in a content field, so we're putting that there.

326. 43:34

     Um, and, um, then the other thing we need to do, um, when you use the OpenAI str- uh, streaming API, uh, it'll, it'll, like, basically chuck out tons and tons of these, uh, output, uh, response, um, events.

327. 43:50

     And, uh, one of them finally at the very end will be, um, of type, um,

328. 43:58

     uh, ba, ba, ba, ba, ba. Um, um.

329. 44:06

     Yeah. I, I think this is a bit hogwash. Um-

330. 44:07

     It's like an end event at the end, no?

331. 44:09

     Huh?

332. 44:09

     When it's done with the-

333. 44:10

     Yeah, yeah. Exactly.

334. 44:11

     Yeah.

335. 44:11

     It, it basically, uh... What we really want here is,

336. 44:16

     um, uh... Um, yeah.

337. 44:27

     You're low on battery, so.

338. 44:29

     Oh, thank you. Um, is that actually right?

339. 44:36

     Um, no. Um, let's just... I don't suppose you have managed to get the-

340. 44:51

     I have pushed workshop two to the-

341. 44:54

     Yeah, but it just doesn't-

342. 44:54

     ... the demo repo, but it's more or less as is.

343. 44:57

     I've got a question. Is this gonna be streamed in? Like, so if-

344. 45:01

     Yes

345. 45:01

     ... pushing the stream doesn't-

346. 45:02

     Yeah. Let, okay. So let's, let's just like, uh, sort of like admit defeat on the thing that I was gonna do, and just like try, try and show you how, how, how it work.

347. 45:09

     Because we, we can get something like very simple working, right? Because we have the extremely simple ping pong processor that, that does work. Uh, and so here we can, uh, we can even, uh, just...

348. 45:21

     Hmm. Um, let's maybe actually start here.

349. 45:32

     Oh, man, I hit, I hit insufficient quote [laughs] running my, uh...

350. 45:41

     Hmm.

351. 45:43

     Joop, joop, joop.

352. 45:47

     Yeah. We, we do now have the same examples on the other repo, but it'll, it'll-

353. 45:51

     Yeah. Ugh. To be honest, I think-

354. 45:55

     ... take the same

355. 45:55

     ... uh, I think we might have to just admit defeat, and we can just talk about something else or give you, give you the time back. Just because the problem is we can't actually run these things. [laughs]

356. 46:03

     Uh, and I think it will take me like probably like 10 minutes of actually concentrating to, to get them runnable. Um, so maybe we should, uh, admit defeat on that.

357. 46:14

     Um, I do wonder if there's like... Does A- Does...

358. 46:20

     Yeah. Do- Does it, does it sort of make sense what the, what the idea is, especially with, with it being distributed? So the... Right? That you can say, "Okay, like I have my processor over here," and your agent can use it, and it can be on a different, uh, server, or it can actually be in a, in

359. 46:36

     a dynamic worker. Um, and, um, yeah. I, I think without showing it, it doesn't really make sense.

360. 46:50

     So is, is the idea you'd, um... So with the stream processing, so you take this event stream from-

361. 46:55

     Yeah

362. 46:55

     ... OpenAI, and-

363. 46:56

     Mm-hmm

364. 46:56

     ... it contains events like output item added, or I, I know the Anthropic model better than anyone, but you've got-

365. 47:02

     It's very similar

366. 47:03

     ... item, and then you've got some series of like text deltas, say.

367. 47:06

     Yeah.

368. 47:06

     And so the reducer here is kind of squashing all of those events-

369. 47:09

     Yeah

370. 47:09

     ... and then materializing that sort of partial response-

371. 47:12

     Yeah

372. 47:13

     ... into an event, right. So then the bit I was trying to understand was sort of with this stream processor-

373. 47:18

     Yeah

374. 47:18

     ... sort of framework that you've built is like these materialized events, is, are those events persisted as new messages on the stream?

375. 47:26

     Um-

376. 47:26

     Are they being declined to consume the raw events, and sort of you're doing the materialization client side? Or is it sort of a part of something that's happening in the stream?

377. 47:34

     So every... Yeah. So, so everything that happens in the system, every streaming chunk, everything becomes a, becomes an event in the stream.

378. 47:42

     Right.

379. 47:42

     And then you have reducers running in many different places. Like, uh, for, for instance, uh, you know, even this thing with like is the stream paused or is the stream not paused, the actual implementation of that is, um...

380. 47:53

     Maybe that, maybe that's actually useful, because this is running in production now. Uh, like, right? Like there is a, a circuit breaker processor, and, and this is like how the whole thing is built, even inside the event service, and the circuit breaker processor has an initial state of paused false, and it has pause reason null.

381. 48:07

     And basically, uh, then it has a really, really simple reducer, um, uh, that just checks what are the last 100 timestamps of events I've seen. If the last 100 timestamps I've seen, uh, seen don't go, uh, you know, don't go more than a second back, then I'm just gonna append a, uh, a- append an event.

382. 48:22

     So the reducer basically just accumulates these timestamps, and then the after append function, uh, here says, "Okay, if I, if I have seen too many timestamps in the last second, uh, I'm gonna, uh, I'm gonna throw the circuit breaker."

383. 48:33

     So this is like a real processor that runs inside my, uh, my, um, my, uh, service. But let me show you another one. Uh, this one here, this UI, this is also a stream processor.

384. 48:43

     What does this UI do? The UI takes all of these events, and for some of the events that it deems like interesting, it chucks- Uh, it, it basically, it, it basically projects them onto what we call feed items.

385. 48:53

     So these things here that have like an, a slightly ni- nicer rendering, they are feed items, and they are not the same as events, but they're derived from events.

386. 49:01

     So, uh, when you're making UI for this thing, you're also just reducing. Um, and because the reducer is just a synchronous function, right? If, if, if you knew, for example, I have, I have this like, uh, a Py agent harness pr- uh, uh, processor, and it comes with a whole bunch of event, uh, event types, um, and

387. 49:18

     it has a reducer. Um, you can just in your processor use their reducer. It's practically free, right? Like, uh, you can just import it and, and run it. Uh, and then you can just sort of like start building abstractions that rely on their event types.

388. 49:30

     Um, and so, um, I think what I'll, what I'll, what I, what I will commit to doing, because this was such a shitty experience, is I will just do, do the whole rundown in video form, uh, on the, on the internet, and that was one of the things I was gonna show.

389. 49:44

     Like, you can just hook up Py or Claude or, or OpenCode as well, and, um, just s- say, okay, like, all... They come with a bunch of events, and you can store them in this event stream, and you can react to them or, or build on them.

390. 49:58

     Um, d- does that answer the question, like, where the reducer runs? Um, um, and, and then, like, one of the things you can do with this is you can also actually just append an event that has some source code in it, and then it runs that particular processor literally automatically for you, horizontally scalable.

391. 50:16

     Um, it's just, uh, yeah.

392. 50:19

     Do you have the example of showing how you do that, where you're appending like a, a function-

393. 50:24

     Yeah, I do have that. I do have that

394. 50:25

     ... even if it's not runnable, I think the, the-

395. 50:27

     I do have that

396. 50:27

     ... idea is somewhat-

397. 50:28

     Yeah. Well, actually, we can do it here. Uh, yeah, I, I think, I think maybe, maybe that is good. Just like focus... Instead of trying to write code, we can, uh, focus on the things that might work.

398. 50:35

     Um, so let's... We can create a new stream. By the way, all streams are created, uh, you know, uh, completely implicitly, so you can just, uh, uh, post to anything under the stream's API, and it'll, it'll make you a new stream.

399. 50:47

     And then, uh, here there's in the UI, and you can also play with this if you want to just see, um, there is a, there's a bunch of sort of preset, um, preset events, uh, and one of them is here.

400. 50:57

     You can append an event of type dynamic worker configured to this thing, and, uh, what does it have? It just has a script in it, and the script is a string, and inside the string is a reducer, uh, and an after append hook.

401. 51:10

     And so this is, um, literally a, uh, like, um, a stream, a stream processor. It would be funny if this, uh, if this is the thing that works, because th- [laughs] this is what...

402. 51:19

     I was basically, I was basically, uh, up most of the night working on this because this is the most exciting bit, but it wasn't really like workshop grade because it was...

403. 51:28

     You know, you get to, get like the first five concepts, and then you're like, "By the way, you can deploy this by just adding an event." Um, uh, but if I, if I just say ping here,

404. 51:37

     it says pong, right? That's pretty crazy. Like, I have here a stream, uh, that knows nothing about the world, and then I have appended this event here that just has a little bit of JavaScript in it, and this little bit of JavaScript, uh, just makes it respond with pong whenever it encounters something with ping.

405. 51:55

     And then there is actually a, a version of this. This is the thing that actually took, took the night [laughs] away, so to speak, is if you have these processor files that export a processor that you define with define processor, you can bundle that into an event effectively.

406. 52:08

     And this AI agent I also got running like that. But of course, you don't want to have your OpenAI API key in there, so that is why we have this like slightly awkward env vars here, where you can put your OpenAI API key outside of the stream, and then, um, it will basically, when it makes a fetch

407. 52:27

     request, it'll substitute the secret in, in the header. But, um,

408. 52:32

     yeah, I'll definitely, I'll definitely post a video of that because that's, that's pretty wild. Like, you, you can write like the 40 lines of code required for a basic AI agent, and then you can append that to any stream, and then that stream becomes an AI agent.

409. 52:46

     I don't know. [laughs] I'm sort of n- looking at like two or three nods, but mostly blank, blank stares. It is very strange. Um, but, uh, but, but I th- does this, does this make sense what happened here at least?

410. 52:56

     Because you can... Y- this one you can just play with. Like, you can just go in the UI. And so, uh, to some extent, uh, actually, you can, uh, you can make increasingly more complicated processors [laughs] for your streams.

411. 53:08

     Uh, you can just, you can just code them in this input field if you want. Or, uh, you know, um, the only problem is it cannot have any dependencies.

412. 53:16

     Like, if you want some npm packages, you have to bundle it into the string, which gets a bit more complicated. But, um, yeah.

413. 53:24

     And can you override?

414. 53:26

     Yeah, so this is the other thing. Like, everything, uh, you, you might have noticed is everything has a slug, and it's just like a URL safe identifier. And so the, the event here is actually called dynamic worker configured because you can override it.

415. 53:37

     And, and what also already works with this event type is you can give instructions to an AI agent to make itself

416. 53:46

     different functionality just by calling append, um, with like a little bit of a different JavaScript.

417. 53:53

     Um, and, uh, you know, you, you could, you can quite easily imagine overcoming the bundling problem, for example, by just saying, "Oh, there's another event type," which is called like a sort of like, uh, unbundled, uh, dynamic worker, which just has a package.json file, uh, uh, field and a script field.

418. 54:07

     And then a different processor takes it, bundles it, makes the fat event that has all of the bundled source code, and suddenly, uh, is, is basically like a, like a IDE.

419. 54:17

     A- a- and that's then where you can, you can hack on the agent harness

420. 54:22

     without having to really like install anything or like bring servers into the mix or, or whatever. Um, but sadly was not able to show it, so bummer.

421. 54:31

     Um, what is, what if there is a error in the JavaScript code you put in the stream?

422. 54:37

     Hmm?

423. 54:38

     So if you pushed a Java-

424. 54:39

     Yeah

425. 54:40

     ... JavaScript code in a stream, and if there is a error in that code or if you made a-

426. 54:46

     Uh-

427. 54:47

     ... made a typing error, what happens then?

428. 54:49

     Uh, well, th- this is, uh, this is, uh, JavaScript. There wouldn't be a typing, a typing error. Um, uh, but it, uh, if there's any kind of error, the only thing that can possibly happen is that you get an event.

429. 55:00

     Yeah.

430. 55:00

     So there would have to be an error event. Um- And there's no, there are no typing errors as implemented right now, but you-- that's because-

431. 55:06

     Yeah

432. 55:06

     ... this is a JavaScript evaluator. But you could also find a way to run a TypeScript compilation step on it first, and then you could emit error saying there was a TypeScript compiler error.

433. 55:17

     Then you could run it through an LLM to try and fix it and emit a new event with the fixed code or do something else with it.

434. 55:22

     Yeah. Um-

435. 55:24

     The idea is that just everything that happens results in an event, and then you can react to those events however you, you see fit.

436. 55:33

     And-

437. 55:34

     These, the event, they are then, um, evaluated and run in your backend, right?

438. 55:39

     Yes. So in this particular case, it's, it's dynamic workers in Cloudflare. So it, it just spins up a, a small, small dynamic worker. Um, so basically, like the deployment story for a processor like this is or, or like one way to think about it is, uh, at wh- wh- when this was still working before I, uh, ask

439. 55:55

     it to be refactored and broken, I had, um... The way I would do it is I would locally on my computer, I would do the SSE subscription and, and run my processor that way.

440. 56:04

     And then once I thought it was cool, uh, I could either deploy it just as a web service on, on Cloudflare Workers or Vercel or, or wherever, uh, and, and just tell, uh, tell the, um, agents, uh, uh, the, uh, tell the streams to notify me whenever a new event needs to be processed, or, uh, alternatively, just

441. 56:21

     use this dynamic worker thing. Um, and like there's, uh, I don't know if this is gonna work. This is why, like, at the moment, it's looking more like it's a really dumb idea, I think, because it's really not coming across.

442. 56:30

     But the-- For example, you could have a plugin for your agent, uh, that runs on another computer, especially if it was like something like really important like a prompt injection protection or something like that.

443. 56:40

     Like you could say, "Oh, okay, the way that my agent uses your prompt injection protection service," it can't really be through MCP because there's no way to, to kind of like proactively hook into the agent loop or, or, uh, um, at least not at the minute.

444. 56:53

     But what it could be is it could say, "Okay, my, my agent loop actually waits for up to like two hundred milliseconds before making any new LLM request, uh, for any safety checkers to say this."

445. 57:02

     And then it can just literally ask your safety checker over there. I don't care how you implement it. You could even charge me for it, right? Um, and, uh, I just don't, yeah, I just don't think that's, uh, currently possible in, in any other way.

446. 57:15

     That's why I, I think it's quite important that the stream can reach out to the processors and say, "You gotta do something," you know? "You need to process some events now."

447. 57:24

     Um, um, there's a, there's like a couple other, uh, potentially... Hmm, I don't, uh, I mean, this might, yeah.

448. 57:31

     Just-

449. 57:32

     Yeah.

450. 57:32

     With the idea, one of the ideas you shared is, uh, my agent might use, like, just-in-time, uh, services of other agent, uh, offering.

451. 57:40

     Yeah.

452. 57:41

     And he might be charged for it.

453. 57:43

     Yeah.

454. 57:44

     Okay.

455. 57:44

     Well, I mean, like, probably in- initially, the human would be charged of it. But, but this is just like the, there is no way... Like let's say, okay, so Claude Code.

456. 57:51

     If I want to make a Claude Code plugin, um, if, if I wanted to make that into a business, think, think about how you would have to do that at the moment.

457. 58:00

     It-- And you need to give some sort of instructions, uh, or, or, or something to, to, uh, to Claude Code, either to proactively call your MCP tools or, or something of that nature, or call maybe some CLI tool that was previously authenticated, where you did like CLI login to, you know, like promptinjectionprotector.com, and then, uh, somehow it

458. 58:18

     works. Where, whereas, um, uh, really, I think you can just plug into the, into the agent stream, like into the event stream directly. Like the, the interface is just somebody, somebody notices that Claude is about to do something and chuck some more events in.

459. 58:32

     And, and like they kind of all work that way, but not quite. Like it's, uh, it's sort of like you, you have to squint a lot to, to see that that's what it all compiles down to.

460. 58:43

     Um, um, another thing maybe worth pointing out if, for, for some of you that have tried to, uh, like tried to make open code or, or, or Py plugins or whatever is, there's no before hook.

461. 58:53

     This is very, very important. There's no like, um... There is actually in the back end, like our built-in processors, the only difference to our processors and your processors is that we can actually stop events from ever being appended.

462. 59:03

     For exam-- That is why the pause feature could not be implemented by a third party. So there's sort of like certain things that are just, um, that, that need to happen before an event goes in the stream.

463. 59:13

     But, um, broadly speaking, I'm very against before hooks. Like I, I think there were some instances in, in OpenClau, for example, some massive performance regressions and, and cost increases where, you know, you can, you can break, uh, um, you can break context caching quite easily with before hooks.

464. 59:27

     You can massively destroy performance. Like it's just way better to think of the whole system as, as being eventually consistent. And, uh, you know, um, doing all these distributed systems things like saying, "Okay, we're gonna wait for up to two hundred milliseconds for somebody to say, 'I've got a little bit of information for you.'

465. 59:42

     But then we're actually gonna make the LLM request, whether you come in with information or not," because this is needs to be a resilient system. So if you think, for example, about RAG pipelines and, and things like that, you can easily have a little processor that says, "I have like a indexed version of the Notion knowledge base

466. 59:57

     or something, and I'm just gonna sit here and try to squeeze in a little bit of extra context if I think it's relevant. But if I don't get there in time, it's like totally chill.

467. 1:00:06

     The, the whole thing still works. It's not like I've, I've kind of delayed the, um, the agent or something."

468. 1:00:12

     Um, yeah.

469. 1:00:16

     What about, um, so you're hosting this, this instance of iterate. Like how would you imagine in the future, would everyone host their own one or is it centralized?

470. 1:00:24

     You mean-- I mean, the, the, the, the streaming database, like, I don't know. I, I, I think, um, I think this could work on almost anything that is like durable stream shape.

471. 1:00:33

     So this was just for the purposes of this, um, exercise. Uh, I do think like there is potentially an interesting scenario where, uh, like, where a, like it's, it's not even really like...

472. 1:00:45

     It, it's a combination of a queue and a pub/sub system and a, and like a streaming database and, uh, uh, that can also run code for you. But, but that, that sort of thing is an interesting infrastructure primitive that, uh, maybe somebody could make.

473. 1:00:57

     Um, I do think that, uh, yeah. But in, in principle, like I, I don't know if you don't know about it yet, you can read up on this, uh, durable streams, uh, API specification that somebody came out with.

474. 1:01:07

     We don't follow it exactly, but ... uh, close enough, and it's, it's basically

475. 1:01:13

     like a, like a very common old idea. Like, um, the, the thing that is more new is that it would be, like, you would also have push subscriptions out of it.

476. 1:01:23

     But there are some things that do is there's a thing called NA- uh, like Jetstream NAT or something, uh, that, that does it. Um, yeah.

477. 1:01:32

     Are you talking about, uh, like the electric SQL?

478. 1:01:35

     Yeah, exactly. They, they have durable streams, yeah. And, uh, like a d- a durable stream is just like an um, append-only event log with, with offset tracking. And, and the, the, the, you know, they want the client to track the last consumed offset.

479. 1:01:46

     That's also how, say, Kafka works or something like that. Uh, uh, but you can also do it the other way around and have the server track the offsets, and those are more nor- more normally like this.

480. 1:01:55

     Google Cloud and AWS, they have these pub/sub systems where they say, "Oh, we'll just like give you the next event and we'll keep track of, of the offset for you."

481. 1:02:04

     Um.

482. 1:02:06

     I've got another question.

483. 1:02:07

     Yeah.

484. 1:02:07

     Uh, say I, say I've got a, I- I've, I've got Michael Bell here or Michael-

485. 1:02:11

     Yeah

486. 1:02:11

     ... and I have a sub path called images, and I-

487. 1:02:13

     Yeah

488. 1:02:13

     ... hook it up to like a local LLM which is generating images, and then anyone could essentially pass a stream to that which would then go to my LLM, generate-

489. 1:02:21

     Yeah

490. 1:02:21

     ... an image, and return it back.

491. 1:02:22

     Yeah.

492. 1:02:23

     How, how would I, like, prevent a certain amount of like basically flooding the system?

493. 1:02:27

     Oh, I mean, you would need to have authentication on it. Right, right [laughs] like this, this entire thing would never, uh, like, uh, right, like this, uh ... I was thinking like it might be fun to just say there's a, there's like-

494. 1:02:37

     Don't do that yet, basically. [laughs]

495. 1:02:38

     Yeah, exactly. Th- it might be fun to just say there's a public namespace, and it just gets cleared every hour or something-

496. 1:02:43

     Yeah

497. 1:02:43

     ... right? Uh, because it is kind of nice that anybody anywhere on the internet can just cURL a thing, uh, or like copy and paste a cURL from somewhere and, and suddenly you have like a flavor of agent.

498. 1:02:52

     I do think that's quite interesting, but it just needs to be deleted every, like quite aggressively. Um, yeah. But, but I think, I think that's more or less a solved problem.

499. 1:02:59

     It's like a bit gnarly, but basically the way I emit vision it, you would do it, um, if you did this properly is on each event, you basically have like client provenance information.

500. 1:03:08

     We say like, "Who was this? Like, uh, what, what were they able to do? Like, how, how sure are we that these people are, you know, the, the HTTP client or the, the stream client was actually entitled to do this operation?"

501. 1:03:19

     Um, and, and again, you can model all of that as events, right? Like, you can, you, you would just have an event that is like, "Make this stream public."

502. 1:03:29

     And until you make that event, it can only be, uh, written to by whoever created it or something.

503. 1:03:37

     Um, all right. I might ask, uh, um, uh, S- Sean if we can, instead of this video, do a video we'll record a little bit later today, um, that shows it more nicely because I do think it's worth showing.

504. 1:03:51

     Yeah.

505. 1:03:51

     Just a little bit too short, short notice.

506. 1:03:59

     And shall we just wrap it then if, uh... I don't, I didn't mean to keep you all of your, uh, like 10 minute la- longer break, or if anybody wants to chat.

507. 1:04:08

     Thank you. [laughs] [clapping] [outro jingle]