Vibe Engineering Effect Apps

Talk transcript

1. 0:00

   [upbeat music] So welcome everybody.

2. 0:17

   Um, just setting up the context for this workshop.

3. 0:22

   I had a lot of ideas to potentially prepare, but at the end, I thought, we are Vibe Engineering. For this to be authentic, it has to be from scratch.

4. 0:35

   So I actually pre-pre-prepared absolutely nothing. That means we can take any path that we want,

5. 0:44

   and let's hope this is real enough. First of all, I'd like to know from the crowd,

6. 0:51

   do you already know Effect? Do you know Zero? Like, what's your level of familiarity with AI tooling and some kind of questions like that. Lucky enough, we're not too many, so I hope this can be as interactive as possible.

7. 1:07

   Maybe let's just start. I know Chris. Hey, guys. Hello. Hello. Hi. I'm Chris Di Paola. Nice to meet you. Familiarity with AI and with Effect? Decent. Some? Decent. Okay, good.

8. 1:22

   Running v4 in production. Running v4 in production. Against advice, by the way.

9. 1:29

   Hi, I'm Conor. Um, I've never actually used Effect, but I like, you know-

10. 1:34

    Good

11. 1:34

    ... functional programming like F# and things like this, but I've not actually used Effect, but.

12. 1:40

    Good.

13. 1:40

    I'm starting with the VibeCon.

14. 1:46

    Yeah. I'm pretty familiar with the agentic workflows. I think I've been... I haven't been, like, writing code by hand since the beginning of this year, like, uh, doing everything in with the agents.

15. 1:59

    And the reason I'm particularly interested with the Effect is like, it's, I don't know, encouraging so much safety, um, so my agents cannot become smoke cannons. And the reason, uh, I, I wanted to use Effect is, was to, like...

16. 2:16

    We had one API client. I transferred that API client from regular, like, promise client to Effect, then built a CLI on top of it.

17. 2:26

    Mm-hmm.

18. 2:27

    And now I'm more interested how you, uh, make the Effect more, uh, discoverable by the agents.

19. 2:33

    Okay.

20. 2:34

    I saw the idea about having the repo cloned as a subtree. It's, like, not pretty convinced with that, so I'm curious how it's-

21. 2:45

    Good

22. 2:45

    ... gonna go at the end.

23. 2:46

    Good. What about you?

24. 2:48

    Yeah. I'm quite familiar with like Claude Code and building stuff, but I've not heard much about Effect yet, so this will be my learning.

25. 2:57

    Sounds good. Sounds good.

26. 3:01

    And hello, everyone. My name is Mirela, and I help with community and design at Effect and Effectful company behind Effect.

27. 3:12

    Good. So pretty heterogeneous crowd, all interested in some sort of how to use agents effectively with Effect, pun intended.

28. 3:24

    You pointed out at a, at a very good thing, which is cloning, uh, giving the repository access to... giving the agent access to the, to the repository. And in reality, this session should just be called, "Just clone the fucking repo"

29. 3:40

    and get... and be done with it. And really,

30. 3:46

    like, I've, I've also have not been coding by hand since about late this summer, so it's been quite a while. I started programming when I was [REDACTED:age], so it's quite an odd feeling to get to the point where, you know, you're no longer writing code by hand.

31. 4:07

    And most of what I do is library level coding. So it's pretty low level, usually fairly complex type machinery stuff that used to require

32. 4:22

    a very good understanding of the language, of how the user interacts with your software, and so on and so forth. Not diminishing in any way up-level development, just that the way you treat a language if you have to build a library versus the way you treat the language if you are building an app on top of it

33. 4:40

    is usually very different. Now, sometimes in app land, you have the same requirements as library land, especially when you need to, you know, generalize, abstract over some patterns, make them, uh, repeatable.

34. 4:54

    Well, um, remove the verbosity from the repetitions, and so on and so forth. So there's some crossing there, but I definitely thought that, uh, AI would be more useful in app land, and I didn't see much usage at library level land, and I was dead wrong 'cause I'm not writing code by hand.

35. 5:14

    I have not wrote any line of code by hand for a while. And I've done that in TypeScript, I've done that in Rust.

36. 5:25

    And the funny element is, given I mostly write libraries,

37. 5:30

    I usually interact with code bases that have zero documentation, that have zero best practices available online.

38. 5:41

    And so I couldn't really use the usual, let's just add an MCP server to get access to the documentation or hoping that the models have been trained on the documentation enough to be directly useful.

39. 5:59

    And the reality is, with LLMs, like people treat them like a human brain, but they are very different

40. 6:09

    We learn continuously. This is a learning experience. Once we get out of this room, hopefully, you're gonna know a little bit more

41. 6:21

    s- from the starting point on, on when you come in, and your brain will keep going and will internalize more and more patterns over time.

42. 6:33

    Then you go to sleep, your brain cleans up a little bit of the mess of irrelevant information that you got during the day, and there's this whole process of transforming experience, so the, the world that we experience every day, into long-term memory.

43. 6:50

    With LLMs, this does not happen. W- with LLMs, you get a pre-training phase where LLMs are trained on all the world of existence, uh, o- of existing knowledge. Usually, they, they get trained on the whole internet, [smacks lips] then they get specialized in some tasks, and then there's the whole post-training phase where models

44. 7:14

    are fine-tuned to act on specific things. For example, coding agents are generic models that have been reinforced, uh, that have had passes of reinforcement learning to operate on code bases.

45. 7:30

    The whole post-training phase of a, of a large language model dedicated to coding is letting the model read through code bases and having evaluations that tells the training phase how is the model performing.

46. 7:48

    Is it doing good? Is it doing bad? Does the code compile after this change? Does the code fail to compile after this change? And so on and so forth.

47. 7:55

    But once that is done, it's done. There's no more knowledge that comes into the model every day. So

48. 8:05

    if you interact with a model today, and you tell, you tell something to the model, and you say, "Hey,

49. 8:10

    I want you to do this in a very specific way," tomorrow, it's not gonna remember.

50. 8:17

    So how do you make it remember? That is the, that is the big question. And

51. 8:24

    models, you have to think of them like

52. 8:31

    you're chatting with them, but the reality is you are basically appending messages onto a fixed size array, which is called the context window, and context window is limited.

53. 8:47

    Now there are models with a one million tokens context window, and that's not necessarily a good idea because

54. 8:56

    the context window of the model is what is pushed to the neural network, and the neural network's gonna try to predict what's coming next. So if you push more information, there's a very good chance you're gonna confuse the model, which is why a one million context window is not necessarily helpful, especially if you're doing multiple things in

55. 9:16

    the same context. That really means we have to architect around

56. 9:25

    a dumb process. We have to architect around some machine that had knowledge of six months ago at best, that it's not gonna remember everything because even if you have one trillion parameters in the model, or even if you have ten trillion parameters in the model, that's not enough to store the, all the human knowledge.

57. 9:49

    So you're always gonna get compressed knowledge. And in the best case scenario, you have some ability of generalization in the model so that the model can say, "Hey, I know A, B.

58. 10:01

    Maybe I can do C because it's similar to A and B," and you have some form

59. 10:08

    of this emergent behavior and, and capability of reasoning on new problems.

60. 10:15

    But models have become very good. I've said it by myself, I'm not writing code by hand since, at a minimum, six to eight months.

61. 10:25

    So that means even if the machine is dumb, it's already at the point where we can leverage it to do good. But how do we do it?

62. 10:35

    Now, if the assumption is the model has outdated knowledge, we need a way for the model to get new knowledge.

63. 10:45

    And we said that those models that we use for coding have been reinforced, have went through re- reinforcement learning to be able to understand your own code base, make changes in your own code base, and replicate patterns that exist in your own code base.

64. 11:08

    They haven't really been trained on reading human documentation. They haven't been trained on using an MCP server that they never seen. They've been trained primarily to consume and produce code.

65. 11:27

    So eight months ago, I was thinking, "What if

66. 11:31

    I just give the model access to code?" That means if I wanna use Effect,

67. 11:37

    I'm gonna add the Effect repository in my directory, just masquerading the Effect code base as my own code base, and maybe I can trick the model into thinking that it's just one big code base and that it, it would explore it and would progressively use it to build up the, the required knowledge and to sort of

68. 12:01

    clone the patterns. And there's various ways of doing that. One could argue the model already has access to library code by having it in NPM, in Node modules.

69. 12:17

    But coding agents have been trained to focus on your own code, not on the code that is on node modules. So if you have it in node modules, the model is de-optimized.

70. 12:28

    It's not gonna look at it with the same frequency as it look at your own code.

71. 12:35

    If you have it in a gitignore directory, the models have been trained not to look at files that are gitignored. For example, Cursor does not index stuff that is gitignored.

72. 12:50

    So there are all of those sort of random restrictions that we figure out while, while developing,

73. 12:59

    and the only way I found the models to be good, regardless of the language, regardless of what you use, is if you just clone the fucking repo, which is the point of this workshop.

74. 13:13

    So this is a completely empty project. I have some ideas of where we could take this. My idea would be to set up, um, a Bun repository, use Vitest for testing, uh, use...

75. 13:30

    build up some kind of HTTP server, ideally providing an OpenAPI documentation for consumption, build kind of type safe client to interact with the backend. Uh, hopefully, if we have enough time, I'm not sure, um, tap into the world of workflows and, and clustering for persistent operations in the, in the backend.

76. 13:53

    And really, I have nothing set it up. So how do I usually start? Well, I would like you to... So I start nice with the model, but as soon as it derails, you're gonna see I'm gonna start to insult the model.

77. 14:11

    And it's fun because it cannot really answer you back. If you don't like the answer, you can just shut it down. It's not like a human that gets offended.

78. 14:20

    Maybe. So I would like to set up a project using Bun.

79. 14:30

    The project should also include setup of Vitest and,

80. 14:41

    uh, TypeScript check script. I'm using GPT 5.4.

81. 14:58

    When I started this journey, I was using Sonnet 4. There's many difference between Sonnet 4 and GPT 5.4, namely Sonnet 4 felt like a kid with a knife running through the house.

82. 15:16

    That's, that's an example that comes from Geoffrey Huntley, the, the author of the Ralph Loops. But even as a kid running through the house with a knife, it was still enough to do coding.

83. 15:32

    And GP... Now we have models like Opus 4.5, GPT 5.4 that are much, much better.

84. 15:44

    But a very interesting element to, to think about is

85. 15:53

    open weights models are kind of lagging behind by three to six months compared to frontier models,

86. 16:02

    which means now we already have models in the open that are smarter compared to Sonnet 4, which I already used in library level development.

87. 16:17

    How long will it take for those open weight models to become good enough to be used in our daily operations? I don't know.

88. 16:28

    It's just one thought that lately I have more and more, especially because, well, Anthropic is putting arbitrary restrictions on how we use their models, so I don't really wanna use Anthropic models.

89. 16:42

    OpenAI is good for now. Who knows what are, what they're gonna do in a, in a year or two.

90. 16:51

    And I like open source, of course. Okay. I don't have a Git repo created. Create an empty

91. 17:01

    Git repo. And by the way, if you have questions, if you wanna interrupt me, this is supposed to be interactive. I'm, I'm here to entertain you for another hour and a half.

92. 17:12

    Initialize Git repo. Okay, this is done. It's amazing that using GPT 5.4 with open code would create by default a Claude.md. [laughs]

93. 17:29

    I think, like, this is, this is from, from Bun. Yeah, let's trash this.

94. 17:35

    It absolutely has nothing to do with this recent leak [laughs] of Claude. [laughs]

95. 17:42

    Perfect marketing strategy. Plus they, they've said it was an Ap- April Fool, and two days after they announce Mythos as the new model.

96. 17:56

    Okay. Uh, create a src and test directory.

97. 18:07

    Let's see what created. Yeah, it types Bun.

98. 18:14

    Bundler mode noemit. That's fine. Strict Skip lib check, that's fine. Implicit override.

99. 18:26

    That's good. Yes. Uh, also actually move the

100. 18:36

     files in the proper directory. Moving the entry file.

101. 18:54

     Good. Seems smart enough. Runs a basic smoke test. Okay.

102. 19:04

     Okay, so now that's a good starting point.

103. 19:09

     We verified with bun run test, bun run type check. Good. Uh, we want to add Effect beta.

104. 19:21

     We're gonna use Effect v4. It's not yet released for production usage, except he uses in production already. So if I have any problem, I'm gonna ask you. It's fine.

105. 19:35

     It's the small library?

106. 19:37

     Yes.

107. 19:38

     It's small.

108. 19:38

     It's Effect Small. Small because it used to be small and evolved to become bigger.

109. 19:46

     Still very, uh, very thin in bundle size.

110. 19:53

     Okay, one percent, fourteen K. There's plenty of context left.

111. 19:59

     Uh, want to add Effect beta, and we want to use

112. 20:07

     effect vitest to write the tests.

113. 20:13

     Trust the LSP and-

114. 20:15

     I will, I will. That's next. That's next.

115. 20:19

     And speaking of that, I wanna try to use the TS Go version of it.

116. 20:30

     Now, I never used this, so...

117. 20:33

     I use it. It's totally ready for production.

118. 20:40

     Well, the point is I haven't used it, so I don't know how to use it.

119. 20:47

     Uh, let's set up as the compiler for... As the type checker. Check the README

120. 21:05

     and set it up. Not sure if this is gonna work or not.

121. 21:15

     I'm not sure how it works with VS Code, but you might want to install the

122. 21:20

     TS Go TypeScript preview extension if you want-

123. 21:25

     Oh, the, the actual base compiler

124. 21:28

     ... 'cause you can use the package repository, uh, and you can use o- only TS Go version for both the VS Code plus the language service and-

125. 21:38

     Yes. I, I don't know if Mattia-

126. 21:41

     Effectful Technologies Inc.

127. 21:47

     Allowed-

128. 21:48

     TypeScript, but it doesn't really use it anymore.

129. 21:51

     Yeah, it does... The, the point is it does not use it, and

130. 21:57

     we could just do an alias install, so install TypeScript as something else, but I'm not sure if

131. 22:08

     he did it. Maybe let's follow the normal, the normal practice. Let's install TypeScript Go instead of

132. 22:22

     TypeScript. Would it be able to do this? Who knows.

133. 22:37

     Usually using the TypeScript Go normal one is that it had a difficulty to use the language service together with

134. 22:46

     the original TS Go I think. Well, we will find out.

135. 22:53

     We will find out. Is this the package?

136. 23:11

     No, I don't think this is the package.

137. 23:18

     No.

138. 23:18

     No. I think they just stole my crypto wallet. [laughs] Except I do not have one, so...

139. 23:35

     Check from here. Oh, [laughs] the NPM, the NPM package name typescript-go is only a placeholder security package,

140. 23:51

     so I use the real preview compiler that provides the TS Go binary. Well, that, that was probably a good idea.

141. 23:58

     Uh, script TS Go no emit. Uh, let's see.

142. 24:05

     Bun exec. TS Go not found. Okay. Uh,

143. 24:16

     okay, type, type check. Uh, bun run typecheck

144. 24:26

     Okay. Set up VS Code to use TS Go,

145. 24:34

     vtsgo-lsp. Will it work? Maybe.

146. 24:43

     Let's see. So there's two settings it needs to set. It has to set it to TS Go enabled and also to the local. I mean, the local only if you want to use, uh, the native service.

147. 25:02

     That's all you need, just the configuration.

148. 25:05

     Yes, that, that I need.

149. 25:09

     It's probably not installing it.

150. 25:15

     There we go. Native preview. That's it. Okay, I did install that.

151. 25:27

     Do I need to reload the window? Most likely.

152. 25:36

     TS Go.

153. 25:40

     Okay.

154. 25:41

     Okay, maybe it worked. Then let's go here and...

155. 25:49

     You can use AI. I think you can just tell it to point it to the repo and tell it to set this up.

156. 25:55

     Yeah, I should be able to do that.

157. 25:59

     Yeah.

158. 26:01

     But also there is a nice, uh... "Will not be loaded if files are specified in command line.

159. 26:26

     No config to skip this error." What?

160. 26:29

     Feed it, feed it to the agent. Using the market.

161. 26:43

     I'm gonna feed it to the agent in a minute.

162. 26:55

     It's

163. 27:25

     Bun that gives issues. Probably, maybe not. Yes, it is Bun.

164. 27:36

     Then let me stop this. Uh, select the TS config to configure this one. This other is a package.json. Installing dev dependencies.

165. 27:51

     Select all. What is this? Uh.

166. 28:16

     That's VS Code. That's fine. Okay, this needs a lot of work.

167. 28:38

     Uh, do we have the effect pattern installed? Oh, gosh.

168. 28:52

     Where is this coming from? Who knows? [clears throat] Okay, okay, okay. Uh,

169. 29:01

     Bun install. Okay, that's installed. Uh, let's see if it catches anything. Import effect from effect.

170. 29:20

     From effect. 100. No, that's a dangling effect. That should be...

171. 29:35

     You need to do the post install for the language service to be connected.

172. 29:42

     I think I've done that. You mean the prepare one?

173. 29:47

     Yeah, that's the one. Mm-hmm.

174. 29:50

     Yeah, I did. Um, maybe I need to reload-

175. 29:56

     Yeah

176. 29:58

     ... reload after that. Yes, that was easy. The Windows solution just restarted.

177. 30:07

     Okay, so we have it. Uh, and now,

178. 30:17

     now we want to... We have some diagnostic severity to suggestion, warning, uh, and so on and so forth For AI, we would like to turn everything into an error so that the, um,

179. 30:39

     the LLM cannot, cannot pass, cannot accept code that has any remote resemblance of an error. So this is,

180. 30:53

     uh, project where we will use AI a lot. We want all diagnostics available for--

181. 31:08

     to be set to error. [door squeaking]

182. 31:51

     I said I should switch from TS Config. Whatever, fine.

183. 32:03

     What is the model doing? Wow.

184. 32:37

     So it did update the TS Config. It did not.

185. 32:45

     Oh, I'm updating the TS Config. Okay.

186. 32:56

     Do you want to use the Effect Solutions?

187. 33:00

     No. No. No. Uh, and that, that's another interesting point. Uh, the Effect Solutions, uh, there is a website called effect.solutions.

188. 33:14

     It's a really nice website. Uh, Kit Langdon did this.

189. 33:21

     And it's kind of a quick start to use Effect in an, in an AI project, and it does install the language service and strict policy defaults and so on and so forth.

190. 33:33

     But then it uses, uh, a CLI to give the model access to the Effect repo,

191. 33:41

     and the model needs to know how to use the CLI.

192. 33:45

     Mm-hmm.

193. 33:46

     So it's kind of a dog biting its tail.

194. 33:50

     I think the CLI, it uses more for the... to access basically what I was in this documentation. For what's in the website that is exposed for CLI for Effect Solution, not the repo.

195. 34:00

     Yes.

196. 34:01

     They use this, uh, link.

197. 34:05

     Yes, but-

198. 34:06

     Links or something

199. 34:06

     ... there, there are some markdown files, but it, it doesn't work as well.

200. 34:15

     Mm-hmm.

201. 34:15

     And if you actually read at some point, it says you should actually just clone the repository.

202. 34:24

     Okay. This, this looks exactly what I had in mind. So we have all the diagnostics set to error,

203. 34:32

     which is good. It's exactly what we want.

204. 34:37

     Reload window. Okay. I also want to-

205. 34:47

     Format on save

206. 34:52

     ... format on save to true just because it's annoying otherwise. Okay. We're at a good point. Uh, commit

207. 35:03

     current. Commit current. Oh. Before I forget. Now I wanna add effects-more as a subtree.

208. 35:27

     Okay. It's committed. Good. Now create a .repos folder and add as a git subtree without history,

209. 35:43

     squashed, uh, in repos effect. Who knows if it's gonna be able to do it.

210. 36:04

     Repos. At least it did okay here. Why is it trying to...

211. 36:14

     Okay. Okay. We have it. Let's just check git log. Yep. It did add it.

212. 36:32

     Okay. And now we are at the point where we can start to do our research. For example, we said we want to create an HTTP API.

213. 36:44

     I would clean up this, open a new session to avoid context pollution.

214. 36:52

     You have access to the Effect repository at repos. Actually, let's do something else before we want...

215. 37:06

     We wanna set up an agents.md. Set up an agents.md listing the commands available, like bun run type check,

216. 37:21

     and specify that you have access to the Effect repository at repos Effect, and you should use that to extract

217. 37:36

     best practices, look at how things works, et cetera.

218. 37:51

     Now, the agents.md, now we're gonna get an initial prototype. As you work in the project, you're gonna evolve that. You're gonna add more commands to it. You're gonna add rules when you spot that some bad patterns are created in code.

219. 38:11

     One thing we have not set it up yet is a linter. Uh, linter is gonna be an essential piece of the back pressure loop that helps the model drive in the right direction.

220. 38:25

     If you want a, a kind of fully working setup, uh, I have a repository of mine that I use for fun, which is called Accountability. Uh, in this repository, you can find, um, a lot of things, but for example, I have an ESLint config with a lot of

221. 38:50

     custom rules, and those are, like, arbitrary. For example, I don't want the model to do an explicit type assertion on things. I want the model to use schema to check for the shape.

222. 39:06

     I have rules prohibiting the usage of X as Z.

223. 39:11

     I have rules prohibiting the usage of any, of unknown.

224. 39:16

     Basically, I'm trying to avoid the model to do dumb stuff that I realized it was doing in my code.

225. 39:24

     Any is, like, easy to disable, but type assertions, I think no, right? Like, you can't tell TypeScript config to-

226. 39:30

     No

227. 39:31

     ... ban as, yeah.

228. 39:32

     Yeah. The same for unknown.

229. 39:34

     Yeah. Unknown, yeah. As unknown or something, yeah.

230. 39:37

     And the funny thing is, initially, I banned unknown because I wanted the model to not do as unknown as X. It found that never is a bottom type, so you can do as never as X. [chuckles]

231. 39:50

     I was like, "Okay, then I, I'm gonna ban as," and, and now it's doing better.

232. 39:57

     Uh, okay. Let's see what it created. Okay. This is... Sure.

233. 40:05

     Use Bun. Okay. Available project commands. That's fine. Test watch. Hmm. This is gonna create issues, I already know,

234. 40:17

     'cause the model is gonna try to run this and get stuck. Same with dev servers.

235. 40:25

     Effect reference repositories, good. Look at the... for repository-specific guidance. Okay, that's enough of a start. Mention in the agents.md that you should never, ever try to run commands,

236. 40:48

     commands in watch mode. For example, you are not allowed to run

237. 41:01

     or a dev server. Otherwise, it's gonna try to run the dev server as the first thing and get stuck.

238. 41:18

     Okay. What I like about OpenAI models is that they are way more concise compared to Anthropic models. The same task with Opus would have probably wrote two hundred lines of agents.md.

239. 41:34

     But that's good. It's enough for... It's enough as a start.

240. 41:39

     So we are back to square zero. We said we wanna create an HTTP API. I know nothing about Effect,

241. 41:49

     so I would like to create an HTTP API that should

242. 41:59

     have OpenAPI documentation and typesafe client generated by default. Explore the Effect repo for patterns on how

243. 42:21

     to do this. Save your research into patterns/httpapi.md

244. 42:39

     Ask me any question you need. Again, I'm, I'm starting from the perspective that

245. 42:53

     I have no idea how to do this in Effect

246. 42:59

     Do you tend to use, uh, plan mode in Open Code, uh, or not really?

247. 43:05

     No, I, I find plan mode to be...

248. 43:10

     Like, the issue with plan mode is that the, the model has crippled access to tools,

249. 43:18

     so it cannot easily do the same things that it does outside of plan mode.

250. 43:24

     Um, so not... I don't make heavy usage of it. I usually do what's called spec-driven development, in the sense that the first task I do with the model is I discuss with the model how to create a spec for something.

251. 43:40

     Then the spec is persisted as a markdown file, which is effectively my plan,

252. 43:47

     and I tell the model then to implement, uh, that. Usually, the, the second step I do in a Ralph loop, 'cause you've seen I already restarted Open Code a few times to clean up the context window.

253. 44:01

     Doing this manually is boring, and you u-usually end up reusing the same context window for multiple things, and it's gonna just deoptimize the model at some point 'cause the context window is limited.

254. 44:16

     You're gonna push a lot of information in, and the earlier information is gonna confuse the, the model for the later information. So I use a very simple bash script that

255. 44:29

     tells the model, pick up a small task, implement a small task, and then exit, and I run that in a loop.

256. 44:39

     It's funny how with, with AI, many times less is more.

257. 44:44

     You can have very complex architectures around context management and so on and so forth. At the end, the dumbest thing ever ends up working better.

258. 44:55

     And we are doing research by ourselves, and it, it looks like

259. 45:02

     there's actually very good margins of improvement, uh, by reducing the number of tools that the model has access to. For example, we have been experimenting with a coding agent that has a single tool call, which is called execute, and it can execute arbitrary TypeScript code, including calling bash through TypeScript.

260. 45:26

     And in that scenario, the model doesn't even have access to a patch. It cannot change files directly. It has to write a TypeScript file that changes the f- the code, and then it ends up doing TypeScript transformers, AST-based transformations.

261. 45:44

     It's like, it's fantastic how you reduce the things that the model can do, and it, it does better. So let's see. Save the research to HTTP API. Good. Main conclusion.

262. 45:56

     For this repo, the strongest default Effect pattern is to define the shared HTTP API. You're absolutely right.

263. 46:05

     Derive OpenAPI from it. Mount the docs. Okay.

264. 46:18

     OpenAPI generation only when you need generated client artifacted. We don't know. We don't, we don't need that. One question before I implement anything further. Do you want the primary pattern here to be shared HTTP API with HTTP client.make?

265. 46:38

     No. Uh, I am fine with a shared HTTP API.

266. 46:50

     I don't need a committed client in the repo itself.

267. 47:00

     Let's see what it did here. For this workshop repo, the best part... Okay. This give you relevant upstream files.

268. 47:12

     Good. It looked at tests. Nice. Okay, this looks like a decent

269. 47:31

     enough. We should probably tell it what we want

270. 47:36

     to do. Um, but this is just generic patterns that we're gonna use as reference.

271. 47:44

     So list the files in patterns in the agents.md so the agent has context of their

272. 48:01

     existence. Model does not care about grammar. And I feel like w-

273. 48:11

     many people, uh, raise the point that a model is not good at something if it d- if it doesn't do good by default.

274. 48:22

     I don't think there's anything more wrong with that statement. The model is good when it can operate a large scale code base using patterns, and it doesn't fail at scale.

275. 48:36

     The zero to one problem is not really... It's a problem for the first ten days or ten hours, depending on what you're building.

276. 48:45

     And Us programmers, if our job is not to write code,

277. 48:54

     our job should be to set up the repositories in ways that the models can act good on it. So what I'm doing now is like most of what I do when I operate a coding agent at scale in a code base, even if the code base has no concept of AI.

278. 49:17

     Like if I start in a project that is brownfield, code base existing from five to 10 years, no context, set it up, the first thing I do is let the model explore the code.

279. 49:28

     Clone the main libraries that are used. If you're using a framework like TanStack or so on and so forth, clone the code of TanStack Router. If you're using Svelte, clone the code base of Svelte.

280. 49:44

     Ask the model to generate best practice files and so on and so forth. Once you have all of it, the model is gonna be much more, uh, efficient.

281. 49:54

     So now that we have a little bit of context on HTTP APIs, we can start implementing one. Uh, I do wanna check something quickly 'cause I'm using Bun and I'm using Vitest.

282. 50:09

     Uh, there's a Vitest run. Does Vitest run actually uses Bun as the runtime, or does it use Node?

283. 50:28

     'Cause if I recall, there was a flag that I had to pass to Vitest to let it use Bun, and I don't want our test setup to defer from our, uh...

284. 50:46

     What is it doing? Uh, no. Add to Vitest [laughs] that it should ignore anything in repos. It was running the effect tests that it found.

285. 51:11

     Yeah, there was no Vitest config whatsoever. Good.

286. 51:29

     Add to the test something that uses a Bun API.

287. 51:42

     I feel like I did it here, so I should have

288. 51:51

     test Vitest run. No? Okay. Was I using Node? Probably.

289. 52:13

     No, it... You should expect it to be defined.

290. 52:21

     'Cause now it did one of the classical mistakes. It had to make the test pass. It changed the test to make it pass.

291. 52:41

     No.

292. 53:14

     Wow.

293. 53:41

     Okay. Okay, it did it. Let's now begin our HTTP API implementation.

294. 53:57

     So we want to implement an HTTP API following the patterns at

295. 54:10

     patterns HTTP API. We want the API to,

296. 54:19

     um, expose a to-do functionality where you can,

297. 54:31

     one, create to-dos. Description. Title, description. Ah Two, update todos.

298. 54:54

     Change title, et cetera. Three, flag a todo as done or not. Four, list todos.

299. 55:13

     Uh, should have done something else. Discuss the plan with me and create a plans todo API.md.

300. 55:29

     So here I'm telling the LLM to read the pattern file that we created before, where it's gonna gather generic knowledge about the Effect ways of doing things.

301. 55:44

     It still has access to the original code base of Effect if it wants to, but now I'm creating a specific plan

302. 55:54

     to implement the API that I would like to, uh, that I would like to implement.

303. 56:30

     Drafting a plan. Okay. Creating a todo shape.

304. 56:55

     That's fine. Initial storage strategy. Okay, let's do something different. For storage, use Effect SQL and, um, SQLite

305. 57:18

     store. Uh, explore the Effect repo for how to do that and create patterns,

306. 57:33

     uh, patterns SQL.md. Okay, I realize we need a persistent strategy, and I don't have a persistent strategy.

307. 57:52

     I know that Effect has some SQL thing,

308. 57:57

     and again, I'm using the same process where I first generate some patterns for it.

309. 58:08

     And this is also useful because you may wanna use something from Effect, but you may not wanna use everything from Effect. So if we were to push all the patterns in your repository by default, you would end up using everything from Effect even if you don't want to.

310. 58:25

     This is kind of self-select, uh, so you can pick and choose whatever you wanna use.

311. 58:34

     Especially in brownfield projects, this is very important because you don't wanna refactor everything you already have.

312. 58:45

     For example, here I could have picked Drizzle to do the persistence just as well.

313. 58:55

     What are your thoughts on potentially, uh, using GitHub trees or something for patterns and, uh, other reusable components, even the agents.md maybe, so that we can use when we start with a project immediately and don't-

314. 59:09

     Most likely we're gonna develop some kind of CLI where

315. 59:15

     you can prefetch some patterns that are already available and still let you pick and choose. And we also want to automate this kind of process of exploring something, create patterns out of it, 'cause the patterns that we have as best practices might not exactly fit your needs.

316. 59:37

     So you would still maybe update them as a, as a second step.

317. 59:42

     Also, the model I use may not be as good as the one that you use, like it also happens. Uh, for example, like the PRs that are submitted to Open Cloud, you go and you read the code, like some of them are just like use the free version or, or like the box version of any model.

318. 59:58

     And the code it generates, all of the documentation, everything it generates is not as good as the one that you use. So it's... I was also going to ask like, like what if the, the framework library authors are providing not like skills like or like solver mode, but this kind of like pattern libraries like Effect solutions, but

319. 1:00:15

     officially like, uh, distributed by the package co-located?

320. 1:00:22

     I feel like generally it's a good idea, but there are some caveats to that. For example, even the agents.md standard-

321. 1:00:30

     Yeah

322. 1:00:31

     ... is kind of not a standard because the way you prompt Claude and the way you prompt GPT is different. For example, you've noticed I never wrote anything in uppercase.

323. 1:00:45

     Uh-huh.

324. 1:00:46

     If I were, if I, if I was using Claude, I would write a lot of stuff in uppercase The reason is GPT gets scared if you scream at them, uh, at it.

325. 1:00:57

     I don't even know how to call the model. And, uh, if you scream at it, it's gonna de-optimize and then be passive and, like, agree on everything. That is not what you want.

326. 1:01:11

     Uh, with Claude, if you scream at it, it's gonna pay attention to that specific sentence.

327. 1:01:18

     So that comes also in these shared patterns. I feel like the patterns should be almost generated with the model you use versus being off the shelf. Now, we can do that for, like, the top three frontier models.

328. 1:01:35

     All the GPT family is very similar, 5.3, 5.4, 5.2. There are-- There are not so many differences. Opus, Sonnet, and Haiku are also very similar. So ideally, we can

329. 1:01:50

     have the CLI where it, where it says, "Which model do you use? Okay, I'm gonna optimize the context for this versus the context for that." And it's annoying because you would obviously like to have a standard. [laughs]

330. 1:02:03

     I wouldn't want to maintain it, uh, as well. So yeah.

331. 1:02:06

     Yes. Yes. [laughs] It's very painful to, to maintain this stuff.

332. 1:02:11

     Maybe you can treat it a, as a derivative so that you have, like, the curated patterns for certain things in a non-specific way, and then by each model, you turn them into useful patterns.

333. 1:02:24

     Our approach is to make the code as good and self-explanatory with examples and everything that any model you use can generate those.

334. 1:02:34

     Mm-hmm.

335. 1:02:34

     And then the CLI would generate them on the spot-

336. 1:02:37

     Right

337. 1:02:37

     ... for the model you use. That's one approach. It may fail, and in six months we provide patterns for everything and just tell you, "Please use either one or two."

338. 1:02:50

     Another very interesting argument is fine-tune an open source model to use effect patterns by default.

339. 1:02:57

     We thought of that.

340. 1:02:59

     It's like a sub-agent to audit and, like, make your code base ready for Effect v4.

341. 1:03:06

     Kinda. Okay, let's see. Uh, update the HTTP. No.

342. 1:03:18

     If you want the next step for me to update... Yes,

343. 1:03:23

     do that. This is the annoying part of GPT models. They are gonna ask constantly for input from you to continue. Opus would have just done it.

344. 1:03:36

     But sometimes it does it wrong, and you have to, like, do it three times, and then your session will be-

345. 1:03:41

     That's why I use GPT 5.4. [laughs] Well, I'd like some sort of fusion and, you know, an inbred fusion of Anthropic models and OpenAI models so that it doesn't ask me all the time.

346. 1:03:55

     'Cause GPT usually, especially in complex tasks, takes its time, but at the end, the output is good. Uh, with Opus is right. Sometimes it, it likes to take these shortcuts, and the funny thing is, if you let one slip,

347. 1:04:15

     it's gonna repeat. Like, if you let one any slip in your code base, and if you have Opus, it's gonna do as any all the time. It's like, "Oh, I can do this.

348. 1:04:24

     Let me do that for everything. I need this to compile. Let's remove the code."

349. 1:04:29

     And you can then do the linter.

350. 1:04:31

     Yes. That's why in, in this project and in accountability, I was using Opus, and I have a lint file of thousands of lines of code to prohibit any shortcut.

351. 1:04:44

     I can start implementing this next? Yes, please.

352. 1:04:48

     I feel like we've spent enough time. Yeah, let's see what it does.

353. 1:05:07

     See, it's, it, it's correctly looking up in the Effect repo

354. 1:05:16

     in the AI docs for, uh, ideas. This most likely is gonna take a little bit,

355. 1:05:37

     which is positive.

356. 1:05:39

     Do you use anything for schemas so, so to support target scheme generation, uh, when you use AI

357. 1:05:51

     to model your domain?

358. 1:05:53

     Kind of.

359. 1:05:56

     It's a bit more...

360. 1:05:57

     In some projects, it was using schema by default, and I didn't need a lot of, uh, back pressure for it.

361. 1:06:09

     Sometimes... Yes, one example is the, is, is the rule in...

362. 1:06:24

     In accountability, I have this, yes, lint rule,

363. 1:06:31

     SQL. Custom ES lint rule to ban SQL type because it would write an SQL query. It would write an interface, and it would just...

364. 1:06:49

     This is the exam- exact same thing as casting.

365. 1:06:55

     And I had to ban this pattern fully.

366. 1:07:00

     And it's using type parameters with SQL template literal provides no runtime validation. Use SQL schema, find one. And you see that the,

367. 1:07:09

     the rule ends up suggesting to use SQL schema.

368. 1:07:18

     So I'm more or less just watching what the model produces, and if there's something I don't like,

369. 1:07:28

     I end up writing lint rules to prohibit that specific pattern.

370. 1:07:33

     For example, in, in schema, many times it would, for example, have a user ID as a string,

371. 1:07:43

     and then it would have an- another ID as a string, and you would, of course, have no type safety whatsoever, and the code would try to pass one into the other.

372. 1:07:53

     So I would force all identifiers to be branded types, and I would then prohibit the usage of typecasting 'cause otherwise it would do, like, this requires a user ID.

373. 1:08:06

     Let me do as user ID. And it's like, eh, it's pointless. [laughs] You should validate the data. So I would ban, uh, the usage of as and force them to use, um, constructors.

374. 1:08:19

     So instead of doing hundred as user ID, user ID.make

375. 1:08:25

     or prohibit usage of constructors in places where you should do validation. For example,

376. 1:08:33

     one case where-- that, that I found was it would do the, the API layer as plain strings

377. 1:08:41

     and then use constructors inside the handler to create the objects, defeating the purpose. Then I would write rules for the model to write validation directly in the schemas so that I was basically saying, "If you use a constructor inside the handler, most likely you're, you're wrong.

378. 1:09:03

     You should improve the starting schema to provide the validation at the edge." It's kind of babysitting a junior developer, uh, with a knife running through the kitchen

379. 1:09:16

     instead of a kid running through the kitchen with a knife.

380. 1:09:20

     Okay, this is still going.

381. 1:09:23

     Michael, you said that you usually use both, uh, Codex and, uh, Claude. How do you, like, decide, like, when I should use Codex and when I should use Claude?

382. 1:09:35

     I... Both models are exceptional. Sometimes one model drives you nuts,

383. 1:09:44

     and you try the other. There's not much of a rule. Uh, lately, I tend to use more OpenAI models because I don't really like to be restricted on the harness that I can use.

384. 1:09:59

     What do you mean by harness?

385. 1:10:01

     Uh, the CLI itself. Uh, I wanna use open code. I wanna use my own TypeScript files that interact with the AI SDK natively, and I'm prohibited from doing that from Anthropic.

386. 1:10:19

     So up until a few months ago, when this was allowed, I would use mostly Opus.

387. 1:10:27

     When they enforced their policies against, uh, open code, I switched to OpenAI models, and now I'm most of the time just using OpenAI model- models.

388. 1:10:39

     There are some small edge cases. For example, when you do UI, Opus is much better than Codex.

389. 1:10:48

     So for... There are some specific things where one is clearly better than the other, but for most of the tasks, they are, they are the same. I just had some experience, for example, where GPT

390. 1:11:06

     thought for half a day on a, on a, on a bug that I had and went nowhere, and Opus one-shotted the solution. But I had the opposite experience too.

391. 1:11:15

     So [chuckles] it's very hard to know, uh, which one is which.

392. 1:11:21

     You can fall back, you know, like, uh, start with one, and if it fails or takes too much time, then kill it and, uh-

393. 1:11:28

     You could

394. 1:11:29

     ... dramatically try. [laughs]

395. 1:11:30

     You could. You could. You definitely could. Okay, let's see what, what, what is this creating. Uh, okay. It created an SQL client.

396. 1:11:42

     The layer looks correct. Uh, has migrations. It decided to

397. 1:11:57

     inline the migrations. Okay, that's a valid choice.

398. 1:12:15

     Okay. It correctly provided the SQL live layer

399. 1:12:24

     to the migration layers. This feels like duplicated.

400. 1:12:39

     There is clear duplication between these two.

401. 1:12:43

     I use, uh, clip for this stuff, like when it creates multiple things that are doing the same thing, and it's not importing it to another thing. It's also a way to fight with the scope

402. 1:12:58

     Sometimes you refactor and it leaves one code in place, and it's like never exported, never used in the same file. It catches it up quite well.

403. 1:13:08

     Okay, good to know. Uh, we are... In our experimentation, another thing we're doing is we're using semantic code search.

404. 1:13:18

     Oh, yeah, yeah.

405. 1:13:19

     'Cause we've noticed that a lot of times the model re-implements the same features because it doesn't find it.

406. 1:13:25

     Yes.

407. 1:13:26

     And with semantic code search, it finds it.

408. 1:13:33

     But okay, here's there's a duplication here. I'll probably tell it that there is a duplication at some point.

409. 1:13:42

     Wanna check the API. Exactly, you see it's using plain strings for identifiers.

410. 1:13:50

     So one of the future things that we might wanna, uh, that we might wanna do is to tell it to use branded stuff.

411. 1:14:04

     Okay, toDoNotFound, it added a schema annotation to flag that toDoNotFound should be a four04.

412. 1:14:22

     This looks decent. Uh, I don't understand why it sometimes creates

413. 1:14:29

     structs instead of classes. I personally prefer to use classes.

414. 1:14:36

     So I would, in the future, um, either create a best practice to prefer classes or, depending on how strict I want, create a lint rule to prohibit usage of schema.struct in specific files and stuff like that.

415. 1:14:55

     For now, it's obviously, it's fine. Doesn't need to...

416. 1:15:00

     Is it not something already part of the LSP, uh, to prefer classes for schema?

417. 1:15:07

     Not sure. There might be, but it's not flagging anything here, so... And the LSP is on.

418. 1:15:19

     Lint all the files. Lint with what? We don't have a linter in place.

419. 1:15:32

     Good point. We also do not have, um, formatter in place.

420. 1:15:38

     Uh, let's ignore for now. Uh, let's see.

421. 1:15:58

     Okay. Client with a base URL. That's good.

422. 1:16:07

     We have the live handler. Server. Index is just exporting everything.

423. 1:16:25

     The index.ts should probably run the server instead of exporting everything.

424. 1:16:40

     Do that in a condition checking that the file is main, so it doesn't

425. 1:16:55

     run when you import the file. I also created some tests.

426. 1:17:13

     Do you have some global agents MP file?

427. 1:17:16

     No. No. What is, what is doing here?

428. 1:17:33

     It created an arbitrary with HTTP to run an effect.

429. 1:17:44

     Make test HTTP live. Okay, it's one way.

430. 1:18:00

     Do the test actually pass? Bun run test. I'd be surprised. Wow.

431. 1:18:09

     Uh, is there a start command? Add a start command to start the API server, and tell me where to find

432. 1:18:26

     the OpenAPI docs. Okay.

433. 1:18:41

     It really likes this pattern. As a future thing, I would probably just tell it to use it.layer instead of

434. 1:18:51

     using the withRepository and with thing. But let's see if, if at least it works

435. 1:19:03

     Bun run start. Good. Way of listing todos.

436. 1:19:32

     Let's check the OpenAPI. Good. There is an OpenAPI

437. 1:19:37

     spec created. This looks decent as a first.

438. 1:19:48

     It shows the schemas properly. Good. Okay, then let's... Let me...

439. 1:20:01

     It did create a database here. Let me maybe gitignore

440. 1:20:09

     the full DB. DB, todos.db.all, todos.db.

441. 1:20:20

     Todo, I guess, because-

442. 1:20:26

     Todo, you're right.

443. 1:20:36

     I want to save it like that.

444. 1:20:38

     Yeah. No longer able to write anything by hand.

445. 1:20:51

     Yes. Okay, let's actually clean up the tests a little bit.

446. 1:21:07

     So clean, uh... You see, I'm fooling myself in wanting to use the same session over and over again.

447. 1:21:18

     That's when rough loops are really useful. We created a lot of mess.

448. 1:21:29

     You.

449. 1:21:30

     You created a lot of mess in tests.

450. 1:21:35

     Clean up everything. This should be the cleanest code you've ever seen, not like the crappy Python code you've been trained on.

451. 1:21:58

     Do not use patterns like... Simply use it.layer with layer.

452. 1:22:20

     And put utilities in their own folder. No offense to Python developers, of course.

453. 1:22:30

     Probably a better approach is to create patterns for it.

454. 1:22:36

     Probably. Now, now I'm winging it. I'm gonna see if it's able to do it. Uh, if it does, once it's done, I'm gonna create a pattern from it. But yes, that would have been a good idea,

455. 1:22:50

     which is why automating the process is very important, 'cause we are lazy. Like now, I was so lazy that I didn't wanna create

456. 1:23:01

     a pattern for it. Maybe I'll use test utils layers. Maybe, maybe.

457. 1:23:12

     So c- can you, can you explain what's bad, the, so bad pattern there? Is the... Yeah, I'm not very familiar with the-

458. 1:23:17

     Oh, the bad pattern?

459. 1:23:18

     Mm-hmm. Yeah.

460. 1:23:19

     It basically created a, a function to provide a layer

461. 1:23:26

     to an effect. It built the layer manually.

462. 1:23:32

     It wrapped everything in Effect.scoped, which is gonna close the layer once this is done.

463. 1:23:41

     And my guess is that it did this because this pattern is actually used to test some layer internals in the code base, but it's completely unnecessary here.

464. 1:23:55

     But if you look at the file, even without knowing

465. 1:23:59

     details of effect, it stinks. There's... Something's not right. Now it cleaned it up, so

466. 1:24:13

     when you see something that isn't e- doesn't look right, usually just ask the model, "Why you did that? Is there any alternative?"

467. 1:24:23

     And in this case, I knew that to provide a layering test, we should just use it.layer,

468. 1:24:29

     so I kind of skipped that. But in reality, I would have... If I didn't know this, I would have d- discussed with the model that I didn't like to see that repeated thing all over.

469. 1:24:44

     And sometimes it's necessary. Sometimes you're wrong and the model is right. That's the way to do it.

470. 1:24:52

     In this case, it was completely unnecessary.

471. 1:24:55

     Do we have describe.layer as well or not really?

472. 1:25:00

     Um-

473. 1:25:02

     It's [inaudible]

474. 1:25:03

     ... no, I think we have it. the... We have it.describe

475. 1:25:09

     So you would, you would do it.layer as a top

476. 1:25:14

     thing, pass the layer, like layer, whatever. Then

477. 1:25:24

     in the closure, do it.describe. Could probably also add an it.describe as a short.

478. 1:25:37

     Models don't care about verbose code. Why should we make it less verbose?

479. 1:25:43

     But does it help do any cleanups between tests?

480. 1:25:46

     Yes.

481. 1:25:46

     So if we do it in the top level, it will poison the other tests.

482. 1:25:53

     Yes. Yes. But you can do it per test still.

483. 1:26:01

     Now, it does poison the other tests.

484. 1:26:11

     I mean, the other alternative is that you just, uh, provide-

485. 1:26:13

     The other alternative is you provide it.layer at every test.

486. 1:26:19

     The reality is whenever you're using a database, in this case it's SQLite, so the argument is kind of moot. But if I were to use a Postgres in a project where you have hundreds of file or hundreds of tests, spinning up a Postgres instance per test is gonna make your test runtime-

487. 1:26:40

     Two days

488. 1:26:40

     ... two days maybe. So usually what I end up doing

489. 1:26:45

     is I end up making tests that are... that can run, that do self-cleanup. Like for example, I run a test within a transaction, and I roll back the transaction as soon as the test finishes so that they are kind of atomic by the fact that they don't leak that.

490. 1:27:07

     It would be another pattern that we can tell the, tell the model to, to do. It would be a matter of creating the transaction and the rollback.

491. 1:27:17

     But there's, there's alternatives and...

492. 1:27:23

     So, and how, how does the model know about the, the Effect library, I mean, related to... Is it using the text seven or something like this? Or is it, are you just relying on the model's knowledge about the library?

493. 1:27:33

     No. We added the re- the Effect code base-

494. 1:27:37

     Ah, I see

495. 1:27:37

     ... in a repository folder. We created an agents.md-

496. 1:27:42

     Yes

497. 1:27:43

     ... that references the, the Effect repo. And then for the features we wanted to use, we asked the model to create patterns by looking at the repo, investigating how things are done in the repo

498. 1:28:01

     as kind of general knowledge. In this case, we did one for SQL, we did one for API. Now the good point is in this session we have best practices about testing, so let's create patterns/testing.md.

499. 1:28:21

     It should include all the best practices of testing Effect-based code, including usage of it.layer,

500. 1:28:41

     year, et cetera. And also update... I'm gonna queue that agents.md to reference all the patterns in .patterns.

501. 1:29:02

     And the next thing that you would do to automate the flow is, for example, OpenCode allows you to create slash commands. Claude Code allows you to do the same.

502. 1:29:14

     You optimize for slash new pattern, whatever you want, and, um-

503. 1:29:21

     Would a skill, for example, can discover it on its own or-

504. 1:29:24

     You can create skills and tag the skills.

505. 1:29:28

     Uh, skills are very useful for these kind of things. Uh,

506. 1:29:34

     I'm kind of against skills in general, not... For these things, they are ideal, but many people think that just by adding a skill, you're gonna make the model good at React.

507. 1:29:46

     You're gonna make the model good at Next.js. The reality is if you put a skill for every single Next.js internal, you're gonna pollute the context and not get anywhere.

508. 1:29:57

     So skills have a very good use case, which is this kind of use case, and I guess they are more general than slash commands. So I tend to do slash commands because I tend to use a single coding agent.

509. 1:30:12

     But definitely if you are, for example, in a team where everybody's free to use their own agent, maybe some people use Cursor, some people use OpenCode, some people use Claude Code, skills are a good baseline.

510. 1:30:29

     Let's see, patterns testing. Use Effect Vitest for all Effect-based tests.

511. 1:30:38

     Use it'll Effect. Use it.layer. Avoid custom wrappers

512. 1:30:45

     that call layer.build. This is a very specific rule. Now, a friend of mine told me whenever you, you, you read a rule book, a legal rule book, or you find those specific rules that are just like when you enter a pub and it's like, "Don't do skateboarding on top of..."

513. 1:31:05

     And you ask yourself, "Why does this rule exist?" Because somebody did that.

514. 1:31:10

     Why does this rule exist? Because the model did that.

515. 1:31:15

     Why this pattern? Okay. You see relevant files. They're all linked.

516. 1:31:22

     Yeah, they need to be, like, maintained. That's the file references that you store?

517. 1:31:28

     Yes, and there's a friend of mine who's writing, um, a linter plugin that checks for existing references.

518. 1:31:38

     Oh, yeah.

519. 1:31:39

     So whe-when you add, uh, when you change code, it runs the, in the CI and says, "Hey, this reference is broken."

520. 1:31:49

     Yes. Also sometimes because of relative path, they give this absolute path-

521. 1:31:54

     Yes

522. 1:31:55

     ... to the file, and this is your name. [laughs]

523. 1:31:56

     Yes.

524. 1:31:58

     So you just-

525. 1:31:58

     The full-

526. 1:31:58

     Yeah, yeah

527. 1:31:58

     ... [laughs] /home/, whatever. Yeah, yeah.

528. 1:32:03

     I guess another approach you can use is that you actually write tests for your patterns,

529. 1:32:10

     if you treat them as their own artifacts, and then you can keep evaluating them.

530. 1:32:15

     How would you write a test for a pattern?

531. 1:32:18

     With a program. Like, you, you have the pattern and then a test.

532. 1:32:23

     Oh, you mean actually write a file?

533. 1:32:25

     You use it. You use the pattern in the test and then evaluate the results of what you expect to be it work and compile and...

534. 1:32:35

     Feel like that could be a way. Sometimes the code that is inside the patterns is not really executable.

535. 1:32:44

     Mm-hmm.

536. 1:32:47

     I guess it, it has pros and cons.

537. 1:32:49

     Mm-hmm.

538. 1:32:49

     It's definitely an interesting idea. For example, maybe with a, with an additional tag like TS execute these

539. 1:32:59

     to flag which of the patterns you actually want executed or, like, references which files you want to be referenced. 'Cause sometimes it mentions files as examples. For example, if you write this feature, use the file called ABC, and that's not a concrete reference,

540. 1:33:20

     so you don't want your program to fail because it read that, um-

541. 1:33:24

     Maybe you can use it in just by

542. 1:33:28

     triggering, uh, have a command using your pattern then to actually write code as part of the test, and then you evaluate the output that it is exactly what you expect.

543. 1:33:38

     So the models, if they change, it can also-

544. 1:33:42

     That's more, more in the direction of evaluations, so evals.

545. 1:33:45

     You can do all kind of testing for, uh, patterns and skills and that sort of thing.

546. 1:33:49

     Yes. That's, at scale, that's very good. I found doing it on a per project basis ends up-

547. 1:33:57

     Yeah. Not on per project. If it's more like your domain, because you spend more time curating these things than actually writing code or eventually reviewing code. So maybe a same sort of quality controls as you would do for code would be useful to do.

548. 1:34:12

     What we are thinking of doing in the Effect repo is, for example, to have evals running once per day

549. 1:34:21

     and generating reports. So any time we do library changes or we add more docs, we add more examples, we see exactly if the outputs are, are better or are, are worse.

550. 1:34:34

     Sometimes in evals it's very hard. Like, even Anthropic a while ago wrote a blog post where the summary of the blog post is, "We don't really know when code is good or bad," because is, is more terse code better?

551. 1:34:54

     Depends. Is more verbose code better? Depends. There are some properties where you can say this is definitely better than not, like code that type check is better than code that doesn't.

552. 1:35:08

     Probably true. But when it comes to style, when it comes to, like, i- is this file

553. 1:35:17

     structure better than another file structure and they both convey meaning? You kind of need a human at the end to say, [laughs] "Yeah, I prefer this." And if you take hundred humans, you're gonna have an eighty-twenty split.

554. 1:35:31

     So we have the same problem now with defining Effect patterns 'cause we are running evals, and evals are kind of our opinion of what's good, and it's not really

555. 1:35:45

     an absolute truth. Uh, let's put it in, let's put it this way.

556. 1:35:49

     So, so, so do you have an LLM check that, that check, check for certain patterns? Like, how, how do you run these evals?

557. 1:35:57

     We have humanly written best practice code. [laughs] We have generated code, and then we have an LLM that matches and says, "Is this too different or not? Give it, give us a score."

558. 1:36:11

     And that's pretty much how you run the eval. Not very... Not a very nice way, uh, to run. But we're trying to figure this out because we are thinking of fine-tuning a model on top of Effect, and for the reinforcement learning part, we are gonna need to have good evals.

559. 1:36:31

     So it's part of what we are researching right now. Uh, there's no right or wrong answer.

560. 1:36:41

     If there was, all the models would perform, would perform the same 'cause [laughs] everybody would have the same evals. Everybody would have the same thing. But now we have all the patterns for what we want, so I feel like we are at the point of saying commit this.

561. 1:37:01

     I'm gonna create a repository and push it

562. 1:37:05

     so that at least you have access to it.

563. 1:37:13

     Gosh, I'm too big. New repository. Is it public?

564. 1:37:29

     Please choose an owner. Sure. Add origin and push.

565. 1:37:54

     Pushing the final repository, so hopefully [laughs] So we haven't got to the point of doing clustering and workflows.

566. 1:38:08

     Just sharing a few words about why you would want those aspects in your code.

567. 1:38:13

     This is a very dumb to-do API. One thing I wanted to add

568. 1:38:20

     would be authentication and registration. For example, when you have a registration, your process is usually write something in the database and then send an email,

569. 1:38:32

     or send an email code and wait for confirmation. Anytime you do two unrelated operation, there is no transaction between them, no database transaction between them, and your server may fail at any random point within your code.

570. 1:38:50

     So it's very hard to guarantee that the email has actually been sent, which is why many time in a registration procedure you see the sentence,

571. 1:39:01

     "If the email did not arrive in thirty minutes, please retry."

572. 1:39:06

     You retry for me. Why should I retry if I haven't received the email? That's the s... That's a symptom of a, of a badly designed system that cannot guarantee that two operations happened.

573. 1:39:19

     Uh, to do that, you have various ways. One way is to implement queues and so on and so forth. The other way is to use something like workflows. You have solutions like Temporal, Ingest.

574. 1:39:32

     There's many workflow solution. Effect has one, uh, implemented on top of what, uh, what is called Effect Cluster, where basically you run a cluster of Bun node, whatever instances, and the system itself guarantees that once a procedure starts, it's gonna finish.

575. 1:39:52

     Even if the server crashes, it's gonna move to a different, uh, location. How I would go about it? Same way as I did now. Uh, ask the model to explore the, the repository, extract the best...

576. 1:40:04

     the, the patterns around how to use Effect Cluster, how to use Effect Workflows, and, um, just

577. 1:40:13

     gone from, uh, from there. It's very interesting. Uh, it's still in the unstable part of Effect, but it's gonna be stable very soon. And, uh, we think especially with, um, if you do...

578. 1:40:30

     if you integrate AI in your app, it's gonna be even more important because with AI, every process becomes more long running. Like LLMs takes minutes to answer. There's a lot of things that can go wrong in a minute.

579. 1:40:46

     If the average response time is ten milliseconds, server is pretty much never gonna fail in that ten milliseconds. If that ten milliseconds becomes a minute, yes, you're pretty sure that server is gonna fail in that minute at some point.

580. 1:41:01

     And usually before the companies that would use workflows were larger scale companies, 'cause at scale every edge case happens twice per day. Uh, with longer response time, even, even if you have ten users, you're pretty much gonna have disruption if your average process takes a minute, and you're gonna have

581. 1:41:26

     failure, uh, all, all over the place. Which is why,

582. 1:41:31

     for example, Temporal became much more interesting in the, in the past twelve months, 'cause everybody's now implementing AI in their own products. Uh, so they have chatbots, they have, uh, any kind of AI, uh, AI-driven process.

583. 1:41:47

     And with Effect, you get workflows, you get clustering, you have AI integrations, you have Discord, Slack integrations, and so on and so forth. So it's system is really composable, and

584. 1:42:03

     the models are pretty decent at it. We have a working API. I've been

585. 1:42:10

     speaking for about an hour and a half, uh, and

586. 1:42:14

     I started with zero Effect knowledge. It was an empty repository,

587. 1:42:20

     and this is why I wanted to call this workshop Just Clone the Fucking Repo.

588. 1:42:27

     That's pretty much it. Uh, if you have any question or anything else, I'm happy to discuss, uh, with you at a later point. And let's get the next speaker set up.

589. 1:42:40

     Thank you so much. [audience applauding] [upbeat music]