1 08:15:48,300 --> 08:15:49,240 What's up, everybody? 2 08:15:50,020 --> 08:15:50,920 How are we all doing? 3 08:15:52,420 --> 08:15:54,900 Give it up for all the fucking organizers out here. 4 08:15:55,120 --> 08:15:56,120 They are killing it. 5 08:15:56,260 --> 08:15:57,820 Swixen team, let's get a round of applause. 6 08:15:58,720 --> 08:16:00,520 All the speakers, keep it going for the speakers. 7 08:16:02,340 --> 08:16:04,800 I knew Quinn was really smart, but that was a really dope talk. 8 08:16:04,940 --> 08:16:06,300 I enjoyed watching it from backstage. 9 08:16:07,380 --> 08:16:08,060 Hi, everybody. 10 08:16:08,260 --> 08:16:08,920 I'm Dex. 11 08:16:08,920 --> 08:16:14,400 This is 12 Factor Factories, which started as a joke title, but then everyone started calling it that. 12 08:16:15,060 --> 08:16:16,180 And the war on slop. 13 08:16:16,900 --> 08:16:18,060 So thank you all for coming. 14 08:16:19,740 --> 08:16:22,520 I like to yap about coding agents on the internet. 15 08:16:22,860 --> 08:16:25,740 And like a year ago, we did the whole context engineering thing. 16 08:16:26,500 --> 08:16:27,620 A lot of views on YouTube. 17 08:16:27,880 --> 08:16:34,120 And up until about a week ago, I was the CEO and co-founder at a company called Human Layer, the multiplayer coding agent workspace. 18 08:16:34,120 --> 08:16:34,240 space. 19 08:16:34,840 --> 08:16:40,140 And for those of you who are blessed to not be quite as terminally online as some of us 20 08:16:40,140 --> 08:16:44,440 may be, we are now a part of factory, which I'm very excited about and I will be touching 21 08:16:44,440 --> 08:16:45,280 on during this talk. 22 08:16:45,620 --> 08:16:46,860 But I want to get right into it. 23 08:16:47,480 --> 08:16:51,340 Last summer was all about this, like, you know, three months ago, it feels like forever. 24 08:16:51,560 --> 08:16:52,560 Factories were all the hype. 25 08:16:52,840 --> 08:16:56,700 And I gave an intuition-based talk back in June of, like, we should probably still read 26 08:16:56,700 --> 08:17:02,320 the code and here's how RL and verifiers are working and, like, exactly as our friends 27 08:17:02,320 --> 08:17:05,300 at Google were saying, you know, models really like just solving the tests, just getting 28 08:17:05,300 --> 08:17:10,500 them to pass. I now have some data beyond just that intuition. There's a benchmark called 29 08:17:10,500 --> 08:17:15,620 Slopcode Bench, which we just released actually today a bunch of new data on from the lab 30 08:17:15,620 --> 08:17:22,540 at University of Wisconsin. But very unsaturated benchmark, you know, GPT 5.5 got a 15%. Astra, 31 08:17:22,600 --> 08:17:27,760 I think, scored 16.3. So like, we are getting better at this, but there's a lot of ways 32 08:17:27,760 --> 08:17:31,640 to go. I'll talk all about that data in a sec. We, of course, have the study from Pharos 33 08:17:31,640 --> 08:17:37,540 of 30% of PRs merging without review, longer comments, more comments, more incidents per PR, 34 08:17:37,940 --> 08:17:42,480 more bugs per developer. And I titled this talk, I don't think it was quite correct. I said, 35 08:17:42,560 --> 08:17:46,940 why software factories fail? And everyone said, oh, Dex is the anti-software factory guy. Dex 36 08:17:46,940 --> 08:17:53,140 hates software factories. I actually love software factories. I'm a huge fan. The title was probably 37 08:17:53,140 --> 08:17:57,180 more along the lines of, there are certain things you could do in your software factory 38 08:17:57,180 --> 08:17:58,680 that would cause it to fail. 39 08:17:59,340 --> 08:18:01,760 And here's what I think those things might be, 40 08:18:01,780 --> 08:18:03,760 and please don't do them. 41 08:18:04,440 --> 08:18:05,760 It'd be the dark factory thing. 42 08:18:07,120 --> 08:18:08,900 But I think the main takeaway here, 43 08:18:08,960 --> 08:18:10,240 and I think we've heard this from a lot of folks, 44 08:18:10,680 --> 08:18:12,540 is software factories is just good engineering. 45 08:18:13,680 --> 08:18:16,280 A friend of the pod, friend of the conference, 46 08:18:16,920 --> 08:18:18,180 Lauren was talking about this yesterday, 47 08:18:18,300 --> 08:18:20,760 of a lot of this is just stuff we've been doing for a long time. 48 08:18:21,240 --> 08:18:22,520 Actually, Eno from Factory had a really, 49 08:18:22,620 --> 08:18:24,140 the post is much longer, but it's like, 50 08:18:24,200 --> 08:18:27,100 the factory mindset is about how do we standardize our process 51 08:18:27,100 --> 08:18:28,420 not just the output. 52 08:18:29,140 --> 08:18:31,160 And there's a whole bunch of stuff we'll dig into there 53 08:18:31,160 --> 08:18:32,520 on how we're thinking about that. 54 08:18:32,800 --> 08:18:34,860 But I try to kind of make the same point, right? 55 08:18:34,980 --> 08:18:36,880 It's like your standard software factory 56 08:18:36,880 --> 08:18:40,200 is going to look something like you have some engineers, 57 08:18:40,580 --> 08:18:43,260 some product managers, maybe a CEO 58 08:18:43,260 --> 08:18:45,000 or someone to drive the vision. 59 08:18:45,660 --> 08:18:48,040 And those people are going to come up with stuff 60 08:18:48,040 --> 08:18:48,820 that we want to build. 61 08:18:48,960 --> 08:18:50,340 And they're going to put it in a tracker. 62 08:18:50,460 --> 08:18:52,220 This is, again, this is pre-AI. 63 08:18:52,700 --> 08:18:54,980 A lot of the things we hear about software factories today 64 08:18:55,920 --> 08:18:57,900 were around for 30 years, right? 65 08:18:58,000 --> 08:18:59,620 You take stuff off the tracker, you build it, 66 08:18:59,720 --> 08:19:01,400 you run some unit tests, automated tests, 67 08:19:01,640 --> 08:19:02,680 maybe you poke it locally. 68 08:19:02,680 --> 08:19:04,480 When you're ready, you make a pull request. 69 08:19:04,780 --> 08:19:06,800 At that point, someone might pull it down, 70 08:19:07,480 --> 08:19:08,840 interact with it, test it. 71 08:19:09,500 --> 08:19:12,100 Your CI is going to run a bunch of checks and static linters, 72 08:19:12,140 --> 08:19:13,660 and none of this is that new. 73 08:19:14,300 --> 08:19:16,140 And if anything is wrong with it, you send it back, 74 08:19:16,160 --> 08:19:17,300 you go fix the problems, 75 08:19:18,960 --> 08:19:21,160 and when it's actually ready to go to prod, 76 08:19:21,240 --> 08:19:24,020 we do some canaries and rollouts and feature flags 77 08:19:24,020 --> 08:19:25,380 and all kinds of fun stuff there. 78 08:19:26,000 --> 08:19:27,920 And eventually it makes contact with our users. 79 08:19:28,340 --> 08:19:29,740 And I love our users. 80 08:19:29,900 --> 08:19:30,660 I love customers. 81 08:19:31,200 --> 08:19:33,900 But they do like to complain about things that are broken 82 08:19:33,900 --> 08:19:36,520 and ask for stuff that we haven't built yet. 83 08:19:36,860 --> 08:19:38,340 And so that goes back to the team. 84 08:19:38,740 --> 08:19:40,760 And at some point, something breaks at 3 in the morning 85 08:19:40,760 --> 08:19:42,980 and the customers are the first one to figure it out. 86 08:19:43,020 --> 08:19:45,060 So we add monitoring so that we can wake people up 87 08:19:45,060 --> 08:19:46,520 at 3 in the morning and they can solve it 88 08:19:46,520 --> 08:19:47,440 before anybody notices. 89 08:19:48,100 --> 08:19:50,180 This is your pre-AI software factory. 90 08:19:50,600 --> 08:19:52,020 And the basic point is, like, 91 08:19:52,780 --> 08:19:54,520 all you need to do to make this agentic 92 08:19:54,520 --> 08:19:56,480 is replace someone builds the thing with 93 08:19:56,480 --> 08:19:58,700 an agent builds the thing. So you have orchestration 94 08:19:58,700 --> 08:20:00,000 I'm not going to go into the details of the sandbox 95 08:20:00,160 --> 08:20:01,320 and the model and all this stuff. 96 08:20:01,940 --> 08:20:05,880 But you can incrementally build this up by adding what I'll call forward pressure. 97 08:20:06,100 --> 08:20:10,820 What causes work to start? Can we take all of the feature requests and just route them straight to our factory? 98 08:20:11,080 --> 08:20:14,440 Can we take all of the incidents that happen so that when I get paged at 3 in the morning, 99 08:20:14,540 --> 08:20:21,120 I'm looking at a pull request that we think is going to solve it, not just a stack trace about what went wrong. 100 08:20:21,920 --> 08:20:24,540 So the forward pressure, what causes new work to start? 101 08:20:25,040 --> 08:20:26,920 and back pressure, which is sort of like, 102 08:20:27,020 --> 08:20:29,420 okay, cool, now we can build things in minutes or hours, 103 08:20:29,900 --> 08:20:32,460 but the review and testing still takes hours or days 104 08:20:32,460 --> 08:20:33,520 if we're doing it in the old way, 105 08:20:33,560 --> 08:20:36,280 so we bring in agentic code review, agentic testing. 106 08:20:36,660 --> 08:20:39,120 Anyone doing agents code reviewing their pull requests? 107 08:20:40,820 --> 08:20:41,180 Okay. 108 08:20:41,440 --> 08:20:42,700 I thought it would be a little bit more, 109 08:20:42,780 --> 08:20:43,260 but that's pretty good. 110 08:20:43,480 --> 08:20:44,420 Y'all are catching up. 111 08:20:45,180 --> 08:20:48,140 And then we invest heavily into our CICD and our linters 112 08:20:48,140 --> 08:20:50,660 and our, anyone doing like lean checks on their code, 113 08:20:50,780 --> 08:20:51,840 like formal verification? 114 08:20:52,420 --> 08:20:52,900 All right. 115 08:20:53,380 --> 08:20:53,740 Hardcore. 116 08:20:53,740 --> 08:20:54,400 I like it. 117 08:20:55,160 --> 08:20:59,180 So the back pressure, how do we let agents hill climb to a better outcome without needing 118 08:20:59,180 --> 08:20:59,980 to involve humans? 119 08:21:00,080 --> 08:21:03,720 How do we just overall increase the quality of what is coming out of our build step? 120 08:21:04,300 --> 08:21:08,900 And agents help here, but a lot of what makes software factories good is stuff we've kind 121 08:21:08,900 --> 08:21:09,860 of had for decades. 122 08:21:10,340 --> 08:21:14,140 And just like context engineering, which was the thing that I was talking about last summer 123 08:21:14,140 --> 08:21:18,960 in like June of 2025, there was all these ideas of like RAG and memory and state and 124 08:21:18,960 --> 08:21:22,820 history and all this stuff, and we tried to like distill out what are the timeless things 125 08:21:22,820 --> 08:21:23,940 that go into building agents. 126 08:21:24,360 --> 08:21:28,440 And so I want to distill out now some of what I think are the timeless things that go into 127 08:21:28,440 --> 08:21:30,180 building and designing a software factory. 128 08:21:31,000 --> 08:21:32,540 Because there's a lot of noise and hype. 129 08:21:32,760 --> 08:21:36,340 Like I said, Terminally Online, Twitter is going to break your brain if you spend too 130 08:21:36,340 --> 08:21:38,000 much time wondering what everyone else is doing. 131 08:21:38,800 --> 08:21:43,080 I'm also going to talk about Slop because I have a very fun project that I've been working 132 08:21:43,080 --> 08:21:47,360 on with Swix and a handful of other speakers here, but we'll get to that later. 133 08:21:48,040 --> 08:21:51,020 We're going to do 12 great practices for software factories. 134 08:21:51,020 --> 08:21:56,140 And I think the first time I started talking about this, it was only like 9, but we are at 12 now, so I'm very excited about this. 135 08:21:56,680 --> 08:22:00,120 I touched on this, but loops was this concept that was very in vogue. 136 08:22:00,260 --> 08:22:05,360 I think this is much more useful if you think about the interplay of forward pressure and back pressure. 137 08:22:06,100 --> 08:22:10,980 So, you know, again, very hyped up, but the actual tactical thing is like, how do you start work automatically? 138 08:22:11,360 --> 08:22:17,520 And how do you make sure that whatever comes out and is touched by a human by the end, all the low-hanging stuff is already fixed. 139 08:22:17,540 --> 08:22:20,140 The outcome is as good as it can be autonomously. 140 08:22:21,120 --> 08:22:22,980 Number two, I will call know your enemy. 141 08:22:23,260 --> 08:22:26,440 I will still posit that unattended models 142 08:22:26,440 --> 08:22:28,100 don't do a great job of maintaining 143 08:22:28,100 --> 08:22:30,180 and improving code base quality over time. 144 08:22:30,320 --> 08:22:31,580 So if you care about having a system 145 08:22:31,580 --> 08:22:33,220 that's going to be still maintainable 146 08:22:33,220 --> 08:22:34,220 in three or six months, 147 08:22:35,400 --> 08:22:36,420 I mean, again, I gave this talk 148 08:22:36,420 --> 08:22:37,540 and I said, please read the code 149 08:22:37,540 --> 08:22:38,620 because a lot of it is slop. 150 08:22:39,160 --> 08:22:41,400 Agents got really good at solving one-off problems 151 08:22:41,400 --> 08:22:43,020 and making video games 152 08:22:43,020 --> 08:22:44,880 and Blender slop and all this stuff. 153 08:22:45,400 --> 08:22:46,540 I don't think, you know, 154 08:22:46,580 --> 08:22:48,040 if you steer a model to fix your code base, 155 08:22:48,100 --> 08:22:49,200 it can go fix your code base. 156 08:22:49,200 --> 08:22:51,980 But then you have to know what is there and what you want. 157 08:22:52,240 --> 08:22:55,580 So if you are just hands-off pushing features into the machine, 158 08:22:56,000 --> 08:22:59,540 you may not find that it autonomously will make things better over time. 159 08:23:00,980 --> 08:23:02,240 Back then I didn't have a benchmark. 160 08:23:02,360 --> 08:23:06,620 Now I have a benchmark that basically has the model solve problems and checkpoints. 161 08:23:06,820 --> 08:23:09,980 Like a model can clone all of Excel or Redis in a couple days, 162 08:23:10,520 --> 08:23:12,840 maybe even a couple hours now with like agent swarms. 163 08:23:13,200 --> 08:23:15,920 But that requires knowing the entire problem in advance. 164 08:23:16,280 --> 08:23:18,260 And most of us don't write software that way. 165 08:23:18,260 --> 08:23:20,840 Most of us ship a feature, and then ship another feature, 166 08:23:21,020 --> 08:23:22,100 and then ship another feature. 167 08:23:22,340 --> 08:23:23,800 And if you're building by hand without AI, 168 08:23:24,000 --> 08:23:25,660 along the way you say, oh, that's a bad abstraction. 169 08:23:25,660 --> 08:23:27,680 We need to centralize this, or we need to remove this. 170 08:23:27,840 --> 08:23:30,460 And so this is a benchmark that measures a model's ability 171 08:23:30,460 --> 08:23:32,900 to evolve a code base over time. 172 08:23:33,800 --> 08:23:37,460 And so Astra, again, is 16%, very unsaturated. 173 08:23:37,800 --> 08:23:38,840 There's a bunch of data here. 174 08:23:38,900 --> 08:23:40,440 I probably don't have time to look at all of it, 175 08:23:40,500 --> 08:23:42,860 but we can look at code quality metrics 176 08:23:42,860 --> 08:23:46,260 like cyclomatic complexity, lines per symbol, 177 08:23:47,340 --> 08:23:49,440 the percentage of functions that have high complexity. 178 08:23:49,660 --> 08:23:51,380 There's more. There's 200 custom rules. 179 08:23:52,000 --> 08:23:53,980 50 of them are based on things we've been doing for a while. 180 08:23:54,120 --> 08:23:56,720 A lot of them are custom based on looking at code that models write, 181 08:23:57,180 --> 08:23:59,020 comparing it to pre-AI repos. 182 08:23:59,760 --> 08:24:01,720 I wrote a long article about that. I've read a bunch. 183 08:24:02,020 --> 08:24:04,280 Gabe Orlansky, who runs the lab that does this benchmark, 184 08:24:04,340 --> 08:24:06,100 just put up this paper today 185 08:24:06,860 --> 08:24:08,700 based on some of the research we did with them. 186 08:24:09,160 --> 08:24:10,260 There's a bunch of stuff in here. 187 08:24:10,300 --> 08:24:12,020 I probably don't have time to go through all of it, 188 08:24:12,060 --> 08:24:15,720 but they also test the model's ability to design good libraries and interfaces. 189 08:24:16,520 --> 08:24:21,940 The most, the craziest one here is like, models did learn to use exceptions instead of returning 190 08:24:21,940 --> 08:24:22,820 tuples in Python. 191 08:24:23,580 --> 08:24:28,500 There was a lot of patterns that were identified as like probably not great, that were way 192 08:24:28,500 --> 08:24:32,400 less present in the, you know, six months ago models than they are in today's models, 193 08:24:32,440 --> 08:24:36,100 so it almost feels like we're like, the slop is amplifying over time. 194 08:24:37,180 --> 08:24:42,460 And Claude and GLM are now writing Python 3 in Python 2 style, so we have, you know, 195 08:24:42,460 --> 08:24:47,040 %s format strings are back, and they will not use fstrings by default anymore. 196 08:24:47,300 --> 08:24:50,260 So things to be aware of. You may not care about this. I care about this. 197 08:24:50,740 --> 08:24:55,220 The jury's still out on Fable 5.2. I love reading the Anthropic blog post about how 198 08:24:55,220 --> 08:24:59,200 to prompt the new models. And so allegedly, if you tell Fable it owns the code base, it 199 08:24:59,200 --> 08:25:02,440 should keep it clean. I haven't tested this yet. We're running the benchmark in the background 200 08:25:02,440 --> 08:25:06,880 right now. But for those of us not building FBS games, shouts out to Ben, built a cool 201 08:25:06,880 --> 08:25:09,160 game. You should go watch this video. It's insane. 202 08:25:09,860 --> 08:25:10,900 you should know your enemy. 203 08:25:11,100 --> 08:25:12,740 You should understand that, like, 204 08:25:12,880 --> 08:25:15,340 work with this in mind, that if you need your codebase 205 08:25:15,340 --> 08:25:17,280 to last a long time, or your codebase has been 206 08:25:17,280 --> 08:25:19,300 around for a long time, we should 207 08:25:19,300 --> 08:25:21,240 probably maintain understanding of the 208 08:25:21,240 --> 08:25:22,540 system and the code. 209 08:25:23,300 --> 08:25:25,360 To do this, we like to plan before we 210 08:25:25,360 --> 08:25:27,360 build. I highly encourage this. You should do 211 08:25:27,360 --> 08:25:29,380 this. We used to do this before in this old software 212 08:25:29,380 --> 08:25:31,560 factory. People realize, especially 213 08:25:31,560 --> 08:25:33,560 when building things took 214 08:25:33,560 --> 08:25:35,000 hours or days, 215 08:25:35,260 --> 08:25:37,360 and when testing and reviewing things 216 08:25:37,360 --> 08:25:39,420 took hours or days, even if this part is faster, 217 08:25:39,480 --> 08:25:41,820 this part is probably still slow if you're doing it properly, 218 08:25:42,260 --> 08:25:43,560 we would have architecture review, 219 08:25:43,720 --> 08:25:44,900 and we would have sprint planning, 220 08:25:44,980 --> 08:25:46,620 and we would talk as a team, and we would align 221 08:25:46,620 --> 08:25:50,180 in the hopes that we might decrease the percent chance 222 08:25:50,180 --> 08:25:51,840 that we'd have to rework something later. 223 08:25:51,900 --> 08:25:54,280 We wanted to be rather sure that we're building 224 08:25:54,280 --> 08:25:55,680 the right thing in the right way 225 08:25:55,680 --> 08:25:58,260 before we actually go and spend time building it, 226 08:25:58,340 --> 08:26:00,460 time in tokens, and then get to review 227 08:26:00,460 --> 08:26:01,420 and have to rework it. 228 08:26:02,080 --> 08:26:04,200 A lot of people overdo this as well. 229 08:26:04,320 --> 08:26:05,600 I've overdone this for sure, 230 08:26:05,600 --> 08:26:07,440 where you spend six hours working on a spec 231 08:26:07,440 --> 08:26:09,340 and you're like, I feel like I could have just YOLO'd this. 232 08:26:10,540 --> 08:26:12,720 Because sometimes you can just YOLO a two-sentence prompt, 233 08:26:12,760 --> 08:26:13,800 and if it's small or you don't care, 234 08:26:13,900 --> 08:26:16,320 there's maybe a 50% chance you send a couple more prompts 235 08:26:16,320 --> 08:26:16,900 and then it's good. 236 08:26:17,320 --> 08:26:20,340 You could spend five hours handwriting a detailed spec, 237 08:26:20,460 --> 08:26:22,980 and maybe there's a 10% of it you have to change afterwards. 238 08:26:23,720 --> 08:26:25,660 Or you could write all the code by hand in 20 hours, 239 08:26:25,820 --> 08:26:27,460 and then there's like a 0% chance. 240 08:26:27,600 --> 08:26:29,100 Well, there's 0% chance you have to rework it 241 08:26:29,100 --> 08:26:30,320 because of something the agent did. 242 08:26:30,420 --> 08:26:31,620 You know, people still make mistakes too. 243 08:26:33,220 --> 08:26:35,000 But I think about this in terms of expected pain. 244 08:26:35,000 --> 08:26:37,440 like how likely will this have to change later 245 08:26:37,440 --> 08:26:40,620 times how hard will it be to change. 246 08:26:41,600 --> 08:26:43,280 I don't argue with the model during planning 247 08:26:43,280 --> 08:26:44,680 about what color the button's gonna be, 248 08:26:44,780 --> 08:26:47,220 but I do wanna make sure we get our database schemas right 249 08:26:47,220 --> 08:26:48,040 and stuff like that, 250 08:26:48,500 --> 08:26:49,840 because it's really hard to change later. 251 08:26:49,940 --> 08:26:51,680 So based on your problem and what you're doing 252 08:26:51,680 --> 08:26:52,400 and your code base, 253 08:26:52,580 --> 08:26:54,980 you should probably decide where on this spectrum 254 08:26:54,980 --> 08:26:55,640 you wanna land. 255 08:26:56,560 --> 08:26:59,280 Align visually, I published a skill called show me, 256 08:26:59,460 --> 08:27:02,140 which is optimized for instead of reading walls of text, 257 08:27:02,460 --> 08:27:04,340 you get HTML mockups, you get tables, 258 08:27:04,340 --> 08:27:09,560 You get charts. You get all this stuff that is optimized for the human brain, which is very visual. 259 08:27:10,400 --> 08:27:15,680 I think Koda said we've been running away from predators and eating fruit for millions of years. 260 08:27:15,800 --> 08:27:18,180 We've only been reading text for about 5,000. 261 08:27:18,700 --> 08:27:20,780 Anyways, you can get this. It's got about 40,000 installs. 262 08:27:21,320 --> 08:27:22,880 You should help agents test their work. 263 08:27:23,580 --> 08:27:26,440 Screenshots, browsers, videos, curl, whatever you can do to help the agent. 264 08:27:26,560 --> 08:27:27,920 Again, he'll climb against an outcome. 265 08:27:28,480 --> 08:27:33,180 If you're building an iOS app, make sure it can build, run, and test that app in whatever environment it's running. 266 08:27:33,860 --> 08:27:36,400 You should fight slop with anyone here writing in TypeScript. 267 08:27:37,580 --> 08:27:38,020 OK. 268 08:27:38,620 --> 08:27:39,580 Some people are feeling lazy. 269 08:27:39,700 --> 08:27:40,900 I feel like it should be more than that. 270 08:27:41,080 --> 08:27:46,020 But this is a repo from Dylan Mulroy, who's a principal engineer at CloudFlare. 271 08:27:46,140 --> 08:27:46,720 Anti-slop. 272 08:27:46,800 --> 08:27:48,560 If you're not using this, you should go check it out. 273 08:27:48,620 --> 08:27:52,400 It's 100-ish rules on stopping agents from building slop. 274 08:27:52,940 --> 08:27:55,640 Factory also has an ESLint plugin that I should probably mention. 275 08:27:56,340 --> 08:28:02,060 And also, again, you can go slop fork all of the Python code quality rules out of Slop 276 08:28:02,060 --> 08:28:04,020 code bench. This exists for your 277 08:28:04,020 --> 08:28:06,060 language. You should use it. Slop 278 08:28:06,060 --> 08:28:07,380 is a tool. Not all slop is bad. 279 08:28:07,800 --> 08:28:09,880 Vibe coding prototypes, side projects. 280 08:28:10,040 --> 08:28:11,980 If you have something behind a feature flag, you just need to like figure 281 08:28:11,980 --> 08:28:14,040 out if people even want it before you 282 08:28:14,040 --> 08:28:15,980 go put in the time. I think it's like, it's great. 283 08:28:16,100 --> 08:28:18,080 I don't think there's like always, always, always, 284 08:28:18,080 --> 08:28:19,840 always every line of code has to be perfect. 285 08:28:20,560 --> 08:28:22,040 You should give models a feature 286 08:28:22,040 --> 08:28:24,040 flagging system. PostHog 287 08:28:24,040 --> 08:28:25,940 or Statsig or LaunchDarkly or whatever it is so they 288 08:28:25,940 --> 08:28:27,800 can try stuff and see if it performs better 289 08:28:27,800 --> 08:28:29,160 and then make decisions about 290 08:28:29,960 --> 08:28:31,660 what the right thing to be in your tool. 291 08:28:32,140 --> 08:28:34,240 You should compound over time, eval everything, 292 08:28:34,520 --> 08:28:35,500 learn from past sessions, 293 08:28:35,780 --> 08:28:37,820 and you should unify your forge. 294 08:28:38,080 --> 08:28:40,000 What I mean by this is we used to send 295 08:28:40,000 --> 08:28:41,640 Git patches around on mailing lists. 296 08:28:41,800 --> 08:28:43,800 I think Linus and the Linux people still do this, 297 08:28:43,820 --> 08:28:45,540 but most people keep all their code in GitHub. 298 08:28:46,600 --> 08:28:48,320 And so before the ANA to forge, 299 08:28:48,420 --> 08:28:49,520 we have centralized patches, 300 08:28:49,840 --> 08:28:52,940 but we still have people pasting 301 08:28:52,940 --> 08:28:55,040 clawed answers back and forth in Slack 302 08:28:55,040 --> 08:28:58,720 and hopefully not doing too many rounds of that 303 08:28:58,720 --> 08:29:00,140 because it's like, I copy out of mine 304 08:29:00,140 --> 08:29:01,020 and send it to my coworker. 305 08:29:01,020 --> 08:29:03,540 He pasted it into his agent, and then they sent it back to me, 306 08:29:03,640 --> 08:29:05,640 and then I pasted it into my agent, and maybe we get an answer. 307 08:29:06,060 --> 08:29:08,480 So centralizing all of your designs and decisions 308 08:29:08,480 --> 08:29:11,320 and whatever you have in Notion and Google Docs 309 08:29:11,320 --> 08:29:14,080 and whatever it is, you should try to bring those things 310 08:29:14,080 --> 08:29:16,520 as close as possible together so you can understand 311 08:29:16,520 --> 08:29:18,320 not just the code about what something was built 312 08:29:18,320 --> 08:29:19,860 but all the decisions that went into it. 313 08:29:20,560 --> 08:29:21,880 You can't buy everything. 314 08:29:21,880 --> 08:29:25,160 I think there's things in this space that just cannot be productized. 315 08:29:25,640 --> 08:29:27,920 Obviously, you can buy VMs or you can bring your own. 316 08:29:28,080 --> 08:29:30,540 Dev environment, I think, is very personal to most teams. 317 08:29:31,040 --> 08:29:33,440 The harness, of course, you can go get a Frontier harness, 318 08:29:33,540 --> 08:29:35,540 or you can use something minimal like OpenCode or Pi 319 08:29:35,540 --> 08:29:37,020 and customize the heck out of it. 320 08:29:37,380 --> 08:29:39,400 And then you have this control plane picture of, like, 321 08:29:39,680 --> 08:29:41,600 how do you collaborate on sessions and plans? 322 08:29:41,600 --> 08:29:43,120 How do you do automations and scheduling? 323 08:29:43,320 --> 08:29:44,540 How do you respond to events? 324 08:29:44,560 --> 08:29:45,740 All this governance stuff. 325 08:29:46,240 --> 08:29:48,760 Part of the reason I'm really excited to be working with Factory on all this stuff 326 08:29:48,760 --> 08:29:51,220 is, like, they have a much bigger view of this picture 327 08:29:51,220 --> 08:29:53,200 and have a lot more stuff solved here. 328 08:29:53,700 --> 08:29:55,840 But there are parts, especially at the top here, 329 08:29:55,920 --> 08:29:59,420 is, like, I don't think anyone can sell you verification of your code 330 08:29:59,420 --> 08:30:00,000 because only you 331 08:30:00,100 --> 08:30:04,640 know, if you have hundreds of engineers and hundreds of repos, you have those opinions and you know how to test your code. 332 08:30:05,980 --> 08:30:07,500 Not just a black box you can buy. 333 08:30:07,620 --> 08:30:09,640 There are building blocks that are undifferentiated. 334 08:30:09,720 --> 08:30:13,280 But the idea of software factory, I think, is a way of thinking about our work. 335 08:30:14,760 --> 08:30:16,020 So that's software factories. 336 08:30:16,160 --> 08:30:17,260 I also want to talk about slop. 337 08:30:20,080 --> 08:30:22,360 Factories are about how do we increase human leverage, right? 338 08:30:22,380 --> 08:30:24,100 How do we get the best return on our time? 339 08:30:24,720 --> 08:30:26,940 and I've talked a little bit about why humans should be involved 340 08:30:26,940 --> 08:30:28,780 and what it means to be a diligent 341 08:30:28,780 --> 08:30:30,760 steward of what we're building and what we're doing 342 08:30:31,240 --> 08:30:33,100 but I think it's bigger than just software 343 08:30:33,100 --> 08:30:35,040 so I'm very excited to announce 344 08:30:35,040 --> 08:30:37,200 the Anti-Slop Manifesto 345 08:30:38,280 --> 08:30:39,120 which you can go 346 08:30:39,120 --> 08:30:40,820 if you go to slopmanifesto.com 347 08:30:40,820 --> 08:30:42,560 you can read it, I'll put the URL up later 348 08:30:43,220 --> 08:30:44,700 but we built this in collaboration 349 08:30:44,700 --> 08:30:46,980 with a lot of people here, both attendees 350 08:30:46,980 --> 08:30:49,020 and speakers, special shout out to 351 08:30:49,020 --> 08:30:49,900 Swix and Phillip 352 08:30:50,340 --> 08:30:52,900 it has signers from Zoom Info 353 08:30:52,900 --> 08:30:56,420 and I think XAI and OpenAI, 354 08:30:56,600 --> 08:30:58,420 and there's a lot of cool people on there. 355 08:30:58,420 --> 08:31:00,340 I encourage you to go read it and check it out 356 08:31:00,340 --> 08:31:01,840 and be a part of, you know. 357 08:31:02,020 --> 08:31:02,800 But I'll tell you what's in it. 358 08:31:03,760 --> 08:31:06,180 We value outcomes over outputs. 359 08:31:06,460 --> 08:31:08,960 We value understanding over speed. 360 08:31:09,360 --> 08:31:11,100 We value our own thinking 361 08:31:11,100 --> 08:31:13,500 over whatever the model came up with as a first draft. 362 08:31:14,020 --> 08:31:17,060 We value better work over easier work. 363 08:31:17,900 --> 08:31:19,660 We value improving the system 364 08:31:19,660 --> 08:31:21,720 over just patching outputs. 365 08:31:22,560 --> 08:31:25,940 And the things we believe are that AI is the most useful tool 366 08:31:25,940 --> 08:31:27,320 most of us have ever worked with, 367 08:31:27,480 --> 08:31:32,000 but it can create a swarm of agents can produce more work in a single day 368 08:31:32,000 --> 08:31:34,520 than a human can review in their entire career. 369 08:31:34,640 --> 08:31:36,420 Look at some of the math stuff coming out of OpenAI. 370 08:31:37,260 --> 08:31:39,880 Slop is when we put less effort into producing something 371 08:31:39,880 --> 08:31:43,800 than we ask their consumer on the other end to spend understanding it. 372 08:31:44,400 --> 08:31:47,360 And using AI doesn't necessarily make something slop, 373 08:31:48,000 --> 08:31:50,540 but it happens when we pass off AI output 374 08:31:50,540 --> 08:31:54,280 without taking responsibility for its quality or for the reader's experience. 375 08:31:55,140 --> 08:31:57,920 I believe strongly, I've been saying this for a year now, 376 08:31:58,040 --> 08:31:59,640 AI cannot replace human judgment. 377 08:31:59,760 --> 08:32:01,640 It can only amplify our judgment, 378 08:32:01,640 --> 08:32:05,100 or maybe perhaps our lack of judgment in certain cases. 379 08:32:05,900 --> 08:32:07,500 And the slop is asymmetric. 380 08:32:07,880 --> 08:32:12,400 The cost of producing a thousand-line PR or a ten-page doc is approaching zero, 381 08:32:12,440 --> 08:32:16,220 but the cost of reviewing it is still fairly constant. 382 08:32:16,520 --> 08:32:20,260 And slop proliferates when we repeatedly work around problems 383 08:32:20,260 --> 08:32:21,500 instead of solving them correctly. 384 08:32:21,720 --> 08:32:23,800 It comes from tools that love to add 385 08:32:23,800 --> 08:32:26,060 and are not biased towards subtracting. 386 08:32:26,800 --> 08:32:27,960 I'm not going to name names. 387 08:32:29,120 --> 08:32:31,420 But I think a lot of slop comes from pressure. 388 08:32:31,940 --> 08:32:33,380 Companies are pushing their people 389 08:32:33,380 --> 08:32:35,940 to redefine what is humanly possible. 390 08:32:36,820 --> 08:32:38,700 And I think slop compounds. 391 08:32:39,140 --> 08:32:41,140 So the more slop sits in our knowledge base, 392 08:32:41,500 --> 08:32:43,080 in chat, in code, 393 08:32:43,340 --> 08:32:46,040 the more of it the next agent reads and copies. 394 08:32:46,720 --> 08:32:48,680 And the more slop flying around our team, 395 08:32:48,680 --> 08:32:52,440 the more we lose the will to fight it, and everything gets worse over time. 396 08:32:52,700 --> 08:32:54,540 Communication and quality and all of this. 397 08:32:55,160 --> 08:32:58,960 So we, the undersigned of the anti-slop manifesto... 398 08:32:58,960 --> 08:33:01,660 Guys, is it a little hot in here? 399 08:33:02,800 --> 08:33:05,120 I might just have to pop off the jack. There we go. 400 08:33:06,180 --> 08:33:08,560 They couldn't give me one of these with a factory logo on it yet, 401 08:33:08,600 --> 08:33:09,780 but I'm sure it's coming soon. 402 08:33:11,120 --> 08:33:14,880 Thus, we, the anti-slop undersigners, 403 08:33:14,880 --> 08:33:18,740 resolve that human time and attention is incredibly valuable. 404 08:33:19,420 --> 08:33:21,580 So we optimize for using it wisely. 405 08:33:22,060 --> 08:33:25,060 We fight slop because AI should make our work better, 406 08:33:25,300 --> 08:33:27,200 not just easier. 407 08:33:27,780 --> 08:33:30,280 We optimize our systems and tools and products 408 08:33:30,280 --> 08:33:32,400 for the efficient elicitation of human intent 409 08:33:32,400 --> 08:33:34,660 and for efficient human understanding. 410 08:33:35,340 --> 08:33:37,700 And we do our own thinking when working with a model. 411 08:33:37,920 --> 08:33:40,060 We explore resources, code, prior art. 412 08:33:40,120 --> 08:33:41,200 We really want to understand the problem 413 08:33:41,200 --> 08:33:42,460 before we actually get to work. 414 08:33:43,040 --> 08:33:46,640 And we own the intent and the taste and the consequences of what we ship. 415 08:33:46,900 --> 08:33:49,900 However much AI helps us to make it, we own it. 416 08:33:50,100 --> 08:33:52,160 And when an agent wrote something that we're sending somebody, 417 08:33:52,420 --> 08:33:55,100 we make that clear, and we actually make sure it's good 418 08:33:55,100 --> 08:33:57,620 before we drop it in Slack or email or whatever it is. 419 08:33:58,380 --> 08:34:01,540 And when someone sends us slop, this is also like we have to push back. 420 08:34:01,680 --> 08:34:03,360 We have to tell people kindly, empathetically, 421 08:34:03,520 --> 08:34:05,360 and then ask them to review it first. 422 08:34:05,920 --> 08:34:07,600 And most of us, it's a journey. 423 08:34:07,780 --> 08:34:11,480 I've been on both sides of this one, so I've had to have that note. 424 08:34:11,480 --> 08:34:15,400 I believe there will always be alpha in reviewing something. 425 08:34:16,260 --> 08:34:18,660 As models get better, we may stop reading most of the code. 426 08:34:19,000 --> 08:34:21,700 We may still review a plan or a design or a result. 427 08:34:22,020 --> 08:34:25,600 But if you don't review anything, if you bring nothing to the table, 428 08:34:25,700 --> 08:34:28,960 no thinking, then you're just going to get the same AI output as everybody else. 429 08:34:30,240 --> 08:34:32,880 Agent reviewers, linters, verifiers, they're very useful, 430 08:34:33,100 --> 08:34:36,580 but they are not sufficient. 431 08:34:36,980 --> 08:34:39,900 So for now, we'll still review some subset of the outputs. 432 08:34:40,340 --> 08:34:42,340 What subset, of course, depends on the stakes. 433 08:34:43,180 --> 08:34:48,220 But what shift, who's using it, and how it's helping them matters far more than lines of 434 08:34:48,220 --> 08:34:52,180 code or PRs merge or, God forbid, the number of agents we're running in parallel. 435 08:34:53,300 --> 08:34:54,980 It's cool if it gets you to the outcome. 436 08:34:55,580 --> 08:34:57,580 But unreviewed output isn't always necessarily bad. 437 08:34:57,780 --> 08:35:02,220 Slop is a cool one-shot prompts, vibe coding. 438 08:35:02,380 --> 08:35:04,680 These are all high-leverage things that are in the right place. 439 08:35:04,900 --> 08:35:06,900 Slop is bad when it undermines our outcome. 440 08:35:06,900 --> 08:35:11,020 If we want to make a good product, a maintainable system, we want to save people time and energy. 441 08:35:11,900 --> 08:35:13,960 If slop undermines that, then it's bad. 442 08:35:14,640 --> 08:35:17,260 And I guess finally, none of us has this figured out. 443 08:35:17,940 --> 08:35:22,260 We're each relearning what we will and won't accept, and there's no single right answer here. 444 08:35:22,340 --> 08:35:23,280 You've got to calibrate yourself. 445 08:35:23,980 --> 08:35:25,240 The world moves fast. 446 08:35:25,740 --> 08:35:30,260 My goal here is these are principles that we can use to embrace our important human role 447 08:35:30,260 --> 08:35:36,040 in harnessing AI safely, responsibly, in a way that creates value for everyone. 448 08:35:37,080 --> 08:35:38,320 for many years to come. 449 08:35:39,240 --> 08:35:40,900 So you can read and sign the manifesto 450 08:35:40,900 --> 08:35:42,460 at slotmanifesto.com. 451 08:35:43,880 --> 08:35:45,860 TLDR, you know, give a shit. 452 08:35:46,200 --> 08:35:47,200 Care about what you're doing. 453 08:35:48,900 --> 08:35:50,320 I'm going to stick around for Tariq's talk, 454 08:35:50,480 --> 08:35:52,920 but then, what is that slide? 455 08:35:53,060 --> 08:35:54,620 All right, I'll be hanging in the hallway afterwards. 456 08:35:55,560 --> 08:35:56,460 So come say hi. 457 08:35:56,860 --> 08:35:58,320 Keep learning, keep giving a shit. 458 08:35:58,860 --> 08:36:00,240 I'm excited to keep building with y'all. 459 08:36:00,400 --> 08:36:01,100 Thank you so much. 460 08:36:01,500 --> 08:36:01,880 Thank you.