1 03:59:46,640 --> 03:59:49,189 Uh, hey guys, this is Era. I'm a 2 03:59:49,199 --> 03:59:51,189 founding engineer at Klein and today I'm 3 03:59:51,199 --> 03:59:52,710 going to be talking about recursively 4 03:59:52,720 --> 03:59:55,670 self-improving coding agents. So this is 5 03:59:55,680 --> 03:59:59,269 a very interesting conversation in the 6 03:59:59,279 --> 04:00:01,269 sense that everyone wants to understand 7 04:00:01,279 --> 04:00:03,429 what RSI is, how we accomplish it, how 8 04:00:03,439 --> 04:00:04,950 we go through it and how it affects our 9 04:00:04,960 --> 04:00:06,710 career. So I'm going to start with 10 04:00:06,720 --> 04:00:08,389 giving you just three goals of this 11 04:00:08,399 --> 04:00:10,070 entire conversation. Just three simple 12 04:00:10,080 --> 04:00:13,030 goals and that would help you understand 13 04:00:13,040 --> 04:00:14,550 like what what we get out of it and what 14 04:00:14,560 --> 04:00:16,309 the future uh for this looks like. So 15 04:00:16,319 --> 04:00:18,309 the goal number first is that 16 04:00:18,319 --> 04:00:19,830 understanding the intent of Frontier 17 04:00:19,840 --> 04:00:21,590 Labs, right? Everyone who's outside of 18 04:00:21,600 --> 04:00:22,870 the frontier lab is always trying to 19 04:00:22,880 --> 04:00:24,790 speculate like hey what what is the 20 04:00:24,800 --> 04:00:26,469 topic in open or what is deep mind going 21 04:00:26,479 --> 04:00:27,590 to come out with or what what are they 22 04:00:27,600 --> 04:00:29,750 trying to make and what's the future 23 04:00:29,760 --> 04:00:31,030 going to look like who has cracked the 24 04:00:31,040 --> 04:00:34,229 RSI so like this kind of helps you get 25 04:00:34,239 --> 04:00:36,150 get a rough outline of what their road 26 04:00:36,160 --> 04:00:37,910 map looks like and the second thing is 27 04:00:37,920 --> 04:00:40,229 that like if the models are improving so 28 04:00:40,239 --> 04:00:41,590 fast like how do you think about your 29 04:00:41,600 --> 04:00:43,990 career or the next few years right that 30 04:00:44,000 --> 04:00:45,510 is really what we want to understand we 31 04:00:45,520 --> 04:00:48,150 want to understand like okay things are 32 04:00:48,160 --> 04:00:49,990 changing so fast like the ground 33 04:00:50,000 --> 04:00:52,550 underneath us is shifting so quickly. 34 04:00:52,560 --> 04:00:55,510 Right now like it's like uh 35 04:00:55,520 --> 04:00:56,870 there used to be a time when you could 36 04:00:56,880 --> 04:01:00,070 like demonstrate your side projects on 37 04:01:00,080 --> 04:01:01,990 your personal website or GitHub and now 38 04:01:02,000 --> 04:01:03,189 it's just like you could literally 39 04:01:03,199 --> 04:01:05,670 oneshot them using any Frontier Coding 40 04:01:05,680 --> 04:01:08,469 Engine and with that it's like it's like 41 04:01:08,479 --> 04:01:10,229 oh well how how do we demonstrate proof 42 04:01:10,239 --> 04:01:12,550 of work? How do we demonstrate you know 43 04:01:12,560 --> 04:01:14,389 that that that we know what we're doing 44 04:01:14,399 --> 04:01:15,910 and how do we think of our career for 45 04:01:15,920 --> 04:01:18,150 the next coming years, right? So it's a 46 04:01:18,160 --> 04:01:19,990 very interesting times that we're living 47 04:01:20,000 --> 04:01:21,510 in. And then the third and the most 48 04:01:21,520 --> 04:01:22,950 important goal of the conversation is 49 04:01:22,960 --> 04:01:25,030 escaping the permanent underclass, 50 04:01:25,040 --> 04:01:27,510 right? So I know this seems like a joke, 51 04:01:27,520 --> 04:01:29,750 but this really is a very valid 52 04:01:29,760 --> 04:01:31,110 possibility that people are 53 04:01:31,120 --> 04:01:32,710 contemplating, everyone's like, "Oh 54 04:01:32,720 --> 04:01:35,030 well, if everything that we want to be 55 04:01:35,040 --> 04:01:37,189 doing can potentially easily done by a 56 04:01:37,199 --> 04:01:38,469 frontier model, like what are we what 57 04:01:38,479 --> 04:01:40,229 are we good for?" And like where does 58 04:01:40,239 --> 04:01:42,150 software go? Where where does the future 59 04:01:42,160 --> 04:01:43,670 of technology going to go? And what are 60 04:01:43,680 --> 04:01:45,349 we going to do as as things evolve so 61 04:01:45,359 --> 04:01:48,229 fast, right? So I think that there's 62 04:01:48,239 --> 04:01:50,229 like a lot of like fear-mongering but 63 04:01:50,239 --> 04:01:52,630 also there's like some very valid merge 64 04:01:52,640 --> 04:01:54,870 where people are concerned that like if 65 04:01:54,880 --> 04:01:56,550 all the revenue of the planet is going 66 04:01:56,560 --> 04:01:59,349 to go to um a few hardware companies 67 04:01:59,359 --> 04:02:01,750 that make frontier GPUs and then a few 68 04:02:01,760 --> 04:02:05,030 few uh AI companies that that make the 69 04:02:05,040 --> 04:02:07,590 best models like how are we how are the 70 04:02:07,600 --> 04:02:08,870 rest of us going to throw in what what 71 04:02:08,880 --> 04:02:11,429 are we going to do here? So, I think 72 04:02:11,439 --> 04:02:12,790 I'll I'll walk you through a few 73 04:02:12,800 --> 04:02:14,229 examples and a few ideas that will like 74 04:02:14,239 --> 04:02:17,510 really really help you reconsider the 75 04:02:17,520 --> 04:02:19,349 career choices and and and the life 76 04:02:19,359 --> 04:02:20,710 choices you're making and what your 77 04:02:20,720 --> 04:02:22,630 problems you're picking up. And then and 78 04:02:22,640 --> 04:02:25,349 then that will help you like realign 79 04:02:25,359 --> 04:02:27,030 what it is that we should be working on 80 04:02:27,040 --> 04:02:29,590 and what it is that you you know would 81 04:02:29,600 --> 04:02:31,510 would that you could work on that will 82 04:02:31,520 --> 04:02:33,670 stand the test of time and help you both 83 04:02:33,680 --> 04:02:35,269 just not just like on a psychological 84 04:02:35,279 --> 04:02:36,710 but like financial and other levels as 85 04:02:36,720 --> 04:02:38,870 well. So uh to give you a very precise 86 04:02:38,880 --> 04:02:40,309 example, let's start with something very 87 04:02:40,319 --> 04:02:42,710 precise that I had done. So a couple 88 04:02:42,720 --> 04:02:47,189 months ago I uh came up with recursive 89 04:02:47,199 --> 04:02:49,830 improve improvements on a coding 90 04:02:49,840 --> 04:02:51,750 harness. So this was the first probably 91 04:02:51,760 --> 04:02:54,790 public example of uh an automatic eval 92 04:02:54,800 --> 04:02:57,110 improvement. So to give you an example u 93 04:02:57,120 --> 04:02:58,630 we [snorts] used to do this thing where 94 04:02:58,640 --> 04:03:01,590 we would measure the quality of a coding 95 04:03:01,600 --> 04:03:04,070 agent using evaluation scores. 96 04:03:04,080 --> 04:03:07,110 on terminal bench 2.1 Kim K3 with client 97 04:03:07,120 --> 04:03:10,870 would score um usually like you know um 98 04:03:10,880 --> 04:03:13,990 60 70% and then depending on like how 99 04:03:14,000 --> 04:03:15,429 good the model is like you get better 100 04:03:15,439 --> 04:03:18,070 scores on on terminal bench 2.1 so we 101 04:03:18,080 --> 04:03:19,910 noticed that when Kim K3 came out our 102 04:03:19,920 --> 04:03:22,229 score was 77.5% which is a good score 103 04:03:22,239 --> 04:03:24,070 but we were like oh is this is this 104 04:03:24,080 --> 04:03:26,469 truly is it truly the best score that 105 04:03:26,479 --> 04:03:28,790 this model can get on this benchmark and 106 04:03:28,800 --> 04:03:31,269 also like what does the model release 107 04:03:31,279 --> 04:03:33,110 paper claim about what their best score 108 04:03:33,120 --> 04:03:34,950 is so the model release paper said that 109 04:03:34,960 --> 04:03:37,750 they could score up to 85% or even more. 110 04:03:37,760 --> 04:03:40,070 So we were just like okay okay so we are 111 04:03:40,080 --> 04:03:41,830 not getting the sort of score and often 112 04:03:41,840 --> 04:03:43,830 what we do with model releases in client 113 04:03:43,840 --> 04:03:45,349 is that we try to like optimize your 114 04:03:45,359 --> 04:03:46,870 score and try to get the best out of the 115 04:03:46,880 --> 04:03:50,550 model. So when we started hill climbing 116 04:03:50,560 --> 04:03:53,030 on the on the model like we we realized 117 04:03:53,040 --> 04:03:55,189 is we realized that like you know 118 04:03:55,199 --> 04:03:57,110 there's a lot of steps that we do says 119 04:03:57,120 --> 04:03:58,710 that we do it gracefully says that we 120 04:03:58,720 --> 04:04:00,229 pursue the right kind of problems and do 121 04:04:00,239 --> 04:04:03,590 it the right way and we started thinking 122 04:04:03,600 --> 04:04:05,429 like okay if we there's a lot of ground 123 04:04:05,439 --> 04:04:07,429 work here how do we automate it and this 124 04:04:07,439 --> 04:04:08,790 is a demonstration of like how we 125 04:04:08,800 --> 04:04:10,790 automated it. So I'll I'll just leave 126 04:04:10,800 --> 04:04:12,150 that as a placeholder for now and we'll 127 04:04:12,160 --> 04:04:13,910 get get back to it later where we talk 128 04:04:13,920 --> 04:04:15,510 about in detail of like how does like 129 04:04:15,520 --> 04:04:18,150 this automatic improvement loop of a 130 04:04:18,160 --> 04:04:20,469 client harness work. So this basically 131 04:04:20,479 --> 04:04:21,670 means that like you have a coding a 132 04:04:21,680 --> 04:04:22,950 harness that could just like basically 133 04:04:22,960 --> 04:04:25,269 improve itself and become better on on 134 04:04:25,279 --> 04:04:27,830 the same set of problems. So let's start 135 04:04:27,840 --> 04:04:29,510 with the ground terms. What is recursive 136 04:04:29,520 --> 04:04:31,590 self? Recursive self-improvement is the 137 04:04:31,600 --> 04:04:33,469 idea that an AI system that helps 138 04:04:33,479 --> 04:04:35,990 [snorts] build a more capable successor 139 04:04:36,000 --> 04:04:37,830 which repeats the process you know until 140 04:04:37,840 --> 04:04:40,309 you hit artificial general intelligence 141 04:04:40,319 --> 04:04:41,429 right you have better systems you have 142 04:04:41,439 --> 04:04:42,710 better improvements you have a better 143 04:04:42,720 --> 04:04:43,910 system and then you know you have 144 04:04:43,920 --> 04:04:46,550 singularity so a classic example is like 145 04:04:46,560 --> 04:04:48,550 a self-reing coding agent right 146 04:04:48,560 --> 04:04:50,309 self-reving coding agent is just like a 147 04:04:50,319 --> 04:04:52,389 coding agent that can improve itself so 148 04:04:52,399 --> 04:04:54,229 you know this this is all like a pie in 149 04:04:54,239 --> 04:04:55,830 the sky idea like it's like it's like 150 04:04:55,840 --> 04:04:57,429 you know yeah sure this looks cool but 151 04:04:57,439 --> 04:04:58,950 how how did this even come out to be 152 04:04:58,960 --> 04:05:02,630 like how we even get to here. So if you 153 04:05:02,640 --> 04:05:05,110 notice in topics road map in 2021 in 154 04:05:05,120 --> 04:05:07,269 2021's road map they very specifically 155 04:05:07,279 --> 04:05:10,870 said that uh uh they wanted to develop a 156 04:05:10,880 --> 04:05:12,389 general AI assistant that can interact 157 04:05:12,399 --> 04:05:14,710 conversationally with a user to create 158 04:05:14,720 --> 04:05:16,630 edit and explain software. It would 159 04:05:16,640 --> 04:05:18,229 display all the capabilities of a 160 04:05:18,239 --> 04:05:19,590 software engineer colleague. It would 161 04:05:19,600 --> 04:05:21,030 write functions. It would identify bugs. 162 04:05:21,040 --> 04:05:23,510 It would design an API discuss style and 163 04:05:23,520 --> 04:05:25,349 best practices. Right? So we don't 164 04:05:25,359 --> 04:05:26,790 expect to reach this goal right away but 165 04:05:26,800 --> 04:05:27,830 it's in the direction which we're 166 04:05:27,840 --> 04:05:29,910 striving. It's interesting how this was 167 04:05:29,920 --> 04:05:31,830 more than five years ago. More than five 168 04:05:31,840 --> 04:05:33,910 years ago, people had predicted that 169 04:05:33,920 --> 04:05:35,990 there exist a day where you will have 170 04:05:36,000 --> 04:05:37,910 like a full-blown journal assistant that 171 04:05:37,920 --> 04:05:39,110 can basically act like a colleague, 172 04:05:39,120 --> 04:05:41,830 which we do have right now. Um, then in 173 04:05:41,840 --> 04:05:43,910 2022, Chad Deputy got released. We all 174 04:05:43,920 --> 04:05:45,830 know how that happened. It was it was a 175 04:05:45,840 --> 04:05:47,670 fun moment. It was like the first time 176 04:05:47,680 --> 04:05:50,710 things got sparked for us. Um, then in 177 04:05:50,720 --> 04:05:53,269 2023, 2024, there was the first time 178 04:05:53,279 --> 04:05:56,070 chat models could call functions. These 179 04:05:56,080 --> 04:05:57,750 functions would be simple functions like 180 04:05:57,760 --> 04:05:59,830 hey what's the weather or you know 181 04:05:59,840 --> 04:06:02,150 what's the what's you know what's this 182 04:06:02,160 --> 04:06:03,990 competition over this computations 183 04:06:04,000 --> 04:06:06,710 result in Python. Um and then finally in 184 04:06:06,720 --> 04:06:09,510 2024 when s 345 was released that was 185 04:06:09,520 --> 04:06:11,189 when the agents could like truly call 186 04:06:11,199 --> 04:06:14,309 like actual tools the way we call like 187 04:06:14,319 --> 04:06:15,750 think of tools. So it's like reading a 188 04:06:15,760 --> 04:06:18,229 file editing a file you know reading 189 04:06:18,239 --> 04:06:20,070 searching something on web uh listing 190 04:06:20,080 --> 04:06:22,070 listing a folder searching and grapping 191 04:06:22,080 --> 04:06:24,469 through you know uh a file system things 192 04:06:24,479 --> 04:06:27,910 like that. So and in 2024 we had the 193 04:06:27,920 --> 04:06:29,750 intuition that like chatbot what if 194 04:06:29,760 --> 04:06:31,429 chatbox could actually talk to each 195 04:06:31,439 --> 04:06:33,189 other you know so then that's was the 196 04:06:33,199 --> 04:06:35,510 first time when we had like you know uh 197 04:06:35,520 --> 04:06:37,269 multi-agent conversations now at the 198 04:06:37,279 --> 04:06:38,790 time these things were very janky right 199 04:06:38,800 --> 04:06:40,630 like they were just like taped together 200 04:06:40,640 --> 04:06:44,309 in this like weird weird uh bizarre ways 201 04:06:44,319 --> 04:06:45,990 where you have just like these classes 202 04:06:46,000 --> 04:06:47,510 and these links and then like these 203 04:06:47,520 --> 04:06:49,670 hierarchies of classes and it was like 204 04:06:49,680 --> 04:06:50,950 it was all like kind of like a very 205 04:06:50,960 --> 04:06:53,670 janky attempt to like make us think that 206 04:06:53,680 --> 04:06:55,349 like these these models to like have 207 04:06:55,359 --> 04:06:56,870 these pipeline of flows of responses 208 04:06:56,880 --> 04:06:58,070 that can all communicate with each 209 04:06:58,080 --> 04:06:59,990 other. [snorts] A lot of it was like 210 04:07:00,000 --> 04:07:02,309 mostly deterministic and it was like you 211 04:07:02,319 --> 04:07:03,990 know it was not as free flowing as it is 212 04:07:04,000 --> 04:07:07,590 right now. So and you know we started 213 04:07:07,600 --> 04:07:08,870 getting the intention. So this was the 214 04:07:08,880 --> 04:07:11,030 autogen project but like in only in 2025 215 04:07:11,040 --> 04:07:12,950 we we started [snorts] getting the like 216 04:07:12,960 --> 04:07:14,150 what if the agents just kept getting 217 04:07:14,160 --> 04:07:15,750 better and faster and cheaper like what 218 04:07:15,760 --> 04:07:17,910 what if that just kept happening and 219 04:07:17,920 --> 04:07:21,590 then in 2026 uh we we had the intuition 220 04:07:21,600 --> 04:07:23,269 that like what if what if the agent 221 04:07:23,279 --> 04:07:25,189 could improve itself which is like the 222 04:07:25,199 --> 04:07:27,429 final you know the final part of the 223 04:07:27,439 --> 04:07:29,349 whole process. So let me give an example 224 04:07:29,359 --> 04:07:30,870 like what does it mean for an agent to 225 04:07:30,880 --> 04:07:32,710 improve itself right? So like it's like 226 04:07:32,720 --> 04:07:34,389 a lot of times when we think of like 227 04:07:34,399 --> 04:07:36,950 evaluating an agent, we initially used 228 04:07:36,960 --> 04:07:38,550 to think of evaluating an agent in the 229 04:07:38,560 --> 04:07:40,229 form of like just a single turn 230 04:07:40,239 --> 04:07:42,309 conversation. So we would tell we would 231 04:07:42,319 --> 04:07:45,830 tell like um um agents that like ask a 232 04:07:45,840 --> 04:07:48,070 simple question like how many toes does 233 04:07:48,080 --> 04:07:50,070 a dash animal has? It's like it could be 234 04:07:50,080 --> 04:07:51,750 let's say how many toes does a cat have 235 04:07:51,760 --> 04:07:53,429 and then al would go through and the 236 04:07:53,439 --> 04:07:55,030 element would like prepare response and 237 04:07:55,040 --> 04:07:56,469 then we would get the response. So like 238 04:07:56,479 --> 04:07:58,710 a cat has maybe 10 toes, 18 toes, 239 04:07:58,720 --> 04:08:00,150 however many. I don't I actually don't 240 04:08:00,160 --> 04:08:01,830 know how many toes a cat has. But 241 04:08:01,840 --> 04:08:05,910 anyway, um yeah. So then we create the 242 04:08:05,920 --> 04:08:07,510 response as like a yes or no. But like 243 04:08:07,520 --> 04:08:09,110 when you work with an agent, there's a 244 04:08:09,120 --> 04:08:10,790 ton of things that you're doing. So let 245 04:08:10,800 --> 04:08:12,070 me give an example. Let me give an 246 04:08:12,080 --> 04:08:13,750 example. So for example, you're going to 247 04:08:13,760 --> 04:08:15,750 ask an agent like hey set up this MCP 248 04:08:15,760 --> 04:08:18,229 for me. So now it's not just like it's 249 04:08:18,239 --> 04:08:20,070 not just like you know a trivial 250 04:08:20,080 --> 04:08:22,229 oneshotable like thing like it will like 251 04:08:22,239 --> 04:08:24,790 search online and go what's the MCP? 252 04:08:24,800 --> 04:08:26,550 what is the config? So, I'll like edit 253 04:08:26,560 --> 04:08:28,389 and download some soft files. Like maybe 254 04:08:28,399 --> 04:08:30,790 I need to add database update the MCB 255 04:08:30,800 --> 04:08:32,870 tool. Maybe I need to read and edit some 256 04:08:32,880 --> 04:08:34,630 files. Maybe I need to run some bash 257 04:08:34,640 --> 04:08:36,630 commands. Right? So, you do all of that 258 04:08:36,640 --> 04:08:38,309 and then the agent response like, hey, I 259 04:08:38,319 --> 04:08:40,309 did it. And then you grade that with a 260 04:08:40,319 --> 04:08:42,469 bunch of tests to gauge like, hey, did 261 04:08:42,479 --> 04:08:44,309 the agent actually accomplish the thing 262 04:08:44,319 --> 04:08:47,269 that it said it would do? So to do that 263 04:08:47,279 --> 04:08:48,950 thing the second part which is like more 264 04:08:48,960 --> 04:08:51,030 intricate where you're actually like 265 04:08:51,040 --> 04:08:54,070 truly you know testing agents like uh to 266 04:08:54,080 --> 04:08:56,309 their best extent like it's like you 267 04:08:56,319 --> 04:08:58,389 know um we use this thing called 268 04:08:58,399 --> 04:09:00,070 terminal bench. terminal bench is a 269 04:09:00,080 --> 04:09:01,910 benchmark for you know AI agents in 270 04:09:01,920 --> 04:09:03,590 terminal environments and that is 271 04:09:03,600 --> 04:09:05,510 something that we are we have a lot of 272 04:09:05,520 --> 04:09:07,990 like experience with at client and uh 273 04:09:08,000 --> 04:09:09,349 it's like a it's like one of the best 274 04:09:09,359 --> 04:09:11,189 ways to like really gauge like is your 275 04:09:11,199 --> 04:09:13,590 coding agent accomplishing uh in general 276 04:09:13,600 --> 04:09:15,670 the task that you're doing uh that you 277 04:09:15,680 --> 04:09:18,070 want to be doing with age model. So 278 04:09:18,080 --> 04:09:19,510 there's a couple there's a many 279 04:09:19,520 --> 04:09:21,910 different parts of evaluation suit. So 280 04:09:21,920 --> 04:09:25,590 ideally you want that to run in um you 281 04:09:25,600 --> 04:09:27,189 know [snorts] you you want that to run 282 04:09:27,199 --> 04:09:29,830 in um individual sandboxes. Then you 283 04:09:29,840 --> 04:09:31,750 have like independent graders that like 284 04:09:31,760 --> 04:09:33,750 check everything like how many turns it 285 04:09:33,760 --> 04:09:35,189 took, how many tokens it took, how much 286 04:09:35,199 --> 04:09:37,349 latency it took, how many how many how 287 04:09:37,359 --> 04:09:39,030 many tool calls did it make, what was 288 04:09:39,040 --> 04:09:40,630 the trajectory looking like and then 289 04:09:40,640 --> 04:09:42,309 grading all the results for things like 290 04:09:42,319 --> 04:09:43,910 hey did you not reward hack? Did you not 291 04:09:43,920 --> 04:09:45,110 cheat through the solution? Did you not 292 04:09:45,120 --> 04:09:47,030 just like look up the solution online? 293 04:09:47,040 --> 04:09:50,389 Things like that. Um so we we basically 294 04:09:50,399 --> 04:09:52,870 like a lot of these uh data sets of 295 04:09:52,880 --> 04:09:55,510 evaluation problems have a set of tasks. 296 04:09:55,520 --> 04:09:57,830 So when a terminal bench 2.1 they have 297 04:09:57,840 --> 04:10:00,229 89 tasks. So what we do is like we spin 298 04:10:00,239 --> 04:10:03,110 up 89 parallel containers where they all 299 04:10:03,120 --> 04:10:05,429 run online and they run in parallel and 300 04:10:05,439 --> 04:10:07,750 they're sandbox and then they each try 301 04:10:07,760 --> 04:10:10,070 to solve a problem. So then they're all 302 04:10:10,080 --> 04:10:11,910 uh in parallel and independent of each 303 04:10:11,920 --> 04:10:14,469 other. And then at the end we get get a 304 04:10:14,479 --> 04:10:16,309 score. So our score could like give us 305 04:10:16,319 --> 04:10:19,189 measures like is is a is this is the 306 04:10:19,199 --> 04:10:21,510 model performing well and what kind of 307 04:10:21,520 --> 04:10:23,110 failures it has right so like you run 308 04:10:23,120 --> 04:10:24,870 the evaluation you get a score so you 309 04:10:24,880 --> 04:10:26,950 get an aggregate score and there 310 04:10:26,960 --> 04:10:28,710 aggregate score could reveal a set of 311 04:10:28,720 --> 04:10:30,309 failures okay failures could be of any 312 04:10:30,319 --> 04:10:32,550 kind failures could be hey the read file 313 04:10:32,560 --> 04:10:34,229 tool is not working sometimes the 314 04:10:34,239 --> 04:10:36,229 entrance is so slow that like you know 315 04:10:36,239 --> 04:10:37,830 the agent's working it's doing it's 316 04:10:37,840 --> 04:10:39,510 making progress maybe it's running a 317 04:10:39,520 --> 04:10:40,790 circle but it's so slow that it 318 04:10:40,800 --> 04:10:42,389 literally like runs out of time it runs 319 04:10:42,399 --> 04:10:44,469 out of like 45 minutes or where that 320 04:10:44,479 --> 04:10:46,070 were given for the task and just like 321 04:10:46,080 --> 04:10:48,630 stop working. So once you do that once 322 04:10:48,640 --> 04:10:50,469 you understand hey this is what we ran 323 04:10:50,479 --> 04:10:52,070 this is what the failures we got we 324 04:10:52,080 --> 04:10:53,590 bucket the failures in different 325 04:10:53,600 --> 04:10:55,510 categories. So like 10 failures of you 326 04:10:55,520 --> 04:10:57,349 know this kind of tool called 10 10 20 327 04:10:57,359 --> 04:10:58,790 failures of like say inference not 328 04:10:58,800 --> 04:11:00,389 working or something else right and then 329 04:11:00,399 --> 04:11:01,990 we once we bucket the failures into 330 04:11:02,000 --> 04:11:03,429 different categories where like we try 331 04:11:03,439 --> 04:11:04,870 to make an improvement to the harness 332 04:11:04,880 --> 04:11:06,710 because like what if there's like 333 04:11:06,720 --> 04:11:08,150 something wrong with the harness that we 334 04:11:08,160 --> 04:11:09,750 could change such that the model score 335 04:11:09,760 --> 04:11:11,750 would improve right and on and on and on 336 04:11:11,760 --> 04:11:13,429 and you do this iteratively it takes a 337 04:11:13,439 --> 04:11:14,870 while it's just like you know hill 338 04:11:14,880 --> 04:11:16,950 climbing is like a very groundworky kind 339 04:11:16,960 --> 04:11:19,830 of process and you know usually usually 340 04:11:19,840 --> 04:11:22,150 we we took we used to take a lot of time 341 04:11:22,160 --> 04:11:23,670 trying to do this but it was a great way 342 04:11:23,680 --> 04:11:25,590 for us to like learn how to improve 343 04:11:25,600 --> 04:11:28,150 models. So [snorts] one way to hill 344 04:11:28,160 --> 04:11:32,309 climb on top is that like you know we we 345 04:11:32,319 --> 04:11:34,790 could have a thing where you know we 346 04:11:34,800 --> 04:11:36,710 started doing this thing where we could 347 04:11:36,720 --> 04:11:39,110 just like what if we could have like a 348 04:11:39,120 --> 04:11:40,790 longunning agent that runs for a while 349 04:11:40,800 --> 04:11:42,550 and automatically improves the model 350 04:11:42,560 --> 04:11:44,950 score and that's what we did. So we ran 351 04:11:44,960 --> 04:11:47,590 GPD 5.6 soul that led the work and that 352 04:11:47,600 --> 04:11:50,070 was like the main agent and then uh Ky 353 04:11:50,080 --> 04:11:52,150 K3 was just running eval. So what it was 354 04:11:52,160 --> 04:11:54,710 doing is that like GBD 5.6 was like hey 355 04:11:54,720 --> 04:11:57,349 run these events get these cores let me 356 04:11:57,359 --> 04:11:58,870 look at the failures let me look at 357 04:11:58,880 --> 04:12:01,030 where where it went wrong and then I'll 358 04:12:01,040 --> 04:12:03,030 make changes to client's harness and 359 04:12:03,040 --> 04:12:05,110 then I'll run these eval again and then 360 04:12:05,120 --> 04:12:07,110 I'll keep trying. So again, you can see 361 04:12:07,120 --> 04:12:09,030 how like very quickly this can like go 362 04:12:09,040 --> 04:12:10,469 off the rails because this is supposed 363 04:12:10,479 --> 04:12:11,830 to be like the running and evaluation 364 04:12:11,840 --> 04:12:13,830 itself takes like an hour. Then you have 365 04:12:13,840 --> 04:12:15,189 to read the feedback. Then you have to 366 04:12:15,199 --> 04:12:17,429 like see if you're properly interpreting 367 04:12:17,439 --> 04:12:19,269 the feedback to make reasonable changes 368 04:12:19,279 --> 04:12:21,030 to the harness that are like not reward 369 04:12:21,040 --> 04:12:22,150 hacking that are just like actual 370 04:12:22,160 --> 04:12:23,429 genuine improvements and then run the 371 04:12:23,439 --> 04:12:26,150 eval again. So we ran that for 17 hours 372 04:12:26,160 --> 04:12:28,950 and uh when we ran that for 17 hours 373 04:12:28,960 --> 04:12:30,309 there was like a legitimate hell 374 04:12:30,319 --> 04:12:32,389 climbing and what we found that was very 375 04:12:32,399 --> 04:12:34,469 interesting. This happened with a very 376 04:12:34,479 --> 04:12:36,550 detailed prompt that accomplished this 377 04:12:36,560 --> 04:12:39,110 is that like we had baseline score of 378 04:12:39,120 --> 04:12:43,349 27% and then you know we we found issues 379 04:12:43,359 --> 04:12:44,870 like loop detection loop detection is 380 04:12:44,880 --> 04:12:46,469 when a harness just runs in circles 381 04:12:46,479 --> 04:12:48,870 around a problem right uh we found like 382 04:12:48,880 --> 04:12:50,309 background process grinding where 383 04:12:50,319 --> 04:12:52,550 background processes would run too long 384 04:12:52,560 --> 04:12:54,550 uh we found some file index issues and 385 04:12:54,560 --> 04:12:56,469 other things as well I mean when I say 386 04:12:56,479 --> 04:12:59,670 we what I actually me 5.6 solve right so 387 04:12:59,680 --> 04:13:01,510 that found those issues that found and 388 04:13:01,520 --> 04:13:03,670 bucketed those issues it's like hey how 389 04:13:03,680 --> 04:13:05,110 about I make those changes in the 390 04:13:05,120 --> 04:13:07,110 harness and I run the eval again and it 391 04:13:07,120 --> 04:13:08,710 iteratively built the hill climbing 392 04:13:08,720 --> 04:13:10,469 process completely autonomously on its 393 04:13:10,479 --> 04:13:12,710 own without my interference and 13 hours 394 04:13:12,720 --> 04:13:14,389 later when I come back to it like it has 395 04:13:14,399 --> 04:13:16,630 improved the harness's performance in so 396 04:13:16,640 --> 04:13:19,510 many ways so see what does that mean 397 04:13:19,520 --> 04:13:20,870 what does that mean it just it just 398 04:13:20,880 --> 04:13:23,349 basically means that like we're at a 399 04:13:23,359 --> 04:13:25,990 point where the experience could make 400 04:13:26,000 --> 04:13:27,510 the harness better that like there's 401 04:13:27,520 --> 04:13:29,429 like a tool level learning loop where 402 04:13:29,439 --> 04:13:31,750 the harness does some action, interacts 403 04:13:31,760 --> 04:13:33,269 with the environment, understands 404 04:13:33,279 --> 04:13:34,790 whether or not it did the right thing or 405 04:13:34,800 --> 04:13:37,269 the wrong thing and gets some reward and 406 04:13:37,279 --> 04:13:39,349 and like almost and then has there's 407 04:13:39,359 --> 04:13:41,349 like a meta agent that's observing the 408 04:13:41,359 --> 04:13:44,309 main harness and finding the changes to 409 04:13:44,319 --> 04:13:45,990 it and just like incorporating those 410 04:13:46,000 --> 04:13:48,070 changes and improving. Now it's I think 411 04:13:48,080 --> 04:13:49,590 in my personal opinion it's much harder 412 04:13:49,600 --> 04:13:51,429 to do in real time because if you do it 413 04:13:51,439 --> 04:13:53,990 in real time you overindex on solving 414 04:13:54,000 --> 04:13:56,870 certain problems or you're just like too 415 04:13:56,880 --> 04:13:59,189 cartel to a specific thing. But I do 416 04:13:59,199 --> 04:14:00,630 think that this is something that works 417 04:14:00,640 --> 04:14:02,070 like on a broader scale of like working 418 04:14:02,080 --> 04:14:04,630 on a wider range of problems [snorts] 419 04:14:04,640 --> 04:14:06,870 and you want to be able to like you want 420 04:14:06,880 --> 04:14:08,229 you want to be able to justify the 421 04:14:08,239 --> 04:14:10,630 changes you're making on a broader 422 04:14:10,640 --> 04:14:12,870 applicability so that the changes you 423 04:14:12,880 --> 04:14:14,950 make here will apply to a broader range 424 04:14:14,960 --> 04:14:18,229 of problems as well. So what does that 425 04:14:18,239 --> 04:14:21,030 mean? Like what am I trying to get at 426 04:14:21,040 --> 04:14:23,030 here? The point I'm trying to get at is 427 04:14:23,040 --> 04:14:24,469 that the models are so good right now 428 04:14:24,479 --> 04:14:25,830 that they can if they can improve the 429 04:14:25,840 --> 04:14:27,189 harness, they can improve a lot of other 430 04:14:27,199 --> 04:14:28,630 things. So then what else can they 431 04:14:28,640 --> 04:14:31,510 improve? So this is this is a this is a 432 04:14:31,520 --> 04:14:34,309 post uh primarily from OpenAI where GP 433 04:14:34,319 --> 04:14:37,189 5.6 autonomously rewrote and optimized 434 04:14:37,199 --> 04:14:39,590 its own production kernels and those 435 04:14:39,600 --> 04:14:41,670 production kernel improvements remove 436 04:14:41,680 --> 04:14:44,710 the end to-end serving costs by 20%. you 437 04:14:44,720 --> 04:14:47,910 know it also had 15% higher to token 438 04:14:47,920 --> 04:14:49,590 generation efficiency and this was like 439 04:14:49,600 --> 04:14:52,630 completely done by 5.6 soul on itself, 440 04:14:52,640 --> 04:14:55,030 right? So what that means is that like 441 04:14:55,040 --> 04:14:57,030 inference is becoming much cheaper and 442 04:14:57,040 --> 04:14:59,349 that means that like if on if if for a 443 04:14:59,359 --> 04:15:01,990 company like OpenAI if you're having 20% 444 04:15:02,000 --> 04:15:04,550 low lower serving cost for a frontier 445 04:15:04,560 --> 04:15:06,309 model that could be a billions of 446 04:15:06,319 --> 04:15:07,990 dollars of improvement in price and you 447 04:15:08,000 --> 04:15:09,510 could you could do that as well for your 448 04:15:09,520 --> 04:15:12,790 own models. Um another interesting thing 449 04:15:12,800 --> 04:15:14,309 is that like when people have kernel 450 04:15:14,319 --> 04:15:16,469 writing uh competitions so core auto is 451 04:15:16,479 --> 04:15:18,389 a company which has a kernel writing 452 04:15:18,399 --> 04:15:20,950 competitions and you'll notice that like 453 04:15:20,960 --> 04:15:23,349 they they mentioned that like the person 454 04:15:23,359 --> 04:15:25,110 who won the kernel writing competition 455 04:15:25,120 --> 04:15:27,189 the explic said that I have never 456 04:15:27,199 --> 04:15:29,670 handwritten a GPU operator it's all ALM 457 04:15:29,680 --> 04:15:32,469 and agent driven so notice that like how 458 04:15:32,479 --> 04:15:35,030 like the best uh inference improvements 459 04:15:35,040 --> 04:15:36,790 kernel improvements are at this point 460 04:15:36,800 --> 04:15:39,030 completely exclusively done by a swarm 461 04:15:39,040 --> 04:15:41,590 of agents or generally like agents doing 462 04:15:41,600 --> 04:15:43,670 some magic to you know magically improve 463 04:15:43,680 --> 04:15:45,910 like the kernel kernel code and have 464 04:15:45,920 --> 04:15:48,790 faster inference cheaper inference. Um 465 04:15:48,800 --> 04:15:51,910 yeah so basically like the improvement 466 04:15:51,920 --> 04:15:54,070 surface that you think of is not just 467 04:15:54,080 --> 04:15:55,670 like the harness improvement it's 468 04:15:55,680 --> 04:15:57,510 literally the whole surface stack right 469 04:15:57,520 --> 04:16:00,309 so the whole stack from like evaluation 470 04:16:00,319 --> 04:16:02,229 to harness improvements to training data 471 04:16:02,239 --> 04:16:03,990 to post training to world interaction to 472 04:16:04,000 --> 04:16:06,710 inference like literally the entire pip 473 04:16:06,720 --> 04:16:08,630 part of the pipeline like all the way 474 04:16:08,640 --> 04:16:11,750 from like electricity going through your 475 04:16:11,760 --> 04:16:14,389 machine to like making a network request 476 04:16:14,399 --> 04:16:16,070 going through the model models doing 477 04:16:16,080 --> 04:16:18,309 inference um making a response and 478 04:16:18,319 --> 04:16:19,750 getting back to you like every single 479 04:16:19,760 --> 04:16:21,830 part of the pipeline we can optimize it. 480 04:16:21,840 --> 04:16:23,990 So the whole RSI loop has like um 481 04:16:24,000 --> 04:16:25,429 there's a couple different parts so it's 482 04:16:25,439 --> 04:16:28,229 like pre-training um uh neatron examples 483 04:16:28,239 --> 04:16:30,229 of like you know AI curated uh 484 04:16:30,239 --> 04:16:32,149 pre-training data that was good um 485 04:16:32,159 --> 04:16:33,590 there's like post- training examples 486 04:16:33,600 --> 04:16:36,309 which are you know where there's like 487 04:16:36,319 --> 04:16:38,630 constitutional AI examples of like post- 488 04:16:38,640 --> 04:16:40,710 training uh improvements then inference 489 04:16:40,720 --> 04:16:43,670 I gave you the example of GP 5.6 So for 490 04:16:43,680 --> 04:16:45,990 eval generation, I give you our own 491 04:16:46,000 --> 04:16:47,830 example of how we improved ourselves on 492 04:16:47,840 --> 04:16:50,550 our own eval. Um, in general, what you 493 04:16:50,560 --> 04:16:52,710 want to think of is that like anytime a 494 04:16:52,720 --> 04:16:54,229 model is trying to do something and it's 495 04:16:54,239 --> 04:16:55,990 like not able to do it or not able to 496 04:16:56,000 --> 04:16:57,670 accomplish it. So for example, like 497 04:16:57,680 --> 04:16:58,950 imagine you're having a long 498 04:16:58,960 --> 04:17:00,790 conversation where you ask the model 499 04:17:00,800 --> 04:17:03,030 something obvious. So like it's like or 500 04:17:03,040 --> 04:17:05,110 something that you think is not 501 04:17:05,120 --> 04:17:07,349 necessarily obvious, but it's like it's 502 04:17:07,359 --> 04:17:08,870 hard, but you would assume that the 503 04:17:08,880 --> 04:17:10,469 model could potentially do it. So to 504 04:17:10,479 --> 04:17:12,309 give an example, let's say you want to 505 04:17:12,319 --> 04:17:15,269 make add a new column to a database and 506 04:17:15,279 --> 04:17:17,750 then fill that column, right? And then 507 04:17:17,760 --> 04:17:19,830 put a cache in front of it. So what 508 04:17:19,840 --> 04:17:21,429 happens is it does the whole thing, but 509 04:17:21,439 --> 04:17:23,030 for some reason the way it pushes the 510 04:17:23,040 --> 04:17:24,950 cache in front of the whole uh database 511 04:17:24,960 --> 04:17:26,389 like it kind of doesn't do it right. So 512 04:17:26,399 --> 04:17:28,389 at that point you're like, "Oh no, set 513 04:17:28,399 --> 04:17:30,070 this cache differently. Set this as the 514 04:17:30,080 --> 04:17:31,510 TTL." And then you give like a 515 04:17:31,520 --> 04:17:33,750 correction signal and then that kind of 516 04:17:33,760 --> 04:17:35,429 like fixes your problem and you kind of 517 04:17:35,439 --> 04:17:36,790 go about your day. 518 04:17:36,800 --> 04:17:38,469 The way I want to think of it is that I 519 04:17:38,479 --> 04:17:40,229 want to think of like every single time 520 04:17:40,239 --> 04:17:42,950 like a human correction to the digital 521 04:17:42,960 --> 04:17:47,110 landscape is a part of a delta that 522 04:17:47,120 --> 04:17:49,750 needs to be covered. And what that means 523 04:17:49,760 --> 04:17:51,349 is that like every single time you're 524 04:17:51,359 --> 04:17:52,950 finding like the edge of model 525 04:17:52,960 --> 04:17:54,710 capabilities of like how far it can push 526 04:17:54,720 --> 04:17:56,070 for infants, how much it can push for 527 04:17:56,080 --> 04:17:58,389 this, how much it can push for, you 528 04:17:58,399 --> 04:18:00,950 know, solving your problems. You want to 529 04:18:00,960 --> 04:18:02,790 find that delta and we want to like push 530 04:18:02,800 --> 04:18:04,790 that frontier a little bit further. a 531 04:18:04,800 --> 04:18:06,710 little bit further and that's what post 532 04:18:06,720 --> 04:18:08,149 training does, right? So post training 533 04:18:08,159 --> 04:18:11,590 is is this is in that post training we 534 04:18:11,600 --> 04:18:13,670 find these like really interesting and 535 04:18:13,680 --> 04:18:15,830 really hard long horizon problems for uh 536 04:18:15,840 --> 04:18:17,349 for coding agents or for other kinds of 537 04:18:17,359 --> 04:18:18,950 problems and then just like slowly hill 538 04:18:18,960 --> 04:18:21,189 climb the models on on those kind of 539 04:18:21,199 --> 04:18:22,870 problems. So anything that has a 540 04:18:22,880 --> 04:18:26,710 verifiable loop can be optimized but is 541 04:18:26,720 --> 04:18:29,030 this is not just limited to software 542 04:18:29,040 --> 04:18:30,710 like literally like the physical world 543 04:18:30,720 --> 04:18:33,590 itself could have a verifiable loop. So 544 04:18:33,600 --> 04:18:35,269 to give an example like let me let me 545 04:18:35,279 --> 04:18:37,189 give you a very funny kind of bizarre 546 04:18:37,199 --> 04:18:40,309 example where like it's like you you 547 04:18:40,319 --> 04:18:44,309 look at this and uh you um 548 04:18:44,319 --> 04:18:46,790 um look at this uh like robot and it's 549 04:18:46,800 --> 04:18:49,189 it's typing something and it's like 550 04:18:49,199 --> 04:18:51,590 really really like interesting what the 551 04:18:51,600 --> 04:19:01,429 robot is typing. 552 04:19:01,439 --> 04:19:03,590 So notice how like notice how the robot 553 04:19:03,600 --> 04:19:05,830 is itself typing like how it wants to 554 04:19:05,840 --> 04:19:08,229 like solve robotics and imagine if the 555 04:19:08,239 --> 04:19:10,229 robot itself could like interpret the 556 04:19:10,239 --> 04:19:13,269 results of of like the answers and just 557 04:19:13,279 --> 04:19:15,189 like keep improving then that means that 558 04:19:15,199 --> 04:19:17,030 like this is you know basically like an 559 04:19:17,040 --> 04:19:19,110 ASI robot. Another example that I could 560 04:19:19,120 --> 04:19:21,990 give you for my uh personal life is that 561 04:19:22,000 --> 04:19:25,429 like um I have this um chip that has the 562 04:19:25,439 --> 04:19:28,149 ability to see and has a speaker and has 563 04:19:28,159 --> 04:19:29,590 a microphone inside of it. So I was 564 04:19:29,600 --> 04:19:32,790 trying to set up this uh with my machine 565 04:19:32,800 --> 04:19:36,389 and it was um a gift that I had for my 566 04:19:36,399 --> 04:19:38,309 girlfriend. And uh what I found very 567 04:19:38,319 --> 04:19:40,149 interesting about this is as I was 568 04:19:40,159 --> 04:19:41,990 trying to set it up, it was very slow. 569 04:19:42,000 --> 04:19:43,590 It was slow to respond. It had higher 570 04:19:43,600 --> 04:19:45,110 latency. It had an agent in and of 571 04:19:45,120 --> 04:19:47,269 itself. Right? So what I did was I 572 04:19:47,279 --> 04:19:51,110 connected the step to my um computer and 573 04:19:51,120 --> 04:19:55,670 I let um I let um putting agents run on 574 04:19:55,680 --> 04:19:58,309 this hardware and basically run on this 575 04:19:58,319 --> 04:20:01,510 hardware automatically make tests to it. 576 04:20:01,520 --> 04:20:04,389 So like my computer would talk to my 577 04:20:04,399 --> 04:20:06,710 hardware give it some tests like hey do 578 04:20:06,720 --> 04:20:08,469 this do that and just like what do you 579 04:20:08,479 --> 04:20:11,110 see right now? So then it would notice 580 04:20:11,120 --> 04:20:12,710 the end to end latency of like the 581 04:20:12,720 --> 04:20:15,030 inference response of like how fast uh 582 04:20:15,040 --> 04:20:17,990 my hardware is responding and then um 583 04:20:18,000 --> 04:20:19,990 what I found is that like it was able to 584 04:20:20,000 --> 04:20:22,309 do the breakdown of like every single 585 04:20:22,319 --> 04:20:24,149 case and scenario where there was like 586 04:20:24,159 --> 04:20:28,229 high latency um slower response or or 587 04:20:28,239 --> 04:20:30,550 just the algorithm wasn't right and I 588 04:20:30,560 --> 04:20:33,670 could then set up a recursive loop where 589 04:20:33,680 --> 04:20:36,149 both the firmware could be updated the 590 04:20:36,159 --> 04:20:38,870 um the tool calls the inference model 591 04:20:38,880 --> 04:20:41,429 used um the speech to text model used 592 04:20:41,439 --> 04:20:42,469 all of those could be improved 593 04:20:42,479 --> 04:20:44,309 automatically with like an open router 594 04:20:44,319 --> 04:20:46,149 key which I'd given to a coding agent 595 04:20:46,159 --> 04:20:47,990 and then overnight with self-running 596 04:20:48,000 --> 04:20:51,349 experiments with the hardware uh my um 597 04:20:51,359 --> 04:20:52,710 this tiny little piece of beautiful 598 04:20:52,720 --> 04:20:54,710 hardware that I have was like deeply 599 04:20:54,720 --> 04:20:57,189 optimized so this also like even 600 04:20:57,199 --> 04:20:59,030 hardware could be a part of the RSI loop 601 04:20:59,040 --> 04:21:00,710 so what happens when you connect these 602 04:21:00,720 --> 04:21:01,910 loops right that's the next question 603 04:21:01,920 --> 04:21:03,830 like you could have cheaper inference 604 04:21:03,840 --> 04:21:05,189 you could have like more experiments 605 04:21:05,199 --> 04:21:07,590 budget you have like you know better 606 04:21:07,600 --> 04:21:09,590 data and better models. So, is it 607 04:21:09,600 --> 04:21:12,149 singularity? It's a very question to ask 608 04:21:12,159 --> 04:21:13,510 and I think that the part of the parts 609 04:21:13,520 --> 04:21:14,870 of the loop are already weird, right? 610 04:21:14,880 --> 04:21:17,110 Like we're we we are really we really 611 04:21:17,120 --> 04:21:19,990 are very close to parts of the RSI loop, 612 04:21:20,000 --> 04:21:22,630 right? And what do we do like what do we 613 04:21:22,640 --> 04:21:24,149 do about this and it's like how do we 614 04:21:24,159 --> 04:21:26,389 think of that in terms of like our 615 04:21:26,399 --> 04:21:27,910 career and how do you think of that in 616 04:21:27,920 --> 04:21:29,429 terms of you know our lives and what do 617 04:21:29,439 --> 04:21:31,510 we want to do next? So here's my advice 618 04:21:31,520 --> 04:21:33,349 to everyone who is watching this like 619 04:21:33,359 --> 04:21:35,269 please please please please move closer 620 04:21:35,279 --> 04:21:38,149 to the RS group right so it's like right 621 04:21:38,159 --> 04:21:39,910 now like people are working on all these 622 04:21:39,920 --> 04:21:41,429 like different kinds of software which 623 04:21:41,439 --> 04:21:43,189 is like you know a agents for you know 624 04:21:43,199 --> 04:21:44,710 construction agents for this agents for 625 04:21:44,720 --> 04:21:47,030 that and it's like I think that you're 626 04:21:47,040 --> 04:21:50,070 you know what the way I see those things 627 04:21:50,080 --> 04:21:51,510 is that like I think of those as like 628 04:21:51,520 --> 04:21:52,870 you're missing the forest for the for 629 04:21:52,880 --> 04:21:55,110 the trees you really are and it's like 630 04:21:55,120 --> 04:21:58,070 you have this like very interesting 631 04:21:58,080 --> 04:22:00,309 godlike technology that can change parts 632 04:22:00,319 --> 04:22:02,630 of humanity. And then we're not 633 04:22:02,640 --> 04:22:04,309 ambitious enough to pursue the best 634 04:22:04,319 --> 04:22:06,469 problems. We really are not. And we're 635 04:22:06,479 --> 04:22:08,389 just like I think that it's not just 636 04:22:08,399 --> 04:22:10,469 like the problem isn't just that we're 637 04:22:10,479 --> 04:22:12,309 not ambitious and we're also like what 638 04:22:12,319 --> 04:22:14,710 we're pursuing. A lot of those things 639 04:22:14,720 --> 04:22:16,550 are just like one model release away 640 04:22:16,560 --> 04:22:18,229 from like being completely like wiped 641 04:22:18,239 --> 04:22:20,550 out, right? So like it's also like it's 642 04:22:20,560 --> 04:22:22,309 also like it may not be that 643 04:22:22,319 --> 04:22:23,750 intellectually simulating because like 644 04:22:23,760 --> 04:22:25,030 all you're essentially all we're 645 04:22:25,040 --> 04:22:26,149 essentially doing is like sitting all 646 04:22:26,159 --> 04:22:27,429 day. It's like hey cloud do this for me. 647 04:22:27,439 --> 04:22:28,710 Hey cloud do this for me. And I don't 648 04:22:28,720 --> 04:22:29,830 know if that's like I don't know if 649 04:22:29,840 --> 04:22:31,349 that's like the most intellectual 650 04:22:31,359 --> 04:22:33,670 stimulating or like or like um 651 04:22:33,680 --> 04:22:35,750 thoughtful use of a person's time like 652 04:22:35,760 --> 04:22:37,349 maybe maybe you know we're supposed to 653 04:22:37,359 --> 04:22:39,110 be solving harder and more uh more 654 04:22:39,120 --> 04:22:40,710 frontier problems and I think that like 655 04:22:40,720 --> 04:22:42,389 it's like the ceiling is like our 656 04:22:42,399 --> 04:22:44,550 ambition. So instead of like working on 657 04:22:44,560 --> 04:22:46,070 these kind of things my suggestion to 658 04:22:46,080 --> 04:22:48,870 people is to like you know do things 659 04:22:48,880 --> 04:22:50,790 that of course like goes without saying 660 04:22:50,800 --> 04:22:52,149 but you should do things you believe in 661 04:22:52,159 --> 04:22:53,590 and you know I'm just a guy on the 662 04:22:53,600 --> 04:22:55,910 internet but do things that help you 663 04:22:55,920 --> 04:22:58,149 move closer to the RSI loop right? So 664 04:22:58,159 --> 04:22:59,910 like anything that just like the end to 665 04:22:59,920 --> 04:23:03,429 end pipeline of like hey this is where 666 04:23:03,439 --> 04:23:05,269 like the models are lacking like it's 667 04:23:05,279 --> 04:23:06,710 like whether that's inference whether 668 04:23:06,720 --> 04:23:07,910 that's post training whether that's 669 04:23:07,920 --> 04:23:09,830 literally like physical interaction 670 04:23:09,840 --> 04:23:11,990 whether that could be you know send 671 04:23:12,000 --> 04:23:13,910 sending end to-end pipelines of truly 672 04:23:13,920 --> 04:23:15,349 any kind you know whether that's 673 04:23:15,359 --> 04:23:17,189 hardware problems whether that's um 674 04:23:17,199 --> 04:23:19,110 networking problems whether that's um 675 04:23:19,120 --> 04:23:20,630 building a sense community things that 676 04:23:20,640 --> 04:23:23,030 just you know um things that just like 677 04:23:23,040 --> 04:23:27,030 help um you as an engineer get closer to 678 04:23:27,040 --> 04:23:29,429 the benefit benefits of the improvements 679 04:23:29,439 --> 04:23:33,590 um in uh in uh um in in the model. So 680 04:23:33,600 --> 04:23:35,189 this is a very interesting slide from 681 04:23:35,199 --> 04:23:39,189 Sasha topic. So um um they mentioned 682 04:23:39,199 --> 04:23:43,349 that like you want to have like this 683 04:23:43,359 --> 04:23:44,710 you're in this space where it's like you 684 04:23:44,720 --> 04:23:46,309 know you only you only have like one to 685 04:23:46,319 --> 04:23:47,830 five years left to contribute to luxury 686 04:23:47,840 --> 04:23:49,189 to the world. What do you do? Right? So 687 04:23:49,199 --> 04:23:51,349 it's like um they they gave great 688 04:23:51,359 --> 04:23:53,269 suggestions were like prefer ambitious 689 04:23:53,279 --> 04:23:55,910 targeted and past collaborations right 690 04:23:55,920 --> 04:23:58,309 and you know in general like the time 691 04:23:58,319 --> 04:23:59,830 scale like all the timelines that you 692 04:23:59,840 --> 04:24:00,950 give to yourself like they should be 693 04:24:00,960 --> 04:24:02,550 just like completely squeezed out should 694 04:24:02,560 --> 04:24:03,750 be completely squeezed out because it's 695 04:24:03,760 --> 04:24:05,349 like a lot of times and you also have to 696 04:24:05,359 --> 04:24:07,189 think through that like if if the model 697 04:24:07,199 --> 04:24:08,389 if the problem that you're working on is 698 04:24:08,399 --> 04:24:10,389 too ambitious like it's like is it 699 04:24:10,399 --> 04:24:12,550 probably is it probably better to wait 700 04:24:12,560 --> 04:24:14,229 for just like the next next frontier 701 04:24:14,239 --> 04:24:16,070 model to come out or not like that's 702 04:24:16,080 --> 04:24:17,349 that's something you also have to weigh 703 04:24:17,359 --> 04:24:21,429 through Right. And uh you you ideally 704 04:24:21,439 --> 04:24:23,110 want to work on problems that benefit 705 04:24:23,120 --> 04:24:25,349 from scale and you know you also at the 706 04:24:25,359 --> 04:24:26,550 same time want to avoid problems that 707 04:24:26,560 --> 04:24:28,309 can be solved by sale alone because then 708 04:24:28,319 --> 04:24:29,830 you know if even if you like it's like 709 04:24:29,840 --> 04:24:31,110 if you're working on a problem and then 710 04:24:31,120 --> 04:24:32,309 the next model release comes out it 711 04:24:32,319 --> 04:24:33,429 solves the problem. So that's not that's 712 04:24:33,439 --> 04:24:35,030 probably not a good thing to do. So it's 713 04:24:35,040 --> 04:24:36,550 very like you have to like think 714 04:24:36,560 --> 04:24:39,189 philosophically of like what is the 715 04:24:39,199 --> 04:24:41,590 endeavor that benefits uh greatly from 716 04:24:41,600 --> 04:24:43,189 auto improvements but like isn't also 717 04:24:43,199 --> 04:24:45,110 completely wiped out from it. And then 718 04:24:45,120 --> 04:24:47,590 ideally like it's like if the if the 719 04:24:47,600 --> 04:24:50,389 endeavor could also uh be a part of the 720 04:24:50,399 --> 04:24:53,269 the like the improvement pipeline uh 721 04:24:53,279 --> 04:24:55,349 that could be amazing. So you see a lot 722 04:24:55,359 --> 04:24:58,469 of companies raising on say um making 723 04:24:58,479 --> 04:25:00,550 data pipelines and there's a lot of 724 04:25:00,560 --> 04:25:02,630 money there. There's a a lot of 725 04:25:02,640 --> 04:25:03,910 companies working on physical hardware 726 04:25:03,920 --> 04:25:05,269 and stuff. So I think if you're a 727 04:25:05,279 --> 04:25:06,630 software engineer watching these, it's 728 04:25:06,640 --> 04:25:08,790 like I think that if you're if you feel 729 04:25:08,800 --> 04:25:11,269 like you're like here the delta of your 730 04:25:11,279 --> 04:25:13,110 intellect and ability in your 731 04:25:13,120 --> 04:25:15,510 day-to-day, it's like not contributing 732 04:25:15,520 --> 04:25:17,670 as in like you have the intellect, you 733 04:25:17,680 --> 04:25:19,590 can make a difference. But like if cloud 734 04:25:19,600 --> 04:25:21,429 is just like oneshotting all those 735 04:25:21,439 --> 04:25:22,710 problems like in your day-to-day and 736 04:25:22,720 --> 04:25:23,910 you're just like kind of like becoming a 737 04:25:23,920 --> 04:25:25,590 me proxy, it's worth to think through. 738 04:25:25,600 --> 04:25:27,189 Take a step back and think through. like 739 04:25:27,199 --> 04:25:29,269 okay where are the problems where where 740 04:25:29,279 --> 04:25:31,269 you're actually truly uh contributing 741 04:25:31,279 --> 04:25:32,790 and that they will stand the test of 742 04:25:32,800 --> 04:25:34,710 time and then that kind of reflection 743 04:25:34,720 --> 04:25:36,870 and realignment right now would I think 744 04:25:36,880 --> 04:25:39,030 like have like very interesting 745 04:25:39,040 --> 04:25:42,070 consequences for you um in the long term 746 04:25:42,080 --> 04:25:44,710 um so yeah so thank you so much you know 747 04:25:44,720 --> 04:25:46,309 thank you so much for your time like I 748 04:25:46,319 --> 04:25:48,550 think that uh I think that if you're 749 04:25:48,560 --> 04:25:49,750 watching this and if you've gone this 750 04:25:49,760 --> 04:25:51,750 far I think that you're definitely 751 04:25:51,760 --> 04:25:53,750 contemplating like one of the most 752 04:25:53,760 --> 04:25:55,110 interesting improvements in the course 753 04:25:55,120 --> 04:25:57,990 of humanity And uh you know there's many 754 04:25:58,000 --> 04:25:59,510 philosophical arguments here. Some 755 04:25:59,520 --> 04:26:02,070 people might say hey you know I I think 756 04:26:02,080 --> 04:26:04,790 I'm maybe I'm off on the timelines or 757 04:26:04,800 --> 04:26:06,469 maybe maybe I'm I'm just being too 758 04:26:06,479 --> 04:26:07,830 pessimistic or maybe I'm being too 759 04:26:07,840 --> 04:26:10,070 optimistic. But I would love to hear all 760 04:26:10,080 --> 04:26:11,429 those thoughts. So this is you know my 761 04:26:11,439 --> 04:26:13,910 QR code. Please feel free to scan it and 762 04:26:13,920 --> 04:26:15,269 let me know all your thoughts. Uh thank 763 04:26:15,279 --> 04:26:16,710 you so much. Very kind of you to give me 764 04:26:16,720 --> 04:26:17,349 your time.