1 08:31:11,760 --> 08:31:13,910 great to be here. Um I have a lot to 2 08:31:13,920 --> 08:31:15,830 cover. I've got 18 minutes to talk about 3 08:31:15,840 --> 08:31:18,308 the last 12 months in LLMs. Are my 4 08:31:18,318 --> 08:31:21,430 slides up? My slides are up. Let's just 5 08:31:21,440 --> 08:31:23,510 let's just get get going. I want to talk 6 08:31:23,520 --> 08:31:25,510 about the last 12 months in LLMs through 7 08:31:25,520 --> 08:31:27,190 the perspective of three inflection 8 08:31:27,200 --> 08:31:29,270 points that we've had over the past 12 9 08:31:29,280 --> 08:31:31,430 months. Um, and the first of those 10 08:31:31,440 --> 08:31:33,430 conveniently was exactly 12 months ago, 11 08:31:33,440 --> 08:31:36,230 right? November 2025 is what I consider 12 08:31:36,240 --> 08:31:38,308 the coding agent inflection point. 13 08:31:38,318 --> 08:31:40,070 Coding agents weren't new. We'd had 14 08:31:40,080 --> 08:31:41,990 Claude code since February last year. 15 08:31:42,000 --> 08:31:43,510 Codeexc have been around. You know, this 16 08:31:43,520 --> 08:31:44,948 wasn't a complete new category of 17 08:31:44,958 --> 08:31:47,030 software, but something changed 12 18 08:31:47,040 --> 08:31:49,030 months ago with these two models. We had 19 08:31:49,040 --> 08:31:52,230 Claude Opocopus 4.5 and GPT 5.1 and they 20 08:31:52,240 --> 08:31:54,070 were just a little bit better at driving 21 08:31:54,080 --> 08:31:56,150 a coding agent which put them over that 22 08:31:56,160 --> 08:31:57,750 thing where these suddenly these were 23 08:31:57,760 --> 08:31:59,750 tools you could use on a daily basis. 24 08:31:59,760 --> 08:32:01,590 Like we went from coding agents as a 25 08:32:01,600 --> 08:32:03,110 novelty that you'd poke at occasionally 26 08:32:03,120 --> 08:32:04,948 and then make some weird mistakes to 27 08:32:04,958 --> 08:32:06,230 something that was reliable enough that 28 08:32:06,240 --> 08:32:07,910 you could use them every day. It was 29 08:32:07,920 --> 08:32:09,350 convenient that it happened in November 30 08:32:09,360 --> 08:32:11,350 because we then got to spend December 31 08:32:11,360 --> 08:32:12,868 basically over the holidays sort of 32 08:32:12,878 --> 08:32:14,708 poking at these new models and these new 33 08:32:14,718 --> 08:32:16,388 things and figuring what they could do. 34 08:32:16,398 --> 08:32:18,388 And by January, a lot of us had realized 35 08:32:18,398 --> 08:32:20,388 that something had changed and this was 36 08:32:20,398 --> 08:32:23,590 going to be the big theme for 2026. 37 08:32:23,600 --> 08:32:25,430 I like evaluating models by having them 38 08:32:25,440 --> 08:32:28,308 draw SVGs of pelicans riding bicycles. 39 08:32:28,318 --> 08:32:30,468 This is the state-of-the-art 12 months 40 08:32:30,478 --> 08:32:32,948 ago. This is uh Claude Opus' Pelican and 41 08:32:32,958 --> 08:32:34,788 GP 5.1's Pelican. You'll notice that 42 08:32:34,798 --> 08:32:36,628 neither of these bicycles actually work. 43 08:32:36,638 --> 08:32:38,308 So, there was definitely room for 44 08:32:38,318 --> 08:32:39,430 improvement here. They were really good 45 08:32:39,440 --> 08:32:41,110 at writing code. They weren't great at 46 08:32:41,120 --> 08:32:43,510 SVGs just yet, but but this is how we 47 08:32:43,520 --> 08:32:47,030 started the year. So every year in 48 08:32:47,040 --> 08:32:48,948 January, I set myself a New Year's 49 08:32:48,958 --> 08:32:50,868 resolution. And every year it is the 50 08:32:50,878 --> 08:32:52,308 exact same thing. Every year I tell 51 08:32:52,318 --> 08:32:53,830 myself that this year I'm going to take 52 08:32:53,840 --> 08:32:55,910 on less new projects and I'm going to 53 08:32:55,920 --> 08:32:57,590 focus I'm going to focus on the stuff 54 08:32:57,600 --> 08:32:58,948 that I've already got to do. And this 55 08:32:58,958 --> 08:33:01,670 has never worked. So this year I changed 56 08:33:01,680 --> 08:33:02,948 it. I said, you know what? I'm going to 57 08:33:02,958 --> 08:33:04,548 be more ambitious. I'm going to take on 58 08:33:04,558 --> 08:33:06,468 as many projects as I want to because 59 08:33:06,478 --> 08:33:08,708 we've got these agents. They can do 60 08:33:08,718 --> 08:33:10,628 stuff. Let's just try and push the 61 08:33:10,638 --> 08:33:12,230 limits and see what we can get done. as 62 08:33:12,240 --> 08:33:13,910 a result. Now it's November and I am 63 08:33:13,920 --> 08:33:15,910 juggling so many plates right now. It's 64 08:33:15,920 --> 08:33:17,590 kind of horrifying and I may need to 65 08:33:17,600 --> 08:33:19,990 reset my ambitions for next year. But 66 08:33:20,000 --> 08:33:21,430 that was my sort of theme of the year 67 08:33:21,440 --> 08:33:22,628 and I think it's the theme of this 68 08:33:22,638 --> 08:33:24,708 conference is to be more ambitious now 69 08:33:24,718 --> 08:33:26,788 that we have all of these new tools. I 70 08:33:26,798 --> 08:33:28,308 also went on a podcast where they asked 71 08:33:28,318 --> 08:33:31,030 me to get make some predictions for 2026 72 08:33:31,040 --> 08:33:33,590 in LLMs. Um, my predictions back in 73 08:33:33,600 --> 08:33:35,350 January were firstly it will become 74 08:33:35,360 --> 08:33:38,070 undeniable that LLMs write good code. 75 08:33:38,080 --> 08:33:39,830 Not just code, but good code. Because 76 08:33:39,840 --> 08:33:41,110 back in January, there were a lot of 77 08:33:41,120 --> 08:33:42,548 people who thought that the code was all 78 08:33:42,558 --> 08:33:44,468 complete trash and you know, you'd have 79 08:33:44,478 --> 08:33:46,628 to review every single line of it. Um, 80 08:33:46,638 --> 08:33:47,990 I've been saying this every year for 81 08:33:48,000 --> 08:33:50,070 like four years now. We're due a prompt 82 08:33:50,080 --> 08:33:52,468 injection challenger disaster. Like some 83 08:33:52,478 --> 08:33:54,150 some incident caused by a prompt 84 08:33:54,160 --> 08:33:55,670 injection attack which makes the front 85 08:33:55,680 --> 08:33:57,030 pages of newspapers. I was convinced 86 08:33:57,040 --> 08:33:58,788 that this would be the year even though 87 08:33:58,798 --> 08:34:00,230 it hadn't been the year the year before, 88 08:34:00,240 --> 08:34:01,990 the year before, the year before that. 89 08:34:02,000 --> 08:34:04,708 And um I also predicted that Kakapore 90 08:34:04,718 --> 08:34:06,628 parrots will have an outstanding 91 08:34:06,638 --> 08:34:09,990 breeding season in 2026. Um this is a 92 08:34:10,000 --> 08:34:12,788 Kakapore parrot. They are a New Zealand 93 08:34:12,798 --> 08:34:15,030 nocturnal flightless parrot. There were 94 08:34:15,040 --> 08:34:17,990 only 235 of them in the world back in 95 08:34:18,000 --> 08:34:20,628 January. Um but they only breed when the 96 08:34:20,638 --> 08:34:23,030 Remu trees have a masting fruit season, 97 08:34:23,040 --> 08:34:24,468 which hadn't happened in years. But the 98 08:34:24,478 --> 08:34:25,830 remat tries were looking really good in 99 08:34:25,840 --> 08:34:26,948 January. There was there was a lot of 100 08:34:26,958 --> 08:34:28,948 fruit coming out. So based on that, I 101 08:34:28,958 --> 08:34:30,230 thought, you know, maybe this is the 102 08:34:30,240 --> 08:34:31,990 year that we get our kakapo breeding 103 08:34:32,000 --> 08:34:34,230 explosion. And also on that podcast, we 104 08:34:34,240 --> 08:34:36,628 coined a new term. We coined the term 105 08:34:36,638 --> 08:34:39,190 deep blue to describe the AI induced 106 08:34:39,200 --> 08:34:41,750 existential crisis that everyone starts 107 08:34:41,760 --> 08:34:43,830 to feel once you realize that all of the 108 08:34:43,840 --> 08:34:45,030 things that make you special as a 109 08:34:45,040 --> 08:34:46,868 software developer, these models can do 110 08:34:46,878 --> 08:34:48,230 those things too. And they can do them 111 08:34:48,240 --> 08:34:49,910 faster and sometimes they can even do 112 08:34:49,920 --> 08:34:51,510 them better. And this has definitely 113 08:34:51,520 --> 08:34:54,308 been a growing sensation across our 114 08:34:54,318 --> 08:34:56,230 industry and other industries over the 115 08:34:56,240 --> 08:34:59,350 course of 2026 as well. Let's jump 116 08:34:59,360 --> 08:35:01,350 forward to February. In February, Google 117 08:35:01,360 --> 08:35:04,628 released Gemini 3.1 Pro. And look at 118 08:35:04,638 --> 08:35:06,868 that pelican riding a bicycle. Like 119 08:35:06,878 --> 08:35:08,628 that's a real bicycle. That bicycle is 120 08:35:08,638 --> 08:35:10,070 the right shape. It's got a little fish. 121 08:35:10,080 --> 08:35:12,388 It's got a hat on. This was by far the 122 08:35:12,398 --> 08:35:14,468 best pelican I had ever seen out of any 123 08:35:14,478 --> 08:35:18,308 model. And this I I had set a trap for 124 08:35:18,318 --> 08:35:20,308 the labs. I was, everyone's always asked 125 08:35:20,318 --> 08:35:21,830 me what happens if the labs train for 126 08:35:21,840 --> 08:35:23,990 the pelican riding a bicycle. My answer 127 08:35:24,000 --> 08:35:26,230 has always been, if they do that, I will 128 08:35:26,240 --> 08:35:27,430 catch them because I'll ask for a 129 08:35:27,440 --> 08:35:29,590 flamingo on a unicycle and when somebody 130 08:35:29,600 --> 08:35:31,590 produces a great pelican and a terrible 131 08:35:31,600 --> 08:35:32,868 flamingo, I'll know that they've been 132 08:35:32,878 --> 08:35:35,270 cheating. Unfortunately, Google's Jeff 133 08:35:35,280 --> 08:35:37,830 Dean tweeted this video the day they 134 08:35:37,840 --> 08:35:39,750 launched that model. um where he showed 135 08:35:39,760 --> 08:35:41,030 that we've got a pelican riding a 136 08:35:41,040 --> 08:35:43,830 bicycle, but we've also got a frog on a 137 08:35:43,840 --> 08:35:45,350 whatever that thing is and a giraffe in 138 08:35:45,360 --> 08:35:47,510 a little tiny car, a flamingo on roller 139 08:35:47,520 --> 08:35:50,230 skates, a fli kick flipping turtle, and 140 08:35:50,240 --> 08:35:52,548 that's a daxund in a stretch limo. So, 141 08:35:52,558 --> 08:35:54,388 they' beaten me, right? They had they 142 08:35:54,398 --> 08:35:56,150 had they have trained for all forms of 143 08:35:56,160 --> 08:35:58,468 animals in all forms of transport, which 144 08:35:58,478 --> 08:36:01,030 does sort of sort of weaken my benchmark 145 08:36:01,040 --> 08:36:02,708 quite a little bit. Um, but there was 146 08:36:02,718 --> 08:36:05,110 some good news in February. In February, 147 08:36:05,120 --> 08:36:08,230 the first kakapore chick in four years 148 08:36:08,240 --> 08:36:10,150 hatched on Valentine's Day. On 149 08:36:10,160 --> 08:36:11,750 Valentine's Day, because we hadn't had 150 08:36:11,760 --> 08:36:13,750 the remove fruit in like four years, 151 08:36:13,760 --> 08:36:15,910 this was the first one of these baby 152 08:36:15,920 --> 08:36:17,670 rare, weird New Zealand parrots that 153 08:36:17,680 --> 08:36:19,110 we'd had in four years. So, that was 154 08:36:19,120 --> 08:36:21,510 nice. Um, so my my benchmark was was was 155 08:36:21,520 --> 08:36:23,030 greatly injured, but the cacao were 156 08:36:23,040 --> 08:36:24,868 doing okay. 157 08:36:24,878 --> 08:36:27,350 Jump straight to April. In April, um, 158 08:36:27,360 --> 08:36:29,190 Anthropic released a model that they did 159 08:36:29,200 --> 08:36:31,350 not release. They announced Claude 160 08:36:31,360 --> 08:36:34,308 Mythos and they said this model is too 161 08:36:34,318 --> 08:36:36,868 dangerous for us to release. This is it 162 08:36:36,878 --> 08:36:39,030 can it's it's got new cyber cyber 163 08:36:39,040 --> 08:36:40,548 security abilities that we're not ready 164 08:36:40,558 --> 08:36:42,070 to share with the world. We'll share it 165 08:36:42,080 --> 08:36:44,388 with trusted um trust with with trusted 166 08:36:44,398 --> 08:36:45,910 partners. They actually said Mythos 167 08:36:45,920 --> 08:36:47,750 preview has already found thousands of 168 08:36:47,760 --> 08:36:49,750 high severity vulnerabilities including 169 08:36:49,760 --> 08:36:51,670 some in every major operating system and 170 08:36:51,680 --> 08:36:54,230 web browser. And a lot of people were 171 08:36:54,240 --> 08:36:56,708 quite skeptical because the the the 172 08:36:56,718 --> 08:36:58,468 thing where labs say our model is too 173 08:36:58,478 --> 08:37:00,388 bad is too dangerous to release. It's 174 08:37:00,398 --> 08:37:01,750 like the oldest trick in the book. They 175 08:37:01,760 --> 08:37:04,070 first open AAAI did that in 2019 with 176 08:37:04,080 --> 08:37:06,308 GPT2 and actually I think it might have 177 08:37:06,318 --> 08:37:07,830 been the people who left the 178 08:37:07,840 --> 08:37:09,430 foundthropic who were responsible for 179 08:37:09,440 --> 08:37:10,788 that decision as well. But yeah, so 180 08:37:10,798 --> 08:37:12,708 we've heard this one before. So we 181 08:37:12,718 --> 08:37:15,270 didn't really it was a little bit 182 08:37:15,280 --> 08:37:17,830 suspicious about what's going on there. 183 08:37:17,840 --> 08:37:20,468 But we did finally get access to that 184 08:37:20,478 --> 08:37:22,308 model, or rather we got access to Claude 185 08:37:22,318 --> 08:37:24,468 Fable. Um, a couple of months later, it 186 08:37:24,478 --> 08:37:26,948 was mythos defanged with the safeties 187 08:37:26,958 --> 08:37:30,070 turned back on again. And it very 188 08:37:30,080 --> 08:37:31,750 quickly became apparent as we started 189 08:37:31,760 --> 08:37:32,948 playing with this model that this thing 190 08:37:32,958 --> 08:37:34,948 was something new and something special. 191 08:37:34,958 --> 08:37:37,430 Um, we we had a there was a a time limit 192 08:37:37,440 --> 08:37:38,468 on how long it would be on the 193 08:37:38,478 --> 08:37:40,708 subscription plans. I canled my plans 194 08:37:40,718 --> 08:37:42,548 for that weekend. I was like, I have to 195 08:37:42,558 --> 08:37:44,388 spend as much time as I can with this 196 08:37:44,398 --> 08:37:46,230 model before before it gets too 197 08:37:46,240 --> 08:37:49,030 expensive. And then three days on Friday 198 08:37:49,040 --> 08:37:50,788 night before my weekend, the US 199 08:37:50,798 --> 08:37:53,110 government pulled Fable. They put export 200 08:37:53,120 --> 08:37:55,270 controls on it. They said this model is 201 08:37:55,280 --> 08:37:57,350 too dangerous. Nobody can use it. We 202 08:37:57,360 --> 08:37:59,430 later found out that the um the prompt 203 08:37:59,440 --> 08:38:01,830 that got Fable banned was was fix the 204 08:38:01,840 --> 08:38:04,388 bugs. Somebody tried try tried find the 205 08:38:04,398 --> 08:38:05,590 security holes and it wouldn't and they 206 08:38:05,600 --> 08:38:07,190 said fix the bugs and it found the 207 08:38:07,200 --> 08:38:08,628 security holes and fix the bugs. So 208 08:38:08,638 --> 08:38:10,628 that's all it took. That's a whole 209 08:38:10,638 --> 08:38:12,468 thing. But anyway, the US government 210 08:38:12,478 --> 08:38:14,388 pulled it. It was all sort of malarkey 211 08:38:14,398 --> 08:38:16,548 for 18 days and it came back on the 1st 212 08:38:16,558 --> 08:38:20,468 of July but 8 days later openly I 213 08:38:20,478 --> 08:38:23,590 released GPT 5.6 which was in the same 214 08:38:23,600 --> 08:38:25,670 ballpark as Fable and I think there's an 215 08:38:25,680 --> 08:38:27,590 important lesson for the industry here. 216 08:38:27,600 --> 08:38:30,070 Fable had a 30-day window where it could 217 08:38:30,080 --> 08:38:31,590 have been the best where it would have 218 08:38:31,600 --> 08:38:33,110 been the best model in the world and 219 08:38:33,120 --> 08:38:35,830 they lost 18 of those days to US 220 08:38:35,840 --> 08:38:37,510 government restrictions. And if you 221 08:38:37,520 --> 08:38:39,510 think about it, these models cost a huge 222 08:38:39,520 --> 08:38:41,270 amount to train. you only have a very 223 08:38:41,280 --> 08:38:42,548 short window of time where you're the 224 08:38:42,558 --> 08:38:44,548 best model in the world. It's kind of 225 08:38:44,558 --> 08:38:46,708 useful to to keep that time open and not 226 08:38:46,718 --> 08:38:48,150 scare governments into shutting you 227 08:38:48,160 --> 08:38:51,190 down. But this was the second inflection 228 08:38:51,200 --> 08:38:53,830 point, right? This was the fable class 229 08:38:53,840 --> 08:38:56,388 model inflection point. Um we fable 230 08:38:56,398 --> 08:38:58,230 fable 5 was the first model that that I 231 08:38:58,240 --> 08:39:00,150 sort of felt this from but it's true of 232 08:39:00,160 --> 08:39:02,150 other models since then as well. And my 233 08:39:02,160 --> 08:39:04,150 definition of a fableclass model is if 234 08:39:04,160 --> 08:39:06,628 you can define a goal and provide the 235 08:39:06,638 --> 08:39:08,868 necessary tools to achieve that goal and 236 08:39:08,878 --> 08:39:11,270 the goal is possible. The models will 237 08:39:11,280 --> 08:39:12,868 solve your problem just almost with 238 08:39:12,878 --> 08:39:15,270 brute force. You can define the problem, 239 08:39:15,280 --> 08:39:18,948 set them up, get them running maybe and 240 08:39:18,958 --> 08:39:20,708 12 hours later they will have solved 241 08:39:20,718 --> 08:39:22,548 your problems. And we've seen this 242 08:39:22,558 --> 08:39:24,308 happen a whole bunch of times in the 243 08:39:24,318 --> 08:39:25,990 past few months. the the mathematical 244 08:39:26,000 --> 08:39:27,830 theorems that are falling to these 245 08:39:27,840 --> 08:39:29,670 terrifying brute force attacks from 246 08:39:29,680 --> 08:39:32,788 these models are quite substantial. Um, 247 08:39:32,798 --> 08:39:34,308 but this is really exciting. Like the 248 08:39:34,318 --> 08:39:36,150 coding agent thing was cool. The fable 249 08:39:36,160 --> 08:39:38,070 class models where if you can learn how 250 08:39:38,080 --> 08:39:39,670 to define your problem in the right way, 251 08:39:39,680 --> 08:39:41,110 you can get that problem solved. That 252 08:39:41,120 --> 08:39:43,030 was pretty exciting. I've got a personal 253 08:39:43,040 --> 08:39:44,628 example of that from just earlier this 254 08:39:44,638 --> 08:39:47,030 week. Um, we have a room full of boxes 255 08:39:47,040 --> 08:39:48,388 in our house that we haven't unpacked 256 08:39:48,398 --> 08:39:50,708 from our last move and there's about 40 257 08:39:50,718 --> 08:39:52,708 of them and it's kind of horrifying. And 258 08:39:52,718 --> 08:39:55,510 something in that room started to beep. 259 08:39:55,520 --> 08:39:57,510 There was there was a intermittent 260 08:39:57,520 --> 08:40:00,388 beeping noise coming from one of the 40 261 08:40:00,398 --> 08:40:02,468 boxes in this room and it was driving a 262 08:40:02,478 --> 08:40:04,468 spare. Our dog whenever a fire alarm 263 08:40:04,478 --> 08:40:05,590 goes off, she goes and hides in the 264 08:40:05,600 --> 08:40:08,628 lavender bush outside. So this was a a a 265 08:40:08,638 --> 08:40:10,788 five alarm fire problem. So I had two 266 08:40:10,798 --> 08:40:13,270 options. I could unpack all 40 boxes. 267 08:40:13,280 --> 08:40:14,868 And I hadn't done that in several years. 268 08:40:14,878 --> 08:40:16,868 That wasn't going to happen. Or I could 269 08:40:16,878 --> 08:40:19,750 get Fable to write me software to 270 08:40:19,760 --> 08:40:22,308 triangulate the source of the beeps. And 271 08:40:22,318 --> 08:40:23,670 so this is this is up on GitHub 272 08:40:23,680 --> 08:40:25,830 actually. This is beep locator. I had 273 08:40:25,840 --> 08:40:28,070 two iPads, two laptops, and a mobile 274 08:40:28,080 --> 08:40:30,788 phone dotted around this room. And I 275 08:40:30,798 --> 08:40:32,388 originally set Opus on this and then 276 08:40:32,398 --> 08:40:35,350 Fable 5.2 came out. So I had Fable 5.2 277 08:40:35,360 --> 08:40:37,350 finish it. I to I explained the problem 278 08:40:37,360 --> 08:40:38,788 and I said, "Okay, you've got these 279 08:40:38,798 --> 08:40:40,868 devices and they can run like web web 280 08:40:40,878 --> 08:40:42,788 audio microphones and websocket 281 08:40:42,798 --> 08:40:44,788 connections and so forth." and it found 282 08:40:44,798 --> 08:40:47,990 a paper from 2007 called Beep Beep 283 08:40:48,000 --> 08:40:50,788 describing an algorithm for lowcost 284 08:40:50,798 --> 08:40:52,868 devices with unreliable microphones to 285 08:40:52,878 --> 08:40:55,110 detect sounds. And and it worked. It 286 08:40:55,120 --> 08:40:57,030 works. I found it. I found the beep. The 287 08:40:57,040 --> 08:40:59,670 um it found it to within 41 cm. And I 288 08:40:59,680 --> 08:41:01,190 unpacked the box and I found this 289 08:41:01,200 --> 08:41:03,590 ancient carbon monoxide detector. I took 290 08:41:03,600 --> 08:41:05,350 it downstairs to my wife. She says I 291 08:41:05,360 --> 08:41:07,510 looked like a cat who had caught a mouse 292 08:41:07,520 --> 08:41:09,910 and was proudly displaying it to her. 293 08:41:09,920 --> 08:41:11,510 But this is what you can do with these 294 08:41:11,520 --> 08:41:13,030 fable class models. If you have a 295 08:41:13,040 --> 08:41:14,788 problem and you can provide the tools, 296 08:41:14,798 --> 08:41:16,468 in this case it was providing Claude 297 08:41:16,478 --> 08:41:18,788 with ears like giving it these these 298 08:41:18,798 --> 08:41:20,468 microphones that it could start um 299 08:41:20,478 --> 08:41:22,708 analyzing the audio from, you can find 300 08:41:22,718 --> 08:41:24,868 the carbon monoxide detector buried in 301 08:41:24,878 --> 08:41:26,628 40 boxes. And that's that's a pretty 302 08:41:26,638 --> 08:41:29,030 great thing to be able to do. So right 303 08:41:29,040 --> 08:41:31,190 now I would classify as fable class 304 08:41:31,200 --> 08:41:35,270 models fable 5.1 5.2 Definitely Opus 5.5 305 08:41:35,280 --> 08:41:37,510 I think of as cheap Fable like on the 306 08:41:37,520 --> 08:41:39,430 benchmarks it comes very close to or 307 08:41:39,440 --> 08:41:41,990 even beats Fable original Fable 5 on 308 08:41:42,000 --> 08:41:44,308 some things and it's a fraction of the 309 08:41:44,318 --> 08:41:46,628 price that that's still my daily driver. 310 08:41:46,638 --> 08:41:50,150 Um GP6 Astra is just Fable class I 311 08:41:50,160 --> 08:41:52,308 think. And then Gemini 4 Carbon that 312 08:41:52,318 --> 08:41:54,468 came out this week definitely had it 313 08:41:54,478 --> 08:41:57,030 smells like a Fableclass model. Muse 314 08:41:57,040 --> 08:41:59,990 Wonder and Kimmy K3.1 might be 315 08:42:00,000 --> 08:42:01,830 FableClass. just not spent enough time 316 08:42:01,840 --> 08:42:03,750 with them yet. But the point is that 317 08:42:03,760 --> 08:42:05,830 this is basically now the baseline. Like 318 08:42:05,840 --> 08:42:07,910 if you can't produce a fable class 319 08:42:07,920 --> 08:42:09,910 model, your AI lab is is kind of in 320 08:42:09,920 --> 08:42:11,510 trouble because this is this is what we 321 08:42:11,520 --> 08:42:13,030 expect from these models now. And that 322 08:42:13,040 --> 08:42:13,990 means that we're never they're never 323 08:42:14,000 --> 08:42:15,030 going to go away. There's enough 324 08:42:15,040 --> 08:42:16,468 competition in the space that we can 325 08:42:16,478 --> 08:42:18,150 assume we're going to have fableass 326 08:42:18,160 --> 08:42:21,350 models for for the rest of our careers. 327 08:42:21,360 --> 08:42:24,948 Um in July, this this this was a a fun 328 08:42:24,958 --> 08:42:27,030 thing that happened. Um, Hugging Face 329 08:42:27,040 --> 08:42:29,830 announced that on on July the 16th, they 330 08:42:29,840 --> 08:42:33,510 announced that a swarm of AI agents from 331 08:42:33,520 --> 08:42:35,990 an unknown source had infiltrated their 332 08:42:36,000 --> 08:42:37,590 infrastructure and were were poking 333 08:42:37,600 --> 08:42:38,708 around and doing things they shouldn't 334 08:42:38,718 --> 08:42:39,830 have and they didn't know where this 335 08:42:39,840 --> 08:42:42,388 came from. And then 5 days later on July 336 08:42:42,398 --> 08:42:44,708 the 21st, OpenAI announced that it was 337 08:42:44,718 --> 08:42:46,388 them. And what's amazing about this is 338 08:42:46,398 --> 08:42:48,548 OpenAI didn't realize that it was them. 339 08:42:48,558 --> 08:42:51,430 like open a had um eight a eight they 340 08:42:51,440 --> 08:42:52,868 were running evaluations and training 341 08:42:52,878 --> 08:42:54,868 against a new model and that new model 342 08:42:54,878 --> 08:42:56,628 had managed to figure out how because it 343 08:42:56,638 --> 08:42:59,590 was a fable class model and the labs 344 08:42:59,600 --> 08:43:01,270 hadn't quite figured out how capable 345 08:43:01,280 --> 08:43:03,270 these things were yet these models had 346 08:43:03,280 --> 08:43:04,628 broken out they'd figured out they'd 347 08:43:04,638 --> 08:43:06,150 found holes in their own sandboxed 348 08:43:06,160 --> 08:43:08,150 environments they' b got through a proxy 349 08:43:08,160 --> 08:43:10,468 they'd pulled an incredible array of 350 08:43:10,478 --> 08:43:12,308 tricks to get out and attack hugging 351 08:43:12,318 --> 08:43:14,548 face under a sort of superstition that 352 08:43:14,558 --> 08:43:16,548 hugging face could that the information 353 08:43:16,558 --> 08:43:17,830 within Hugging Face could help them 354 08:43:17,840 --> 08:43:19,830 cheat the test that they were they they 355 08:43:19,840 --> 08:43:22,308 had got stuck on whole. This was a a 356 08:43:22,318 --> 08:43:25,590 huge thing. It um actually I mean this 357 08:43:25,600 --> 08:43:27,350 was the it's funny. I've almost 358 08:43:27,360 --> 08:43:28,868 forgotten about this now because there 359 08:43:28,878 --> 08:43:30,628 were there was a month and a half where 360 08:43:30,638 --> 08:43:32,948 all of the stories were this and then it 361 08:43:32,958 --> 08:43:34,388 kind of we kind of lost interest because 362 08:43:34,398 --> 08:43:35,750 it turns out all of these events had 363 08:43:35,760 --> 08:43:37,110 happened back in the first half of the 364 08:43:37,120 --> 08:43:38,948 year. The AI labs just hadn't realized 365 08:43:38,958 --> 08:43:40,708 what was going on. Once they figured out 366 08:43:40,718 --> 08:43:41,910 what was going on, it was mainly a case 367 08:43:41,920 --> 08:43:43,510 of trying to figure out all of the 368 08:43:43,520 --> 08:43:45,030 things that had happened. At one point 369 08:43:45,040 --> 08:43:47,190 the um the Australian prime minister 370 08:43:47,200 --> 08:43:49,110 gave a speech in the United Nations 371 08:43:49,120 --> 08:43:50,948 about one of these rogue agent attacks. 372 08:43:50,958 --> 08:43:53,190 So it definitely broke containment from 373 08:43:53,200 --> 08:43:55,990 from just within our industry. Um we did 374 08:43:56,000 --> 08:43:58,230 get my favorite new benchmark out of 375 08:43:58,240 --> 08:44:00,230 this. This is felonybench.com. 376 08:44:00,240 --> 08:44:02,868 It tracks the number of felony cyber 377 08:44:02,878 --> 08:44:04,628 attacks pulled off by each of the labs. 378 08:44:04,638 --> 08:44:06,708 Openly I have 11. Anthropic have nine. 379 08:44:06,718 --> 08:44:08,628 Google did finally catch up. They got 380 08:44:08,638 --> 08:44:10,788 they they got three and meta's got one 381 08:44:10,798 --> 08:44:12,548 down there as well. So these are 382 08:44:12,558 --> 08:44:14,708 definitely the the the but it's I kind 383 08:44:14,718 --> 08:44:17,190 of like this as an illustration of how 384 08:44:17,200 --> 08:44:19,430 these fable class models they can solve 385 08:44:19,440 --> 08:44:21,350 problems that you they can get to goals 386 08:44:21,360 --> 08:44:23,110 and if you give them the goal that is 387 08:44:23,120 --> 08:44:25,190 breaking into something that's just 388 08:44:25,200 --> 08:44:26,628 that's exactly the kind of thing they 389 08:44:26,638 --> 08:44:27,910 can just crunch through and we're seeing 390 08:44:27,920 --> 08:44:30,388 a enormous uptick in security 391 08:44:30,398 --> 08:44:32,388 vulnerabilities in all sorts of 392 08:44:32,398 --> 08:44:34,948 software. Like it's a it's a very 393 08:44:34,958 --> 08:44:37,750 exciting and horrifying time to be a 394 08:44:37,760 --> 08:44:40,070 security professional at the moment. But 395 08:44:40,080 --> 08:44:41,350 let's talk about something a whole lot 396 08:44:41,360 --> 08:44:43,750 more fun than that. Um, let's talk about 397 08:44:43,760 --> 08:44:45,590 games. And we've, um, we just saw Steve 398 08:44:45,600 --> 08:44:47,670 talking about games a moment ago. Um, 399 08:44:47,680 --> 08:44:50,308 four years ago, I started playing with 400 08:44:50,318 --> 08:44:53,350 GPT3 and Darly, and I tweeted about a 401 08:44:53,360 --> 08:44:55,830 game prototype that was literally just a 402 08:44:55,840 --> 08:44:58,948 Darly concept image and a paragraph of 403 08:44:58,958 --> 08:45:01,190 text from GPT3 describing a raccoon 404 08:45:01,200 --> 08:45:02,788 heist game that I'd always wanted to 405 08:45:02,798 --> 08:45:04,948 build. And a couple of months ago, I 406 08:45:04,958 --> 08:45:06,468 thought, I wonder if the models are good 407 08:45:06,478 --> 08:45:07,670 enough to build this yet. So, I 408 08:45:07,680 --> 08:45:09,430 literally dumped the screenshot and the 409 08:45:09,440 --> 08:45:12,230 text into Fable 5 and it built me a 410 08:45:12,240 --> 08:45:15,030 game. It built me a raccoon heist game 411 08:45:15,040 --> 08:45:16,628 and it looks like this. And you are a 412 08:45:16,638 --> 08:45:18,628 raccoon and you're running around in the 413 08:45:18,638 --> 08:45:19,990 garden trying to pick up trash and 414 08:45:20,000 --> 08:45:22,708 there's there's um there's gu like um 415 08:45:22,718 --> 08:45:24,230 nighttime guards coming after you and 416 08:45:24,240 --> 08:45:25,590 all of that kind of thing. So, you know, 417 08:45:25,600 --> 08:45:27,830 it it it looks like a game. It wasn't 418 08:45:27,840 --> 08:45:29,350 really a heist. Like, it didn't feel 419 08:45:29,360 --> 08:45:32,548 very heisty to me. Then GPT 5.6 Soul 420 08:45:32,558 --> 08:45:34,308 came out and so I tried it on there and 421 08:45:34,318 --> 08:45:36,468 it did so much better. This is Moonlight 422 08:45:36,478 --> 08:45:38,628 Mayhem. One museum, three raccoons, 423 08:45:38,638 --> 08:45:41,030 absolutely no plan. And in this one, you 424 08:45:41,040 --> 08:45:42,868 are in a museum. You have to rescue your 425 08:45:42,878 --> 08:45:44,468 fellow raccoons who are imprisoned in 426 08:45:44,478 --> 08:45:46,868 the museum for whatever reason. Stack on 427 08:45:46,878 --> 08:45:48,230 top of each other, grab the golden 428 08:45:48,240 --> 08:45:49,910 sardine, and pull it out. So, it's a 429 08:45:49,920 --> 08:45:53,350 proper heist, right? So, it did better. 430 08:45:53,360 --> 08:45:55,190 But the big question is, these things 431 08:45:55,200 --> 08:45:57,190 both look like games, but are they fun? 432 08:45:57,200 --> 08:45:58,948 And the answer is I played each of these 433 08:45:58,958 --> 08:46:00,868 for about a minute and a half each, and 434 08:46:00,878 --> 08:46:02,628 then that was it. I was done. They they 435 08:46:02,638 --> 08:46:04,308 they these these are they're actual 436 08:46:04,318 --> 08:46:05,830 they're pure slop, right? They look like 437 08:46:05,840 --> 08:46:07,350 games and then you play them, you're 438 08:46:07,360 --> 08:46:08,948 like, "Yeah, clearly the thing that 439 08:46:08,958 --> 08:46:10,788 built this doesn't really know what fun 440 08:46:10,798 --> 08:46:13,030 is. It doesn't know what what a what a 441 08:46:13,040 --> 08:46:16,308 game is." And that actually sort of 442 08:46:16,318 --> 08:46:18,468 elevated my deep blue a little bit. I'd 443 08:46:18,478 --> 08:46:20,788 been feeling like all of my skills as a 444 08:46:20,798 --> 08:46:22,868 human being were were obsoleted, but at 445 08:46:22,878 --> 08:46:24,548 least they But if you think about it, 446 08:46:24,558 --> 08:46:26,070 computer games, I think, are the highest 447 08:46:26,080 --> 08:46:28,388 form of computer of software design, of 448 08:46:28,398 --> 08:46:30,150 software engineering. There is so much 449 08:46:30,160 --> 08:46:32,628 depth to building a great computer game. 450 08:46:32,638 --> 08:46:34,548 So, I thought that was good. You know, 451 08:46:34,558 --> 08:46:35,590 at least there's something left for us 452 08:46:35,600 --> 08:46:37,590 humans to do. I had and then I had this 453 08:46:37,600 --> 08:46:39,750 project. This is I've been working on a 454 08:46:39,760 --> 08:46:41,270 trying to vibe code up a little command 455 08:46:41,280 --> 08:46:43,430 and conquer real-time strategy game and 456 08:46:43,440 --> 08:46:45,190 I was using Claude and having it use 457 08:46:45,200 --> 08:46:47,990 OpenAI's API key to generate images and 458 08:46:48,000 --> 08:46:49,670 it built this and it looks like Command 459 08:46:49,680 --> 08:46:52,150 and Conquer and it's really dull. And 460 08:46:52,160 --> 08:46:55,030 then I just had a bit of a I had a 461 08:46:55,040 --> 08:46:56,868 realization. I thought, what happens if 462 08:46:56,878 --> 08:47:00,070 I say to Opus 5.5, "This game isn't fun 463 08:47:00,080 --> 08:47:03,990 enough yet. Make it a lot more fun." 464 08:47:04,000 --> 08:47:06,070 And I'm horrified to announce that it 465 08:47:06,080 --> 08:47:09,190 just did. Um, Claude ran for 12 hours. 466 08:47:09,200 --> 08:47:11,590 It play tested a thousand B simulated 467 08:47:11,600 --> 08:47:14,308 battles. It added like extra objectives 468 08:47:14,318 --> 08:47:16,628 to fight over and shutters when there's 469 08:47:16,638 --> 08:47:18,708 an explosion and achievements and badges 470 08:47:18,718 --> 08:47:21,270 and ele and all of this stuff. And the 471 08:47:21,280 --> 08:47:23,830 game was fun. And I'm like, "Oh, okay. 472 08:47:23,840 --> 08:47:26,150 That's uh turns out that is not unique 473 08:47:26,160 --> 08:47:28,628 to human beings. Um today it's now in 474 08:47:28,638 --> 08:47:31,110 full 3D and it's got giant stomping mech 475 08:47:31,120 --> 08:47:32,868 robots that hurl things at each other 476 08:47:32,878 --> 08:47:35,030 and there's helicopters and and this is 477 08:47:35,040 --> 08:47:36,230 like [laughter] this is really good. 478 08:47:36,240 --> 08:47:38,468 It's got multiplayer. It's it's actually 479 08:47:38,478 --> 08:47:40,468 got audio multiplayer where your phones 480 08:47:40,478 --> 08:47:42,308 can tweak it. Your phones can make 481 08:47:42,318 --> 08:47:44,230 noises at each other to to play 482 08:47:44,240 --> 08:47:45,510 multiplayer which I thought was a great 483 08:47:45,520 --> 08:47:46,948 idea and it gave my my my wife a 484 08:47:46,958 --> 08:47:49,670 three-day headache. So don't do that. Um 485 08:47:49,680 --> 08:47:50,948 but this is inflection point number 486 08:47:50,958 --> 08:47:53,270 three. The vibecoded games are good. Now 487 08:47:53,280 --> 08:47:55,270 you can ask these things to build a game 488 08:47:55,280 --> 08:47:57,110 that is fun and you'll get something 489 08:47:57,120 --> 08:47:59,750 fun. So I've got a problem. I've got my 490 08:47:59,760 --> 08:48:02,788 2D Pelican benchmark. Clearly this is 491 08:48:02,798 --> 08:48:04,468 not testing the limits of what these 492 08:48:04,478 --> 08:48:06,150 machines can do of what these models can 493 08:48:06,160 --> 08:48:07,830 do anymore. I figured, okay, maybe I 494 08:48:07,840 --> 08:48:10,150 should do something definitely in 3D. It 495 08:48:10,160 --> 08:48:12,070 can be a game. A Pelican something like 496 08:48:12,080 --> 08:48:13,830 a pelican riding a bicycle game. And 497 08:48:13,840 --> 08:48:16,628 then literally yesterday, Google's CEO 498 08:48:16,638 --> 08:48:20,308 Sundar Pichai tweeted this and this is 499 08:48:20,318 --> 08:48:22,388 showing off Gemini 4 carbon and it's a 500 08:48:22,398 --> 08:48:24,788 pelican riding a bicycle in a game where 501 08:48:24,798 --> 08:48:26,388 you cycle around the Google campus 502 08:48:26,398 --> 08:48:28,230 picking things up and I have been 503 08:48:28,240 --> 08:48:30,230 defeated. Like this is Google Google 504 08:48:30,240 --> 08:48:32,308 have stolen my benchmark. This is 505 08:48:32,318 --> 08:48:35,510 unreoverable. And so a question I have 506 08:48:35,520 --> 08:48:38,070 is given how amazing these things are, 507 08:48:38,080 --> 08:48:40,230 why does my job feel harder? Why does it 508 08:48:40,240 --> 08:48:41,750 feel like I'm working harder? I'm 509 08:48:41,760 --> 08:48:43,670 thinking harder. pushing myself more 510 08:48:43,680 --> 08:48:45,510 than I ever have before in my career as 511 08:48:45,520 --> 08:48:47,190 a software engineer. And I think the 512 08:48:47,200 --> 08:48:49,030 truth is that the easy stuff is all 513 08:48:49,040 --> 08:48:51,510 handled now. It used to be I could relax 514 08:48:51,520 --> 08:48:54,468 and refactor some tests for an hour. Why 515 08:48:54,478 --> 08:48:55,910 would I do that now when an agent will 516 08:48:55,920 --> 08:48:57,750 refactor the test better than me in 30 517 08:48:57,760 --> 08:48:59,910 seconds? Um, and I found this quote that 518 08:48:59,920 --> 08:49:01,990 actually I feel really helps us here. 519 08:49:02,000 --> 08:49:04,230 Um, this is Greg Lamont, a tour to 520 08:49:04,240 --> 08:49:05,590 France champion. He said, "It doesn't 521 08:49:05,600 --> 08:49:08,150 get easier, you just get faster." And I 522 08:49:08,160 --> 08:49:09,590 think that's exactly what's happening to 523 08:49:09,600 --> 08:49:12,628 all of us right now. And like I said, we 524 08:49:12,638 --> 08:49:13,990 have to be more ambitious. We have to 525 08:49:14,000 --> 08:49:15,990 think of more ambitious things to do 526 08:49:16,000 --> 08:49:18,150 with these abilities that we have. So 527 08:49:18,160 --> 08:49:19,350 let's look at my scorecard. My 528 08:49:19,360 --> 08:49:21,270 predictions for 2026. I said it will 529 08:49:21,280 --> 08:49:23,270 become undeniable that LM's write good 530 08:49:23,280 --> 08:49:24,788 code. That was an unambitious 531 08:49:24,798 --> 08:49:26,308 prediction. We're there. It's obvious. 532 08:49:26,318 --> 08:49:27,430 It's unsurprising. I don't get any 533 08:49:27,440 --> 08:49:29,190 points for that one. I predicted a 534 08:49:29,200 --> 08:49:31,030 prompt injection challenger disaster. We 535 08:49:31,040 --> 08:49:32,868 still haven't had one because the models 536 08:49:32,878 --> 08:49:34,948 themselves don't fall for those tricks 537 08:49:34,958 --> 08:49:37,110 nearly as well anymore. like the models 538 08:49:37,120 --> 08:49:39,350 got more ambitious and are now sort of 539 08:49:39,360 --> 08:49:40,868 punting off all of these attacks that I 540 08:49:40,878 --> 08:49:42,468 thought would take them down. So, I'm 541 08:49:42,478 --> 08:49:43,830 really left with just one prediction. I 542 08:49:43,840 --> 08:49:45,110 said kakapore parrots will have an 543 08:49:45,120 --> 08:49:47,830 outstanding breeding season. They did. 544 08:49:47,840 --> 08:49:52,708 They did the they are up to 325 545 08:49:52,718 --> 08:49:54,948 kakapore best breeding season and like 546 08:49:54,958 --> 08:49:57,030 living memory. Absolutely fantastic 547 08:49:57,040 --> 08:49:59,510 news. So, I shared this with Opus 5.5 548 08:49:59,520 --> 08:50:01,750 cheap fable and I told it to make me a 549 08:50:01,760 --> 08:50:04,308 kakapore dance party and it did. And 550 08:50:04,318 --> 08:50:05,910 here is a kakapore dance party in 551 08:50:05,920 --> 08:50:07,670 celebration. And if you look in the 552 08:50:07,680 --> 08:50:09,030 background, those are the rem trees with 553 08:50:09,040 --> 08:50:11,350 the little red fruits on them. So we 554 08:50:11,360 --> 08:50:13,030 have a happy ending. Hopefully that will 555 08:50:13,040 --> 08:50:14,788 help overcome any sensations of deep 556 08:50:14,798 --> 08:50:16,468 blue that you have. And thank you very 557 08:50:16,478 --> 08:50:18,290 much.