2026 in LLMs

8:31:11great to be here. Um I have a lot to

8:31:13cover. I've got 18 minutes to talk about

8:31:15the last 12 months in LLMs. Are my

8:31:18slides up? My slides are up. Let's just

8:31:21let's just get get going. I want to talk

8:31:23about the last 12 months in LLMs through

8:31:25the perspective of three inflection

8:31:27points that we've had over the past 12

8:31:29months. Um, and the first of those

8:31:31conveniently was exactly 12 months ago,

8:31:33right? November 2025 is what I consider

8:31:36the coding agent inflection point.

8:31:38Coding agents weren't new. We'd had

8:31:40Claude code since February last year.

8:31:42Codeexc have been around. You know, this

8:31:43wasn't a complete new category of

8:31:44software, but something changed 12

8:31:47months ago with these two models. We had

8:31:49Claude Opocopus 4.5 and GPT 5.1 and they

8:31:52were just a little bit better at driving

8:31:54a coding agent which put them over that

8:31:56thing where these suddenly these were

8:31:57tools you could use on a daily basis.

8:31:59Like we went from coding agents as a

8:32:01novelty that you'd poke at occasionally

8:32:03and then make some weird mistakes to

8:32:04something that was reliable enough that

8:32:06you could use them every day. It was

8:32:07convenient that it happened in November

8:32:09because we then got to spend December

8:32:11basically over the holidays sort of

8:32:12poking at these new models and these new

8:32:14things and figuring what they could do.

8:32:16And by January, a lot of us had realized

8:32:18that something had changed and this was

8:32:20going to be the big theme for 2026.

8:32:23I like evaluating models by having them

8:32:25draw SVGs of pelicans riding bicycles.

8:32:28This is the state-of-the-art 12 months

8:32:30ago. This is uh Claude Opus' Pelican and

8:32:32GP 5.1's Pelican. You'll notice that

8:32:34neither of these bicycles actually work.

8:32:36So, there was definitely room for

8:32:38improvement here. They were really good

8:32:39at writing code. They weren't great at

8:32:41SVGs just yet, but but this is how we

8:32:43started the year. So every year in

8:32:47January, I set myself a New Year's

8:32:48resolution. And every year it is the

8:32:50exact same thing. Every year I tell

8:32:52myself that this year I'm going to take

8:32:53on less new projects and I'm going to

8:32:55focus I'm going to focus on the stuff

8:32:57that I've already got to do. And this

8:32:58has never worked. So this year I changed

8:33:01it. I said, you know what? I'm going to

8:33:02be more ambitious. I'm going to take on

8:33:04as many projects as I want to because

8:33:06we've got these agents. They can do

8:33:08stuff. Let's just try and push the

8:33:10limits and see what we can get done. as

8:33:12a result. Now it's November and I am

8:33:13juggling so many plates right now. It's

8:33:15kind of horrifying and I may need to

8:33:17reset my ambitions for next year. But

8:33:20that was my sort of theme of the year

8:33:21and I think it's the theme of this

8:33:22conference is to be more ambitious now

8:33:24that we have all of these new tools. I

8:33:26also went on a podcast where they asked

8:33:28me to get make some predictions for 2026

8:33:31in LLMs. Um, my predictions back in

8:33:33January were firstly it will become

8:33:35undeniable that LLMs write good code.

8:33:38Not just code, but good code. Because

8:33:39back in January, there were a lot of

8:33:41people who thought that the code was all

8:33:42complete trash and you know, you'd have

8:33:44to review every single line of it. Um,

8:33:46I've been saying this every year for

8:33:48like four years now. We're due a prompt

8:33:50injection challenger disaster. Like some

8:33:52some incident caused by a prompt

8:33:54injection attack which makes the front

8:33:55pages of newspapers. I was convinced

8:33:57that this would be the year even though

8:33:58it hadn't been the year the year before,

8:34:00the year before, the year before that.

8:34:02And um I also predicted that Kakapore

8:34:04parrots will have an outstanding

8:34:06breeding season in 2026. Um this is a

8:34:10Kakapore parrot. They are a New Zealand

8:34:12nocturnal flightless parrot. There were

8:34:15only 235 of them in the world back in

8:34:18January. Um but they only breed when the

8:34:20Remu trees have a masting fruit season,

8:34:23which hadn't happened in years. But the

8:34:24remat tries were looking really good in

8:34:25January. There was there was a lot of

8:34:26fruit coming out. So based on that, I

8:34:28thought, you know, maybe this is the

8:34:30year that we get our kakapo breeding

8:34:32explosion. And also on that podcast, we

8:34:34coined a new term. We coined the term

8:34:36deep blue to describe the AI induced

8:34:39existential crisis that everyone starts

8:34:41to feel once you realize that all of the

8:34:43things that make you special as a

8:34:45software developer, these models can do

8:34:46those things too. And they can do them

8:34:48faster and sometimes they can even do

8:34:49them better. And this has definitely

8:34:51been a growing sensation across our

8:34:54industry and other industries over the

8:34:56course of 2026 as well. Let's jump

8:34:59forward to February. In February, Google

8:35:01released Gemini 3.1 Pro. And look at

8:35:04that pelican riding a bicycle. Like

8:35:06that's a real bicycle. That bicycle is

8:35:08the right shape. It's got a little fish.

8:35:10It's got a hat on. This was by far the

8:35:12best pelican I had ever seen out of any

8:35:14model. And this I I had set a trap for

8:35:18the labs. I was, everyone's always asked

8:35:20me what happens if the labs train for

8:35:21the pelican riding a bicycle. My answer

8:35:24has always been, if they do that, I will

8:35:26catch them because I'll ask for a

8:35:27flamingo on a unicycle and when somebody

8:35:29produces a great pelican and a terrible

8:35:31flamingo, I'll know that they've been

8:35:32cheating. Unfortunately, Google's Jeff

8:35:35Dean tweeted this video the day they

8:35:37launched that model. um where he showed

8:35:39that we've got a pelican riding a

8:35:41bicycle, but we've also got a frog on a

8:35:43whatever that thing is and a giraffe in

8:35:45a little tiny car, a flamingo on roller

8:35:47skates, a fli kick flipping turtle, and

8:35:50that's a daxund in a stretch limo. So,

8:35:52they' beaten me, right? They had they

8:35:54had they have trained for all forms of

8:35:56animals in all forms of transport, which

8:35:58does sort of sort of weaken my benchmark

8:36:01quite a little bit. Um, but there was

8:36:02some good news in February. In February,

8:36:05the first kakapore chick in four years

8:36:08hatched on Valentine's Day. On

8:36:10Valentine's Day, because we hadn't had

8:36:11the remove fruit in like four years,

8:36:13this was the first one of these baby

8:36:15rare, weird New Zealand parrots that

8:36:17we'd had in four years. So, that was

8:36:19nice. Um, so my my benchmark was was was

8:36:21greatly injured, but the cacao were

8:36:23doing okay.

8:36:24Jump straight to April. In April, um,

8:36:27Anthropic released a model that they did

8:36:29not release. They announced Claude

8:36:31Mythos and they said this model is too

8:36:34dangerous for us to release. This is it

8:36:36can it's it's got new cyber cyber

8:36:39security abilities that we're not ready

8:36:40to share with the world. We'll share it

8:36:42with trusted um trust with with trusted

8:36:44partners. They actually said Mythos

8:36:45preview has already found thousands of

8:36:47high severity vulnerabilities including

8:36:49some in every major operating system and

8:36:51web browser. And a lot of people were

8:36:54quite skeptical because the the the

8:36:56thing where labs say our model is too

8:36:58bad is too dangerous to release. It's

8:37:00like the oldest trick in the book. They

8:37:01first open AAAI did that in 2019 with

8:37:04GPT2 and actually I think it might have

8:37:06been the people who left the

8:37:07foundthropic who were responsible for

8:37:09that decision as well. But yeah, so

8:37:10we've heard this one before. So we

8:37:12didn't really it was a little bit

8:37:15suspicious about what's going on there.

8:37:17But we did finally get access to that

8:37:20model, or rather we got access to Claude

8:37:22Fable. Um, a couple of months later, it

8:37:24was mythos defanged with the safeties

8:37:26turned back on again. And it very

8:37:30quickly became apparent as we started

8:37:31playing with this model that this thing

8:37:32was something new and something special.

8:37:34Um, we we had a there was a a time limit

8:37:37on how long it would be on the

8:37:38subscription plans. I canled my plans

8:37:40for that weekend. I was like, I have to

8:37:42spend as much time as I can with this

8:37:44model before before it gets too

8:37:46expensive. And then three days on Friday

8:37:49night before my weekend, the US

8:37:50government pulled Fable. They put export

8:37:53controls on it. They said this model is

8:37:55too dangerous. Nobody can use it. We

8:37:57later found out that the um the prompt

8:37:59that got Fable banned was was fix the

8:38:01bugs. Somebody tried try tried find the

8:38:04security holes and it wouldn't and they

8:38:05said fix the bugs and it found the

8:38:07security holes and fix the bugs. So

8:38:08that's all it took. That's a whole

8:38:10thing. But anyway, the US government

8:38:12pulled it. It was all sort of malarkey

8:38:14for 18 days and it came back on the 1st

8:38:16of July but 8 days later openly I

8:38:20released GPT 5.6 which was in the same

8:38:23ballpark as Fable and I think there's an

8:38:25important lesson for the industry here.

8:38:27Fable had a 30-day window where it could

8:38:30have been the best where it would have

8:38:31been the best model in the world and

8:38:33they lost 18 of those days to US

8:38:35government restrictions. And if you

8:38:37think about it, these models cost a huge

8:38:39amount to train. you only have a very

8:38:41short window of time where you're the

8:38:42best model in the world. It's kind of

8:38:44useful to to keep that time open and not

8:38:46scare governments into shutting you

8:38:48down. But this was the second inflection

8:38:51point, right? This was the fable class

8:38:53model inflection point. Um we fable

8:38:56fable 5 was the first model that that I

8:38:58sort of felt this from but it's true of

8:39:00other models since then as well. And my

8:39:02definition of a fableclass model is if

8:39:04you can define a goal and provide the

8:39:06necessary tools to achieve that goal and

8:39:08the goal is possible. The models will

8:39:11solve your problem just almost with

8:39:12brute force. You can define the problem,

8:39:15set them up, get them running maybe and

8:39:1812 hours later they will have solved

8:39:20your problems. And we've seen this

8:39:22happen a whole bunch of times in the

8:39:24past few months. the the mathematical

8:39:26theorems that are falling to these

8:39:27terrifying brute force attacks from

8:39:29these models are quite substantial. Um,

8:39:32but this is really exciting. Like the

8:39:34coding agent thing was cool. The fable

8:39:36class models where if you can learn how

8:39:38to define your problem in the right way,

8:39:39you can get that problem solved. That

8:39:41was pretty exciting. I've got a personal

8:39:43example of that from just earlier this

8:39:44week. Um, we have a room full of boxes

8:39:47in our house that we haven't unpacked

8:39:48from our last move and there's about 40

8:39:50of them and it's kind of horrifying. And

8:39:52something in that room started to beep.

8:39:55There was there was a intermittent

8:39:57beeping noise coming from one of the 40

8:40:00boxes in this room and it was driving a

8:40:02spare. Our dog whenever a fire alarm

8:40:04goes off, she goes and hides in the

8:40:05lavender bush outside. So this was a a a

8:40:08five alarm fire problem. So I had two

8:40:10options. I could unpack all 40 boxes.

8:40:13And I hadn't done that in several years.

8:40:14That wasn't going to happen. Or I could

8:40:16get Fable to write me software to

8:40:19triangulate the source of the beeps. And

8:40:22so this is this is up on GitHub

8:40:23actually. This is beep locator. I had

8:40:25two iPads, two laptops, and a mobile

8:40:28phone dotted around this room. And I

8:40:30originally set Opus on this and then

8:40:32Fable 5.2 came out. So I had Fable 5.2

8:40:35finish it. I to I explained the problem

8:40:37and I said, "Okay, you've got these

8:40:38devices and they can run like web web

8:40:40audio microphones and websocket

8:40:42connections and so forth." and it found

8:40:44a paper from 2007 called Beep Beep

8:40:48describing an algorithm for lowcost

8:40:50devices with unreliable microphones to

8:40:52detect sounds. And and it worked. It

8:40:55works. I found it. I found the beep. The

8:40:57um it found it to within 41 cm. And I

8:40:59unpacked the box and I found this

8:41:01ancient carbon monoxide detector. I took

8:41:03it downstairs to my wife. She says I

8:41:05looked like a cat who had caught a mouse

8:41:07and was proudly displaying it to her.

8:41:09But this is what you can do with these

8:41:11fable class models. If you have a

8:41:13problem and you can provide the tools,

8:41:14in this case it was providing Claude

8:41:16with ears like giving it these these

8:41:18microphones that it could start um

8:41:20analyzing the audio from, you can find

8:41:22the carbon monoxide detector buried in

8:41:2440 boxes. And that's that's a pretty

8:41:26great thing to be able to do. So right

8:41:29now I would classify as fable class

8:41:31models fable 5.1 5.2 Definitely Opus 5.5

8:41:35I think of as cheap Fable like on the

8:41:37benchmarks it comes very close to or

8:41:39even beats Fable original Fable 5 on

8:41:42some things and it's a fraction of the

8:41:44price that that's still my daily driver.

8:41:46Um GP6 Astra is just Fable class I

8:41:50think. And then Gemini 4 Carbon that

8:41:52came out this week definitely had it

8:41:54smells like a Fableclass model. Muse

8:41:57Wonder and Kimmy K3.1 might be

8:42:00FableClass. just not spent enough time

8:42:01with them yet. But the point is that

8:42:03this is basically now the baseline. Like

8:42:05if you can't produce a fable class

8:42:07model, your AI lab is is kind of in

8:42:09trouble because this is this is what we

8:42:11expect from these models now. And that

8:42:13means that we're never they're never

8:42:14going to go away. There's enough

8:42:15competition in the space that we can

8:42:16assume we're going to have fableass

8:42:18models for for the rest of our careers.

8:42:21Um in July, this this this was a a fun

8:42:24thing that happened. Um, Hugging Face

8:42:27announced that on on July the 16th, they

8:42:29announced that a swarm of AI agents from

8:42:33an unknown source had infiltrated their

8:42:36infrastructure and were were poking

8:42:37around and doing things they shouldn't

8:42:38have and they didn't know where this

8:42:39came from. And then 5 days later on July

8:42:42the 21st, OpenAI announced that it was

8:42:44them. And what's amazing about this is

8:42:46OpenAI didn't realize that it was them.

8:42:48like open a had um eight a eight they

8:42:51were running evaluations and training

8:42:52against a new model and that new model

8:42:54had managed to figure out how because it

8:42:56was a fable class model and the labs

8:42:59hadn't quite figured out how capable

8:43:01these things were yet these models had

8:43:03broken out they'd figured out they'd

8:43:04found holes in their own sandboxed

8:43:06environments they' b got through a proxy

8:43:08they'd pulled an incredible array of

8:43:10tricks to get out and attack hugging

8:43:12face under a sort of superstition that

8:43:14hugging face could that the information

8:43:16within Hugging Face could help them

8:43:17cheat the test that they were they they

8:43:19had got stuck on whole. This was a a

8:43:22huge thing. It um actually I mean this

8:43:25was the it's funny. I've almost

8:43:27forgotten about this now because there

8:43:28were there was a month and a half where

8:43:30all of the stories were this and then it

8:43:32kind of we kind of lost interest because

8:43:34it turns out all of these events had

8:43:35happened back in the first half of the

8:43:37year. The AI labs just hadn't realized

8:43:38what was going on. Once they figured out

8:43:40what was going on, it was mainly a case

8:43:41of trying to figure out all of the

8:43:43things that had happened. At one point

8:43:45the um the Australian prime minister

8:43:47gave a speech in the United Nations

8:43:49about one of these rogue agent attacks.

8:43:50So it definitely broke containment from

8:43:53from just within our industry. Um we did

8:43:56get my favorite new benchmark out of

8:43:58this. This is felonybench.com.

8:44:00It tracks the number of felony cyber

8:44:02attacks pulled off by each of the labs.

8:44:04Openly I have 11. Anthropic have nine.

8:44:06Google did finally catch up. They got

8:44:08they they got three and meta's got one

8:44:10down there as well. So these are

8:44:12definitely the the the but it's I kind

8:44:14of like this as an illustration of how

8:44:17these fable class models they can solve

8:44:19problems that you they can get to goals

8:44:21and if you give them the goal that is

8:44:23breaking into something that's just

8:44:25that's exactly the kind of thing they

8:44:26can just crunch through and we're seeing

8:44:27a enormous uptick in security

8:44:30vulnerabilities in all sorts of

8:44:32software. Like it's a it's a very

8:44:34exciting and horrifying time to be a

8:44:37security professional at the moment. But

8:44:40let's talk about something a whole lot

8:44:41more fun than that. Um, let's talk about

8:44:43games. And we've, um, we just saw Steve

8:44:45talking about games a moment ago. Um,

8:44:47four years ago, I started playing with

8:44:50GPT3 and Darly, and I tweeted about a

8:44:53game prototype that was literally just a

8:44:55Darly concept image and a paragraph of

8:44:58text from GPT3 describing a raccoon

8:45:01heist game that I'd always wanted to

8:45:02build. And a couple of months ago, I

8:45:04thought, I wonder if the models are good

8:45:06enough to build this yet. So, I

8:45:07literally dumped the screenshot and the

8:45:09text into Fable 5 and it built me a

8:45:12game. It built me a raccoon heist game

8:45:15and it looks like this. And you are a

8:45:16raccoon and you're running around in the

8:45:18garden trying to pick up trash and

8:45:20there's there's um there's gu like um

8:45:22nighttime guards coming after you and

8:45:24all of that kind of thing. So, you know,

8:45:25it it it looks like a game. It wasn't

8:45:27really a heist. Like, it didn't feel

8:45:29very heisty to me. Then GPT 5.6 Soul

8:45:32came out and so I tried it on there and

8:45:34it did so much better. This is Moonlight

8:45:36Mayhem. One museum, three raccoons,

8:45:38absolutely no plan. And in this one, you

8:45:41are in a museum. You have to rescue your

8:45:42fellow raccoons who are imprisoned in

8:45:44the museum for whatever reason. Stack on

8:45:46top of each other, grab the golden

8:45:48sardine, and pull it out. So, it's a

8:45:49proper heist, right? So, it did better.

8:45:53But the big question is, these things

8:45:55both look like games, but are they fun?

8:45:57And the answer is I played each of these

8:45:58for about a minute and a half each, and

8:46:00then that was it. I was done. They they

8:46:02they these these are they're actual

8:46:04they're pure slop, right? They look like

8:46:05games and then you play them, you're

8:46:07like, "Yeah, clearly the thing that

8:46:08built this doesn't really know what fun

8:46:10is. It doesn't know what what a what a

8:46:13game is." And that actually sort of

8:46:16elevated my deep blue a little bit. I'd

8:46:18been feeling like all of my skills as a

8:46:20human being were were obsoleted, but at

8:46:22least they But if you think about it,

8:46:24computer games, I think, are the highest

8:46:26form of computer of software design, of

8:46:28software engineering. There is so much

8:46:30depth to building a great computer game.

8:46:32So, I thought that was good. You know,

8:46:34at least there's something left for us

8:46:35humans to do. I had and then I had this

8:46:37project. This is I've been working on a

8:46:39trying to vibe code up a little command

8:46:41and conquer real-time strategy game and

8:46:43I was using Claude and having it use

8:46:45OpenAI's API key to generate images and

8:46:48it built this and it looks like Command

8:46:49and Conquer and it's really dull. And

8:46:52then I just had a bit of a I had a

8:46:55realization. I thought, what happens if

8:46:56I say to Opus 5.5, "This game isn't fun

8:47:00enough yet. Make it a lot more fun."

8:47:04And I'm horrified to announce that it

8:47:06just did. Um, Claude ran for 12 hours.

8:47:09It play tested a thousand B simulated

8:47:11battles. It added like extra objectives

8:47:14to fight over and shutters when there's

8:47:16an explosion and achievements and badges

8:47:18and ele and all of this stuff. And the

8:47:21game was fun. And I'm like, "Oh, okay.

8:47:23That's uh turns out that is not unique

8:47:26to human beings. Um today it's now in

8:47:28full 3D and it's got giant stomping mech

8:47:31robots that hurl things at each other

8:47:32and there's helicopters and and this is

8:47:35like [laughter] this is really good.

8:47:36It's got multiplayer. It's it's actually

8:47:38got audio multiplayer where your phones

8:47:40can tweak it. Your phones can make

8:47:42noises at each other to to play

8:47:44multiplayer which I thought was a great

8:47:45idea and it gave my my my wife a

8:47:46three-day headache. So don't do that. Um

8:47:49but this is inflection point number

8:47:50three. The vibecoded games are good. Now

8:47:53you can ask these things to build a game

8:47:55that is fun and you'll get something

8:47:57fun. So I've got a problem. I've got my

8:47:592D Pelican benchmark. Clearly this is

8:48:02not testing the limits of what these

8:48:04machines can do of what these models can

8:48:06do anymore. I figured, okay, maybe I

8:48:07should do something definitely in 3D. It

8:48:10can be a game. A Pelican something like

8:48:12a pelican riding a bicycle game. And

8:48:13then literally yesterday, Google's CEO

8:48:16Sundar Pichai tweeted this and this is

8:48:20showing off Gemini 4 carbon and it's a

8:48:22pelican riding a bicycle in a game where

8:48:24you cycle around the Google campus

8:48:26picking things up and I have been

8:48:28defeated. Like this is Google Google

8:48:30have stolen my benchmark. This is

8:48:32unreoverable. And so a question I have

8:48:35is given how amazing these things are,

8:48:38why does my job feel harder? Why does it

8:48:40feel like I'm working harder? I'm

8:48:41thinking harder. pushing myself more

8:48:43than I ever have before in my career as

8:48:45a software engineer. And I think the

8:48:47truth is that the easy stuff is all

8:48:49handled now. It used to be I could relax

8:48:51and refactor some tests for an hour. Why

8:48:54would I do that now when an agent will

8:48:55refactor the test better than me in 30

8:48:57seconds? Um, and I found this quote that

8:48:59actually I feel really helps us here.

8:49:02Um, this is Greg Lamont, a tour to

8:49:04France champion. He said, "It doesn't

8:49:05get easier, you just get faster." And I

8:49:08think that's exactly what's happening to

8:49:09all of us right now. And like I said, we

8:49:12have to be more ambitious. We have to

8:49:14think of more ambitious things to do

8:49:16with these abilities that we have. So

8:49:18let's look at my scorecard. My

8:49:19predictions for 2026. I said it will

8:49:21become undeniable that LM's write good

8:49:23code. That was an unambitious

8:49:24prediction. We're there. It's obvious.

8:49:26It's unsurprising. I don't get any

8:49:27points for that one. I predicted a

8:49:29prompt injection challenger disaster. We

8:49:31still haven't had one because the models

8:49:32themselves don't fall for those tricks

8:49:34nearly as well anymore. like the models

8:49:37got more ambitious and are now sort of

8:49:39punting off all of these attacks that I

8:49:40thought would take them down. So, I'm

8:49:42really left with just one prediction. I

8:49:43said kakapore parrots will have an

8:49:45outstanding breeding season. They did.

8:49:47They did the they are up to 325

8:49:52kakapore best breeding season and like

8:49:54living memory. Absolutely fantastic

8:49:57news. So, I shared this with Opus 5.5

8:49:59cheap fable and I told it to make me a

8:50:01kakapore dance party and it did. And

8:50:04here is a kakapore dance party in

8:50:05celebration. And if you look in the

8:50:07background, those are the rem trees with

8:50:09the little red fruits on them. So we

8:50:11have a happy ending. Hopefully that will

8:50:13help overcome any sensations of deep

8:50:14blue that you have. And thank you very

8:50:16much.