1 00:40:09,440 --> 00:40:11,349 Hi everyone, my name is Michelle Chen. 2 00:40:11,359 --> 00:40:13,109 I'm so excited to be here today at AI 3 00:40:13,119 --> 00:40:14,950 Engineer Code Summit chatting a little 4 00:40:14,960 --> 00:40:17,270 bit more about why Cloudflare became a 5 00:40:17,280 --> 00:40:19,589 Neolab. So at Cloudflare I am the group 6 00:40:19,599 --> 00:40:21,829 PM here. I run on the product team 7 00:40:21,839 --> 00:40:24,069 working on products like workers AI, AI 8 00:40:24,079 --> 00:40:26,550 search, AI gateway as well. And so today 9 00:40:26,560 --> 00:40:27,430 we're going to go through this little 10 00:40:27,440 --> 00:40:29,270 journey of how Cloudflare came to be. 11 00:40:29,280 --> 00:40:30,630 You might have heard that we launched 12 00:40:30,640 --> 00:40:32,550 and trained some models recently. I'm 13 00:40:32,560 --> 00:40:33,589 going to tell you a little bit more 14 00:40:33,599 --> 00:40:36,230 about why and what you can be doing with 15 00:40:36,240 --> 00:40:37,750 it. And so we're going to go through a 16 00:40:37,760 --> 00:40:39,510 little bit of a philosophical journey. 17 00:40:39,520 --> 00:40:41,670 Uh when I was preparing for this talk, I 18 00:40:41,680 --> 00:40:43,109 wanted to use this quote and then 19 00:40:43,119 --> 00:40:45,750 Anthropic like launched a blog post last 20 00:40:45,760 --> 00:40:47,349 weekend and used my quote, but I'm still 21 00:40:47,359 --> 00:40:50,230 going to use it anyways. Um so the quote 22 00:40:50,240 --> 00:40:53,910 goes 23 00:40:53,920 --> 00:40:56,069 in one second. Um the quote is from 24 00:40:56,079 --> 00:40:58,150 Kirkagard and it says, "Life can only be 25 00:40:58,160 --> 00:40:59,829 understood backwards, but it must be 26 00:40:59,839 --> 00:41:01,589 lived forwards." And I think this is 27 00:41:01,599 --> 00:41:03,589 really really apt quote here for us 28 00:41:03,599 --> 00:41:05,349 because at Cloudflare we're a 29 00:41:05,359 --> 00:41:07,190 16-year-old company and we've done so 30 00:41:07,200 --> 00:41:09,270 many things over the years but somehow 31 00:41:09,280 --> 00:41:11,109 we got all some of the pieces right and 32 00:41:11,119 --> 00:41:13,349 we ended up having to become a Neolab. 33 00:41:13,359 --> 00:41:14,230 So we're going to go through a little 34 00:41:14,240 --> 00:41:15,829 bit journey. The first thing we're going 35 00:41:15,839 --> 00:41:20,550 to do is look backwards and talk through 36 00:41:20,560 --> 00:41:23,030 why Cloudflare had to uh became a 37 00:41:23,040 --> 00:41:25,990 Neolab. 38 00:41:26,000 --> 00:41:29,829 Give me one second here. 39 00:41:29,839 --> 00:41:31,638 Perfect. We're going to go backwards 40 00:41:31,648 --> 00:41:32,630 [laughter] and talk through why 41 00:41:32,640 --> 00:41:35,589 Cloudflare became a Neolab. So, 42 00:41:35,599 --> 00:41:38,790 Cloudflare is a 16-year-old CDN company. 43 00:41:38,800 --> 00:41:41,750 We just celebrated our 16th birthday uh 44 00:41:41,760 --> 00:41:43,990 at birthday week this year uh in 45 00:41:44,000 --> 00:41:46,309 September. 46 00:41:46,319 --> 00:41:48,870 And we first started as a CDN. So, as a 47 00:41:48,880 --> 00:41:50,550 CDN company, what you have to do is you 48 00:41:50,560 --> 00:41:52,550 have to serve content as quick to your 49 00:41:52,560 --> 00:41:55,030 users as possible. you sit between the 50 00:41:55,040 --> 00:41:57,589 users request and the origin and you 51 00:41:57,599 --> 00:41:59,190 help uh sit in the middle and you can do 52 00:41:59,200 --> 00:42:01,030 things like cache and protect do 53 00:42:01,040 --> 00:42:02,390 security and things like that. So that's 54 00:42:02,400 --> 00:42:05,270 kind of how Cloudflare started. But 55 00:42:05,280 --> 00:42:07,829 Cloudflare's infrastructure had to be 56 00:42:07,839 --> 00:42:09,990 adaptable in order to become the CDN. 57 00:42:10,000 --> 00:42:12,069 And so what we did was 16 years ago in 58 00:42:12,079 --> 00:42:13,510 2010, we launched with our own 59 00:42:13,520 --> 00:42:16,069 infrastructure. Our fast CDN had to be 60 00:42:16,079 --> 00:42:17,510 everywhere. And this looks really 61 00:42:17,520 --> 00:42:19,430 different than a traditional hyperscaler 62 00:42:19,440 --> 00:42:21,510 where compute is centralized in US- East 63 00:42:21,520 --> 00:42:23,589 one or US West one. Cloudflare's 64 00:42:23,599 --> 00:42:25,270 infrastructure is a completely different 65 00:42:25,280 --> 00:42:27,589 paradigm. We have data centers all 66 00:42:27,599 --> 00:42:29,829 around the world as close to the user as 67 00:42:29,839 --> 00:42:31,589 possible. And this is really important 68 00:42:31,599 --> 00:42:34,150 in becoming a CDN. But now we have these 69 00:42:34,160 --> 00:42:35,990 computers basically running as close to 70 00:42:36,000 --> 00:42:37,430 the user as possible. So we thought to 71 00:42:37,440 --> 00:42:39,349 ourselves, what else can we be doing 72 00:42:39,359 --> 00:42:40,950 with these computers that people would 73 00:42:40,960 --> 00:42:44,870 want to build with? And so came. We 74 00:42:44,880 --> 00:42:45,990 started launching these compute 75 00:42:46,000 --> 00:42:48,309 primitives as well. And so in 2017, we 76 00:42:48,319 --> 00:42:49,829 launched a product called workers, which 77 00:42:49,839 --> 00:42:51,990 is basically like serverless functions 78 00:42:52,000 --> 00:42:54,870 that you can run. It contrasted VMs at 79 00:42:54,880 --> 00:42:56,550 that time where you had to rent a whole 80 00:42:56,560 --> 00:42:58,550 machine and pay for the whole machine. 81 00:42:58,560 --> 00:43:00,470 Either you're using it or not using it. 82 00:43:00,480 --> 00:43:02,390 And so with workers, you just execute 83 00:43:02,400 --> 00:43:04,309 some code and then the worker goes away. 84 00:43:04,319 --> 00:43:05,910 You don't have to pay for it. It takes 85 00:43:05,920 --> 00:43:07,910 milliseconds to start up, not seconds as 86 00:43:07,920 --> 00:43:09,430 well. And then we followed with other 87 00:43:09,440 --> 00:43:11,109 compute primitives like durable objects 88 00:43:11,119 --> 00:43:12,790 which are more stateful. And then in 89 00:43:12,800 --> 00:43:14,630 2025, we launched containers as well 90 00:43:14,640 --> 00:43:16,230 because there are some workloads that do 91 00:43:16,240 --> 00:43:18,069 need something like a VM and we want all 92 00:43:18,079 --> 00:43:19,589 of those workloads to be able to run on 93 00:43:19,599 --> 00:43:22,710 Cloudflare. So now you know 2017 start 94 00:43:22,720 --> 00:43:24,630 of our developer platform we have these 95 00:43:24,640 --> 00:43:26,470 compute primitives people started using 96 00:43:26,480 --> 00:43:28,630 it people started building more like 97 00:43:28,640 --> 00:43:31,430 static websites and things like that so 98 00:43:31,440 --> 00:43:32,870 we thought to ourselves how do we 99 00:43:32,880 --> 00:43:34,470 elevate this and take it to the next 100 00:43:34,480 --> 00:43:38,150 level so we introduced storage as well 101 00:43:38,160 --> 00:43:39,589 and so we have these storage primitives 102 00:43:39,599 --> 00:43:41,589 on cloudflare's developer platform we 103 00:43:41,599 --> 00:43:44,309 have KV we have R2 do object storage we 104 00:43:44,319 --> 00:43:46,550 have databases like D1 as well and the 105 00:43:46,560 --> 00:43:48,230 idea here is that people should be able 106 00:43:48,240 --> 00:43:50,630 to build full stack applications 107 00:43:50,640 --> 00:43:53,030 all on Cloudflare stateful applications 108 00:43:53,040 --> 00:43:54,950 where people can go on and start 109 00:43:54,960 --> 00:43:58,309 building um full applications and web 110 00:43:58,319 --> 00:44:00,150 apps and platforms on Cloudflare's 111 00:44:00,160 --> 00:44:02,630 developer platform. So we have this 112 00:44:02,640 --> 00:44:04,069 infrastructure, we added compute 113 00:44:04,079 --> 00:44:05,910 primitives to it, we added storage 114 00:44:05,920 --> 00:44:09,109 primitives to it and then came 2023. And 115 00:44:09,119 --> 00:44:10,390 if you guys remember what happened in 116 00:44:10,400 --> 00:44:13,190 2023, chat GPT kind of blew up a little 117 00:44:13,200 --> 00:44:15,430 bit and it changed how we think about 118 00:44:15,440 --> 00:44:19,190 AI, right? And I remember it being 119 00:44:19,200 --> 00:44:21,349 birthday week and uh three weeks before 120 00:44:21,359 --> 00:44:22,790 birthday week a lot of us sat together 121 00:44:22,800 --> 00:44:24,710 in a room and we were like what should 122 00:44:24,720 --> 00:44:26,790 we do now that chat GPT is on the rise? 123 00:44:26,800 --> 00:44:28,950 What can Cloudflare play in here with 124 00:44:28,960 --> 00:44:31,030 our infrastructure and our primitives in 125 00:44:31,040 --> 00:44:35,109 order to be in this AI game? So in 2023 126 00:44:35,119 --> 00:44:37,030 we launched a product called workers AI 127 00:44:37,040 --> 00:44:39,589 where we added GPUs to our network. And 128 00:44:39,599 --> 00:44:41,430 so again this is really distributed 129 00:44:41,440 --> 00:44:43,349 where GPUs are running as close to the 130 00:44:43,359 --> 00:44:45,430 user as possible. It's not centralized 131 00:44:45,440 --> 00:44:47,910 like a US East1 hyperscaler for example, 132 00:44:47,920 --> 00:44:49,589 but it's running as close to the edge as 133 00:44:49,599 --> 00:44:51,990 possible. And we had a hypothesis when 134 00:44:52,000 --> 00:44:54,870 we started workers AI. Our hypothesis 135 00:44:54,880 --> 00:44:56,470 was that models were going to get 136 00:44:56,480 --> 00:44:58,390 smaller, so that network latency was 137 00:44:58,400 --> 00:45:01,190 going to become more and more important. 138 00:45:01,200 --> 00:45:02,870 We launched with small models that could 139 00:45:02,880 --> 00:45:04,790 fit on our small hardware around the 140 00:45:04,800 --> 00:45:06,470 world that were super distributed. And 141 00:45:06,480 --> 00:45:08,630 we thought that people would have models 142 00:45:08,640 --> 00:45:10,550 get better and faster and smarter and 143 00:45:10,560 --> 00:45:12,470 that network latency was going to play a 144 00:45:12,480 --> 00:45:15,670 much more bigger impact here. 145 00:45:15,680 --> 00:45:17,990 Well, uh actually the opposite happened 146 00:45:18,000 --> 00:45:20,230 and models got larger and larger. So we 147 00:45:20,240 --> 00:45:22,309 went from something like 7 billion 148 00:45:22,319 --> 00:45:24,150 parameters which was frontier scale at 149 00:45:24,160 --> 00:45:26,309 the time to being the three trillion 150 00:45:26,319 --> 00:45:27,510 parameter models that were getting 151 00:45:27,520 --> 00:45:30,710 launched. And I think this is not so 152 00:45:30,720 --> 00:45:32,309 shocking when you look back in time 153 00:45:32,319 --> 00:45:34,470 because you know as we're on the way to 154 00:45:34,480 --> 00:45:37,910 AGI um if you're following scaling law 155 00:45:37,920 --> 00:45:39,910 as you train on more data as the models 156 00:45:39,920 --> 00:45:41,589 get larger they're more generalizable 157 00:45:41,599 --> 00:45:43,510 and so on the path to AGI it makes sense 158 00:45:43,520 --> 00:45:44,870 that we're pushing the frontier of 159 00:45:44,880 --> 00:45:47,109 research here in order to get larger and 160 00:45:47,119 --> 00:45:49,670 larger and smarter models. 161 00:45:49,680 --> 00:45:51,990 So this goes against our hypothesis 162 00:45:52,000 --> 00:45:53,270 right that the models were going to get 163 00:45:53,280 --> 00:45:55,030 smaller. the models actually ended up 164 00:45:55,040 --> 00:45:56,710 being larger and the network latency 165 00:45:56,720 --> 00:45:58,870 mattered less and less. So what did we 166 00:45:58,880 --> 00:46:01,030 do? 167 00:46:01,040 --> 00:46:03,349 We ended up 168 00:46:03,359 --> 00:46:05,670 pivoting. So cloud got really good at 169 00:46:05,680 --> 00:46:07,589 serving these giant models as well. So 170 00:46:07,599 --> 00:46:09,990 in March 2026, we launched Kimmy and 171 00:46:10,000 --> 00:46:11,430 then we followed up with GLM and 172 00:46:11,440 --> 00:46:13,109 DeepSeek and these large trillion 173 00:46:13,119 --> 00:46:15,430 parameter models um hosted on our 174 00:46:15,440 --> 00:46:16,790 infrastructure. And so this is 175 00:46:16,800 --> 00:46:18,870 serverless. People hit an API call and 176 00:46:18,880 --> 00:46:20,950 get a token back. It actually took us a 177 00:46:20,960 --> 00:46:23,190 while in order to decide to do this, 178 00:46:23,200 --> 00:46:26,150 right? We March 2026 is actually pretty 179 00:46:26,160 --> 00:46:27,750 late in the game to to serving these 180 00:46:27,760 --> 00:46:29,349 model, but we were trying to make a 181 00:46:29,359 --> 00:46:31,190 conscious decision here on does this 182 00:46:31,200 --> 00:46:32,630 actually make sense for us to be serving 183 00:46:32,640 --> 00:46:34,230 it. It didn't really fit with our 184 00:46:34,240 --> 00:46:36,069 infrastructure story, but it fit very 185 00:46:36,079 --> 00:46:39,349 well with the developer platform. 186 00:46:39,359 --> 00:46:42,390 So in 2026, we had all these primitives 187 00:46:42,400 --> 00:46:44,710 in place. We had the infrastructure 188 00:46:44,720 --> 00:46:46,309 which is unique to Cloudflare running 189 00:46:46,319 --> 00:46:48,069 close to users. We had compute 190 00:46:48,079 --> 00:46:49,430 primitives where people were building 191 00:46:49,440 --> 00:46:51,750 agents on top of so workers and durable 192 00:46:51,760 --> 00:46:53,510 objects right as your agent needs to 193 00:46:53,520 --> 00:46:56,309 complete a task it goes on and it spins 194 00:46:56,319 --> 00:46:57,910 and does a task and then sleeps when it 195 00:46:57,920 --> 00:46:59,910 doesn't. We had storage primitives so 196 00:46:59,920 --> 00:47:01,990 your agents can actually interact with 197 00:47:02,000 --> 00:47:04,230 data and storage and then we had models 198 00:47:04,240 --> 00:47:05,750 these large models that were powering 199 00:47:05,760 --> 00:47:08,470 agents. And so all of this fit into the 200 00:47:08,480 --> 00:47:10,870 Cloudflare agent cloud story. And so we 201 00:47:10,880 --> 00:47:12,390 started telling this narrative that we 202 00:47:12,400 --> 00:47:13,990 want to be the agent cloud. We want 203 00:47:14,000 --> 00:47:16,309 agents to be deployed on us powered by 204 00:47:16,319 --> 00:47:18,230 models on us using our compute and data 205 00:47:18,240 --> 00:47:19,589 primitives all built on our 206 00:47:19,599 --> 00:47:21,910 infrastructure and our network. But it 207 00:47:21,920 --> 00:47:24,710 never felt quite right. We felt like we 208 00:47:24,720 --> 00:47:26,710 were always missing a piece here because 209 00:47:26,720 --> 00:47:28,230 the infrastructure pattern didn't 210 00:47:28,240 --> 00:47:30,550 necessarily fit in with what workers AI 211 00:47:30,560 --> 00:47:33,589 was designed for. 212 00:47:33,599 --> 00:47:35,030 And so I'm going to sit here and give 213 00:47:35,040 --> 00:47:36,870 you my second philosophical quote which 214 00:47:36,880 --> 00:47:39,589 is a risk considering that I'm a girl in 215 00:47:39,599 --> 00:47:41,030 Chinese and AI, but I'm going to give 216 00:47:41,040 --> 00:47:43,682 you this Chinese quote anyways. Um, 217 00:47:43,692 --> 00:47:44,309 [laughter] 218 00:47:44,319 --> 00:47:48,870 and uh, it basically says, 219 00:47:48,880 --> 00:47:50,790 which means everything is ready here. 220 00:47:50,800 --> 00:47:53,510 All that's missing is the east wind. 221 00:47:53,520 --> 00:47:55,910 And one of the things uh about this is 222 00:47:55,920 --> 00:47:57,349 like you can't really predict where the 223 00:47:57,359 --> 00:47:59,030 wind is going to come from or how it's 224 00:47:59,040 --> 00:48:00,470 going to come, but you need to be very 225 00:48:00,480 --> 00:48:02,470 reactive when it happens. And so in 226 00:48:02,480 --> 00:48:05,349 September 2026, the east wind came. 227 00:48:05,359 --> 00:48:08,069 Typesafe launched a model called Jev and 228 00:48:08,079 --> 00:48:09,910 introduced this idea of system one 229 00:48:09,920 --> 00:48:11,829 models and it contrasted what we 230 00:48:11,839 --> 00:48:13,829 typically knew with LLMs which were auto 231 00:48:13,839 --> 00:48:15,829 reggressive and taking a long time to 232 00:48:15,839 --> 00:48:18,230 generate tokens. It scales quadratically 233 00:48:18,240 --> 00:48:20,630 etc. A system one model is a completely 234 00:48:20,640 --> 00:48:22,549 different paradigm where it actually 235 00:48:22,559 --> 00:48:24,710 just takes one forward pass through the 236 00:48:24,720 --> 00:48:27,109 inputs and generates an output that you 237 00:48:27,119 --> 00:48:30,150 can use to make decisions with it. 238 00:48:30,160 --> 00:48:31,829 And so I really think that this came 239 00:48:31,839 --> 00:48:33,910 down to perfect timing. I think people 240 00:48:33,920 --> 00:48:35,510 were starting to get fed up a little bit 241 00:48:35,520 --> 00:48:37,829 with LLMs because they were so costly. 242 00:48:37,839 --> 00:48:40,309 It cost so many tokens. It's s time 243 00:48:40,319 --> 00:48:41,750 consuming. And I don't know about you, 244 00:48:41,760 --> 00:48:43,270 but when I read a reasoning trace, 245 00:48:43,280 --> 00:48:44,710 sometimes my blood pressure rises when 246 00:48:44,720 --> 00:48:46,710 the LLM doesn't get the right the things 247 00:48:46,720 --> 00:48:48,309 right. And so I think we were sending 248 00:48:48,319 --> 00:48:49,910 everything to LLMs and started getting 249 00:48:49,920 --> 00:48:51,510 frustrated with it. And so when this 250 00:48:51,520 --> 00:48:53,349 system one and Jev paradigm started 251 00:48:53,359 --> 00:48:55,430 coming on board, we were like this makes 252 00:48:55,440 --> 00:48:57,589 so much sense for us to adopt, right? We 253 00:48:57,599 --> 00:48:59,109 don't have to send every request to an 254 00:48:59,119 --> 00:49:01,430 LLM. We can send some of it to a system 255 00:49:01,440 --> 00:49:04,230 one model. And so the system one models 256 00:49:04,240 --> 00:49:06,150 actually end up taking work off of the 257 00:49:06,160 --> 00:49:08,230 giant LLMs and you give it to the small 258 00:49:08,240 --> 00:49:09,670 fast models and the two are supposed to 259 00:49:09,680 --> 00:49:11,349 work in tandem. They're supposed to 260 00:49:11,359 --> 00:49:13,349 complement each other and get to the end 261 00:49:13,359 --> 00:49:16,390 goal faster. And so if you look we are 262 00:49:16,400 --> 00:49:18,950 back at these small and fast models 263 00:49:18,960 --> 00:49:21,109 right this was the original workers AI 264 00:49:21,119 --> 00:49:23,750 hypothesis 265 00:49:23,760 --> 00:49:26,390 and so Cloudflare was not wrong here we 266 00:49:26,400 --> 00:49:28,390 actually were just early with these 267 00:49:28,400 --> 00:49:29,990 small and fast models the network 268 00:49:30,000 --> 00:49:32,390 latency became important again we want 269 00:49:32,400 --> 00:49:34,390 the model to be responding fast but also 270 00:49:34,400 --> 00:49:36,470 the network latency to be even faster 271 00:49:36,480 --> 00:49:39,670 and so we decided to do something here 272 00:49:39,680 --> 00:49:43,109 but Jev was closed source cloudflare 273 00:49:43,119 --> 00:49:44,790 couldn't actually host the Jev models 274 00:49:44,800 --> 00:49:45,910 and take advantage of our 275 00:49:45,920 --> 00:49:48,390 infrastructure. So what did we decide to 276 00:49:48,400 --> 00:49:51,510 do? We built our own version of Jev. So 277 00:49:51,520 --> 00:49:53,990 Jev launched on September 15th. Um on 278 00:49:54,000 --> 00:49:55,510 September 18th, just three days later, 279 00:49:55,520 --> 00:49:57,589 we started experimenting and we launched 280 00:49:57,599 --> 00:49:59,910 our version of Jev. Uh our first 281 00:49:59,920 --> 00:50:01,990 experiment of it. And then it took us a 282 00:50:02,000 --> 00:50:04,150 week of deliberation to decide do we 283 00:50:04,160 --> 00:50:05,750 actually want to be going into this 284 00:50:05,760 --> 00:50:08,390 Neolab territory. It's completely new to 285 00:50:08,400 --> 00:50:10,630 us. We don't know what we need to know 286 00:50:10,640 --> 00:50:12,230 it. you know, are we going to set the 287 00:50:12,240 --> 00:50:13,430 expectation that we're want to be a 288 00:50:13,440 --> 00:50:15,510 Neolab and are we going to live up to 289 00:50:15,520 --> 00:50:17,750 those expectations? But we decided to do 290 00:50:17,760 --> 00:50:19,750 it. I remember on a Friday evening, we 291 00:50:19,760 --> 00:50:21,430 were like, let's just go for it. And we 292 00:50:21,440 --> 00:50:23,030 did it. And we ended up training the 293 00:50:23,040 --> 00:50:25,030 model over 48 hours over the weekend. 294 00:50:25,040 --> 00:50:26,390 And then we launched it the following 295 00:50:26,400 --> 00:50:28,069 week. And so we ended up with a model 296 00:50:28,079 --> 00:50:30,309 called KFF. 297 00:50:30,319 --> 00:50:33,030 KFF gave power back to the agents to be 298 00:50:33,040 --> 00:50:34,870 able to make decisions on their own. So 299 00:50:34,880 --> 00:50:37,510 agents could ask act more autonomously. 300 00:50:37,520 --> 00:50:39,910 You didn't need to have a LLM reason 301 00:50:39,920 --> 00:50:41,430 through for minutes on if it should make 302 00:50:41,440 --> 00:50:43,190 this decision or not. You also didn't 303 00:50:43,200 --> 00:50:44,790 need to put a human in the loop and be 304 00:50:44,800 --> 00:50:46,630 interrupted by like a human's sleep 305 00:50:46,640 --> 00:50:48,069 schedule when you want your agents to be 306 00:50:48,079 --> 00:50:49,910 running 24/7. So you can now 307 00:50:49,920 --> 00:50:52,950 programmatically in code decide make 308 00:50:52,960 --> 00:50:54,870 decisions for your agent to take action 309 00:50:54,880 --> 00:50:57,349 on. 310 00:50:57,359 --> 00:50:59,750 So these decision models countered LLM 311 00:50:59,760 --> 00:51:01,349 models because they were a lot faster 312 00:51:01,359 --> 00:51:02,950 and they were also a lot cheaper as 313 00:51:02,960 --> 00:51:04,950 well. And so it actually as an inference 314 00:51:04,960 --> 00:51:06,630 provider takes a lot of pressure off the 315 00:51:06,640 --> 00:51:08,309 LLMs as well because the LLMs were 316 00:51:08,319 --> 00:51:10,790 reserved for work for meaningful work 317 00:51:10,800 --> 00:51:13,030 whereas the cleft and uh decision models 318 00:51:13,040 --> 00:51:15,670 could be used for fast um intuitive 319 00:51:15,680 --> 00:51:18,710 thinking and work right 320 00:51:18,720 --> 00:51:20,870 and so we also went back and thought to 321 00:51:20,880 --> 00:51:23,430 ourselves you know what else do agents 322 00:51:23,440 --> 00:51:26,230 and people use for LLMs for that are 323 00:51:26,240 --> 00:51:28,870 overkill. So we came up with decisions 324 00:51:28,880 --> 00:51:31,109 and um there must be other workloads out 325 00:51:31,119 --> 00:51:33,750 there that are unsuitable for an LLM 326 00:51:33,760 --> 00:51:35,109 both because they are either too 327 00:51:35,119 --> 00:51:37,430 expensive or take too long or maybe that 328 00:51:37,440 --> 00:51:39,190 it just would get the answer wrong 329 00:51:39,200 --> 00:51:40,630 because the auto reggressive nature of 330 00:51:40,640 --> 00:51:43,829 an LLM means that it would hallucinate. 331 00:51:43,839 --> 00:51:46,069 So we also launched a model 3 weeks 332 00:51:46,079 --> 00:51:47,750 later called cleric. We actually ended 333 00:51:47,760 --> 00:51:49,910 up launching four models in the span of 334 00:51:49,920 --> 00:51:51,270 three weeks. So I think we did pretty 335 00:51:51,280 --> 00:51:53,750 good job as a new lab. Um but cleric is 336 00:51:53,760 --> 00:51:55,990 a new model paradigm. So it's completely 337 00:51:56,000 --> 00:51:57,750 new. We're calling it an extraction 338 00:51:57,760 --> 00:51:59,589 model, but it's a system one model in 339 00:51:59,599 --> 00:52:01,190 that in the sense that it is just as 340 00:52:01,200 --> 00:52:03,829 fast and just as cheap, but it is also 341 00:52:03,839 --> 00:52:05,349 doing like one forward pass through the 342 00:52:05,359 --> 00:52:07,589 user. And the purpose of cleric is when 343 00:52:07,599 --> 00:52:08,950 you're interacting with the world, 344 00:52:08,960 --> 00:52:10,630 you're not interacting in perfectly 345 00:52:10,640 --> 00:52:12,630 typed JSON objects, right? Sometimes 346 00:52:12,640 --> 00:52:14,230 you're getting an email, a tweet, a 347 00:52:14,240 --> 00:52:16,230 slack message, but those people are not 348 00:52:16,240 --> 00:52:17,910 filling out perfect forms for you to 349 00:52:17,920 --> 00:52:20,230 extract inputs in out of. So what cleric 350 00:52:20,240 --> 00:52:22,309 does is it can take unstructured input 351 00:52:22,319 --> 00:52:25,190 and it can extract, compute, estimate 352 00:52:25,200 --> 00:52:27,430 and forecast values from those inputs 353 00:52:27,440 --> 00:52:29,270 and it gives you back a perfectly typed 354 00:52:29,280 --> 00:52:31,829 JSON objects which means the agents are 355 00:52:31,839 --> 00:52:33,750 able to do things with it. So we're kind 356 00:52:33,760 --> 00:52:35,670 of giving agents the ability to read 357 00:52:35,680 --> 00:52:40,390 here and take action with cleft as well. 358 00:52:40,400 --> 00:52:42,470 And so in code again you're giving 359 00:52:42,480 --> 00:52:44,150 cleric you know typed values from 360 00:52:44,160 --> 00:52:46,150 unstructured data. You can pass it in 361 00:52:46,160 --> 00:52:47,829 whatever kind of unstructured data. It 362 00:52:47,839 --> 00:52:49,510 can also take image input as well. And 363 00:52:49,520 --> 00:52:51,510 so you can just tell it to read 364 00:52:51,520 --> 00:52:53,589 basically what's in text here, pass it 365 00:52:53,599 --> 00:52:55,510 to KFT, decide what to do with this 366 00:52:55,520 --> 00:52:57,510 text, and then pass it to an LLM to 367 00:52:57,520 --> 00:53:01,430 decide what to do with the agent. 368 00:53:01,440 --> 00:53:03,589 And so Cloudflare had to become a neolab 369 00:53:03,599 --> 00:53:05,510 here. This was our missing bit. We had 370 00:53:05,520 --> 00:53:07,109 the infrastructure. We had the compute 371 00:53:07,119 --> 00:53:08,870 primitives. We had the data primitives. 372 00:53:08,880 --> 00:53:10,470 We had the large models powering the 373 00:53:10,480 --> 00:53:12,549 agents. But in order to take advantage 374 00:53:12,559 --> 00:53:14,309 of all of that and our infrastructure 375 00:53:14,319 --> 00:53:16,230 that we were built on, we had to be a 376 00:53:16,240 --> 00:53:18,069 new lab and ship these system one models 377 00:53:18,079 --> 00:53:19,270 that fit perfectly with our 378 00:53:19,280 --> 00:53:20,950 infrastructure. And I think one of the 379 00:53:20,960 --> 00:53:22,710 more important things is that we're not 380 00:53:22,720 --> 00:53:24,950 limited to this. Being a new lab means 381 00:53:24,960 --> 00:53:27,270 that we can create what we are missing, 382 00:53:27,280 --> 00:53:29,430 right? It means that we're not uh if 383 00:53:29,440 --> 00:53:31,270 we're missing a system one model because 384 00:53:31,280 --> 00:53:33,349 we're um at risk of it things being 385 00:53:33,359 --> 00:53:35,430 closed sourced or regulatory risks or 386 00:53:35,440 --> 00:53:37,109 things like that, we can actually create 387 00:53:37,119 --> 00:53:38,790 and invent things. So I think being a 388 00:53:38,800 --> 00:53:40,390 Neolab has actually opened a lot of 389 00:53:40,400 --> 00:53:42,390 doors for Cloudflare to do more in the 390 00:53:42,400 --> 00:53:45,030 AI world. 391 00:53:45,040 --> 00:53:46,630 So we went through backwards. We went 392 00:53:46,640 --> 00:53:47,510 through a little bit of a history 393 00:53:47,520 --> 00:53:49,510 lesson. I want to go forwards as well. I 394 00:53:49,520 --> 00:53:51,510 want to teach you guys what you guys can 395 00:53:51,520 --> 00:53:53,270 be doing with KF and Cleric. And I'm 396 00:53:53,280 --> 00:53:55,510 cognizant that I am at AI engineer code 397 00:53:55,520 --> 00:53:57,510 summit. So in a in a room full of 398 00:53:57,520 --> 00:53:59,190 developers and engineers, how can you 399 00:53:59,200 --> 00:54:01,430 guys leverage KFF and cleric and system 400 00:54:01,440 --> 00:54:03,430 one models in order to better use your 401 00:54:03,440 --> 00:54:05,589 workflows? 402 00:54:05,599 --> 00:54:08,230 So we'll do a quick demo here. Um it is 403 00:54:08,240 --> 00:54:10,470 an let's think you have an agent you're 404 00:54:10,480 --> 00:54:12,710 triaging like a CI build that went wrong 405 00:54:12,720 --> 00:54:15,109 right um let's say you have 40 tool 406 00:54:15,119 --> 00:54:17,190 tools that are available to this agent 407 00:54:17,200 --> 00:54:19,270 you have your PR that went wrong um and 408 00:54:19,280 --> 00:54:20,950 then you have your agent maybe it's 409 00:54:20,960 --> 00:54:22,710 running PI durable on a durable object 410 00:54:22,720 --> 00:54:25,190 says all of this is built on Cloudflare 411 00:54:25,200 --> 00:54:27,750 so this is the setup here tech typically 412 00:54:27,760 --> 00:54:29,109 what you would do is you would take an 413 00:54:29,119 --> 00:54:31,030 LLM and in the LLM would take a tool 414 00:54:31,040 --> 00:54:32,630 make a tool call the tool call would go 415 00:54:32,640 --> 00:54:34,309 back to the LLM the LM would make 416 00:54:34,319 --> 00:54:36,309 another tool call etc etc until the 417 00:54:36,319 --> 00:54:38,870 issue is resolved So this is like five 418 00:54:38,880 --> 00:54:41,430 back and forths with GLM 5.3. You're 419 00:54:41,440 --> 00:54:44,150 sending 40 tools every single time and 420 00:54:44,160 --> 00:54:45,750 maybe you're blowing up the context, 421 00:54:45,760 --> 00:54:47,030 right? Because you're sending the same 422 00:54:47,040 --> 00:54:48,710 kind of inputs back and forth to the 423 00:54:48,720 --> 00:54:51,430 LLM. And so what if we augment this with 424 00:54:51,440 --> 00:54:54,390 system one models 425 00:54:54,400 --> 00:54:56,870 with system one models? Uh this could 426 00:54:56,880 --> 00:54:58,470 look a lot different. And so let's say 427 00:54:58,480 --> 00:55:00,390 we were we want speed. So we pick cleft 428 00:55:00,400 --> 00:55:02,470 flash. Cle can actually help you prune 429 00:55:02,480 --> 00:55:04,549 your tool list. Maybe it only needs 10 430 00:55:04,559 --> 00:55:06,230 out of the 40 tools that are available. 431 00:55:06,240 --> 00:55:08,069 So you only need to pick 10 of those. 432 00:55:08,079 --> 00:55:10,150 Then let's say you take cleric and you 433 00:55:10,160 --> 00:55:12,470 use it to extract the values from the 434 00:55:12,480 --> 00:55:14,150 input. So it can get the PR number 435 00:55:14,160 --> 00:55:15,829 without reasoning and thinking about it 436 00:55:15,839 --> 00:55:18,069 too long. Um and then let's say you just 437 00:55:18,079 --> 00:55:19,750 you know orchestrate some code here to 438 00:55:19,760 --> 00:55:22,710 execute these API calls and uh you can 439 00:55:22,720 --> 00:55:24,230 actually just you know get those answers 440 00:55:24,240 --> 00:55:26,950 back. Then you ask cleric again um to 441 00:55:26,960 --> 00:55:29,030 pull the data from the logs. So normally 442 00:55:29,040 --> 00:55:30,630 when you're using with working with an 443 00:55:30,640 --> 00:55:32,230 LLM you probably just paste in all of 444 00:55:32,240 --> 00:55:34,950 the logs right like all 300 600 lines of 445 00:55:34,960 --> 00:55:37,030 logs. But Cleric can actually just go 446 00:55:37,040 --> 00:55:38,790 through and read those and say, "Here's 447 00:55:38,800 --> 00:55:40,710 the exact error. Here's the error 448 00:55:40,720 --> 00:55:42,390 message. Here are the lines of code it's 449 00:55:42,400 --> 00:55:43,910 affecting." And you're basically 450 00:55:43,920 --> 00:55:45,829 shrinking your context a lot. And then 451 00:55:45,839 --> 00:55:47,990 finally, you use an LLM to actually do 452 00:55:48,000 --> 00:55:51,270 take action and um make decisions and 453 00:55:51,280 --> 00:55:53,030 take action and extract values 454 00:55:53,040 --> 00:55:55,270 altogether. This same agent can run the 455 00:55:55,280 --> 00:55:57,109 system one. Let's do a little bit of a 456 00:55:57,119 --> 00:55:59,510 live demo. Let's see how it goes. We're 457 00:55:59,520 --> 00:56:01,430 going to have to click run both here. 458 00:56:01,440 --> 00:56:03,670 And this is a live demo. And uh I will 459 00:56:03,680 --> 00:56:05,670 say that 460 00:56:05,680 --> 00:56:09,510 Cloudflare our workers AI is uh has a 461 00:56:09,520 --> 00:56:11,510 very dal usage pattern. So we're in the 462 00:56:11,520 --> 00:56:12,870 middle of the workday. So we'll see how 463 00:56:12,880 --> 00:56:14,789 this how this goes. But it'll give you 464 00:56:14,799 --> 00:56:16,789 some illustrative example here of how it 465 00:56:16,799 --> 00:56:18,710 goes. Uh and we're also dealing with 466 00:56:18,720 --> 00:56:20,630 conference Wi-Fi. So so bear with me 467 00:56:20,640 --> 00:56:22,950 here. All right. The system one model 468 00:56:22,960 --> 00:56:25,829 side finished already and the all LLM 469 00:56:25,839 --> 00:56:27,910 one is is going. So I think even 470 00:56:27,920 --> 00:56:30,630 illustratively you can see that the uh 471 00:56:30,640 --> 00:56:32,470 system one set of models is a lot 472 00:56:32,480 --> 00:56:34,630 faster. Even by just this simple 473 00:56:34,640 --> 00:56:35,910 waterfall chart, you can see that the 474 00:56:35,920 --> 00:56:38,549 system one part takes so much less time 475 00:56:38,559 --> 00:56:41,030 than the uh actual generation which is 476 00:56:41,040 --> 00:56:43,349 the purple bars here. And I think they 477 00:56:43,359 --> 00:56:44,950 both ended up getting to the answer at 478 00:56:44,960 --> 00:56:46,309 the end. So the quality was not 479 00:56:46,319 --> 00:56:49,589 affected, but it took a lot less time. 480 00:56:49,599 --> 00:56:51,750 It was a lot cheaper. Six LLM calls 481 00:56:51,760 --> 00:56:54,789 versus one. 40,000 LLM tokens versus 482 00:56:54,799 --> 00:56:57,270 1,800. And so you can see how you can 483 00:56:57,280 --> 00:56:59,270 use these kind of system one and system 484 00:56:59,280 --> 00:57:00,870 two models even in like developer 485 00:57:00,880 --> 00:57:03,190 workflows in order to get you to your 486 00:57:03,200 --> 00:57:06,309 answer faster. 487 00:57:06,319 --> 00:57:08,390 So the same answer but it took a 488 00:57:08,400 --> 00:57:10,470 fraction of the LLM time and cost. 489 00:57:10,480 --> 00:57:13,030 Right? I ran this when we were not doing 490 00:57:13,040 --> 00:57:15,510 conference Wi-Fi but we had about 9.2 491 00:57:15,520 --> 00:57:18,789 seconds and 17.2 seconds on the LLM. It 492 00:57:18,799 --> 00:57:22,789 was almost two times faster. It was 72% 493 00:57:22,799 --> 00:57:25,349 less expensive. We shrunk our LLM calls 494 00:57:25,359 --> 00:57:27,589 from five down to one and it was about 495 00:57:27,599 --> 00:57:31,829 94% less tokens. And so this is an 496 00:57:31,839 --> 00:57:34,069 illustrative example of how you could be 497 00:57:34,079 --> 00:57:35,750 using these system one and system 2 498 00:57:35,760 --> 00:57:38,309 models in tandem. It's not intended to 499 00:57:38,319 --> 00:57:40,309 completely replace your LLM usage, but 500 00:57:40,319 --> 00:57:42,549 it can augment it and make it better by 501 00:57:42,559 --> 00:57:45,430 making allowing you to free up time and 502 00:57:45,440 --> 00:57:47,910 compute for your other more important 503 00:57:47,920 --> 00:57:50,150 tasks here. And you can h now do you 504 00:57:50,160 --> 00:57:51,750 know higher concurrency tasks because 505 00:57:51,760 --> 00:57:54,309 these models are able and capable to do 506 00:57:54,319 --> 00:57:56,390 so much more. 507 00:57:56,400 --> 00:57:57,829 And so that was one illustrative 508 00:57:57,839 --> 00:57:59,190 example. I just wanted to flash on the 509 00:57:59,200 --> 00:58:00,549 screen really quickly as well. What else 510 00:58:00,559 --> 00:58:01,990 are people building with cleft and 511 00:58:02,000 --> 00:58:04,069 cleric right? There's a lot of um 512 00:58:04,079 --> 00:58:06,309 routers that you can build and in making 513 00:58:06,319 --> 00:58:08,230 decisions there. Um there's fraud that 514 00:58:08,240 --> 00:58:10,870 we've been seeing a lot like spam or 515 00:58:10,880 --> 00:58:12,710 fishing. It can make decisions on there. 516 00:58:12,720 --> 00:58:14,309 With Cleric, you can use it to extract 517 00:58:14,319 --> 00:58:16,630 data from like web pages or support 518 00:58:16,640 --> 00:58:18,470 tickets and things like that. So, I'm 519 00:58:18,480 --> 00:58:20,230 excited to see uh we actually have a new 520 00:58:20,240 --> 00:58:21,910 launch coming out for Cleric. Uh we 521 00:58:21,920 --> 00:58:23,190 released an early preview, but it's 522 00:58:23,200 --> 00:58:24,630 coming out. So, I think these use cases 523 00:58:24,640 --> 00:58:27,109 will grow as uh people are trying it out 524 00:58:27,119 --> 00:58:29,670 more and more. But the idea is that 525 00:58:29,680 --> 00:58:31,589 there's so much in the world. We're not 526 00:58:31,599 --> 00:58:33,750 limited to just LLMs. Now, these new 527 00:58:33,760 --> 00:58:35,670 paradigms have been introduced and I 528 00:58:35,680 --> 00:58:36,710 think we should all take advantage of 529 00:58:36,720 --> 00:58:39,349 that and try building with it. 530 00:58:39,359 --> 00:58:41,270 And so we went backwards, we went 531 00:58:41,280 --> 00:58:42,950 forwards. But I think the most important 532 00:58:42,960 --> 00:58:45,670 thing here is that we go onwards, right? 533 00:58:45,680 --> 00:58:47,990 We have to be very reactive in this 534 00:58:48,000 --> 00:58:50,309 industry. Love it or hate it. Things 535 00:58:50,319 --> 00:58:53,430 just change on a whim. You know, if we 536 00:58:53,440 --> 00:58:55,910 as Cloudflare um back in the day said 537 00:58:55,920 --> 00:58:58,470 our hypothesis was wrong, uh we have to 538 00:58:58,480 --> 00:59:01,109 go and actually pivot and start serving 539 00:59:01,119 --> 00:59:03,270 these giant LLMs in order to take 540 00:59:03,280 --> 00:59:04,789 advantage of the fact that we want to be 541 00:59:04,799 --> 00:59:06,309 an agent cloud and that it makes sense 542 00:59:06,319 --> 00:59:09,349 for our story. when an east wing wind 543 00:59:09,359 --> 00:59:12,470 comes and we need to um have decision 544 00:59:12,480 --> 00:59:14,230 models because it fit perfectly with our 545 00:59:14,240 --> 00:59:16,150 infrastructure story, we had to pivot 546 00:59:16,160 --> 00:59:18,710 and be a neolab as well. So it's not 547 00:59:18,720 --> 00:59:20,630 it's one thing to sit here and look back 548 00:59:20,640 --> 00:59:22,870 but the more important thing is that we 549 00:59:22,880 --> 00:59:25,030 took what we had in stride and moved 550 00:59:25,040 --> 00:59:26,549 forwards. 551 00:59:26,559 --> 00:59:27,910 So I'll leave you with one last quote 552 00:59:27,920 --> 00:59:30,390 here 553 00:59:30,400 --> 00:59:32,710 which is a story from tool story. Uh 554 00:59:32,720 --> 00:59:34,549 remember then there's only one time that 555 00:59:34,559 --> 00:59:36,789 is important now. It is most important 556 00:59:36,799 --> 00:59:38,630 time because the only time when we have 557 00:59:38,640 --> 00:59:40,549 any power and so I'll leave you with 558 00:59:40,559 --> 00:59:42,549 this quote and a question of what are 559 00:59:42,559 --> 00:59:44,600 you going to build today?