Machine-generated source transcript; recognition errors remain. Timestamps and subtitles use the selected source recording’s clock, not clip-relative time. Section links use cited source moments; sections without a verified moment return to the talk start.
0:40:09Hi everyone, my name is Michelle Chen.
0:40:11I'm so excited to be here today at AI
0:40:13Engineer Code Summit chatting a little
0:40:14bit more about why Cloudflare became a
0:40:17Neolab. So at Cloudflare I am the group
0:40:19PM here. I run on the product team
0:40:21working on products like workers AI, AI
0:40:24search, AI gateway as well. And so today
0:40:26we're going to go through this little
0:40:27journey of how Cloudflare came to be.
0:40:29You might have heard that we launched
0:40:30and trained some models recently. I'm
0:40:32going to tell you a little bit more
0:40:33about why and what you can be doing with
0:40:36it. And so we're going to go through a
0:40:37little bit of a philosophical journey.
0:40:39Uh when I was preparing for this talk, I
0:40:41wanted to use this quote and then
0:40:43Anthropic like launched a blog post last
0:40:45weekend and used my quote, but I'm still
0:40:47going to use it anyways. Um so the quote
0:40:50goes
0:40:53in one second. Um the quote is from
0:40:56Kirkagard and it says, "Life can only be
0:40:58understood backwards, but it must be
0:40:59lived forwards." And I think this is
0:41:01really really apt quote here for us
0:41:03because at Cloudflare we're a
0:41:0516-year-old company and we've done so
0:41:07many things over the years but somehow
0:41:09we got all some of the pieces right and
0:41:11we ended up having to become a Neolab.
0:41:13So we're going to go through a little
0:41:14bit journey. The first thing we're going
0:41:15to do is look backwards and talk through
0:41:20why Cloudflare had to uh became a
0:41:23Neolab.
0:41:26Give me one second here.
0:41:29Perfect. We're going to go backwards
0:41:31[laughter] and talk through why
0:41:32Cloudflare became a Neolab. So,
0:41:35Cloudflare is a 16-year-old CDN company.
0:41:38We just celebrated our 16th birthday uh
0:41:41at birthday week this year uh in
0:41:44September.
0:41:46And we first started as a CDN. So, as a
0:41:48CDN company, what you have to do is you
0:41:50have to serve content as quick to your
0:41:52users as possible. you sit between the
0:41:55users request and the origin and you
0:41:57help uh sit in the middle and you can do
0:41:59things like cache and protect do
0:42:01security and things like that. So that's
0:42:02kind of how Cloudflare started. But
0:42:05Cloudflare's infrastructure had to be
0:42:07adaptable in order to become the CDN.
0:42:10And so what we did was 16 years ago in
0:42:122010, we launched with our own
0:42:13infrastructure. Our fast CDN had to be
0:42:16everywhere. And this looks really
0:42:17different than a traditional hyperscaler
0:42:19where compute is centralized in US- East
0:42:21one or US West one. Cloudflare's
0:42:23infrastructure is a completely different
0:42:25paradigm. We have data centers all
0:42:27around the world as close to the user as
0:42:29possible. And this is really important
0:42:31in becoming a CDN. But now we have these
0:42:34computers basically running as close to
0:42:36the user as possible. So we thought to
0:42:37ourselves, what else can we be doing
0:42:39with these computers that people would
0:42:40want to build with? And so came. We
0:42:44started launching these compute
0:42:46primitives as well. And so in 2017, we
0:42:48launched a product called workers, which
0:42:49is basically like serverless functions
0:42:52that you can run. It contrasted VMs at
0:42:54that time where you had to rent a whole
0:42:56machine and pay for the whole machine.
0:42:58Either you're using it or not using it.
0:43:00And so with workers, you just execute
0:43:02some code and then the worker goes away.
0:43:04You don't have to pay for it. It takes
0:43:05milliseconds to start up, not seconds as
0:43:07well. And then we followed with other
0:43:09compute primitives like durable objects
0:43:11which are more stateful. And then in
0:43:122025, we launched containers as well
0:43:14because there are some workloads that do
0:43:16need something like a VM and we want all
0:43:18of those workloads to be able to run on
0:43:19Cloudflare. So now you know 2017 start
0:43:22of our developer platform we have these
0:43:24compute primitives people started using
0:43:26it people started building more like
0:43:28static websites and things like that so
0:43:31we thought to ourselves how do we
0:43:32elevate this and take it to the next
0:43:34level so we introduced storage as well
0:43:38and so we have these storage primitives
0:43:39on cloudflare's developer platform we
0:43:41have KV we have R2 do object storage we
0:43:44have databases like D1 as well and the
0:43:46idea here is that people should be able
0:43:48to build full stack applications
0:43:50all on Cloudflare stateful applications
0:43:53where people can go on and start
0:43:54building um full applications and web
0:43:58apps and platforms on Cloudflare's
0:44:00developer platform. So we have this
0:44:02infrastructure, we added compute
0:44:04primitives to it, we added storage
0:44:05primitives to it and then came 2023. And
0:44:09if you guys remember what happened in
0:44:102023, chat GPT kind of blew up a little
0:44:13bit and it changed how we think about
0:44:15AI, right? And I remember it being
0:44:19birthday week and uh three weeks before
0:44:21birthday week a lot of us sat together
0:44:22in a room and we were like what should
0:44:24we do now that chat GPT is on the rise?
0:44:26What can Cloudflare play in here with
0:44:28our infrastructure and our primitives in
0:44:31order to be in this AI game? So in 2023
0:44:35we launched a product called workers AI
0:44:37where we added GPUs to our network. And
0:44:39so again this is really distributed
0:44:41where GPUs are running as close to the
0:44:43user as possible. It's not centralized
0:44:45like a US East1 hyperscaler for example,
0:44:47but it's running as close to the edge as
0:44:49possible. And we had a hypothesis when
0:44:52we started workers AI. Our hypothesis
0:44:54was that models were going to get
0:44:56smaller, so that network latency was
0:44:58going to become more and more important.
0:45:01We launched with small models that could
0:45:02fit on our small hardware around the
0:45:04world that were super distributed. And
0:45:06we thought that people would have models
0:45:08get better and faster and smarter and
0:45:10that network latency was going to play a
0:45:12much more bigger impact here.
0:45:15Well, uh actually the opposite happened
0:45:18and models got larger and larger. So we
0:45:20went from something like 7 billion
0:45:22parameters which was frontier scale at
0:45:24the time to being the three trillion
0:45:26parameter models that were getting
0:45:27launched. And I think this is not so
0:45:30shocking when you look back in time
0:45:32because you know as we're on the way to
0:45:34AGI um if you're following scaling law
0:45:37as you train on more data as the models
0:45:39get larger they're more generalizable
0:45:41and so on the path to AGI it makes sense
0:45:43that we're pushing the frontier of
0:45:44research here in order to get larger and
0:45:47larger and smarter models.
0:45:49So this goes against our hypothesis
0:45:52right that the models were going to get
0:45:53smaller. the models actually ended up
0:45:55being larger and the network latency
0:45:56mattered less and less. So what did we
0:45:58do?
0:46:01We ended up
0:46:03pivoting. So cloud got really good at
0:46:05serving these giant models as well. So
0:46:07in March 2026, we launched Kimmy and
0:46:10then we followed up with GLM and
0:46:11DeepSeek and these large trillion
0:46:13parameter models um hosted on our
0:46:15infrastructure. And so this is
0:46:16serverless. People hit an API call and
0:46:18get a token back. It actually took us a
0:46:20while in order to decide to do this,
0:46:23right? We March 2026 is actually pretty
0:46:26late in the game to to serving these
0:46:27model, but we were trying to make a
0:46:29conscious decision here on does this
0:46:31actually make sense for us to be serving
0:46:32it. It didn't really fit with our
0:46:34infrastructure story, but it fit very
0:46:36well with the developer platform.
0:46:39So in 2026, we had all these primitives
0:46:42in place. We had the infrastructure
0:46:44which is unique to Cloudflare running
0:46:46close to users. We had compute
0:46:48primitives where people were building
0:46:49agents on top of so workers and durable
0:46:51objects right as your agent needs to
0:46:53complete a task it goes on and it spins
0:46:56and does a task and then sleeps when it
0:46:57doesn't. We had storage primitives so
0:46:59your agents can actually interact with
0:47:02data and storage and then we had models
0:47:04these large models that were powering
0:47:05agents. And so all of this fit into the
0:47:08Cloudflare agent cloud story. And so we
0:47:10started telling this narrative that we
0:47:12want to be the agent cloud. We want
0:47:14agents to be deployed on us powered by
0:47:16models on us using our compute and data
0:47:18primitives all built on our
0:47:19infrastructure and our network. But it
0:47:21never felt quite right. We felt like we
0:47:24were always missing a piece here because
0:47:26the infrastructure pattern didn't
0:47:28necessarily fit in with what workers AI
0:47:30was designed for.
0:47:33And so I'm going to sit here and give
0:47:35you my second philosophical quote which
0:47:36is a risk considering that I'm a girl in
0:47:39Chinese and AI, but I'm going to give
0:47:41you this Chinese quote anyways. Um,
0:47:43[laughter]
0:47:44and uh, it basically says,
0:47:48which means everything is ready here.
0:47:50All that's missing is the east wind.
0:47:53And one of the things uh about this is
0:47:55like you can't really predict where the
0:47:57wind is going to come from or how it's
0:47:59going to come, but you need to be very
0:48:00reactive when it happens. And so in
0:48:02September 2026, the east wind came.
0:48:05Typesafe launched a model called Jev and
0:48:08introduced this idea of system one
0:48:09models and it contrasted what we
0:48:11typically knew with LLMs which were auto
0:48:13reggressive and taking a long time to
0:48:15generate tokens. It scales quadratically
0:48:18etc. A system one model is a completely
0:48:20different paradigm where it actually
0:48:22just takes one forward pass through the
0:48:24inputs and generates an output that you
0:48:27can use to make decisions with it.
0:48:30And so I really think that this came
0:48:31down to perfect timing. I think people
0:48:33were starting to get fed up a little bit
0:48:35with LLMs because they were so costly.
0:48:37It cost so many tokens. It's s time
0:48:40consuming. And I don't know about you,
0:48:41but when I read a reasoning trace,
0:48:43sometimes my blood pressure rises when
0:48:44the LLM doesn't get the right the things
0:48:46right. And so I think we were sending
0:48:48everything to LLMs and started getting
0:48:49frustrated with it. And so when this
0:48:51system one and Jev paradigm started
0:48:53coming on board, we were like this makes
0:48:55so much sense for us to adopt, right? We
0:48:57don't have to send every request to an
0:48:59LLM. We can send some of it to a system
0:49:01one model. And so the system one models
0:49:04actually end up taking work off of the
0:49:06giant LLMs and you give it to the small
0:49:08fast models and the two are supposed to
0:49:09work in tandem. They're supposed to
0:49:11complement each other and get to the end
0:49:13goal faster. And so if you look we are
0:49:16back at these small and fast models
0:49:18right this was the original workers AI
0:49:21hypothesis
0:49:23and so Cloudflare was not wrong here we
0:49:26actually were just early with these
0:49:28small and fast models the network
0:49:30latency became important again we want
0:49:32the model to be responding fast but also
0:49:34the network latency to be even faster
0:49:36and so we decided to do something here
0:49:39but Jev was closed source cloudflare
0:49:43couldn't actually host the Jev models
0:49:44and take advantage of our
0:49:45infrastructure. So what did we decide to
0:49:48do? We built our own version of Jev. So
0:49:51Jev launched on September 15th. Um on
0:49:54September 18th, just three days later,
0:49:55we started experimenting and we launched
0:49:57our version of Jev. Uh our first
0:49:59experiment of it. And then it took us a
0:50:02week of deliberation to decide do we
0:50:04actually want to be going into this
0:50:05Neolab territory. It's completely new to
0:50:08us. We don't know what we need to know
0:50:10it. you know, are we going to set the
0:50:12expectation that we're want to be a
0:50:13Neolab and are we going to live up to
0:50:15those expectations? But we decided to do
0:50:17it. I remember on a Friday evening, we
0:50:19were like, let's just go for it. And we
0:50:21did it. And we ended up training the
0:50:23model over 48 hours over the weekend.
0:50:25And then we launched it the following
0:50:26week. And so we ended up with a model
0:50:28called KFF.
0:50:30KFF gave power back to the agents to be
0:50:33able to make decisions on their own. So
0:50:34agents could ask act more autonomously.
0:50:37You didn't need to have a LLM reason
0:50:39through for minutes on if it should make
0:50:41this decision or not. You also didn't
0:50:43need to put a human in the loop and be
0:50:44interrupted by like a human's sleep
0:50:46schedule when you want your agents to be
0:50:48running 24/7. So you can now
0:50:49programmatically in code decide make
0:50:52decisions for your agent to take action
0:50:54on.
0:50:57So these decision models countered LLM
0:50:59models because they were a lot faster
0:51:01and they were also a lot cheaper as
0:51:02well. And so it actually as an inference
0:51:04provider takes a lot of pressure off the
0:51:06LLMs as well because the LLMs were
0:51:08reserved for work for meaningful work
0:51:10whereas the cleft and uh decision models
0:51:13could be used for fast um intuitive
0:51:15thinking and work right
0:51:18and so we also went back and thought to
0:51:20ourselves you know what else do agents
0:51:23and people use for LLMs for that are
0:51:26overkill. So we came up with decisions
0:51:28and um there must be other workloads out
0:51:31there that are unsuitable for an LLM
0:51:33both because they are either too
0:51:35expensive or take too long or maybe that
0:51:37it just would get the answer wrong
0:51:39because the auto reggressive nature of
0:51:40an LLM means that it would hallucinate.
0:51:43So we also launched a model 3 weeks
0:51:46later called cleric. We actually ended
0:51:47up launching four models in the span of
0:51:49three weeks. So I think we did pretty
0:51:51good job as a new lab. Um but cleric is
0:51:53a new model paradigm. So it's completely
0:51:56new. We're calling it an extraction
0:51:57model, but it's a system one model in
0:51:59that in the sense that it is just as
0:52:01fast and just as cheap, but it is also
0:52:03doing like one forward pass through the
0:52:05user. And the purpose of cleric is when
0:52:07you're interacting with the world,
0:52:08you're not interacting in perfectly
0:52:10typed JSON objects, right? Sometimes
0:52:12you're getting an email, a tweet, a
0:52:14slack message, but those people are not
0:52:16filling out perfect forms for you to
0:52:17extract inputs in out of. So what cleric
0:52:20does is it can take unstructured input
0:52:22and it can extract, compute, estimate
0:52:25and forecast values from those inputs
0:52:27and it gives you back a perfectly typed
0:52:29JSON objects which means the agents are
0:52:31able to do things with it. So we're kind
0:52:33of giving agents the ability to read
0:52:35here and take action with cleft as well.
0:52:40And so in code again you're giving
0:52:42cleric you know typed values from
0:52:44unstructured data. You can pass it in
0:52:46whatever kind of unstructured data. It
0:52:47can also take image input as well. And
0:52:49so you can just tell it to read
0:52:51basically what's in text here, pass it
0:52:53to KFT, decide what to do with this
0:52:55text, and then pass it to an LLM to
0:52:57decide what to do with the agent.
0:53:01And so Cloudflare had to become a neolab
0:53:03here. This was our missing bit. We had
0:53:05the infrastructure. We had the compute
0:53:07primitives. We had the data primitives.
0:53:08We had the large models powering the
0:53:10agents. But in order to take advantage
0:53:12of all of that and our infrastructure
0:53:14that we were built on, we had to be a
0:53:16new lab and ship these system one models
0:53:18that fit perfectly with our
0:53:19infrastructure. And I think one of the
0:53:20more important things is that we're not
0:53:22limited to this. Being a new lab means
0:53:24that we can create what we are missing,
0:53:27right? It means that we're not uh if
0:53:29we're missing a system one model because
0:53:31we're um at risk of it things being
0:53:33closed sourced or regulatory risks or
0:53:35things like that, we can actually create
0:53:37and invent things. So I think being a
0:53:38Neolab has actually opened a lot of
0:53:40doors for Cloudflare to do more in the
0:53:42AI world.
0:53:45So we went through backwards. We went
0:53:46through a little bit of a history
0:53:47lesson. I want to go forwards as well. I
0:53:49want to teach you guys what you guys can
0:53:51be doing with KF and Cleric. And I'm
0:53:53cognizant that I am at AI engineer code
0:53:55summit. So in a in a room full of
0:53:57developers and engineers, how can you
0:53:59guys leverage KFF and cleric and system
0:54:01one models in order to better use your
0:54:03workflows?
0:54:05So we'll do a quick demo here. Um it is
0:54:08an let's think you have an agent you're
0:54:10triaging like a CI build that went wrong
0:54:12right um let's say you have 40 tool
0:54:15tools that are available to this agent
0:54:17you have your PR that went wrong um and
0:54:19then you have your agent maybe it's
0:54:20running PI durable on a durable object
0:54:22says all of this is built on Cloudflare
0:54:25so this is the setup here tech typically
0:54:27what you would do is you would take an
0:54:29LLM and in the LLM would take a tool
0:54:31make a tool call the tool call would go
0:54:32back to the LLM the LM would make
0:54:34another tool call etc etc until the
0:54:36issue is resolved So this is like five
0:54:38back and forths with GLM 5.3. You're
0:54:41sending 40 tools every single time and
0:54:44maybe you're blowing up the context,
0:54:45right? Because you're sending the same
0:54:47kind of inputs back and forth to the
0:54:48LLM. And so what if we augment this with
0:54:51system one models
0:54:54with system one models? Uh this could
0:54:56look a lot different. And so let's say
0:54:58we were we want speed. So we pick cleft
0:55:00flash. Cle can actually help you prune
0:55:02your tool list. Maybe it only needs 10
0:55:04out of the 40 tools that are available.
0:55:06So you only need to pick 10 of those.
0:55:08Then let's say you take cleric and you
0:55:10use it to extract the values from the
0:55:12input. So it can get the PR number
0:55:14without reasoning and thinking about it
0:55:15too long. Um and then let's say you just
0:55:18you know orchestrate some code here to
0:55:19execute these API calls and uh you can
0:55:22actually just you know get those answers
0:55:24back. Then you ask cleric again um to
0:55:26pull the data from the logs. So normally
0:55:29when you're using with working with an
0:55:30LLM you probably just paste in all of
0:55:32the logs right like all 300 600 lines of
0:55:34logs. But Cleric can actually just go
0:55:37through and read those and say, "Here's
0:55:38the exact error. Here's the error
0:55:40message. Here are the lines of code it's
0:55:42affecting." And you're basically
0:55:43shrinking your context a lot. And then
0:55:45finally, you use an LLM to actually do
0:55:48take action and um make decisions and
0:55:51take action and extract values
0:55:53altogether. This same agent can run the
0:55:55system one. Let's do a little bit of a
0:55:57live demo. Let's see how it goes. We're
0:55:59going to have to click run both here.
0:56:01And this is a live demo. And uh I will
0:56:03say that
0:56:05Cloudflare our workers AI is uh has a
0:56:09very dal usage pattern. So we're in the
0:56:11middle of the workday. So we'll see how
0:56:12this how this goes. But it'll give you
0:56:14some illustrative example here of how it
0:56:16goes. Uh and we're also dealing with
0:56:18conference Wi-Fi. So so bear with me
0:56:20here. All right. The system one model
0:56:22side finished already and the all LLM
0:56:25one is is going. So I think even
0:56:27illustratively you can see that the uh
0:56:30system one set of models is a lot
0:56:32faster. Even by just this simple
0:56:34waterfall chart, you can see that the
0:56:35system one part takes so much less time
0:56:38than the uh actual generation which is
0:56:41the purple bars here. And I think they
0:56:43both ended up getting to the answer at
0:56:44the end. So the quality was not
0:56:46affected, but it took a lot less time.
0:56:49It was a lot cheaper. Six LLM calls
0:56:51versus one. 40,000 LLM tokens versus
0:56:541,800. And so you can see how you can
0:56:57use these kind of system one and system
0:56:59two models even in like developer
0:57:00workflows in order to get you to your
0:57:03answer faster.
0:57:06So the same answer but it took a
0:57:08fraction of the LLM time and cost.
0:57:10Right? I ran this when we were not doing
0:57:13conference Wi-Fi but we had about 9.2
0:57:15seconds and 17.2 seconds on the LLM. It
0:57:18was almost two times faster. It was 72%
0:57:22less expensive. We shrunk our LLM calls
0:57:25from five down to one and it was about
0:57:2794% less tokens. And so this is an
0:57:31illustrative example of how you could be
0:57:34using these system one and system 2
0:57:35models in tandem. It's not intended to
0:57:38completely replace your LLM usage, but
0:57:40it can augment it and make it better by
0:57:42making allowing you to free up time and
0:57:45compute for your other more important
0:57:47tasks here. And you can h now do you
0:57:50know higher concurrency tasks because
0:57:51these models are able and capable to do
0:57:54so much more.
0:57:56And so that was one illustrative
0:57:57example. I just wanted to flash on the
0:57:59screen really quickly as well. What else
0:58:00are people building with cleft and
0:58:02cleric right? There's a lot of um
0:58:04routers that you can build and in making
0:58:06decisions there. Um there's fraud that
0:58:08we've been seeing a lot like spam or
0:58:10fishing. It can make decisions on there.
0:58:12With Cleric, you can use it to extract
0:58:14data from like web pages or support
0:58:16tickets and things like that. So, I'm
0:58:18excited to see uh we actually have a new
0:58:20launch coming out for Cleric. Uh we
0:58:21released an early preview, but it's
0:58:23coming out. So, I think these use cases
0:58:24will grow as uh people are trying it out
0:58:27more and more. But the idea is that
0:58:29there's so much in the world. We're not
0:58:31limited to just LLMs. Now, these new
0:58:33paradigms have been introduced and I
0:58:35think we should all take advantage of
0:58:36that and try building with it.
0:58:39And so we went backwards, we went
0:58:41forwards. But I think the most important
0:58:42thing here is that we go onwards, right?
0:58:45We have to be very reactive in this
0:58:48industry. Love it or hate it. Things
0:58:50just change on a whim. You know, if we
0:58:53as Cloudflare um back in the day said
0:58:55our hypothesis was wrong, uh we have to
0:58:58go and actually pivot and start serving
0:59:01these giant LLMs in order to take
0:59:03advantage of the fact that we want to be
0:59:04an agent cloud and that it makes sense
0:59:06for our story. when an east wing wind
0:59:09comes and we need to um have decision
0:59:12models because it fit perfectly with our
0:59:14infrastructure story, we had to pivot
0:59:16and be a neolab as well. So it's not
0:59:18it's one thing to sit here and look back
0:59:20but the more important thing is that we
0:59:22took what we had in stride and moved
0:59:25forwards.
0:59:26So I'll leave you with one last quote
0:59:27here
0:59:30which is a story from tool story. Uh
0:59:32remember then there's only one time that
0:59:34is important now. It is most important
0:59:36time because the only time when we have
0:59:38any power and so I'll leave you with
0:59:40this quote and a question of what are
0:59:42you going to build today?
