AI Engineer Summit 2025
Why Agent Engineering
Read the talk
Why Agent Engineering
As models improve and inference gets cheaper, AI engineering is moving toward a more specific deliverable: agents that do useful work for users.
From a talk by swyx (Shawn Wang)
What does an AI engineer build?
How is AI engineering doing? There is now an O’Reilly book: Chip Huyen’s AI Engineering. There is also a hype-cycle chart placing the field near the peak—a setup for swyx’s joke that it can only go downhill from here. Recognition and inflated expectations have arrived together.
The earlier task was to establish that the discipline existed. The Rise of the AI Engineer named the emerging role; the first AI Engineer Summit explored three types of AI engineer; the following World’s Fair examined its maturation across disciplines. Those frameworks helped people organize teams, but a label that has gained acceptance still needs a distinct account of the work.
Two neighboring professions offer competing descriptions. From machine learning engineering, AI engineering can look like machine learning plus a few prompts. From software engineering, it can look like software plus a few LLM API calls. Neither description leaves much room for an independent discipline. Swyx’s own current heuristic is 90% software engineering and 10% AI, but he expects the AI portion to grow as the work develops its own practices.
Professional language already reveals different priorities. A machine learning engineer might say test-time compute, while an AI engineer says inference-time compute because inference is the product’s everyday work. A software engineer might simply call it reasoning. This is an anthropological view of engineering: people describe their work, form groups around shared concerns, and eventually establish industries. Agent engineering is the next proposed focus for that process.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From frameworks to production
Making the Summit an agent engineering conference required excluding RAG, open models, and GPUs as standalone topics. That was a choice about the program’s scope, not a rejection of those technologies. Narrowing the subject opened space for more specific kinds of agent work, and the previous year’s strongest-performing YouTube talks suggested that the audience wanted that focus.
Audience interest alone did not solve the selection problem. The available speakers were largely people who built agent frameworks for a living, while the unanswered question was who actually used agents in production. The new rule was no vendor pitches. Recruiting practitioners made curation harder: people running systems have less incentive to appear onstage than people selling the tools used to build them. Their experience was precisely what the conference needed.
Agents can extend existing application categories. Agent plus RAG, agent plus code generation, and agent plus search were swyx’s compact formula for commercial opportunity in 2025. The accompanying slide expands the combinations to vision, voice, wearables, finance, and money. An agent focus therefore need not isolate the field from the rest of AI engineering; it changes what builders try to accomplish with those components.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reconsidering the year of agents
Calling 2025 the year of agents does not make it so. Predictions often confuse what their authors want with what will happen. Swyx names Satya Nadella, Roman, Greg Brockman, and Sam Altman as advocates of that expectation, while acknowledging that he and his Latent.Space co-host Alessio had been skeptical themselves.
The audience had its own fatigue: in the previous day’s Family Feud exercise, agents ranked second among buzzwords people were tired of hearing. In a March 2024 conversation with David Luan, swyx and Alessio had advised people to remove agents from their branding. Now swyx was advising them to put it back. The reversal sets up two questions: what counts as an agent, and what has changed enough to make one useful?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Several definitions, no settled boundary
The definition depends partly on where the engineer starts. Machine learning brings reinforcement learning environments, actions, and goals. Software engineering offers the deliberately reductive joke: put it in a for loop. AI engineering has not yet settled on its own preferred boundary. These perspectives emphasize different parts of a system, even when they describe the same application.
Simon Willison’s crowdsourcing gathered hundreds of responses about what an agent is. Swyx organizes the recurring definitions around six properties:
| Emphasis | Defining concern |
|---|---|
| Goals | Autonomous pursuit of an objective |
| Tools | An LLM using tools |
| Control flow | An LLM directing what happens next |
| Long-running processes | Work that persists over time |
| Delegated authority | Permission to act on someone’s behalf |
| Multistep completion | Carrying a task through several steps |
These are different criteria, not a single checklist that every definition requires.
The full slide also includes metaphors, reductive descriptions, and specific examples. That last approach offers a practical test: collect systems people broadly agree are agents, then check whether a proposed definition includes them. It grounds a terminology debate in recognizable cases.
Definitions also matter because they shape platforms. OpenAI had introduced another agent definition in the previous day’s presentation, and swyx urges attention to it because the company will build around that understanding. The significance here is the connection between a definition and the systems it encourages, rather than a final resolution of the terminology.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Why agents may work now
Why should agents work now when they struggled a year or two earlier? The first answer is capability. Swyx presents trajectories from 2023 to 2025 approaching human baselines. Without named tasks and evaluation conditions, that is a directional argument about improving models, not a claim that agents have reached human performance across arbitrary work.
He separates three familiar contributors:
- Better reasoning: stronger model capability for working through a task.
- Better tool use: improved ability to use external capabilities.
- Better tools: a stronger ecosystem for connecting models to useful functions and information, including the Model Context Protocol, or MCP.
The distinction matters: a model’s ability to use a tool and the availability of a useful tool are separate parts of the engineering problem.
Model diversity adds another source of opportunity. Swyx estimates that OpenAI’s share fell from roughly 95% two years earlier to 50%, without specifying the market denominator. He also points to two possible frontier-lab challengers emerging in the preceding week. The argument is that builders face a more varied competitive landscape; which providers will prevail remains uncertain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Cheaper intelligence, more operating options
Capability is only part of viability; the cost of using it matters too. Swyx calls the decline in intelligence costs “super Moore’s law.” He claims that the cost of GPT-4-level intelligence fell 1,000-fold over 18 months, without specifying the benchmark or pricing basis. He sees a similar curve beginning for o1-level intelligence. The proposed enabling change is that an application can afford intelligence that would previously have been too expensive.
Reinforcement learning fine-tuning introduces another option for builders. Swyx explicitly says he has no personal experience with it and points to Will’s later presentation for a practitioner’s account. It belongs in the list of emerging capabilities, rather than as a technique demonstrated in this keynote.
The remaining changes concern both delivery and economics. A conversation with Brett Taylor raised charging for outcomes instead of costs, shifting the commercial unit toward the result a customer wants. Work on multiple agents expands how systems can organize tasks, while better hardware enables faster inference. Swyx points to Latent.Space conversations for the supporting discussions; here, these are additional contributors to agent viability, not detailed implementation recipes.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Useful delegation and unwanted delegation
Where does that combination already create value? Referring to Barry Zhang’s Building effective agents, swyx identifies coding and support agents as having product-market fit. He adds deep research to that category and anticipates further use cases in the conference program. These are his assessments of adoption and usefulness, not quantified results for every product in those categories.
His anti-use cases are more personal: stop repeating flight-booking agent demos. He wants to choose his own flights and place his own Instacart orders. That objection exposes a limit on indiscriminate delegation: a task can be technically automatable without being something a user wants to hand over. He also explicitly objects to astroturfing. The opportunity is to find valuable work users want agents to do, not merely another action an agent can perform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Adoption evidence and a growth forecast
The final example is ChatGPT’s growth. Swyx brings in a newly published user milestone and describes asking deep research to research OpenAI and draw the growth chart. Reuters reported more than 400 million weekly active users in February 2025, compared with 300 million in December 2024—approximately 33% growth over roughly two months. That is a shorter interval than the three months described onstage. Swyx frames the larger trajectory as going from zero to that scale in approximately two and a half years.
He reads the chart as a history of model-driven demand: a period he characterizes as a year without growth, followed by acceleration when agentic models arrived. Swyx credits the o1 models with doubling ChatGPT usage. That is his causal interpretation of the adoption curve; the reported user counts do not establish which releases caused the increase.
Extending the curve, swyx predicts one billion ChatGPT weekly active users by the end of 2025, five times the September 2024 level. The slide makes that extrapolation explicit with a December 2025 billion-user callout alongside model-release annotations. It is a forecast made in the talk, not an observed milestone.
The product thesis follows from that interpretation: stronger reasoning and more agents shipped to users can drive demand for AI products. Swyx frames the projected billion users as roughly one-eighth of the world’s population and sees substantial commercial room for other builders. The forecast supplies the scale of his ambition; the engineering question is which capabilities will become useful products.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The AI engineer’s evolving deliverable
Machine learning engineers build models; software engineers build software; AI engineers increasingly build agents. This is swyx’s proposed direction for the profession, not an exclusive boundary between jobs. It gives the emerging discipline a concrete deliverable: systems that put model capabilities to work on tasks users are willing to delegate.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The original essay proposing AI engineering as a discipline built around applying foundation models.
The original MCP announcement explains how clients and servers connect AI applications to external tools and data.
Further reading
- Building effective agentsArticle
Anthropic’s guide to workflows, autonomous agents, tool design and practical coding and support applications.
Contemporaneous reporting on the February 2025 weekly-user milestone and its December baseline.
Updates since the talk
- Agent EngineeringArticle
Swyx’s expanded companion to the keynote, covering agent definitions, enabling trends and conference examples.
Read the complete timestamped transcript
- 0:00
[on-hold music] Hi.
- 0:17
Good morning, everyone. [cheering] [clapping] Love that. Love that. Um, I'm gonna get right into it. One of the challenges we have with Summit is that we actually ask our spe- speakers to do very short talks.
- 0:27
So I, as the leader of Summit, I have to do even shorter talks. [laughing] So let's go. Uh, you can see a lot of these. There'll be a lot of show notes and homework.
- 0:34
You can see it on the live stream. How is AI engineering doing? Uh, it's pretty good. We have an O'Reilly book. That's pretty cool. Yeah. [laughing] Um, uh, Chip is actually a good friend, and she's actually speaking at, uh, she's giving our keynote for the workshop session, uh, tomorrow, which is pretty cool.
- 0:48
Uh, Gartner hates us. Gartner thinks we've, we've hit the peak. [laughing] So it's only downhill from here, guys. I'm sorry to inform you that AI engineering is over. Uh, there's no, there's nowhere else, else, uh, else to go but down.
- 0:58
Um, a lot of, uh, what I try to do with these eight, with these, uh, talks that I do at, at each conference is to try to landmark the se- the, the state of the art or the state of the industry.
- 1:10
Um, so with Latent.Space, I, I did the rise of the AI engineer. With the first AI Engineer Summit, we talked about the three types of AI engineer. And with last year's AI Engineer World's Fair, we talked about how the discipline of AI engineering was maturing and spreading across different disciplines.
- 1:26
Um, uh, I think this is starting to get a little stale. Uh, by now a few million people have seen this and, like, you know, uh, used this to form their teams, and I think that was the intended effect.
- 1:33
What I a- am encountering these days is the two resistance from two sides of the AI engineer spectrum. Uh, if you come from an MLE point of view, you think that the AI engineer is just, like, mostly an MLE plus a few prompts.
- 1:46
If you come from the software engineering point of view, you think that it's mostly software engineering and, uh, calling a few LLM APIs. Um, I think over time, it- the AI engineering is gonna basically emerge as its own discipline, and it's still not there yet.
- 1:59
It's still very, very early. I still say things like, "Oh yeah, AIE is ninety percent software engineering and ten percent AI." I think that will grow over time, and I think this is the year when it starts to spread out, and that's, that's what I'm here to talk about a little bit today.
- 2:13
Um, so for example, uh, I, I think, like, what I try to do with AIE is also, like, it's a, it's a work in anthropology, like how people describe themselves, form groups, form identities, and form industries.
- 2:24
Uh, so MLE, you know, it leaks out in your language. Um, they say test-time compute because the only reason to run inference is to test it. Uh, AIE will maybe say inference-time compute because we actually really care about inference.
- 2:34
Um, software engineers may be reas- reasoning. Um, a- a- and I think you see these differences, and I'm trying to articulate them over time. Um, part of what I wanna do here to set context is to explain why we've kind of pivoted AI Engineer Summit to be the Agent Engineering Conference.
- 2:48
Um, [coughs] it's not a decision that we made lightly because, uh, we're saying no to all these things. We're saying no to RAG. We're saying no to open models, uh, GPUs, and we're just saying, uh, you know, th- this is the only thing that we're gonna do today.
- 3:03
Um, and, uh, but, like, closing all those doors actually opens up others. So when we put out the call for speakers, we, uh, made up all this list of, uh, you know, other a- agent engineering disciplines.
- 3:13
Uh, and I soon realized we didn't have to. I'll talk about this in a bit. Um, I also looked at last year's top-performing talks on YouTube, and you guys told us, uh, that, you know, you really wanted all the, all the agentic things.
- 3:25
Now, the only problem with this is that we only got speakers who basically made agent frameworks for a living, uh, and everyone's asking the, the, the real question, "Who's putting this in production?"
- 3:35
So we had a new rule this year of, all right, no more vendor pitches. Um, you know, you, you complain about-- Yeah. Let's-- [clapping] Oh, thank you. [laughs] [clapping] Uh, as a, as a curator, it makes it so mu- infinitely harder because, uh, basically the people that you're about to see have no incentive to come on stage and share what
- 3:53
they're sharing, uh, but somehow we talked them into it. So, uh, I hope you're looking forward to that.
- 3:58
Uh, the, the other thing also I realized that, like, everything plus agent works basically. So agent plus RAG works. Agent plus code gen works. Agent plus search works. Um, and this is kind of like the simple formula for, like, making money in twenty twenty-five.
- 4:10
Uh- [laughing] Most of these, most of these names you'll see in the talks that, uh, that will follow, uh, in, in the sessions.
- 4:18
Um, some of you have heard this one before, twenty twenty-five is the year of agents, right? If you say it often enough, it might be true. Uh, I think that when people make predictions, oftentimes they confuse what they want to happen for what will actually happen.
- 4:31
Um, so maybe you believe Satya Nadella, maybe you believe Roman, maybe you believe Greg Brockman, maybe you believe Sam Altman. All of them want you to believe that twenty twenty-five is the year of agents.
- 4:41
Uh, and I'll be very honest, uh, me and my co-host, Alessio, I think I saw you over there. Hey. [clapping] Um, uh, [laughs] we were pretty skeptical as well. We were on the record being skeptical.
- 4:51
Actually, actually, all of you are being o- on the record because last-- yesterday, uh, Barr played, uh, Family Feud with, with our, with the audience, and the number two, uh, buzzword that everyone is tired of hearing is agents.
- 5:02
Um, but fortunately, you guys are not tired enough 'cause you came to today. I have you for one more day of, uh, of agents talk. Uh, but we're on record March twenty twenty-four with David Luan, uh, the former VP of Eng of OpenAI, uh, saying that we, we tell people to take agents off of their branding.
- 5:18
Uh, now we tell them to put it back on. [laughing]
- 5:20
So okay. The, um, I, I, I think, I mean, I'm doing this as a public service. To start any agents conference, we have to define the word agents. Are you guys ready?
- 5:29
Yeah. [laughs] All right. I actually have one. I, I-- it's a monumental task. I could do it in one slide. Um, so if you talk-- Again, this, this is a very POV, sort of anthropological point of view.
- 5:41
The machine learning people will talk about some kind of reinforcement learning environments. They wanna talk about actions, achieving goals, and all that. Um, AIE, we don't know what they, what they want yet.
- 5:49
Uh, the, the software engineers are very reductive. They're just, you know, put it in a for loop.[audience applauding] [laughs]
- 5:56
Okay, you-- It seems like you agree. Um, so, uh, fortunately, you know, I think every AIE conference needs to invoke the name of Simon Willison. Uh, he is our, uh, patron saint.
- 6:05
Um, he's actually gone and crowdsourced three hundred, uh, definitions of what an agent is. So I didn't have to survey all of you. I, I was thinking about asking every single speaker to start with what is your definition?
- 6:14
Uh, it doesn't matter. Um, there's-- here's six of them, right? You-- It's either about goals, it's about tools, it's about control flow, it's about long-running processes, it's about delegated authority, uh, it's about multi-step task completion.
- 6:26
Yeah, I see all the phones coming out. Don't worry, it's on the live stream, right? There's, like, twenty thousand people, uh, watching along. Um, and then there's a, there's a bunch of other things.
- 6:33
Uh, I think, I think the last one on the bottom left, bottom right is, uh, is an interesting one. Like, just have some things that everyone defines, uh, agrees is an agent, and make sure that they're sort of-- your agent definition is passing those things.
- 6:46
Um, except, so that was my one slide. That was my slide, uh, of, like, what, what defining an agent. And then yesterday, OpenAI went and dropped a new agent's definition, uh, on the live stream, uh, that you can w- well, watch yesterday as well.
- 6:58
Um, so this is something that they're obviously going to work with. Um, and, uh, I, I think you should, you should definitely pay attention to, to this because they're, they're building on top of this, uh, new definition as well. [lips smack]
- 7:08
So that's defining agents. Why now? Why is, why are agents working now when they did not work a year ago, two years ago? Um, I have a rough idea.
- 7:18
So the people are ta-ta-talking about capabilities, and so, uh, you can see that capabilities, even, even on a trajectory of twenty twenty-three, twenty twenty-five, um, have been, have been really growing, and they start to ru- around to hit human baselines, uh, right about now. [lips smack]
- 7:32
Um, and I also have a map of other cap-- [phone beeping] uh, reasons as well. So I'll just bring you through each of them. Most people will say, "Oh yeah, we have better reasoning now, we have better tool use now, we have better tools," um, including MCP, which, which you're doing a workshop on, uh, tomorrow. [lips smack]
- 7:47
Uh, but I think there are some other less appreciated things which I'm gonna bring up to you right now. Model diversity, right? Uh, the OpenAI market share has gone from, like, let's say ninety-five percent two years ago, now down to fifty percent.
- 7:56
It's a much more diverse, uh, uh, landscape, including, like, this, this, this past week, um, two frontier model labs that are possible challenges to OpenAI have emerged, and which I think, which I think is, um, really exciting for twenty twenty-five.
- 8:09
We, we don't actually know what is gonna shake out to it by the end of the year. Uh, the second thing is, uh, that the cost of intelligence is super Moore's law, is what I call it.
- 8:17
Um, it's, it's gone, uh... The cost of GPT-4 level intelligence has gone down one thousand times in the last eighteen months. Um, and you can see the same curve starting for the o1 level intelligence. [lips smack]
- 8:26
Um, uh, and also we now start to have RL, RL fine-tuning options. Um, I have zero experience in this area. [laughs]
- 8:33
But fortunately, one of our speakers, Will, uh, is gonna tell us, talk to us, uh, later today about this, about this. Um, so we have all these reasons. We have-- Uh, I have a few more.
- 8:42
Uh, you know, in our conversation with Brett Taylor, um, he talked about, uh, co- charging for outcomes instead of c-- for, uh, instead of costs. Um, there's a lot of work on multi-agents as well as, uh, faster inference as well that's coming out from the, the better hardware that we have.
- 8:53
Um, there's more homework there if you want. Uh, this is all sourced and, uh, you know, co-- has, has, has some backing in our, in our Latent.Space conversations, uh, but I don't really have time for that.
- 9:02
Okay, so one last thing for you guys on agent use cases. So, uh, I think most people agree w-with, like, Barri-um, Barry's, uh, building effective agents talk. Um, he's, he's gonna talk about how coding agents and support agents have product-market fit.
- 9:14
I think now it's fair to say deep research has PMF. Um, but also I will say up and coming are some of these use cases, uh, some of which you'll, you're gonna see in the con-- the talks later.
- 9:23
But I also wanna s-- offer anti-use cases. Can we please stop demoing agents that book flights? Yeah?
- 9:28
Yeah.
- 9:29
No more flight booking agents. [audience applauding] Uh, [laughs] I wanna book my own flights, thank you very much. I wanna, I wanna book my own Instacart orders, and also please don't astroturf for it.
- 9:38
Right. Uh, okay. So, uh, one, uh, yeah, and I think the reason that the tell that, uh, you know... This is, this is a headline that I saw yesterday.
- 9:45
I had to put this in. Um, OpenAI reported four hundred million users, uh, which is a thirty-three percent growth from three months ago. Um, and then you can ask deep research to research OpenAI and draw this chart of ChatGPT growth, uh, going from, uh, zero to, uh, four hundred million users in two years-- in two and a
- 10:01
half years. Um, so, uh, I, I, I remember this chart very well because Open-- Uh, ChatGPT spent a year not growing. And why did it spend a year not growing?
- 10:09
Because they didn't ship any, any agentic models. Um, and if you actually just look at the, uh, the sort of weekly active user chart and stretch it out, you actually get this chart, uh, which is actually super interesting because it basically shows that one, one, um, the sort of o1 models have doubled ChatGPT usage.
- 10:27
And if you stretch it out, um, ChatGPT is gonna hit a billion users by the end of this year. This year. Uh, it's basically going to quintuple the number of users it had, uh, as of September of last year.
- 10:37
Um, and so, like, the, the, the, the, the growth of, of ChatGPT and the growth of any AI product is gonna be very, very tied to reasoning capabilities and the amount of agents that you can ship for your users.
- 10:46
Um, it is, it is real. It is, it is, uh, huge, huge numbers. This is one-eighth of the world population that's gonna be using ChatGPT by the end of this year, and I think there's a lot of money left on the table for everyone else.
- 10:56
So, um, I hope you enjoy doing that. Um, I'm well past time, so I'm gonna skip all this. But basically, I, I think that the job of AI is now evolving towards building agents in the same way that MLEs build models, software engineers build software.
- 11:07
Um, so, uh, I'm gonna skip all that. You can see all, you can see all that on, on the, on the live stream. Uh, but we're actually, uh, you know, just here to welcome you to the show.
- 11:15
Um, and, uh, I'm really excited to introduce you to everyone. So, um, thank you, and I hope you enjoy. [upbeat music] [audience applauding]
- 11:25
Thank you.