AI Engineer Summit 2023
The 1,000x AI Engineer: Swyx
Read the talk
The 1,000x AI Engineer
AI engineering combines better tools, software built around models, and increasingly capable agents—but its largest multiplier may be how engineers teach and organize one another.
From a talk by Swyx
What if the clicker had more dimensions?
A presentation clicker moves forward and backward. Swyx arrives carrying a Magic Trackpad instead: what could a presenter do with more dimensions? His Tome slides extend the experiment into two-dimensional navigation. Before the argument begins, the presentation itself is an invitation to reconsider a familiar tool.
The deliberately provocative 1,000x title asks engineers to make a similar adjustment to their ambitions. It is an invitation to think in higher orders of magnitude, not a measured productivity result. Even the AI-generated artwork belongs to the experiment: at an AI conference, the medium should participate in the subject.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Being just in time
When would you want to be alive if you were a mathematician? Swyx nominates the era around 600 AD, crediting Brahmagupta with inventing zero after a roughly four-thousand-year wait. That compressed historical example introduces a broader idea: some periods make participation in a field unusually consequential. The mathematics slide widens the view from geometry and primes to zero and algebra, then calculus and probability.
For physics, he points to 1905 and 1927. The Solvay Conference supplies an explicit inspiration for this gathering: Einstein, Marie Curie, and figures familiar to an audience that has recently seen Oppenheimer. His other suggested windows are 1900–1930 for cars and 1980–2010 for personal computing products. The common thread is a period when foundational changes create room for many people to contribute.
That framing answers the internet lament about being born too late to explore Earth and too early to explore the stars. Swyx adds a demographic claim: humanity is roughly 73% of the way through what he calls “all concurrent intelligences,” conditional on neither expanding our intelligence nor going to other planets. The denominator is not defined, so the percentage cannot establish where we stand. The underlying invitation is clearer: AI offers a field in which the audience may be just in time.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From technology cycles to training compute
Carlota Perez’s Technological Revolutions and Financial Capital provides the next frame: technological revolutions have installation and deployment periods. This longer perspective matters when enthusiasm coexists with the suspicion that AI might be another fad, or another Web3. Swyx places the Industrial Revolution, railways, heavy engineering and steel, oil, and the recent technology revolution in that historical sequence. He describes these technology cycles as lasting roughly 50–70 years.
Where should the AI cycle begin? Swyx chooses AlexNet in 2012, roughly a decade before the talk. Unlike a schematic technology curve with an undefined vertical axis, the chart he describes uses the amount of compute devoted to training models. He identifies AlexNet as the blue-dot inflection where the field began recognizing that scale worked. Training compute measures resources committed to model development; it is not itself a measure of intelligence or engineer productivity. The practical instruction is to take continued scaling seriously when deciding what to build.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three reasons to plan for more
Swyx groups three scaling arguments around the number six:
- Compute investment. An unnamed investor recommends planning for roughly six orders of magnitude more compute by the end of the decade. This is a planning forecast, accompanied by an expectation of continued investment in language models.
- Conceptual progress. Swyx credits John Carmack with the view that six key insights remain on the path to AGI. More hardware and new ideas are distinct sources of progress; neither establishes the other.
- Human-scale analogy. Swyx attributes to George Hotz an analogy in which GPT-3 required about one person-year of compute and GPT-4 about 100 person-years. He then extends the comparison across six further model generations to a hypothetical GPT-10, suggesting compute beyond that of every human who has ever lived. The human-compute conversion and the scaling formula are not specified in the talk, so this remains a conditional analogy rather than a model forecast established by measurement.
These arguments serve a planning purpose: an engineer’s future working environment will be shaped by investment and research far larger than any individual project. Being present at the right moment makes it possible to build on those changes rather than having to produce every improvement oneself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The engineering work around the model
At the organizational level, AI engineering occupies an opening among ML researchers, ML engineers, and software engineers. Swyx makes a supply-and-demand argument: he estimates roughly 100,000 data-science and ML engineers, against GitHub’s claimed 100 million registered developers. He explicitly allows that the relevant developer population might instead be 40–50 million. These are not matched occupational counts; his prediction that AI engineers will outnumber ML engineers rests on the much larger pool of people who already write software.
The companion essay, The Rise of the AI Engineer, develops the emerging role. Here, the immediate distinction is engineering rather than prompting alone. In the talk’s framing, LLMs are not yet AGIs: useful products require code that coordinates models inside larger software systems.
The slide makes that design space concrete. One diagram places an LLM core inside a shell of code; another shows code coordinating multiple LLM components. In the first arrangement, software surrounds a model. In the second, software also determines how model components work together. That surrounding coordination is a substantial part of the product—and it uses skills software engineers already have.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Three meanings of AI engineer
The label covers three different relationships between software engineers and AI. First comes the engineer using AI tooling, illustrated by Copilot. Second comes the engineer building an AI product, illustrated by Midjourney. Third comes an AI product that could itself perform the engineer’s work, illustrated prospectively by AutoGPT and possibly Replit Ghostwriter. Ghostwriter is a historical coding-assistance example here; its inclusion in the replacement category expresses a possibility, not a demonstrated replacement capability.
To distinguish assistance from replacement, Swyx borrows the language of level-two and level-three self-driving systems. The consequential question is whether the human remains in the loop or serves as the fallback. He then names the three categories:
| Category | Who does the engineering? | AI’s role |
|---|---|---|
| AI-enhanced engineer | A human software engineer | Assists the engineer |
| AI product engineer | A human building an AI product | Supplies product capabilities |
| AI engineer agent | A nonhuman agent | Potentially performs engineering work |
The categories separate the use of AI tools, the construction of AI products, and the delegation of engineering work to AI.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Stacking improvements
Swyx playfully casts enhanced engineer, product engineer, and engineer agent as a career-ladder progression. The framing draws inspiration from Amjad Masad’s appearance on a16z’s The 1000x Developer; Amjad is also the next speaker. Swyx attributes to Sam Altman the observation that he sees 1,000x engineers at OpenAI every day. The anecdote motivates a decomposition into stackable improvements rather than supplying a measured comparison.
The 1,000x ambition becomes three compounding 10x improvements:
The multiplication does not identify three proven, independent productivity gains. It gives the audience a way to look for improvements at different layers, with speakers over the next two days addressing different parts of that stack. The personal question is where AI can increase productivity in one’s own work and life.
The same ambition extends to the gathering itself. Swyx sees a small room that could grow by 100x or 1,000x, and treats participation as part of that growth: meet other attendees, talk with speakers, and engage with sponsors. Tools and presentations alone do not create a field; the people making connections also contribute to its capacity.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The multiplier that travels home
The final account of engineering impact is deliberately nontechnical: a 10x engineer can be someone who teaches ten other people what they know. This shifts attention from how much one person produces to how much capability they help others acquire.
Swyx expresses the widening reach through O(N), O(N²), and O(2ᴺ). These are network-growth analogies, not measured outcomes for the conference. At the first level, attendees learn from talks, consume content, and welcome others into conversations through the Pac-Man rule. At the next level, they help others learn. His own first blog post came from encouragement at a conference like this one—and its subject was machine learning.
The final step happens after the event: go home, build local networks of AI engineers, and help create further networks of learning. The opportunity is not confined to absorbing knowledge in this room. It includes giving other people a place to begin, so that experiencing the start of an industry becomes something participants can carry into their own communities.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The 2023 essay explaining AI engineering as software development around foundation models.
Carlota Perez's 2002 book on technological revolutions, financial capital, and installation and deployment periods.
Amjad Masad's podcast conversation with Steph Smith about AI tools and developer impact, with episode links and timestamps.
The current AutoGPT repository, with source code and project setup information.
Further reading
- AI and computeArticle
OpenAI's 2018 analysis of training-compute growth in large AI experiments beginning in 2012.
- Commoditizing the PetaflopArticle
A George Hotz interview with a transcript discussing tinygrad, hardware and the person-of-compute analogy.
- Announcing Ghostwriter ChatArticle
Replit's 2023 introduction to Ghostwriter Chat and its code-assistance features.
Read the complete timestamped transcript
- 0:00
[upbeat music] [audience applauding] A few logistical things.
- 0:16
One, I'm ha- I'm carrying a magic trackpad because everyone has clickers. What if we had multiple dimensions? So we're gonna experiment with this today. [laughs] [laughing]
- 0:24
And, uh, and two, I'm also-- I'm using, like, AI, like, fancy new everything, right? Like, so this is Tome, um, and we're gonna go two-dimensional with our, uh, slides as well.
- 0:34
So I'm here to talk about the AI engineer. You're all here because you believe that there's some value to this idea, um, and then I just put, like, a ridiculous 1,000x on this.
- 0:42
Um, but I do think there is some meaning towards thinking about hires, or higher orders of magnitude towards raising your ambitions, and that's what I would like t- all of you to do today and to do with your friends back home.
- 0:55
So, um, and obv-obviously, a lot of AI-generated art because, I mean, it's an AI conference. We gotta do, we gotta do that. Um, first of all, I wanna congratulate you on being here.
- 1:04
Um, not just-- I'm, I'm not talking about here location-wise, physically. I'm talking about here in terms of the point in time. Uh, imagine if you were a mathematician. When was the best time to be born?
- 1:14
Uh, I, I would propose, uh, around about 600 AD. This dude, Brahmagupta, he invented zero. [laughing] Pretty, a pretty, pretty novel invention that took us only four thousand years to do that.
- 1:27
Um, but there's a, there's, there's certain times where, like, if you're in that field, you sh- you have to be there. You-- That's the thing. If you're alive during that time, you have to be doing that thing.
- 1:36
Uh, physics, when was the best time to be born? There's a right answer, 1905, 1927. Um, and this conference kind of is, um, inspired by the Solvay Conference. Um, that's Albert Einstein, Marie Curie, and a lot of people that you just saw in the Oppenheimer movie. [laughs]
- 1:53
Um, same thing. If you, if you made cars, there was a right time, 1900 to 1930. If you made personal computing products, 1980 to 2010.
- 2:02
Uh, if you ever get this, like, if you're a millennial, if you're very online, you ever get these memes like you're born too late to explore the Earth, born too early to explore the stars?
- 2:10
Um, you're not too late. We are here. Uh, this is based on demographics and history, the approximate timeline of all of humanity. Um, we know that we're roughly about seventy-three percent of all concurrent intelligences if we don't expand our own intelligences or go to other, other planets.
- 2:28
Um, so my argument or my message to you today is that you are just in time, and the timing is right to 1,000x.
- 2:36
Um, I think a lot, a lot of my technology and industrial organization thinking is informed by Carlota Perez, one of the most influential thinkers on tech revolutions. So she wrote this book about the installation and deployment periods of tech cycles, and we're definitely going through one today.
- 2:53
Uh, a lot of you on your mind here, you're-- I, I know you're here, but also mentally you're back home thinking, "How much of this is a fad? How much of this is Web3 again?"
- 3:01
Um- [laughing] And we've seen this over and over. Uh, the, the great historians greater than us have explored this, um, over the Industrial Revolution, the age of railways, age of heavy engineering and steel, oil, uh, and most recently the tech revolution.
- 3:18
Uh, funny enough, I f- I recently put a, I, I put a, put a... It's, it's, it-- They all roughly span between fifty to seventy years, and, uh, like, if you're around in that time, that's the field to be pursuing.
- 3:27
Um, so when did the AI revolution start? Um, we're very lucky. It's very hard historically to place a start point on something that changes ci- human civilization. We have a moment, 2012, AlexNet.
- 3:39
We're roughly ten years on. Um, and we can put numbers to it, right? So the, most of the time, these, these curves are sort of theoretical. They're just kind of like, "Ugh."
- 3:49
Like, Y-axis is just, "Ugh." Um, [laughs] here we can actually just put the amount of compute we put, uh, we're using towards training the models. There's a huge inflection. That's AlexNet right on the, the blue dot over there.
- 4:00
That's a huge inflection, uh, where we realized, gradually realized, it took too long to realize, but scale was starting to work. Um, and if you actually take this out, um, I-- A lot of people have been taking this out, and I want you to take scaling seriously.
- 4:14
There's three reasons why six is a magic number. Um, there's a very famous investor who I shall not name [laughs] who says, "Imagine roughly six orders of magnitude more compute by the end of the decade and plan for that."
- 4:23
Um, so there, there is more of this coming linear projection-wise, um, and you can plan f- on a lot more investment in language models. Uh, John Carmack says there's six key insights towards AGI.
- 4:35
And lastly, uh, George Hotz has these, these, like, really nice analogies. "GPT-3 took about one person year of compute. GPT-4 took about one hundred person years of compute." You stretch that out to GPT-10, the difference between GPT-4 and GPT-10, uh, again under the sixfold inc- increments in GPT advancements, and that would be more compute than the equivalent
- 4:55
compute of every human ever who ever lived.
- 4:58
Um, so just being in the right moment, you will get to ex- ex, uh, live on top of these mega, mega trends that is greater than any single one of us.
- 5:06
Um, and I think you're all here thinking about the AI engineer, and I put it in a very, very small sort of local context of, hey, what's the org chart and where do the ML engineers seat, sit, where the ML researchers sit, where the software engineers sit, and what's the gap that's opening?
- 5:21
It's the AI engineer. Um, it's a very much of a demand and supply argument. Um, there's something like a hundred thousand card-carrying data science machine learning engineers, and GitHub claims to have a hundred million, [laughs] uh, registered developers.
- 5:34
Uh, I don't know how, I don't know what the real number is. You can debate forty or fifty million to a hundred million. It's orders of magnitude more. Um, so we are, we think there's gonna be much more AI engineers than ML engineers.
- 5:46
There's all these reasons why, same reasons that I mentioned in the blog post that you've all read. Um, and also why engineering and not just prompting is because, um, LLMs themselves are not AGIs yet, right?
- 5:57
Like, we actually have to coordinate them in, in systems of software. We have to write code around them and orchestrate them with code, uh, in order to do, do something useful, and we already know how to do code.
- 6:08
So I wanna s-sort it out a little bit more. I think that the c- the conversation on AI engineer, um, has, like, sort of a vague, uh, discrepancy. And, and I wanna basically split it out into three areas of AI engineer.
- 6:20
Software engineer enhanced by AI tooling, like Copilot. Software g- software engineer building AI products, like Midjourney. AI product that replaces human engineer, potentially like AutoGPT and maybe replicate Ghost- Ghostwriter. [laughs]
- 6:33
Um, so let's give these guys a name. Uh, and in case you're wondering, like, well, enhanced by versus replaces, I think about it vers- uh, in, very much like the self-driving car terms, like level two, level three.
- 6:43
Uh, there's a difference between whether human's in the loop, whether or human's as the fallback.
- 6:48
So let's name it. Uh, three major types of AI engineer, the AI-enhanced engineer for people who are enhanced by AI, uh, people who build AI products, AI product engineer, and then the AI engineer agents who is not human.
- 7:01
Um, and naturally, of course, if you're interested in sort of progressing up the career ladder, there's AI-enhanced engineer, then products engineer, engineer agent. Um, so this talk was really inspired by, actually, Amjad, who's speaking next, [laughs] uh, where he did, did a recent talk on, uh, with the A- A6D podcast, and Sam Altman, who actually sees, um, 1,000X
- 7:20
engineers in OpenAI every day. Um, and it's really a set of stackable 10 by 10 by 10 improvements. Over the course of the next two days, I think you'll be seeing a, a lot of the speakers will be working on different parts of this stack.
- 7:33
Uh, so I really encourage you to think about where in, where in your life this AI movement, um, can improve and, uh, increase your productivity.
- 7:42
Um, I'm very, very honored to have ach- uh, drawn, um, from, from, you know, all over the world, um, the leading lights of the AI engineering movement. Um, we are a very small room today.
- 7:53
Um, I do think we can 100 and 1,000X from here. Um, and it's not just about tools and speakers, it's also about you. So I highly encourage you to take part in all the opportunities that we have for you to mix and mingle with each other, with the speakers, um, and with the sponsors as well.
- 8:08
So there's that. The final word, um, I do want to offer you, um, is effectively what I think in terms of non-technical terms, the AI, the, the 1,000X engineer could offer.
- 8:18
Um, my favorite advice for what a 10X engineer could look like is a, uh, an engineer that teaches 10 other people what they know, right? Um, that's, that's not a technical term, but it is, uh, very useful.
- 8:30
Um, and there's all these scaling laws for networks, which I, which I really keep in mind. So, uh, you can go from O of N to O N squared to O 2 to the power of N.
- 8:38
But really, um, what O N is, is you attending all the talks and learn- consuming all the content and letting people in with your Pac-Man rule. Um, O N squared is helping others learn.
- 8:49
Um, my very first blog post was at exactly a conference like this where I was encouraged to write something, and of course it was on machine learning. [laughs]
- 8:57
Uh, and finally going home and then building your own, uh, networks, uh, of AI engineers and helping, uh, to grow networks of, of, uh, of learning as well. So I hope you take that with you, uh, in your AI engineer journey.
- 9:09
I hope that over the next few days you get a sense of what it's like, uh, to be at the start of an industry, and I'm just glad to be here with you.
- 9:15
Thanks so much. [upbeat music]