# Vibes won't cut it

Chris Kelly

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/Dc3qOA9WOnE-vibes-wont-cut-it) · [Video](https://www.youtube.com/watch?v=Dc3qOA9WOnE)

## Vibes won’t cut it: changing production software safely with AI

Generating code is only part of engineering. Production work still requires architectural judgment, reliable feedback, meaningful review, and a repeatable way to direct AI.

Familiarity with code review, automated tests, and deploying software will help you connect the guidance to your own workflow.

### What a code-generation percentage leaves out

[0:19](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=19s)

If AI generates 30% of the code in a mature codebase, how much of the engineering work has it actually done? Chris Kelly opens with this challenge to forecasts that AI coding will soon make engineers unnecessary. The percentage is a claim he critiques, not a measured productivity gain.

![Three headlines cite 30% of Microsoft code, a prediction of 90% from Anthropic’s CEO, and most of Meta’s code within 12 to 18 months.](https://ai.engineer/api/talk-essay-media/53ffc66cee22efc8d86d10c0080e3322fd55760fa0f85ff47d3587f6c94ceaf6)

Headlines about AI writing code at Microsoft, Anthropic, and Meta.

Consider thousands of newly generated lines surrounded by millions of existing lines. The surrounding system already embodies decisions about architecture, infrastructure, and expected behavior. Those decisions constrain what the new code can and should do. Generating an addition inside those boundaries is different from establishing the boundaries themselves.

Kelly illustrates that constraint with a deliberately pointed anecdote: a Meta engineer spending six months building a button in an advertising platform. This is his characterization, not a company performance measurement. The button may sit inside an enormous system, but much of its design space has already been settled. A model completing that narrowly defined work inherits the engineering that made it narrowly defined.

### Generated code still has to run

[1:58](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=118s)

AI produces code in familiar programming languages, and that code still has to run somewhere. A correct line does not establish the correctness of a complex system: interactions create emergent behavior that is invisible when examining individual lines. Someone must operate the system, investigate failures, and understand the nuances well enough to repair it. Changing who writes the code does not remove those responsibilities.

Kelly has heard predictions of engineering’s disappearance before. During the cloud and DevOps transition, the sysadmins he knew moved from racking machines and booting kernels to work he describes as more valuable, better paid, and more satisfying. His farming analogy makes the qualification explicit: tractors displaced particular forms of labor, but they did not eliminate farming. A new abstraction can substantially change an industry without eliminating its underlying work.

### Code is an artifact of engineering

[3:14](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=194s)

In Kelly’s definition, **vibe coding** means letting AI write and reason through the code without examining it yourself. The feedback loop is behavioral: request something, see whether it does what you want, and keep going. The code itself does not receive the scrutiny that would let you explain or deliberately change its implementation.

Kelly frames production as software with four-nines availability expectations, thousands of users, and gigabytes of data. These are examples of operational stakes, not universal thresholds for calling software production. At those stakes, observing that a feature appears to work is insufficient; the implementation carries obligations beyond the immediate interaction.

**Code is the artifact of software engineering, much as a blueprint is an artifact of architecture.** The work includes deciding what the software should do, what to build, and which packages to bring into the system. Producing the artifact is necessary, but counting its lines misses the decisions that give it meaning. That is why generating code and engineering production software cannot be treated as interchangeable accomplishments.

![A beige slide pairs the statement about code being an artifact of work with a pink brain built from toy bricks.](https://ai.engineer/api/talk-essay-media/0b0b80ee5541f373b0663dd640d60b5a750e6ce84bd45ab3e526883cf79d278f)

“Code isn’t the job. It’s an artifact of the work.”

### More code creates more obligations

[4:47](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=287s)

Kelly invokes Jeff Atwood’s [“The Best Code is No Code At All”](https://blog.codinghorror.com/the-best-code-is-no-code-at-all/): every line creates something to maintain, debug, and take responsibility for. Making code cheaper to generate does not make those obligations disappear. Optimizing for the amount generated can therefore move in the wrong direction; the useful objective is to accomplish the required behavior with as little code to own as possible.

The obligations begin with trade-offs. A package has performance characteristics; an implementation approach brings its own constraints. Take the same flight-booking system and build it as a monolith, as microservices, or as an event-driven system. The user-facing goal can remain the same while the architecture demands many different decisions. Naming the desired application does not resolve those choices.

Kelly’s concern is that generating familiar patterns does not settle the judgment required by a particular production system. His example is the codebase where only Bob knows how one part works and only Jane can fix another, although she left the project years ago. Such idiosyncrasies accumulate into system-specific knowledge. When an outage arrives at two in the morning, someone still has to diagnose what this system is doing. His critique is about the limits of relying on patterns in that setting, not a demonstrated theorem about everything an LLM can or cannot do.

### The objective is safe change

[6:37](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=397s)

**Software engineering means changing software safely.** Adding functionality or modifying existing code must preserve the outcomes users depend on: the service remains available, the widget ships, and data stays secure. That definition shifts attention from how quickly code appears to what happens when the change enters the system.

The industry already has several ways to make change safer:

- **Codebase knowledge:** understand the system before modifying it.
- **Version control:** keep changes under control and inspectable.
- **Tests:** detect whether changing one part breaks behavior elsewhere.
- **Type systems and deployment strategies:** provide additional safeguards around implementation and release.

AI can contribute to this work if its context helps it understand more of the codebase. Kelly identifies that context as central to Augment’s approach. But assistance with understanding or generating a change does not transfer responsibility for production away from the engineers operating it.

### Reassessing tools as their capabilities change

[7:49](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=469s)

Kelly finds professional engineers unusually hesitant to adopt AI compared with earlier developer tools. From his experience at GitHub during its early growth, engineers eagerly adopted Git; he recalls similar enthusiasm for cloud infrastructure. Their reluctance around AI is his observation rather than survey evidence, and he does not offer a settled explanation for it.

The tools themselves have changed quickly in his account. Early AI coding often amounted to limited tricks. He identifies [**Claude 3.5 Sonnet**](https://www.anthropic.com/news/claude-3-5-sonnet) as a substantial improvement in coding quality, followed by a wave of agent announcements, including [Augment’s agent](https://www.augmentcode.com/blog/meet-augment-agent). These are milestones relative to the recording, not a claim about today’s model rankings. The practical question is how to bring that improving capability into an engineering workflow.

### Make the repository and the task understandable

[9:14](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=554s)

Start with the conditions under which AI will write code. Kelly recommends the same clarity and working infrastructure that a human engineer needs:

- **Documented standards and direction:** state which packages and practices to use. When a codebase is transitioning between alternatives, document where it is going so that existing examples do not become the only guide.
- **Reproducible environments:** make it straightforward to start a development environment instead of depending on a bespoke setup.
- **Accessible testing:** make local tests easy to run and fast enough to support iteration.
- **Clear boundaries:** identify the scope of the intended change. Asking for a module extraction using the [Strangler pattern](https://martinfowler.com/bliki/StranglerFigApplication.html) names an approach but still leaves substantial decisions unspecified.
- **Defined tasks:** explain what behavior should change. A vague request to make a button do something different would be inadequate for a senior engineer or a junior engineer, and it is inadequate for an AI agent.

These requirements turn an open-ended request into work that can be understood and checked.

![A slide lists documented standards, simple reproducible environments, accessible automatic testing infrastructure, clear boundaries, and clearly defined tasks and work.](https://ai.engineer/api/talk-essay-media/b5b193790c8d2536c25eb7e977d16c0da7b58d2c664fd554df0c67235b5f6616)

Five practices for building software that’s easy for AI to write.

None of this requires a separate discipline invented for AI. Bespoke test infrastructure and unique developer environments already hurt human productivity. Preparing a codebase for AI exposes the same weaknesses: unclear direction, expensive setup, and tasks whose acceptance criteria exist only in someone’s head.

### Give generated code a feedback loop

[10:46](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=646s)

A human engineer does not reliably produce flawless code on the first attempt. Kelly describes his own normal process: write code, run a test, discover a mistake, and use tools such as linters to correct problems. Expecting an AI to skip that process sets a standard that engineering practice itself does not meet.

Give AI access to the same feedback that makes human work reliable. The loop is to generate a change, run checks, correct failures, and repeat. A useful AI coding setup must support that iteration rather than treating the first response as a finished implementation.

### Review the change, not the alphabet

[11:21](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=681s)

As agents produce more code, **code review** becomes a central engineering skill: can you read someone else’s implementation and explain why it is good or bad? Kelly argues that hiring should test this ability rather than concentrating on esoteric coding puzzles. Producing an answer and evaluating an unfamiliar answer are different skills, and agent-assisted work increases the importance of the latter.

Current review interfaces can make that evaluation harder. A lexicographically sorted list asks the reviewer to inspect file A, then file B, even though alphabetical order says nothing about how the software’s behavior changes. Understanding a change requires connecting its parts, not merely traversing filenames. Kelly anticipates better review tools for this purpose; he does not demonstrate one here. Meanwhile, the immediate skill to develop is reading changes for their meaning.

### A fluent explanation is not an action log

[12:09](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=729s)

For engineers still unsure how much to trust AI, Kelly starts with its conversational surface. He recounts challenging an assistant about a file and receiving the explanation, “I just scanned that file, I didn't read it.” The apology sounds like something a human colleague might say, but it does not explain which operation the software performed or what information it actually received.

Kelly interprets the response as imitation of familiar human apologies: language about skimming a document offers a plausible conversational answer when someone complains that it was not read carefully. That account is his interpretation, not evidence about the assistant’s specific training examples. The operational lesson is firmer: **generated descriptions of actions are claims, not proof that those actions occurred.** Read an assistant’s explanation with that distinction in mind.

### Separate quality from personal style

[13:25](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=805s)

Generated code may look different from the code you would have written. So does a colleague’s code. Before spending effort making it resemble your own, ask whether the difference makes it better or worse, or merely different. You can direct a model toward your preferences, but that work has a cost and should have a reason.

Linters, rule systems, and style guides exist to settle recurring conventions without repeatedly arguing over them. Put shared expectations there, then reserve review attention for differences that matter. Accepting an unfamiliar but sound implementation is part of making AI assistance useful.

### Carry a saved plan through define, create, refine

[14:04](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=844s)

Make your expectations persistent. Kelly starts projects with a **rules file** describing the stack and the guidelines he wants followed, then includes it in the model’s context. This is generic project guidance, not a requirement for a particular tool-specific filename or rules feature. It gives subsequent requests a shared foundation instead of requiring every prompt to restate the project’s conventions.

His preferred workflow is a repeated **define, create, refine** loop:

1. **Define:** ask the model to help write a specification or plan describing the intended work. Save it as a Markdown file so that the plan becomes an explicit artifact.
2. **Create:** supply the saved plan as context and have the agent implement against it. Make edits to the plan as needed so that the written direction reflects what you want built.
3. **Refine:** inspect and adjust the implementation, using direct edits or code completions for targeted changes. Repeat the loop as the work develops.

The plan carries intent into implementation, while refinement makes room for the corrections that normal engineering requires.

![A slide titled “I have no idea what I’m doing” displays ten numbered tips beside a toy-brick dog at a desk with a laptop and laboratory equipment.](https://ai.engineer/api/talk-essay-media/f6de6c2d9925270a69a6409c3b44cc9e682841eed5805d564676407232e69a83)

Ten tips, ending with “Write a rules file” and “Define-create-refine loop.”

Repeating this process helps you learn how to prompt for the behavior you need and where to intervene afterward. Kelly describes it as an efficient way to work, without offering a quantitative benchmark. Its value also depends on the earlier distinction between quality and preference: the target is good, functional code, not an exact reproduction of how you personally would have written it.

## Resources

- [The Best Code is No Code At All](https://blog.codinghorror.com/the-best-code-is-no-code-at-all/) — Jeff Atwood explains why every additional line carries maintenance and comprehension costs.
- [Claude 3.5 Sonnet launch announcement](https://www.anthropic.com/news/claude-3-5-sonnet) — Anthropic's historical announcement describes the model's coding capabilities and internal evaluation results.
- [Meet Augment Agent](https://www.augmentcode.com/blog/meet-augment-agent) — Augment's agent announcement explains its use of codebase context, persistent memories, tools, and checkpoints.
- [Strangler Fig](https://martinfowler.com/bliki/StranglerFigApplication.html) — Martin Fowler explains gradual replacement of legacy systems through incremental delivery.
- [Introducing Augment Rules](https://www.augmentcode.com/changelog/introducing-augment-rules) — Instructions for organizing project rules and controlling when an agent receives them.
- [Why the future of software engineering is beyond the IDE](https://www.augmentcode.com/blog/beyond-the-ide) — Kelly's later essay explores engineering workflows centered on reviewing and directing agents.
- [Alphabet's April 2025 coding-assistance claim](https://abc.xyz/investor/events/event-details/2025/2025-Q1-Earnings-Call/) — The earnings-call transcript records Pichai's explanation of Google's AI-assisted code check-ins.

## Slides

### Innovation Partner and Platinum Sponsors

[0:10](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=10s) [AWS is the innovation partner; six platinum sponsor logos appear below.](https://assets.ai.engineer/sites/aiecode2025/sha256/3d78c64be6a2781979150dc6d263140efd066d2bc3c24d387a2d222c11434a30.webp)

INNOVATION PARTNER

aws

PLATINUM SPONSORS

Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### Headlines on AI-Generated Code

The speaker questions AI coding headlines from Microsoft, Anthropic and Meta, arguing that their leaders may underestimate the realities of production systems.

[0:28](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=28s) [Three headlines attribute claims about AI writing code to leaders at Microsoft, Anthropic and Meta.](https://assets.ai.engineer/sites/aiecode2025/sha256/226431cf88eb6ab1e55237b032b016019d82be082265b9289a4284df46a267c7.webp)

Satya Nadella says as much as 30% of Microsoft code is written by AI
PUBLISHED TUE, APR 29 2025 · 9:33 PM EDT | UPDATED TUE, APR 29 2025 · 9:58 PM EDT

Anthropic's CEO says that in 3 to 6 months, AI will be writing 90% of the code software developers were in charge of
Mar 13, 2025, 11:27 PM PT

Mark Zuckerberg 'predicts' AI will write most of Meta's code within 12 to 18 months
Updated Wed, Apr 30, 2025 · 2 min read

[illegible] code

Interface text:
AIE
World's Fair
aws

### The reports of my death have been greatly exaggerated

The speaker argues that AI-generated code builds on massive existing codebases, where prior architectural and infrastructure decisions constrain what new code can do.

[0:58](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=58s) [A quote attributed to Twain appears above point 1: AI writes code based on massive codebases.](https://assets.ai.engineer/sites/aiecode2025/sha256/1c1d71eccd01d9bf58d21c15704e00451c723444ad1bbc1856b26002c06a319f.webp)

The reports of my death have been greatly exaggerated. – Twain

1. Their AI is writing code based on massive codebases.

[illegible] code

Interface text: AIE; Microsoft; smol ai; AI Engineer World's Fair

[1:52](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=112s) [A quote attributed to Twain and a first point stating that AI writes code based on massive codebases.](https://assets.ai.engineer/sites/aiecode2025/sha256/bd3106200080caea578293916fea5489b8c8fba3a3463b59bdbbb4690112e50c.webp)

The reports of my death have been greatly exaggerated. – Twain

1. Their AI is writing code based on massive codebases.

[illegible] code

Interface text: AIE; Microsoft; smol ai; AI Engineer World's Fair

### AI code still needs to run

AI-generated code still needs to run in production, where complex systems can fail in ways that aren't apparent from individual lines of code.

[1:56](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=116s) [The slide adds that AI code is still code and does not run itself.](https://assets.ai.engineer/sites/aiecode2025/sha256/ea771d7a18868b40cbc5ff6de2b4a6acb71f343c8b835504568121ce3d165a27.webp)

The reports of my death have been greatly exaggerated. - Twain

1. Their AI is writing code based on massive codebases.
2. AI code is still code, and code doesn’t run itself.

code

Interface text: AIE; AI Engineer World's Fair; aws

### The reports of my death have been greatly exaggerated

AI-generated code still needs people to run and troubleshoot complex systems. The speaker points to sysadmins’ changing work during the DevOps and cloud transition as a precedent.

[2:18](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=138s) [The slide states that AI writes code based on massive codebases and that code does not run itself.](https://assets.ai.engineer/sites/aiecode2025/sha256/07ec646a6a57acd1568bfbcd9c308a5eb17ddcb60b33d7bcacee30a9b65f451f.webp)

The reports of my death have been greatly exaggerated. - Twain

1. Their AI is writing code based on massive codebases.
2. AI code is still code, and code doesn’t run itself.

Other visible text:
AIE
code
aws
AI Engineer
World’s Fair

[2:27](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=147s) [A Twain quotation above three points about AI-generated code, operating code, and history repeating itself.](https://assets.ai.engineer/sites/aiecode2025/sha256/8f615630d8ffcc3de7207f59586e6e4c31736f28fc73b5224a2f4db91dc68bca.webp)

The reports of my death have been greatly exaggerated. - Twain

1. Their AI is writing code based on massive codebases.
2. AI code is still code, and code doesn’t run itself.
3. History repeats itself, repeatedly.

Other visible text:
AIE
code
Microsoft
smol ai
AI Engineer
World’s Fair

### Vibes

A sunglasses-wearing toy coder introduces vibe coding: letting AI write and reason through the code, then checking whether it works without examining the code itself.

[3:12](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=192s) [A toy figure wearing sunglasses sits at a computer beneath the word Vibes.](https://assets.ai.engineer/sites/aiecode2025/sha256/9344bab5b2cffbdc7e9badb30ffd640cfd5ad1691f7c1ee549d13bce67a0e7a5.webp)

Vibes.

Interface text:
AIE
code
Microsoft
smol ai
AI Engineer
World’s Fair

### Code you can ship to production

The speaker describes production code as serving thousands of users and gigabytes of data with four nines. He argues that letting AI write code without examining it misses the nuances of production.

[3:46](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=226s) [A toy sailing ship floats among blue blocks beneath the words Code you can ship to production.](https://assets.ai.engineer/sites/aiecode2025/sha256/e38feded41a4c876ad4bbb51914be7be65fb8f65f14155333e03d53785f9cb56.webp)

Code you can ship
to production.

Interface text:
AIE
code
aws
AI Engineer
World’s Fair

[3:53](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=233s) [The words “Code you can ship to production” beside a toy sailing ship on blue blocks.](https://assets.ai.engineer/sites/aiecode2025/sha256/0759cbcc7272f93f976826a81fa1681391e599e983e20cf60c361ca08a78f156.webp)

Code you can ship
to production.

Interface text:
AIE
code
aws
AI Engineer
World’s Fair

### Code is an artifact of the work

Like an architect’s blueprints, code is an artifact of the job. Software developers also decide what their software should do and which packages to use.

[4:08](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=248s) [The words “Code isn’t the job. It’s an artifact of the work.” beside a model brain.](https://assets.ai.engineer/sites/aiecode2025/sha256/428f44b34cf3e1735c7a5214848280e2c3c1945dbb458801ba1ab693d62a2f4a.webp)

Code isn’t the job.
It’s an artifact of the work.

Interface text:
AIE
code
aws
AI Engineer World’s Fair

### The best code is no code at all

Atwood’s quote frames each new line of code as a continuing responsibility: someone must debug, understand and support it. The speaker argues that generating more code increases that burden.

[4:46](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=286s) [Slide quoting Atwood on the burden of debugging, understanding, and supporting code.](https://assets.ai.engineer/sites/aiecode2025/sha256/857f521ee66c88333da34c0e2c43490aa0330da705b339105b96e73a594ef277.webp)

The best code is no code at all. — Atwood

Every new line of code you willingly bring into the world is code that has to be debugged, code that has to be read and understood, code that has to be supported.

Interface text:
AIE
code
Microsoft
smol ai
AI Engineer
World’s Fair

### All code is trade-offs

Shipley argues that mastering programming means understanding the trade-offs in every decision. The speaker points to performance characteristics and choices among monolith, microservices, and event-driven architectures.

[5:19](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=319s) [Slide quoting Shipley on recognizing and understanding trade-offs in programming decisions.](https://assets.ai.engineer/sites/aiecode2025/sha256/ededcd0d9c3e43a4af705b6c35db9ba393703d9ffcc65444dcd20c5711086a02.webp)

All code is trade-offs.

The fundamental nature of coding is that our task, as programmers, is to recognize that every decision we make is a trade-off. To be a master programmer is to understand the nature of these trade-offs, and be conscious of them in everything we write. – Shipley

Interface text:
AIE
code
AI Engineer World's Fair
aws

[5:22](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=322s) [A quotation describes programming decisions as trade-offs.](https://assets.ai.engineer/sites/aiecode2025/sha256/669101016da0b32e0e04de26f7f1b7d637977313c8f7ac4f848683977e75441e.webp)

All code is trade-offs.

The fundamental nature of coding is that our task, as programmers, is to recognize that every decision we make is a trade-off. To be a master programmer is to understand the nature of these trade-offs, and be conscious of them in everything we write. – Shipley

Interface text:
AIE
code
AI Engineer World's Fair
Microsoft
smol ai

### Monolith, microservices, and event-driven architecture

A monolith packs blocks together, microservices separate them, and an event-driven system connects them through a hub. Building a flight booking system across these architectures involves thousands of individual decisions, the speaker says.

[5:30](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=330s) [Colored building blocks illustrate monolith, microservices, and event-driven architectures.](https://assets.ai.engineer/sites/aiecode2025/sha256/ea146cbaca5bbc26fbb3a213cbc08854644b83800880a10dbaa13f4f5e52dd87.webp)

Monolith
Microservices
Event-driven

Interface text:
AIE
code
Microsoft
smol ai
AI Engineer World’s Fair

### Fixing an airplane while it is in flight

A mechanic hanging from an airplane illustrates the challenge of changing running software safely. Version control, testing, type systems, and deployment strategies help manage that work; tests can catch changes that break other code.

[6:42](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=402s) [A toy mechanic hangs from an airplane holding a wrench.](https://assets.ai.engineer/sites/aiecode2025/sha256/c99b511d8b63214e6f96bb7dafe5b672ef2621f9a724d75c361a4c9fa9f1268a.webp)

Fixing an airplane while it is in flight.

Interface text:
AIE
code
Microsoft
smol ai
AI Engineer World’s Fair

[7:03](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=423s) [A toy airplane mechanic illustrates changing software safely, with version control, testing, type systems, and deployment strategies listed.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa0cd0045a6b0b896ac57d222ce62281f447f74e8a5b0878d1973192466ba094.webp)

Fixing an airplane while it is in flight.

How do we change software, safely?
1. Version control
2. Testing
3. Type systems
4. Deployment strategies

Interface text:
AIE
code
aws
AI Engineer World’s Fair

### AI writing code

A shrugging toy figure poses the question of AI writing code. The speaker assumes AI will be used while emphasizing that production still matters.

[7:35](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=455s) [A toy figure shrugs beneath the question, “So I guess AI is going to be writing code now?”](https://assets.ai.engineer/sites/aiecode2025/sha256/a80f93dfe1f62ab12cba484b6e88f51dd07506b91eb333bd628ceb54dbeb1519.webp)

So I guess AI is going to be writing code now?

Interface text:
AIE
Microsoft
smol ai
AI Engineer
World’s Fair

### The future is already here

The speaker sees professional software engineers among the last to adopt AI—a reversal from their enthusiasm for earlier shifts like Git and the cloud. He has hypotheses, but doesn't fully understand why.

[7:45](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=465s) [The quotation “The future is already here — it’s just not evenly distributed,” attributed on the slide to Gibson.](https://assets.ai.engineer/sites/aiecode2025/sha256/23ef9f484b55e13eee269a6e775955a3a4992eab7cf66d3685ff4623afd56ba6.webp)

The future is already here —
it's just not evenly distributed.
– Gibson

Interface text:
AIE
Microsoft
smol ai
AI Engineer
World’s Fair

### AI coding

Bricks, a robotic arm and a robot trace AI coding from early parlor tricks to a quality jump around a year ago and a push toward agents four weeks ago.

[8:27](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=507s) [Toy bricks, a robotic arm, and a robot illustrate AI coding at two years ago, twelve months ago, and four weeks ago.](https://assets.ai.engineer/sites/aiecode2025/sha256/b98fd486d8dbeb7a684b79567211fe4a243caac6c7b13dc6874ef6dbc83d8944.webp)

AI coding

2 years ago
12 months ago
4 weeks ago

Interface text:
AIE
Microsoft
smol ai
AI Engineer World's Fair

### Building software that’s easy for AI to write

Document the codebase’s direction so AI can follow it, and make developer environments reproducible and tests easy to run locally.

[9:13](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=553s) [Title slide introducing practices for building software that is easy for AI to write.](https://assets.ai.engineer/sites/aiecode2025/sha256/b394d4d597e02d789119b2a89ce4e94457f5ee4238fbe6fe5f84ec744de28fa6.webp)

Building software that’s easy for AI to write.

AIE
aws
AI Engineer
World’s Fair

### Building software that’s easy for AI to write

Make development environments reproducible and tests easy to run locally. Give AI clear boundaries for what it should build, alongside documented standards and practices.

[10:00](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=600s) [Four recommendations cover documented standards, reproducible environments, automatic testing and clear boundaries.](https://assets.ai.engineer/sites/aiecode2025/sha256/c503c3d162c451a8772d7e86d2e17e6ca673919996b3ca3356bf1b87337d7d60.webp)

Building software that’s easy for AI to write.

1. Have documented standards and practices.
2. Have simple, reproducible environments.
3. Have accessible and automatic testing infra.
4. Establish clear boundaries.

Interface text:
AIE
aws
AI Engineer
World’s Fair

### Building software that’s easy for AI to write

The speaker pairs documented standards, reproducible environments and automated tests with clear boundaries and defined tasks. Vague requests leave AI without enough direction.

[10:08](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=608s) [Five recommendations cover documented standards, reproducible environments, automatic testing, clear boundaries and clearly defined tasks.](https://assets.ai.engineer/sites/aiecode2025/sha256/185846f6fc9d7854bed2a7e53cb77ffabbd637686f06b238e33080edaf2b9b2f.webp)

Building software that’s easy for AI to write.

1. Have documented standards and practices.
2. Have simple, reproducible environments.
3. Have accessible and automatic testing infra.
4. Establish clear boundaries.
5. Clearly define the tasks and work.

Interface text:
AIE
Microsoft
smol ai
AI Engineer
World’s Fair

## Full transcript

[0:00](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=0s) [upbeat music] Thanks for coming, by the way, and, uh, for sticking around for a little while.

[0:19](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=19s) Um, if youuu aren't prepared, I hate to break it to you, but this time next year, half of us won't even be here anymore. That's basically if you listen to whatever the hype is about AI and AI coding.

[0:35](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=35s) You know, there's lots of fanfare. No disrespect to very intelligent people, um, that made these quotes, um, but I think they're probably wrong. Not because I don't think AI coding is going somewhere important, but probably because they haven't actually touched a production system in a very, very long time.

[0:52](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=52s) And so maybe generating code at thirty percent isn't really what they think it is. Because really, AI code is still code. One thing they don't really recognize in this space is that they're working in very, very, very large code bases that have basically every decision that's ever needed to be made about that code, about that architecture, about

[1:15](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=75s) that infrastructure, has already been made for them. So if I'm generating thirty percent of my code against, you know, let's say they're doing thousands of lines a day against millions of lines of new code or of, of existing code, there's not a lot of wiggle room for what that code can do or should do, right?

[1:32](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=92s) Like, it's just another button. If you've ever... No offense. Anyone from Meta here before I start ripping on Meta? [laughing]

[1:38](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=98s) No. No Meta. Great. I'll rip on Meta for a minute. If you've ever talked to a Meta engineer, they will talk about how they built a button in the ads platform for six months.

[1:46](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=106s) Like, that's, that's what they worked on for six months. That was their job. So, like, there's very little definition about, like, what this can-- needs to do that has wiggle room that AI really stru-- can, can influence.

[1:58](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=118s) Next, AI is still writing code. This is code we have written for fifty years. Same programming languages, nothing is different there. That code still needs to run in production somewhere.

[2:08](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=128s) And if you're not familiar with-- If you haven't run a large production system, even if you write a great line of code, in complex systems, things fail. Like, complex systems have emergent behavior that don't show up in just single lines of code.

[2:20](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=140s) And so we still have to run this stuff. And so who's gonna fix it? Who's gonna examine it? Who's gonna understand those nuances when, uh, when-- if you don't have software engineers?

[2:30](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=150s) So I think we're still gonna have software engineers. And then also, history repeats itself. This is not the first time I've been told that my career is over. Um, I'm getting in on the, on the years at this point in time.

[2:42](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=162s) But anyone, uh, around, like, the DevOps transformation from fifteen years ago, cloud, all the, all the sysadmins that I know that were racking boxes and, and booting kernels all got pay raises and all work on much more valuable things now.

[2:56](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=176s) They're much happier doing the work they're doing. So this isn't new. This is just a different level of abstraction, right? Like, tractors didn't get rid of farms. They just got rid of farmhands and horses.

[3:06](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=186s) Yeah. Like, yes, there will be change in an industry for certain, but tractors didn't get rid of farming. We still have to farm.

[3:14](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=194s) So vibes. Anyone here a vibe coder? It's okay if you are. No shame. Anyone-- And who here thinks call-- would call themselves a softwa-- a professional software engineer?

[3:25](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=205s) Great. So vibe coding, if you're not familiar with the term, I think everyone in this room is, but I'll do a quick summary, is basically letting the AI write all of the code and think through the code and not really examining the code at all.

[3:35](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=215s) Just, does it do what I need it to do? And then in that case, like, I just keep going and let it keep going, let it keep going. Don't, don't examine the code and edit it that way.

[3:45](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=225s) But that's not what we're talking about. We're ta-- I'm talking about how can we write code that's for production. And when I say production, I mean you. You have four nines.

[3:54](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=234s) Do you even know what four nines means? That's production code. You have thousands of users, gigabytes of data. We're talking software that runs the internet today. Vibes don't cut that because there's a lot of, like, nuances on what goes into code.

[4:09](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=249s) So first, let's clear up one misconception. Code is not the job, in the same way that blueprints are not the job of an architect. That's an artifact of being an architect.

[4:20](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=260s) The artifact of being a, a software developer is code. Sure, yeah, I have to output some code, but I make thousands of decisions about what my software is supposed to do, what kind of-- what I'm writing, what packages I'm bringing in.

[4:32](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=272s) So let's stop conflating generating code with the art and the f- the craft of doing software engineering. Those are different things. And so yes, LLMs are great at producing code, but are they-- is that the same as writing production software?

[4:47](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=287s) So a very smart person that, uh, started Stack Overflow, Jeff Atwood, rest in peace, Stack Overflow, um, said, "The best code is no code at all." And I think that's true because every line of code comes with, comes with a burden, right?

[5:00](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=300s) I have to maintain that code. I have to debug that code. So every line of code I generate, I have to be responsible for. And so we've been s- we've spent so much time thinking about, like, how much code can AI generate.

[5:11](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=311s) Who cares how much it can generate? The more it generates, the worse off I end up being, worse off the system ends up being. We want to put as little code as possible in there.

[5:20](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=320s) Because all code has trade-offs, right? Like, we recognize this package has this performance characteristic, this, this methodology has this. The best example I can give you, which I'll, I'll try to get through so you can all get back to snacks, is the difference between like a monolith, uh, and microservices architecture and an event-driven system.

[5:39](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=339s) How many decisions go into those kinds of architectures and the choices you have to make to do the same thing? Like, if you've ever-- If you built a flight booking system in all three of these, you have thousands of individual decisions that have to get made.

[5:53](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=353s) LLMs don't make decisions. They generate text. They generate patterns. And at some scale, at some point, there's no pattern for my software. Anyone here run a piece of production software that's kind of a bit of a snowflake?

[6:06](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=366s) Has a bunch of like idiosyncrasies in it that like, "Well, only Bob knows how that works, and only Jane can fix that. She wrote it six years ago and hasn't been on the project in..."

[6:16](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=376s) That is software. So at some scale, pattern matching doesn't work anymore because all of the nuances that go into all that software can't be pattern matched again. So when, when the software's going down, this is my like trauma of like carrying a pager.

[6:29](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=389s) When, when the software goes down at 2:00 in the morning, vibes aren't gonna fix the bug. Like, someone has to diagnose that problem.

[6:37](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=397s) So what is the work of software engineering? For me, that's changing software safely. That's been my job for 20 years, is how can I make changes to software, whether that's adding new functionality or, or changing existing code, and how do I do it safely so that software doesn't go down, so my users get their...

[6:53](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=413s) get the thing that they're getting, the, the widget ships, that, um, you know, data is secure. So how do we d- how have we done that so far in the industry?

[7:01](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=421s) We've done lots of different things to solve that problem. One, there's just my own knowledge. I have to like learn a shit ton about a code base before I can make changes safely.

[7:09](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=429s) Uh, we use version control to do that. We know testing. You know, like that's why we write tests, to catch if like, if I change this, did this thing break over here?

[7:17](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=437s) Um, we use type systems. We use deployment strategies. Can AI start to help us? Can context that an AI has about and can understand more of the code base help us?

[7:28](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=448s) Probably. You know, at Augment, we believe that context is like the most important part of all AI generation in code. So yeah, we think we can solve that problem.

[7:36](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=456s) That doesn't change that I still have to care about production. So let's just assume that we're writing code. Like, okay, the world is... the future is here. I have to use AI.

[7:49](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=469s) Um, the thing I find the most interesting about this space, I've been in it a long ti- in this, in dev tools for a long time, is professional software engineers are the last people I see adopting AI, and I've never seen that before.

[8:02](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=482s) Like, I've seen version control systems change. I was at GitHub in the early days, and so like massive jumps to Git, uh, the cloud transformation. Uh, pick your, pick your like innovation and developers are like, "Hell yes, give me that new thing."

[8:15](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=495s) I don't know why. Well, I have some hypotheses, but I don't unders- fully understand why software developers are like, "I'm not touching code. I'm not touching AI." It's like, it can't do what it's gonna do.

[8:26](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=506s) So I wanna talk about that for just like, just a couple minutes. So a few years ago, AI coding was mostly just parlor tricks. Like, it kind of worked, but really didn't do much.

[8:37](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=517s) Um, about a year ago, you know, years and change when Sonnet, uh, 3.5 came out, that's really when we saw a massive explosion, um, in, in AI coding because the quality substantially jumped.

[8:50](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=530s) Um, and then, uh, four weeks ago, if you weren't watching the news, literally every AI coding tool said like, "Agents are the future." Uh, we did. I'm very proud of that.

[9:00](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=540s) But like, now agents are everywhere, right? And so this transformation's happened, uh, very, very quickly. Uh, and so I wanna talk about how we can talk, do software engineering with this new future that we're seeing.

[9:14](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=554s) Um, so how do you build software that's easy for AI to write? A couple tips for you. Um, have some documented standards and practices, right? Like what do you use?

[9:25](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=565s) Every code base I know is in some sort of flux, right? So like, are we using this package or this package? Okay. Well, document that. Let the AI know that this is the next direction of your code base.

[9:35](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=575s) Um, have reproducible environments, right? Like, can you easily spin up a developer environment? Is your developer environment very bespoke and unique? Um, you wanna have it reproducible. We wanna have easy testing, right?

[9:48](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=588s) Like, can you run your test locally? Can... Is that fast? Uh, you know, that's pretty s- pretty standard stuff. Um, establish clear boundaries of what you're gonna do. You're never gonna give AI the idea of like, uh, extract this module using the Strangler pattern.

[10:03](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=603s) What does that even mean to an AI? Like, that's a whole... You have to give clear boundaries of what you're trying to build and how to get the AI to do it.

[10:10](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=610s) And lastly, like have clearly defined tasks and work, um, because AI is gonna be as-- You know, I wouldn't give any engineer on my team, whether they're senior staff, junior, just a vague task of like, "Could you make this button do something different?"

[10:24](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=624s) Like, I see too many engineers prompting in that way. And what's interesting about this to me is like, this just sounds like software engineering, right? Like if we don't...

[10:35](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=635s) If ideally, your software engineering stack looks, has these qualities, and if it doesn't, you're like, "Our productivity sucks because we have, you know, bespoke testing infrastructure. We have bespoke developer environments."

[10:46](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=646s) You have to give AI the same tools that engineers need because it's doing exactly the same job. It's writing code. I've never one-shotted a piece of code in my life personally.

[10:56](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=656s) Like, I've always made mistakes in the code I write. I run a test, it fails. I've got a linter, it fixes it. Whatever thing is. But we've had this expectation that AI can like write perfect code, and I don't know why.

[11:08](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=668s) And so when you're thinking about adopting AI as a software engineer, you have to make sure that your, your systems work like you would expect any other engineer to work, because that's how it's writing code.

[11:21](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=681s) The next thing is code review is by far the most important skill. Um, I think we've probably forgotten that skill as an industry. Um, we probably should have been interviewing for code review and not like, "Here's this esoteric LeetCode problem that you can solve," but like, "Can you read somebody else's code and comment on why it's

[11:37](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=697s) good or bad?" Um, I think this is gonna become far more important as agents are writing more and more code, and our code review tools today frankly suck. Like, I'm getting a list of changed files.

[11:48](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=708s) It's lexicographically sorted, so like great, file A changed. Let me read what changed in file A. What changed in file B? That's not a way to think about how software changes is in the order of the files.

[11:58](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=718s) And so I think we're gonna see a pretty big explosion on the way code review can happen, um, and that's the skill we need to be interviewing for, you need to be, um, again, brushing up on.

[12:09](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=729s) All right. I'm running quick on time, so I do wanna give you a couple highlights of, like, if you are so- one of those software engineers that's like, "I don't, you know, I'm not-- I don't trust AI, I need some tips," I wanna give-- leave you a few of these.

[12:21](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=741s) Um, feel free to take a screenshot. One, the most important thing I can imagine is AI talks like a human, but is actually a machine. I had this interaction with AI the other day where I was like, uh, you know, I was yelling at it because that's what you do at AI when it doesn't do what you

[12:36](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=756s) want it to do. And it's like, "Oh, I'm sorry, I just scanned that file, I didn't read it." I'm like, "The fuck does that even mean?" [laughing]

[12:43](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=763s) What-- How does a so- piece of software scan a file? It just, like, skimmed it? Like, that's not a thing in software. It's because LLMs are trained on all the data in the world, right?

[12:53](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=773s) There are thousands of emails that it has read that said like, "Oh sorry, I didn't read your document thoroughly. I actually just skimmed it." It's like, "Oh, that's what I should say when someone's yelling at me about not reading a file."

[13:02](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=782s) Right? And so we have, we have to distrust some of the things that LLMs are saying they're doing because it's not actually doing that. Don't forget, it's just generating text.

[13:11](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=791s) It's not always doing exactly what you're s- what it's outputting in that text. So keep that in mind when you're writ- when you're reading through LLM output. Um, let's see, what other tip, quick tips can I give you?

[13:25](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=805s) Sometimes code is just different. It's okay if the LLM outputs code differently than you would. I can't expect it to produce the code that I would exactly. In the same way that, like, the person that sits next to me also writes code a little differently than I do.

[13:38](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=818s) And so just accept that that's okay. And if you wanna force it to write code like you, you can spend that energy. But know the difference of is the code better or is it just different?

[13:48](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=828s) And so let go of some of that, like, this is how... You know, we-- this is why we have linters. This is why we have rule systems. This is why we have style guides, so we can stop arguing about, like, is that how you define a function or is this how you define a function?

[14:02](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=842s) Let some of that go if you can.

[14:04](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=844s) Um, write a rules file. You know, give it-- tell it what you want it to do. Have a file. I always start all of my projects with, like, here's the stack I'm using.

[14:14](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=854s) Here's the, like, guidelines I want you to use. And that's always ends up being part of the context I send with the LLM. Um, and then lastly, I like the define, create, refine loop.

[14:24](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=864s) A spec. Create a, a document of some kind. Have the LLM help you generate it. Like, I'm doing this thing. Write a markdown file that would, like, lay out the plan.

[14:33](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=873s) Here's my plan. Save that as a markdown file. Then use that again in your context. Have it create the thing. Have the agent run against that file. You know, you can make your edits to it and then refine it.

[14:44](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=884s) Then you're going in with, you know, code completions or whatever to, like, just tweak the things that you wanna tweak. And just do that loop over and over and over again.

[14:51](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=891s) Make a plan, have it create it, and then just make your, make your tweaks. And you'll get a lot more comfortable, h- one, how to prompt the LLM to get the code you want to do, as well as, um, that's just a very efficient way of coding.

[15:03](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=903s) And if you've letting go-- if you let go of that code has to be how I would write it, not just good functional code, then you can get a lot more productive in that way.

[15:13](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=913s) Um, thanks again. Uh, I blasted through that. I'm happy to talk. Augment is in the expo hall. I'll be around there to chat or over here. Um, I hope to see you here next year.

[15:22](https://www.youtube.com/watch?v=Dc3qOA9WOnE&t=922s) Hope we're all here next year. Uh, I think the jobs aren't going anywhere. Um, but happy coding. Really appreciate it. [outro music]
