Vibes won't cut it

If AI generates 30% of the code in a mature codebase, how much of the engineering work has it actually done? Chris Kelly opens with this challenge to forecasts that AI coding will soon make engineers unnecessary. The percentage is a claim he critiques, not a measured productivity gain.

Consider thousands of newly generated lines surrounded by millions of existing lines. The surrounding system already embodies decisions about architecture, infrastructure, and expected behavior. Those decisions constrain what the new code can and should do. Generating an addition inside those boundaries is different from establishing the boundaries themselves.

Kelly illustrates that constraint with a deliberately pointed anecdote: a Meta engineer spending six months building a button in an advertising platform. This is his characterization, not a company performance measurement. The button may sit inside an enormous system, but much of its design space has already been settled. A model completing that narrowly defined work inherits the engineering that made it narrowly defined.

AI produces code in familiar programming languages, and that code still has to run somewhere. A correct line does not establish the correctness of a complex system: interactions create emergent behavior that is invisible when examining individual lines. Someone must operate the system, investigate failures, and understand the nuances well enough to repair it. Changing who writes the code does not remove those responsibilities.

Kelly has heard predictions of engineering’s disappearance before. During the cloud and DevOps transition, the sysadmins he knew moved from racking machines and booting kernels to work he describes as more valuable, better paid, and more satisfying. His farming analogy makes the qualification explicit: tractors displaced particular forms of labor, but they did not eliminate farming. A new abstraction can substantially change an industry without eliminating its underlying work.

In Kelly’s definition, vibe coding means letting AI write and reason through the code without examining it yourself. The feedback loop is behavioral: request something, see whether it does what you want, and keep going. The code itself does not receive the scrutiny that would let you explain or deliberately change its implementation.

Kelly frames production as software with four-nines availability expectations, thousands of users, and gigabytes of data. These are examples of operational stakes, not universal thresholds for calling software production. At those stakes, observing that a feature appears to work is insufficient; the implementation carries obligations beyond the immediate interaction.

Code is the artifact of software engineering, much as a blueprint is an artifact of architecture. The work includes deciding what the software should do, what to build, and which packages to bring into the system. Producing the artifact is necessary, but counting its lines misses the decisions that give it meaning. That is why generating code and engineering production software cannot be treated as interchangeable accomplishments.

Kelly invokes Jeff Atwood’s “The Best Code is No Code At All”: every line creates something to maintain, debug, and take responsibility for. Making code cheaper to generate does not make those obligations disappear. Optimizing for the amount generated can therefore move in the wrong direction; the useful objective is to accomplish the required behavior with as little code to own as possible.

The obligations begin with trade-offs. A package has performance characteristics; an implementation approach brings its own constraints. Take the same flight-booking system and build it as a monolith, as microservices, or as an event-driven system. The user-facing goal can remain the same while the architecture demands many different decisions. Naming the desired application does not resolve those choices.

Kelly’s concern is that generating familiar patterns does not settle the judgment required by a particular production system. His example is the codebase where only Bob knows how one part works and only Jane can fix another, although she left the project years ago. Such idiosyncrasies accumulate into system-specific knowledge. When an outage arrives at two in the morning, someone still has to diagnose what this system is doing. His critique is about the limits of relying on patterns in that setting, not a demonstrated theorem about everything an LLM can or cannot do.

Software engineering means changing software safely. Adding functionality or modifying existing code must preserve the outcomes users depend on: the service remains available, the widget ships, and data stays secure. That definition shifts attention from how quickly code appears to what happens when the change enters the system.

The industry already has several ways to make change safer:

• Codebase knowledge: understand the system before modifying it.
• Version control: keep changes under control and inspectable.
• Tests: detect whether changing one part breaks behavior elsewhere.
• Type systems and deployment strategies: provide additional safeguards around implementation and release.

AI can contribute to this work if its context helps it understand more of the codebase. Kelly identifies that context as central to Augment’s approach. But assistance with understanding or generating a change does not transfer responsibility for production away from the engineers operating it.

Kelly finds professional engineers unusually hesitant to adopt AI compared with earlier developer tools. From his experience at GitHub during its early growth, engineers eagerly adopted Git; he recalls similar enthusiasm for cloud infrastructure. Their reluctance around AI is his observation rather than survey evidence, and he does not offer a settled explanation for it.

The tools themselves have changed quickly in his account. Early AI coding often amounted to limited tricks. He identifies Claude 3.5 Sonnet as a substantial improvement in coding quality, followed by a wave of agent announcements, including Augment’s agent. These are milestones relative to the recording, not a claim about today’s model rankings. The practical question is how to bring that improving capability into an engineering workflow.

Start with the conditions under which AI will write code. Kelly recommends the same clarity and working infrastructure that a human engineer needs:

• Documented standards and direction: state which packages and practices to use. When a codebase is transitioning between alternatives, document where it is going so that existing examples do not become the only guide.
• Reproducible environments: make it straightforward to start a development environment instead of depending on a bespoke setup.
• Accessible testing: make local tests easy to run and fast enough to support iteration.
• Clear boundaries: identify the scope of the intended change. Asking for a module extraction using the Strangler pattern names an approach but still leaves substantial decisions unspecified.
• Defined tasks: explain what behavior should change. A vague request to make a button do something different would be inadequate for a senior engineer or a junior engineer, and it is inadequate for an AI agent.

These requirements turn an open-ended request into work that can be understood and checked.

None of this requires a separate discipline invented for AI. Bespoke test infrastructure and unique developer environments already hurt human productivity. Preparing a codebase for AI exposes the same weaknesses: unclear direction, expensive setup, and tasks whose acceptance criteria exist only in someone’s head.

A human engineer does not reliably produce flawless code on the first attempt. Kelly describes his own normal process: write code, run a test, discover a mistake, and use tools such as linters to correct problems. Expecting an AI to skip that process sets a standard that engineering practice itself does not meet.

Give AI access to the same feedback that makes human work reliable. The loop is to generate a change, run checks, correct failures, and repeat. A useful AI coding setup must support that iteration rather than treating the first response as a finished implementation.

As agents produce more code, code review becomes a central engineering skill: can you read someone else’s implementation and explain why it is good or bad? Kelly argues that hiring should test this ability rather than concentrating on esoteric coding puzzles. Producing an answer and evaluating an unfamiliar answer are different skills, and agent-assisted work increases the importance of the latter.

Current review interfaces can make that evaluation harder. A lexicographically sorted list asks the reviewer to inspect file A, then file B, even though alphabetical order says nothing about how the software’s behavior changes. Understanding a change requires connecting its parts, not merely traversing filenames. Kelly anticipates better review tools for this purpose; he does not demonstrate one here. Meanwhile, the immediate skill to develop is reading changes for their meaning.

For engineers still unsure how much to trust AI, Kelly starts with its conversational surface. He recounts challenging an assistant about a file and receiving the explanation, “I just scanned that file, I didn't read it.” The apology sounds like something a human colleague might say, but it does not explain which operation the software performed or what information it actually received.

Kelly interprets the response as imitation of familiar human apologies: language about skimming a document offers a plausible conversational answer when someone complains that it was not read carefully. That account is his interpretation, not evidence about the assistant’s specific training examples. The operational lesson is firmer: generated descriptions of actions are claims, not proof that those actions occurred. Read an assistant’s explanation with that distinction in mind.

Generated code may look different from the code you would have written. So does a colleague’s code. Before spending effort making it resemble your own, ask whether the difference makes it better or worse, or merely different. You can direct a model toward your preferences, but that work has a cost and should have a reason.

Linters, rule systems, and style guides exist to settle recurring conventions without repeatedly arguing over them. Put shared expectations there, then reserve review attention for differences that matter. Accepting an unfamiliar but sound implementation is part of making AI assistance useful.

Make your expectations persistent. Kelly starts projects with a rules file describing the stack and the guidelines he wants followed, then includes it in the model’s context. This is generic project guidance, not a requirement for a particular tool-specific filename or rules feature. It gives subsequent requests a shared foundation instead of requiring every prompt to restate the project’s conventions.

His preferred workflow is a repeated define, create, refine loop:

1. Define: ask the model to help write a specification or plan describing the intended work. Save it as a Markdown file so that the plan becomes an explicit artifact.
2. Create: supply the saved plan as context and have the agent implement against it. Make edits to the plan as needed so that the written direction reflects what you want built.
3. Refine: inspect and adjust the implementation, using direct edits or code completions for targeted changes. Repeat the loop as the work develops.

The plan carries intent into implementation, while refinement makes room for the corrections that normal engineering requires.

Repeating this process helps you learn how to prompt for the behavior you need and where to intervene afterward. Kelly describes it as an efficient way to work, without offering a quantitative benchmark. Its value also depends on the earlier distinction between quality and preference: the target is good, functional code, not an exact reproduction of how you personally would have written it.