AI Engineer World's Fair 2025

Ship Production Software in Minutes, Not Months

Eno Reyes· Co-founder and CTO, Factory16:06

Read the talk

Ship Production Software in Minutes, Not Months

Delegating production work takes more than a coding agent: it requires shared context, collaborative planning, reliable validation, and clear responsibility for what agents do.

From a talk by Eno Reyes

Before you start: Familiarity with Git branches, pull requests, continuous integration, and basic incident response will help you follow the workflows.

What changes when agents do the work?

What should a software development environment look like when humans no longer write every individual line of code? Eno Reyes began working with LLMs around the arrival of GPT-3.5, when agentic systems were becoming plausible. At Factory, that possibility became a product direction: move from human-driven development toward delegating work to agents across the software lifecycle.

Adding AI to an editor gives developers useful assistance, but it leaves the original interaction pattern largely intact: a human operates tools designed for humans to write code. Reyes invokes the faster-horses saying attributed to Henry Ford to ask whether improving that familiar interface is enough. Delegating most development tasks requires an environment built around delegation, rather than an accumulation of AI features.

That environment needs four capabilities working together:

  • Task management: an intuitive interface for assigning work and managing agents.
  • Shared context: centralized information from engineering tools and data sources.
  • Reliable outputs: agents that consistently produce work worth reviewing and using.
  • Parallel infrastructure: capacity for thousands of agents to work concurrently.

These are the platform requirements Reyes draws from Factory’s two years of work with large organizations, not a claim that a coding model alone supplies them.

0:150:39
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:15 · section reference included

Production software still needs engineering judgment

Karpathy’s framing of English as a programming language captures the excitement around natural-language development. But being able to ask for software does not remove the engineering demands of a consequential system. Reyes illustrates the gap with a hypothetical legacy Java 7 application handling 5% of global bank transactions: it is an example of the stakes, not an identified customer or verified transaction share.

His analogy is climbing equipment. Agents make a difficult ascent more accessible, while building production software remains like scaling Mount Everest. Better equipment extends what an experienced climber can do; the climber still needs judgment about how to use it. Existing engineering expertise is what makes the new tools useful on difficult terrain.

Slide titled “The new way to build software,” with the quote “The hottest new programming language is English” attributed to Andrej Karpathy above line drawings of mountains.Open full source frame
“The new way to build software” pairs Karpathy’s English programming quote with a mountain illustration.
2:513:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:51 · section reference included

From a delegated task to a pull request

The first demonstration makes delegation concrete. Factory calls its agents Droids. Once given a task, the Droid begins by establishing where it is and what it can do; it does not immediately start editing files. The workflow proceeds in this order:

  1. Inspect the environment. Search the codebase, determine the Git branch, and check the machine’s available capabilities.
  2. Recover relevant history. Review recent code changes and retrieve memories of interactions with the user and across the organization.
  3. Propose and clarify. Return a plan, then ask the user to resolve uncertainties before implementation.
  4. Execute and validate. After the user supplies that information, write code, run pre-commit hooks and linting, and generate a pull request that Reyes describes as passing CI.

The clarification step is part of the engineering work. A useful agent should question an incomplete request and help improve it, rather than treat the first instruction as a complete specification. The endpoint shown here is a CI-passing pull request, not a production deployment.

3:433:55
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:43 · section reference included

Failure often begins with missing context

A successful demonstration raises the harder question: how do you make the outcome repeatable? Start by treating an LLM as an intelligence with only a slice of the real world available to it. What looks like a prompting problem may actually be a mismatch between what the developer knows and what the agent has been told.

Factory’s investigation of thousands of Droid-assisted sessions led Reyes to a recurring heuristic: an agent often fails because crucial context is missing, rather than because the underlying model cannot solve the problem. This is an observation from those sessions, not a universal diagnosis of agent failure. More capable models may make failures less frequent, but they cannot recover organizational facts that nobody has supplied.

The missing information may never have reached a repository or issue tracker. An LLM does not know what changed in the morning standup, what someone explained in an ad hoc meeting, or what the team drew on a whiteboard. Transcribed notes and uploaded whiteboard photos can make that information available. Native connections to engineering systems address another part of the problem, but access and use are separate requirements: the agent must be able to find the context and apply it to the task. Thinking of the system as somewhere between a coworker and a platform helps expose what it still needs to know.

4:515:06
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:51 · section reference included

Research what to build before asking for code

Using agents throughout development means involving them before implementation. In the next example, a Droid researches a new model release and plans its integration into an existing chat application. The task requires several kinds of evidence to meet in one place:

ContextContribution to the plan
Internet searchUp-to-date information about the model release
Existing codebaseHow the application currently works
Organizational memoryThe product’s goals
Recent design documentThe technical architecture

The difficult question is what to build within the existing product, not simply how to generate the integration code.

Planning becomes collaborative when the agent does the groundwork and research, then the developer explores possibilities with it. Asking for a finished design document in one step skips that interaction. The useful handoff is an informed starting point that people and agents can refine together.

The resulting plan can be exported to Notion, Confluence, or Jira through integrations. Reyes describes a platform experience without per-developer setup, while pointing out the friction of asking every developer to install servers, configure connections, and handle API keys. Model Context Protocol (MCP) standardizes integration interfaces; the no-setup experience is a platform claim, not a property guaranteed by the protocol. Removing that friction gets information moving, but the next improvement is organizational: standardize how the team expresses its reasoning.

6:306:42
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:30 · section reference included

Build the PRD from evidence and constraints

Factory’s planning for a cloud development environment feature shows what that reasoning can look like. The team started with roughly three months of feedback transcripts from enterprise and individual users. Reyes says Factory transcribes every interaction and meeting. They combined those notes with a Droid’s access to the architecture and an engineer’s ad hoc meeting notes captured with Granola.

Instead of asking the Knowledge Droid to plan the feature immediately, they asked it to investigate two things:

  • Customer patterns: Which patterns in the feedback correspond to the team’s assumptions?
  • Technical constraints: What limitations in the current architecture could inform a better design?

Those questions produced four or five intermediate results. The team then used the resulting documents to iterate toward a final product requirements document (PRD). The intermediate analysis matters: it gives the team evidence and constraints to examine before committing to the feature’s shape.

“Planning with AI” slide with three columns covering business context, architecture and meeting notes; patterns, constraints and iterative refinement; and templates, language polishing and visual enhancements.Open full source frame
Planning with AI: gathering context, collaborative discovery, and final deliverables.
8:198:29
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:19 · section reference included

Turn the plan into coordinated work

Once the PRD exists, a Droid with Linear or Jira tools can translate it into a roadmap by creating and modifying tickets and epics. Reyes sketches eight tickets assigned across eight code Droids, with dependencies determining which work can proceed in parallel. The important operation is not merely splitting a document into tasks: it is preserving the relationships that govern when those tasks can run. The developer moves from executing each item to orchestrating the systems doing the work.

This makes ordinary process artifacts more valuable. PRDs, engineering design documents, root-cause analysis templates, quarterly engineering and product roadmaps, and meeting transcripts form a knowledge base for agents. They describe how the team thinks and works. Recording why a decision was made, alongside the decision itself, gives future developers and agents the context needed to follow the team’s practices. Documentation becomes a continuing conversation with both audiences.

9:269:40
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:26 · section reference included

Connect incident evidence before proposing a mitigation

The same context problem appears in site reliability engineering. Reyes explicitly rejects the idea that all incident response and root-cause analysis can already be automated. The demonstration has a narrower scope: a Droid takes a Sentry incident and turns it into a root-cause analysis (RCA) and mitigation plan. Producing that plan is distinct from executing a fix or resolving the incident.

Traditional incident investigation is a puzzle with pieces spread across systems: logs in one place, metrics elsewhere, historical context in documents, and some knowledge only in people’s heads. With appropriate tools, a Droid can gather relevant system logs, past incidents, runbooks in Notion or Confluence, and Slack discussions. This retrieval work supplies the evidence needed to interpret the alert.

Reyes says that access to these tools and sources can reduce incident search effort from hours to minutes. He does not provide a benchmark methodology. His next step—an agent offering an explanation and proposed fix as soon as an incident occurs—is an aspiration for immediate assistance, not a measured guarantee of zero response time. User and organization memory could then help identify recurring issues and familiar response patterns, allowing the system to propose process improvements beyond the current mitigation.

10:4910:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:49 · section reference included

Make incident response a learning cycle

The RCA is an input to subsequent work. An agent can generate runbooks for newly recognized patterns, update response workflows, and capture team knowledge for automatic sharing. Connecting these steps makes incident response part of a learning cycle rather than a sequence of isolated investigations.

Reyes reports seeing unnamed teams cut incident response time in half because context is immediately available. No sample size or measurement method accompanies that report. He also describes repeated incidents prompting suggestions for underlying fixes: by the third occurrence, a Droid might flag that the recurring problem deserves a more durable solution. For a new engineer, the same organizational memory provides a way to ask how the team handles a particular situation without first finding the person who remembers it.

As operational history accumulates, patterns across incidents can inform future action. Reyes frames this as a shift from reactive toward predictive operations: each incident becomes an opportunity to improve the system’s reliability. The agent’s role is to amplify the engineer’s ability to connect evidence, recognize repetition, and improve the response process.

12:3212:47
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:32 · section reference included

The work moves outward, and responsibility remains

The developer’s role expands beyond the inner loop of writing individual lines in an IDE. Reyes describes developers managing agents that work concurrently, organizing the surrounding systems, and building reusable patterns for how development happens. Working effectively with agents remains an active discipline; expecting the AI to do everything is not the same as directing it well. In that broader role, he places clear thinking and effective communication with humans and AI at the center of engineering practice.

The closing invitation was specific to the recording: attendees could scan a QR code to create an account with twenty million free tokens, and Reyes recommended a laptop because the mobile experience was not yet optimized. These were talk-time access details, not a statement of today’s offer.

Enterprise adoption adds questions that a successful code demo cannot answer. Where are the audit logs? Who owns the work? Who is responsible if an agent performs a destructive action? Reyes raises recursive deletion as a hypothetical and says Factory’s Droids do not do that; the example introduces accountability questions rather than demonstrating a safeguard.

Security, ownership, auditability, and legal indemnification all need explicit consideration. Reyes warns against unrestricted “YOLO mode” inside an enterprise and closes by inviting people to inspect the platform’s controls and send questions by email. Delegation changes who performs an action, but organizations still have to establish who authorizes it, how it can be audited, and who bears responsibility for its consequences.

13:5114:11
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:51 · section reference included

Resources

From the talk

Updates since the talk

Transcript download
Full transcript