Devin 2.0 and the Future of SWE

Ask GPT-3 to do something, and it might produce only a few acceptable words before you need to redirect it. GPT-3.5 and GPT-4 extend that useful stretch. Scott Wu opens with this practical measure of capability: how much work can an agent complete before a human must intervene or steer it? For software, the progression runs from individual lines and functions toward tasks representing hours of human work.

Wu cites roughly seven-month doubling for agent task length generally and seventy-day doubling for coding tasks. These are trend estimates, not a universal coding law or a Devin-specific performance measurement. The distinction between task length and runtime matters: related METR research measures the human duration of tasks an agent can complete at a specified success probability, primarily 50%, rather than elapsed agent runtime before intervention. Its broader estimate already covers software and reasoning tasks; the faster SWE-bench Verified trend depends on human-time estimates that assume repository familiarity.

Wu translates four to six annual doublings into a claimed 16–64× increase in coding work capacity per year over the preceding couple of years. The product change is easier to recognize than the multiplier: in his account, eighteen months earlier, tab completion was the only coding experience with product-market fit. The useful interaction was to supply the current file and predict the next line. Delegating an engineering task asks for something much larger.

Longer tasks change both the engineering bottleneck and the interface. A tool that predicts a line needs different support from one that investigates a bug, and an agent that can investigate a bug needs different support from one that takes responsibility for a backlog. Wu describes the period from Cognition’s late-2023 beginnings through June 2025 as a sequence of these capability tiers, each arriving on roughly the timescale of a few months.

The first broadly adopted use case Wu describes is repetitive migration: JavaScript to TypeScript, an Angular upgrade, or a Java version upgrade. A large repository needs the same procedure applied file by file. Upgrade documentation may spell out the steps, but applying them still requires judgment about the code in front of you. The task is explicit without necessarily being reducible to one deterministic transformation.

That combination made migrations a useful reliability target. An agent might occasionally solve something harder, but occasional success is not enough when a procedure must work repeatedly across, in Wu’s illustrative example, ten thousand files. Repetition also makes the work tedious for humans. Delegating it frees attention for more creative engineering while giving the agent a bounded procedure to execute.

The immediate capability requirement was instruction following. Cognition built Playbooks so a user could spell out a sequence of steps and have Devin follow it. That fits migration work particularly well; much of software engineering cannot be specified as a fixed sequence with no decisions left to make.

Repeated execution then exposed a second requirement: persistent feedback. If a human corrects the same omission on one migration, that correction should inform the next one. Wu describes Knowledge or memory as retaining instructions such as remembering a particular action whenever a certain situation appears. Playbooks supply the procedure; memory carries forward what people learn while applying it. He notes that Cognition subsequently rebuilt this memory system multiple times.

By late summer and early fall, the useful task expanded from repeated procedures to contained bugs and features. Wu’s example is a repository selector: put the currently selected repositories at the top of the dropdown instead of leaving checked items scattered throughout it. The desired behavior is clear, but the agent must find and change the relevant implementation.

For a TypeScript implementation of that behavior, a stable partition keeps the existing order within each group and leaves the repository records unchanged:

type Repository = {
  id: string;
  selected: boolean;
};

function selectedFirst(repositories: readonly Repository[]): Repository[] {
  return [
    ...repositories.filter(repo => repo.selected),
    ...repositories.filter(repo => !repo.selected),
  ];
}

const repositories: Repository[] = [
  { id: "api", selected: false },
  { id: "web", selected: true },
  { id: "docs", selected: false },
];

const ordered = selectedFirst(repositories);
// ordered IDs: web, api, docs
// repositories remains in its original order.

This makes the small change concrete: ordering changes, while identity and selection state are preserved.

Wu compares this scope to work you might assign an intern, typically involving one or two files. Yet a small diff still needs a functioning development environment. Devin must be able to set up the repository, run lint, and run CI checks to obtain basic evidence that its change works.

The infrastructure response was a prepared repository environment with snapshots that could be loaded again or rolled back. A clean remote VM supplied the place to execute the linter and CI checks. This broadened Devin’s value beyond a specialized migration tool: it could become a junior collaborator for ordinary, contained engineering work.

Later in the fall, broader bugs and requests introduced a different problem. A report might identify a symptom without identifying its cause. The agent must investigate, determine which files participate in the behavior, and coordinate a change that may span hundreds of lines. Reading the file containing the symptom is no longer enough.

Wu points to several ways to treat a repository as a connected system:

• Call hierarchies and language servers expose relationships between definitions and uses.
• Git commit history helps explain how files relate through earlier changes.
• Lint and cross-file references provide additional context for checking and coordinating edits.

The goal is consistency across the affected files. An individually plausible edit can still be wrong if the corresponding changes elsewhere are missing.

Once broader issues became practical to delegate, Slack became a natural interface: it was already where teams discussed bugs and requested work. A person could tag Devin in the discussion and ask it to investigate or build something. Wu connects this capability tier with Devin’s general availability, when a newly configured agent could take on a wider range of tasks.

The underlying requirement resembles onboarding a human engineer. Someone on their first day may understand the language but lack the repository-specific knowledge needed to choose a safe change. Devin likewise needed to build a representation of the codebase, use it to plan the task, identify all affected files, and then produce the diff. Delegation became useful as codebase understanding improved.

By spring, more difficult tasks exposed an ambiguity on the human side: the person assigning the work might not yet know exactly what should change. Improving architecture requires understanding its constraints. Speeding up a slow function requires profiling. Better error handling requires enumerating possible failures and deciding what each should do. A short prompt cannot fully specify decisions that have not been made.

DeepWiki made Devin’s codebase understanding useful to the human as well. Wu describes it as exposing a representation Devin already needed internally, so people could understand the repository and ask questions about it. Search supported the closely related activity of investigating a particular part of the codebase before committing to a change.

The workflow became iterative:

1. Ask questions about the relevant code and behavior.
2. Explore the repository with the agent until the intended change is clear.
3. Define the task, then send the agent off to implement it.

The Search slide shows that handoff explicitly: a codebase answer sits beside source code, followed by a prompt requesting implementation and unit tests. Exploration supplies the context for execution.

This alternation also shaped the Devin 2.0 IDE experience. The human could work closely with Devin at the points requiring judgment, then let it continue independently. Wu illustrates the workflow with 10–20% of a task closely monitored and the remaining 80–90% performed autonomously; these are example proportions, not measured supervision requirements.

Wu’s June tier is handing over many backlog tasks at once. This combines the earlier capabilities rather than replacing them. Linear and Jira provide the tickets, but integrating with a tracker is only the entry point: the agent still has to determine what each ticket means and whether the task is sufficiently specified. June marks the capability tier in this retrospective, not the first release of every supporting integration.

In an organization with multiple repositories, even choosing where to work becomes part of the assignment. Devin must identify the right repository, locate the relevant subsystem, and coordinate edits across files. Before doing that autonomously, it also needs to distinguish two states: enough confidence in the task to proceed, or enough uncertainty to ask a human for clarification or approval. Knowing when to stop and ask is part of useful autonomy.

Asynchronous work raises the standard for what the agent should deliver. If it is returning an entire pull request, particularly for a larger task, it needs to test its own changes rather than hand the human an unchecked diff. The Ticket to PR slide makes this concrete with a checklist labeled “Local testing complete” above an application screenshot.

The agent needs an iterative loop: run the code in its local execution environment, examine the result, and use that feedback to continue the work. But an environment capable of running commands solves only part of the problem. The agent must also know what to test and what to look for. Choosing the relevant behavior and interpreting the result require substantial task and codebase context. Testing becomes a reasoning problem as well as an execution capability.

The next question is whether an agent can take on an entire project rather than an individual task. Wu presents that as the next frontier, not an established capability demonstrated here. From a distance, every doubling may look like another increment on the same curve. In practice, each increase introduces different work.

Single-line completion could be treated as text prediction over the current file. Longer engineering work requires participation in the human workflow through Linear, Slack, and Jira; accepting feedback and steering; and helping people plan before implementation begins. It also requires execution tooling for autonomous tests, longer-term decisions, debugging generated outputs, and shell commands that gather useful feedback. These are distinct systems that must work together for a longer assignment to succeed.

Wu closes by forecasting another 16–64× increase in coding-agent work capacity over the following twelve months. That is his forecast, not a verified outcome. The engineering challenge behind it is concrete: each larger unit of delegated work requires new ways to establish intent, obtain feedback, and decide what to do next.