AI Engineer World's Fair 2025
Devin 2.0 and the Future of SWE
Read the talk
Devin 2.0 and the Changing Bottlenecks of Software Engineering
As coding agents take on longer tasks, the hard problem shifts from following instructions to understanding repositories, clarifying intent, and testing their own work.
From a talk by Scott Wu
Before you start: Familiarity with repositories, pull requests, linting, and continuous integration will help you follow the engineering examples.
How much work can an agent do before you intervene?
Ask GPT-3 to do something, and it might produce only a few acceptable words before you need to redirect it. GPT-3.5 and GPT-4 extend that useful stretch. Scott Wu opens with this practical measure of capability: how much work can an agent complete before a human must intervene or steer it? For software, the progression runs from individual lines and functions toward tasks representing hours of human work.
Wu cites roughly seven-month doubling for agent task length generally and seventy-day doubling for coding tasks. These are trend estimates, not a universal coding law or a Devin-specific performance measurement. The distinction between task length and runtime matters: related METR research measures the human duration of tasks an agent can complete at a specified success probability, primarily 50%, rather than elapsed agent runtime before intervention. Its broader estimate already covers software and reasoning tasks; the faster SWE-bench Verified trend depends on human-time estimates that assume repository familiarity.
Wu translates four to six annual doublings into a claimed 16–64× increase in coding work capacity per year over the preceding couple of years. The product change is easier to recognize than the multiplier: in his account, eighteen months earlier, tab completion was the only coding experience with product-market fit. The useful interaction was to supply the current file and predict the next line. Delegating an engineering task asks for something much larger.
Longer tasks change both the engineering bottleneck and the interface. A tool that predicts a line needs different support from one that investigates a bug, and an agent that can investigate a bug needs different support from one that takes responsibility for a backlog. Wu describes the period from Cognition’s late-2023 beginnings through June 2025 as a sequence of these capability tiers, each arriving on roughly the timescale of a few months.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Repetitive migrations: follow the procedure, retain the corrections
The first broadly adopted use case Wu describes is repetitive migration: JavaScript to TypeScript, an Angular upgrade, or a Java version upgrade. A large repository needs the same procedure applied file by file. Upgrade documentation may spell out the steps, but applying them still requires judgment about the code in front of you. The task is explicit without necessarily being reducible to one deterministic transformation.
That combination made migrations a useful reliability target. An agent might occasionally solve something harder, but occasional success is not enough when a procedure must work repeatedly across, in Wu’s illustrative example, ten thousand files. Repetition also makes the work tedious for humans. Delegating it frees attention for more creative engineering while giving the agent a bounded procedure to execute.
The immediate capability requirement was instruction following. Cognition built Playbooks so a user could spell out a sequence of steps and have Devin follow it. That fits migration work particularly well; much of software engineering cannot be specified as a fixed sequence with no decisions left to make.
Repeated execution then exposed a second requirement: persistent feedback. If a human corrects the same omission on one migration, that correction should inform the next one. Wu describes Knowledge or memory as retaining instructions such as remembering a particular action whenever a certain situation appears. Playbooks supply the procedure; memory carries forward what people learn while applying it. He notes that Cognition subsequently rebuilt this memory system multiple times.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Small fixes still need a working repository
By late summer and early fall, the useful task expanded from repeated procedures to contained bugs and features. Wu’s example is a repository selector: put the currently selected repositories at the top of the dropdown instead of leaving checked items scattered throughout it. The desired behavior is clear, but the agent must find and change the relevant implementation.
For a TypeScript implementation of that behavior, a stable partition keeps the existing order within each group and leaves the repository records unchanged:
typescript
type Repository = {
id: string;
selected: boolean;
};
function selectedFirst(repositories: readonly Repository[]): Repository[] {
return [
...repositories.filter(repo => repo.selected),
...repositories.filter(repo => !repo.selected),
];
}
const repositories: Repository[] = [
{ id: "api", selected: false },
{ id: "web", selected: true },
{ id: "docs", selected: false },
];
const ordered = selectedFirst(repositories);
// ordered IDs: web, api, docs
// repositories remains in its original order.
This makes the small change concrete: ordering changes, while identity and selection state are preserved.
Wu compares this scope to work you might assign an intern, typically involving one or two files. Yet a small diff still needs a functioning development environment. Devin must be able to set up the repository, run lint, and run CI checks to obtain basic evidence that its change works.
The infrastructure response was a prepared repository environment with snapshots that could be loaded again or rolled back. A clean remote VM supplied the place to execute the linter and CI checks. This broadened Devin’s value beyond a specialized migration tool: it could become a junior collaborator for ordinary, contained engineering work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Broader bugs require relationships, not just source text
Later in the fall, broader bugs and requests introduced a different problem. A report might identify a symptom without identifying its cause. The agent must investigate, determine which files participate in the behavior, and coordinate a change that may span hundreds of lines. Reading the file containing the symptom is no longer enough.
Wu points to several ways to treat a repository as a connected system:
- Call hierarchies and language servers expose relationships between definitions and uses.
- Git commit history helps explain how files relate through earlier changes.
- Lint and cross-file references provide additional context for checking and coordinating edits.
The goal is consistency across the affected files. An individually plausible edit can still be wrong if the corresponding changes elsewhere are missing.
Once broader issues became practical to delegate, Slack became a natural interface: it was already where teams discussed bugs and requested work. A person could tag Devin in the discussion and ask it to investigate or build something. Wu connects this capability tier with Devin’s general availability, when a newly configured agent could take on a wider range of tasks.
The underlying requirement resembles onboarding a human engineer. Someone on their first day may understand the language but lack the repository-specific knowledge needed to choose a safe change. Devin likewise needed to build a representation of the codebase, use it to plan the task, identify all affected files, and then produce the diff. Delegation became useful as codebase understanding improved.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Explore together before handing off the task
By spring, more difficult tasks exposed an ambiguity on the human side: the person assigning the work might not yet know exactly what should change. Improving architecture requires understanding its constraints. Speeding up a slow function requires profiling. Better error handling requires enumerating possible failures and deciding what each should do. A short prompt cannot fully specify decisions that have not been made.
DeepWiki made Devin’s codebase understanding useful to the human as well. Wu describes it as exposing a representation Devin already needed internally, so people could understand the repository and ask questions about it. Search supported the closely related activity of investigating a particular part of the codebase before committing to a change.
The workflow became iterative:
- Ask questions about the relevant code and behavior.
- Explore the repository with the agent until the intended change is clear.
- Define the task, then send the agent off to implement it.
The Search slide shows that handoff explicitly: a codebase answer sits beside source code, followed by a prompt requesting implementation and unit tests. Exploration supplies the context for execution.
This alternation also shaped the Devin 2.0 IDE experience. The human could work closely with Devin at the points requiring judgment, then let it continue independently. Wu illustrates the workflow with 10–20% of a task closely monitored and the remaining 80–90% performed autonomously; these are example proportions, not measured supervision requirements.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A backlog adds routing and escalation decisions
Wu’s June tier is handing over many backlog tasks at once. This combines the earlier capabilities rather than replacing them. Linear and Jira provide the tickets, but integrating with a tracker is only the entry point: the agent still has to determine what each ticket means and whether the task is sufficiently specified. June marks the capability tier in this retrospective, not the first release of every supporting integration.
In an organization with multiple repositories, even choosing where to work becomes part of the assignment. Devin must identify the right repository, locate the relevant subsystem, and coordinate edits across files. Before doing that autonomously, it also needs to distinguish two states: enough confidence in the task to proceed, or enough uncertainty to ask a human for clarification or approval. Knowing when to stop and ask is part of useful autonomy.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Complete pull requests need an iterative testing loop
Asynchronous work raises the standard for what the agent should deliver. If it is returning an entire pull request, particularly for a larger task, it needs to test its own changes rather than hand the human an unchecked diff. The Ticket to PR slide makes this concrete with a checklist labeled “Local testing complete” above an application screenshot.
The agent needs an iterative loop: run the code in its local execution environment, examine the result, and use that feedback to continue the work. But an environment capable of running commands solves only part of the problem. The agent must also know what to test and what to look for. Choosing the relevant behavior and interpreting the result require substantial task and codebase context. Testing becomes a reasoning problem as well as an execution capability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The next doubling asks a different engineering question
The next question is whether an agent can take on an entire project rather than an individual task. Wu presents that as the next frontier, not an established capability demonstrated here. From a distance, every doubling may look like another increment on the same curve. In practice, each increase introduces different work.
Single-line completion could be treated as text prediction over the current file. Longer engineering work requires participation in the human workflow through Linear, Slack, and Jira; accepting feedback and steering; and helping people plan before implementation begins. It also requires execution tooling for autonomous tests, longer-term decisions, debugging generated outputs, and shell commands that gather useful feedback. These are distinct systems that must work together for a longer assignment to succeed.
Wu closes by forecasting another 16–64× increase in coding-agent work capacity over the following twelve months. That is his forecast, not a verified outcome. The engineering challenge behind it is concrete: each larger unit of delegated work requires new ways to establish intent, obtain feedback, and decide what to do next.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The December 2024 release describes Slack delegation, first-draft pull requests, and practical task-scoping guidance.
Introduces interactive planning, parallel cloud IDE sessions, Devin Search, and repository wikis.
Explains the public repository documentation and codebase question-answering service built from Devin Wiki and Search.
Current guidance for reusable task instructions, procedures, success criteria, and iterative improvement.
Further reading
Explains human-duration task horizons, success thresholds, exponential trends, and limits of extrapolation.
The NeurIPS 2025 paper details the evaluation methodology and the approximately 70-day SWE-bench Verified trend.
- Devin's 2024 release historyDocumentation
Dated records of Playbooks, Knowledge, snapshots, machine access, and Slack workflow changes.
- Devin's 2025 release historyDocumentation
Tracks interactive planning, DeepWiki, Linear and Jira integration, and confidence-based approval behavior.
Read the complete timestamped transcript
- 0:00
[upbeat music] Yeah. Well, thank you guys so much for having me.
- 0:16
It's exciting to be back. It's, uh, I, I was last here at AI Engineer one year ago. Um, and it's kinda crazy. I, I've always been-- I, I've been telling Swix that we need to have these conferences way more often if it's gonna be about AI software engineering.
- 0:28
Probably should be, like, every two months or something like that [chuckles] with the pace that everything's done. But, but, but gonna be fun to, to talk a little bit about, um, you know, what we've seen in the space and, and what we've learned over the last twelve or eighteen months, uh, building Devin over this time.
- 0:43
And I wanna start this off with, um, Moore's law for AI agents. And so you can kinda think of the, the, the capability or the capacity of an AI by how much work it can do uninter-uninterrupted until you have to come in and step in and intervene or steer it or whatever it is, right?
- 1:02
And, um, you know, in GPT-3, for example, it's if you were to go and ask GPT-3 to do something, you know, it could probably get through a few words or so, and then it'll say something where it's like, "Okay, you know, this is probably not the right thing to say." [chuckles]
- 1:14
Um, and GPT-3.5 was better, and GPT-4 was better, right? Um, and, and so people talk about these lengths of tasks, and what you see in general is that that doubling time is about every seven months, which already is pretty crazy, actually.
- 1:27
But in code, it's actually even faster. It's every seventy days, which is two or three months. And so, you know, if you look at various software engineering tasks that start from the simplest single functions or single lines, and you go all the way to, you know, we're doing tasks now that take hours of humans' time, and, and
- 1:46
an AI agent is able to just do all of that, right? Um, and if you think about doubling every seventy days, I mean, basically, you know, every two to three months means you get four to six doublings every year.
- 1:56
Um, which means [chuckles] that the amount of work that an AI agent can do in code goes something between sixteen and sixty-four x in a year, every year, at least for the last couple years that we've seen.
- 2:08
Um, and it's kinda crazy to think about, but, but it-- that sounds about right, actually, for, for what we've seen. You know, eighteen months ago, I would say the only really-- the only product experience that had PMF in code was just tab completion, right?
- 2:21
It was just, like, here's what I have so far. Predict the next line for me. That was kind of all you really could do, um, in, in a way that really worked.
- 2:29
And we've gone from that, obviously, to full AI engineer that goes and just do-does, does all these tasks for you, right, and implements a ton of these things. And people ask all the time, what is the, um, you know, what, what, what is the, the future interface or what is the right way to do this or what
- 2:46
are the most important capabilities to solve for? And I think funnily enough, the answer to all these questions actually is it changes every two or three months. Like, every time you get to the next tier, the, the, the bottleneck that you're running into or the most important capability or the right way you should be interfacing with it,
- 3:02
like, all these actually change at, at each point. And so I wanted to talk a bit about some of the, the, those tiers for us over the last year or so.
- 3:13
Um, and, you know, over the course of that time, obviously, you know, when, when we got started, um, in the end of twenty twenty-three, obviously, agents were not even a concept.
- 3:20
Um, and now everyone has, you know, everyone's talking about coding agents. People are doing more and more and more. Uh, and, and it's very cool to see. Um, and, and each of these has kind of been almost a, a discrete tier for us.
- 3:31
Um, and so right, right around a year ago when we were doing the last AI Engineer talk actually, um, the, the biggest use case that we really saw that, that was getting broad adoption was what I'll kind of call these repetitive migrations.
- 3:43
And so I'm talking, like, JavaScript to TypeScript or, like, upgrading your Angular version from this one to that one or going from this Java version to that Java version or something like that.
- 3:54
Um, and those, those kinds of tasks in particular, what you typically see is you are--
- 4:01
you, you have some massive code base that you wanna apply this whole migration for. You have to go file by file and do every single one, and usually, the set of steps is pretty clear, right?
- 4:10
If you go to the Angular we-website or something like that, it'll tell you, "All right. Here's what you have to do, this, this, this, this, this." And, um, you wanna go and execute each of these steps.
- 4:19
It's not so routine that there, you know, there's no classical deterministic program that solves that. But there's kind of a clear set of steps, and if you can follow those steps very well, then you can do the task.
- 4:29
And, you know, this was the thing for us because that was all you could really trust agents to do at the time. You know? You could do harder things once in a while, and you could do some really cool stuff occasionally.
- 4:40
But as far as something that was consistent enough that you could do it over and over and over, um, these kinds of, like, repetitive migrations that you would be doing for, you know, ten thousand files were, you know, in, in, in many ways, the, the, the easiest thing.
- 4:52
Which was cool actually because it, it, it was also kind of the, the most annoying thing for humans to do. [chuckles] And I think that's generally been the trend where, um, AI has always done these more boilerplate tasks and the more tedious stuff, the more repetitive stuff, and we get to do the, the, the more fun, creative stuff.
- 5:08
Um, and obviously, as time has gone on, it's, it's taken on more and more of that boilerplate. But for a problem like this one, a lot of what you need to do is you need Devin to be able to go and execute a set of steps super reliably.
- 5:22
And so a lot of this was, you know, I would say the big capabilities problems to solve was mostly instruction following. And so we built this system called Playbooks where basically you could just outline a very clear set of steps, have it follow each of those step-by-step, and then do exactly what's said.
- 5:37
Now, if you think about it, obviously, a lot of software engineering does not fall under the category of literally just follow ten steps step-by-step and do exactly what it said.
- 5:44
But migration does. And it allowed us to go and actually do these, and, and this was kind of, I would say, the first big use case of Devin that really, um, that really came up.
- 5:53
I think one of the other big systems that got built around that time, which we've since rebuilt many times, is knowledge or memory, right? Which is, you know, if you're doing the same task over and over and over again, then often the human will have feedback on, "Hey, by the way, you have to remember to do X
- 6:08
thing," or, "You have to, you know-- you, you need to do Y thing every time."
- 6:13
When you see this, right? Um, and so basically an ability to, to just maintain and understand the learnings from that and use that to improve the agent in every future one.
- 6:22
And those were kind of the, the big problems of the time. You know, and that was summer of last year. And around end of summer or fall or so, you know, I think the, the, the kind of big thing that started coming up was as these systems got more and more capable, instead of just doing the most
- 6:38
routine migrations, you could do, you know, these more s- still pretty isolated, but, but, but, but a bit broader of these general kind of bugs or features where you can actually just tell it what you want to do and have you- have it do it, right?
- 6:50
And so for example, "Hey Devin, in this, uh, repo select dropdown, can you please just list the currently selected ones at the top? Like, having the checkboxes throughout is, it's just doesn't really..."
- 7:00
And, and Devin will just go and do that, right? And so if you think about it, it's,
- 7:04
y- you know, it's, it's, it's something like the kind of a level of task that you would give an intern.
- 7:09
And there are a few particular things that you have to solve for, um, with this. First of all, usually these, these, these changes are pretty isolated and pretty contained, and so it's one, maybe two files that you really have to look at and change to do a task like this.
- 7:23
But at least you do still need to be able to set up the repo and work with the repo, right? And so you wanna be able to run lint, you wanna be able to run CI, all of these other things, so you know, at, to at least have the basic checks of whether things work.
- 7:36
One of the big things that we built around then was the ability to really set up your repository, uh, ahead of time and build a snapshot, um, that, that you could start off, that you could reload, that you could roll back and all of these kinds of primitives as well, right?
- 7:48
So having a, this clean remote VM that could run all these things, it could run your CI, it could run your linter, uh, and, and so on. Um, but that's when we started to really see, I would say, a bit more broader value, right?
- 8:00
I mean, migrations is one particular thing, and for that particular thing, we were showing a ton of value, and then we started to see where, you know, with these bug fixes or things like that, you would be able to just generally get value from Devin as, as almost like a junior buddy of yours.
- 8:14
And then in fall, things really moved towards just much broader bugs and requests. And here it's, you know,
- 8:23
most, most changes, again, you know, where you, you jumping another order of magnitude. Most changes don't just contain themselves to one file, right? Often you have to go and look, see what's going on.
- 8:31
You have to diagnose things. You have to figure out what's happening. You have to work across files and make the right changes. Often these changes are, you know, hundreds of lines.
- 8:39
If it's like, "Hey, I've got this bug, let's figure out what's going on, let's solve it," right?
- 8:44
And you know, there, there are a lot of things here that, that really started to make sense and really started to be important, but, but one in particular I'll just point out was there's a lot of stuff that you can do with not just looking at the code as text, but thinking of it as this whole hierarchy,
- 8:58
right? So, so understanding call hierarchies, running a language server, uh, is a big deal. You have Git commit history, which you can look at, which informs how, how these different files relate to one another.
- 9:09
You have, um, um, obviously you have, like, your linter and things like that, but, but you're able to kind of reference things across files. And so, like, one of the big problems here, I think, was, uh, kind of working with the context of it and getting to the point where it could make changes across several files.
- 9:25
It could be consistent across those changes. It would be able to understand across the codebase. And here was really the point, I would say, where you started to be able to just tag it and have it do an issue and just have it build it for you.
- 9:36
Um, and so Slack was a, it was, you know, a, a huge part of the workflow then. Um, and, and, and it was just,
- 9:42
it, it, it, it made sense because it's where you discuss your issues and it's where you set these things up, right? So you would tag Devin in Slack and say, "Hey, by the way, we've got this bug.
- 9:49
Please take a look." Or, you know, "Could you please go build this thing?" Uh, this is especially fun part for us because this is right around when we went GA, uh, and a lot of that was because it was, it got to the point where you truly could just get set up with Devin and ask it a
- 10:03
lot of these broad tasks and, and just have it do it. Um, but, but a lot of these, you know, a, a lot of the work that we did was around having Devin have better and better understanding of the codebase, right?
- 10:13
And if you think about it, you know, from the human lens, it's the same way, where on your first day on the job, for example, being super fresh on the codebase, it's kinda tough to know exactly what you're supposed to do.
- 10:22
Like, a lot of these details are things that you understand over time or that a, a representation of the codebase that you build over time, right? Um, and Devin had to do the same thing and had to understand, "How do I plan this task out before I solve it?
- 10:34
How do I understand all the files that need to be changed? How do I go from there and make that diff?"
- 10:41
And around spring of this year, um, again, every, every gap is, like, two or three months. You know, the, we, we got to an interesting point, which is once you start to get to harder and harder tasks, you as the human don't necessarily know everything that you want done at the time that you're giving the task, right?
- 10:59
If you're saying, "Hey, you know, I, I'd like to go and, um, improve the architecture of this," or, you know, "This, this function is slow. Like, let's, let's profile it and look into it and see what needs to be done."
- 11:09
Or, "Hey, like, you know, we really should, should handle this, this error case better, but, like, let's look at all the possibilities and see what we should, you know, what the right logic should be in each of these," right?
- 11:19
And basically what it meant is that this whole idea of taking a two-line prompt or a three-line prompt or something and then just having that result in a, a Devin task was, was not sufficient, and you wanted to really be able to work with Devin and specify a lot more.
- 11:34
And around this time, along with this kind of, like, better codebase intelligence, um, we had a few different things that, that, that, that came up, and so we released DeepWiki, for example.
- 11:42
Um, and the whole idea of DeepWiki was, you know, funnily enough, is Devin had its own internal representation of the codebase, but it turns out that for humans it, it was great to look at that too, to be able to understand what was going on or to be able to ask questions quickly about the codebase.
- 11:56
Um, closely related to that was with Search, which was the ability to really just ask questions about a codebase and understand, um, some, some piece of this. A- and a lot of the workflow that really started to come up was actually basically this, this more iterative workflow where the first thing that you would do is you would
- 12:12
ask a few questions. You would understand. You would basically have a more L2 experience where you can go and explore the codebase with your agent.
- 12:20
Figure out what has to be done in the task, and then set your agent off to go do that. Because for these more complex tasks, you kind of needed that, right?
- 12:29
Um, and so, so, you know, that was a, I, I would say, kind of like a big paradigm shift for us then is, is understanding, you know, this is what also came along with Devin 2.0, for example, and the in IDE experience where often, yeah, you wanna be able to have points where you closely monitor Devin for
- 12:43
10% of the task, 20% of the task, and then have it do, uh, work on its own for the other 80, 90%.
- 12:51
Um, and then lastly, most recently in June [laughs] which is now, uh, it was kind of, yeah, the really the ability to just truly just kill your backlog and hand it a ton of tasks and have it do all these at once.
- 13:02
And, you know, if you think about this task, in, in many ways, I would say it's, it's almost like a culmination of, of many of these different things that, that had to be done in the past.
- 13:09
You have to work with all of these systems, obviously. You have to integrate into all these. Certainly, you wanna be able to, to work with Linear or with Jira or systems like that.
- 13:15
But- Right ... you have to be able to scope out a task to understand what's meant by what's going on. You have to decide when to go to the human for more approval or for questions or things like that.
- 13:25
You have to work across several different files. Often you have to understand even what repo is the right repo to make the change in if, if your, if your org has multiple repos, or what part of the code base is the right part of the code base that needs to change.
- 13:37
Um, and to really get to the point where you can go and do this more autonomously,
- 13:42
first of all, um, you, you have to have, like, a, a really great sense of confidence, right? And so, um, you know, rather than just going off and doing things immediately, you have to be able to say, "Okay, I am quite sure that this is the task, and I'm gonna go execute it now," versus, "I don't understand
- 13:57
what's going on. Human, please give me help," [laughs] basically, right? But, but the other piece of it is
- 14:04
this is, I think, the era where testing and this asynchronous testing gets really, really important, right? Which is if you want something to just deliver entire PRs for you for tasks that you do, especially for these larger tasks, you wanna know that it is ha- can, can test it itself, and often the, the agent actually needs this
- 14:20
iterative loop to be able to go and do that, right? So it needs to be able to run all the code locally. It needs to know what to test.
- 14:26
It needs to know what to look for. Um, and in many ways, it's just a, a much higher context problem to solve for, right, is this testing itself.
- 14:34
And that brings us to now. And obviously it's a, it's a pretty fun time to see because now what we're thinking about is, hey, maybe if instead of it doing it just one task, it's, you know, how, how do we think about tackling an entire project, right?
- 14:46
And after we do a project, you know, what, what goes after that? A- A- A- And maybe one point that I would just p- make here is
- 14:53
we talk about all these 2Xs, you know, that happen every couple months. And I think from a kind of cosmic perspective, all the 2Xs look the same, right? But in practice, every 2X actually is a different one, right?
- 15:04
And so when we were just doing, you know, tab completion, line, single line completion, it really was just a text problem. It is just, like, taking the single file so far and just predict what the line is next, right?
- 15:15
Over the last year or year and a half, we've had to think about so much more. How do you, how do you work with the human in Linear or Slack or Jira?
- 15:22
How do you take in feedback or steering? Um, how, how do you help the human plan out and do all these things, right? And moreover, obviously, there's a ton of this, the tooling and the capabilities work that have to be done of how does, how does Devin test on its own?
- 15:35
How does Devin, um, uh, you know, make a lot of these longer term decisions on its own? How does it debug its own outputs or, or run the right shell commands to figure out what the feedback is, uh, and go from there?
- 15:47
And so it's super exciting now that there's a lot more, uh, there's a lot more coding agents in the space. It's, uh, it's, it's very fun to see, and I think that, you know, we- we're [laughs] gonna see another 16 to 64 X over the next twelve months as well.
- 15:59
And, uh, and, and so, yeah, sup- super, super excited.
- 16:03
Awesome. Well, that's all. Thank you guys so much for having me. [audience applauding] [upbeat music]