How Forward Deployed Engineering is done at Cognition

[upbeat music] It's nice to be here. Uh, appreciate you all. My name is Jia.

I'm a deployed engineering lead at Cognition, and today hopefully what you'll take away from this is that how we deploy Devin in the field is very much a function of how we view s- uh, deployed engineering at Cognition.

So, how the forward deployed motion makes AI engineering actually real. Before I start, how many people, like, have heard of Devin or, like, know of Devin? Okay, cool. And I'm not talking about, like, the Devin of today, like, I'm talking about the Devin back in twenty twenty-four when we first released, and it was like, oh, SWE-bench thirteen

percent. We're so, we're so back. And as engineers, we were like, "We're so cooked." But, I mean, after a week, everyone's like, oh, this is actually, like, not that useful.

Um, I would only use this if I was desperate and out of ideas. [laughing] That's twenty twenty-four. Uh, and let it be said that we have a sense of humor because we took this and we ran with it.

I'm sure you've seen all these ads around SF. We're actually good now. [laughs] So, the reason why we're good is... I'll talk a little bit about the product surface area a little bit, just to give you folks, uh, a little bit of context for who might not have used Devin before, uh, who might not have exposure to something

like Cognition. So, if you've used Claude Code, we also expose a CLI. If you've used something like, you know, Cursor or Windsurf, we also expose an interface such as an IDE.

Uh, actually, how many people, like, know of Windsurf or have used Windsurf in the past? Sweet.

So I come over from the Windsurf side after the Windsurf acquisition. Um, great times. [laughs] And then what we're actually known for specifically is for Devin Cloud or the Devin Cloud agent.

And I'm not gonna bore you to death talking about, like, all the components and features and everything that comprise of the actual product. I'm not here to sell you on that.

What I am here to sell you on is we are one of the premier software engineering functions across the enterprise, and we're actually able to deliver impact on a global scale.

What do I mean by that? We can take a pause and take a look at this figure. So internally at Cognition, over the last six months, for better or for worse, we might have been behind on hiring, but using our agent, we were able to ship almost an order of magnitude more good quality, robust PRs across the

organization. So it's a step function increase in the amount of engineering leverage that we can have by deploying our own agent. You don't have to take our word for this.

If we take a look-- If we can take a look at the specifics of how we're actually being consumed, how we're being utilized across the enterprise, it's a parabolic growth of how companies are adopting our agent, deploying our agent, and using it in multiple different use cases and multiple different scenarios.

So what does that mean, right? Like, how does this actually happen? Well, it can only happen with the forward deployed engineers at Cognition. And I'm gonna frame up the problem from, like, a couple of, like, buckets, right?

So there's two circles in front of you on the screen. One of them can represent the domain of a product, right? As a business, as a software engineering organization, obviously you have a product.

You obviously also, on the other side, on the left-hand side, or right-hand side for you guys, you specifically have, like, a bucket of problems that you're looking to solve, right?

You have a product, you have problems that you're trying to solve, and the intersection of these two, or whatever you would call it, is the product market fit, right?

Hopefully, you have a pretty good overlap in the sense that whatever it is that your company does, you can actually bring value to customers. And hopefully whatever problems the customer has, you can solve with your company's stuff.

So the forward deploying motion at Cognition essentially aims to maximize the overlap between the products that we typically build and the problems that we are experiencing across the enterprise.

So what does that mean, right? The first fundamental concept that I would like to convey is that forward deployed engineers at Cog deeply understand the problem space at hand.

And specifically, right, like if we think of the problem of software engineering, and I'm just gonna, like, mask the features at the bottom, we don't really care about those.

But if we think about when you go ahead to take some sort of code base, some sort of implementation, and you need to, like, build features, you need to maintain that software, uh, you need to, like, review, deploy, maintain that software.

All of these steps, all of these functions have a lot of business value behind them, right? You can only do-- Like, it would be great if we could build from zero to one and just, like, prompt stuff and not worry about, like, legacy code, but that's not the reality of the situation.

It would be great if our product engineers or our product managers could just take user stories and say, like, "These are the things that we need to build in order to get value and revenue."

And then we come to coding. Coding itself, at least from our perspective, is a mostly solved problem, right? These models are so good now that, like, with any type of context, with enough context engineering, you can get the code blocks that you really care about.

But the problem isn't, like, writing code faster. That's usually only twenty percent of the problem. The problem really just becomes, like, how do you test this code? How do you review and deploy this code?

And how do you maintain this code across the enterprise? So that's the premise of the problem. Now, I will predicate it by saying that, like, we also have the solution.

I'm not gonna bore you to death about talking to you about the solution. But forward deployed engineers at Cognition, we map Devin's capabilities specifically to the customer problem. So if the software development life cycle is extremely complex, deploying the agents for-- with, like, no specific direction, you are straight up just token maxing, right?

Like, you are wasting tokens. You are burning, you are burning spend. You're not getting any tangible outcomes. So we try to identify when we partner with customers, when we take meetings...

And for example, right, like, my day might look like four or five hours of customer calls and then four or five hours of, like, actual hands-on keyboard work. Those four or five hours of calls actually allow us to understand very deeply what strategic initiatives are the highest leverage for the business, right?

Once we've identified that, right, how can we automate ourselves out of the job in the sense that we set up the agent in a way that it runs all of the automations for us, right?

We don't have to be there manually triggering the agents. It can respond to, like, specific alerts, specific events.

But most importantly, as a forward deployed engineer, how do you measure the return on investment?

And it's very ambiguous, and it's an unsolved problem because the company that will solve this will be, um, you know, five trillion dollars market cap. So specifically, our forward deployed engineers will embed in the customer's ecosystems, right?

We take a look at, you know, the backlog of stuff that needs to be built. We take a look at the remediations that need to be done. We take a look at all of, like, the delayed code that never ships or the tests that nobody writes, uh, or the automatic triage of specific alerts, and we map our

product capabilities into those problems. That being said, if we all do our job and we solve the customer's problems one hundred percent and the customer is very happy, that is only half of the s- equation, right?

Not only do we have to solve the customer's problem, we also need to solve for our product. What do I mean by that, right? If we are taking a, a union of, like, the problems that the customers have and the products that we are building, right?

Solving the problem only shifts, like, part of the Venn diagram. But in order to get that true feedback loop where we unify, like, the maximal overlap between what we're doing and what the customers need, that is forward deployed engineering at Cog.

So the second part of this that I wanna emphasize is that at our company, we map the customer problems back to the capabilities at hand. Now, how do we do that, right?

A lot of engineering challenges come in similar shapes, um, from the field, right? We have the highest fidelity evaluation set that comes back from our customers, right? We are in the field every single day.

We are hearing about the c- problems, we are hearing about whatever the customers are doing, and we have to take that context and bring it back to product in a sensible way.

So what are the ways in which these enterprise challenges, you know, manifest? Are they common across the entire enterprise or are they unique to a specific user? Should, like, workarounds or, like, hacks or bugs in th- in, in what we're building become features, right?

Because ultimately what we want to do as a company is we want to de-risk our roadmap. At the end of the day, it would be great I- if, like, I as an engineer knew exactly what to build to get Y percentage of revenue from a particular customer and that's exactly what the problem is that we're trying to

solve for because we are ultimately at the end of the day the heralds of the change, right? We are the bridge between products, we are the bridge between problems and the feedback is actually like half of the loop that makes the next deployment better than the previous deployment.

So if we think about the T-shape or personas of the folks that we hire,

and forward deployed engineering is just so... It's so flexible in terms of how you actually define it. Like, are you a sales engineer? Are you a solutions architect? What are you, right?

So if you think about all of the skills that FDEs typically are expected to have, right, you probably wanna go wide. You probably wanna have, you know, good people skill, like, business skill, uh, good process, customer skill, or even technology, right?

In order to be deployed in this space, you need to know, like, what- whatever the tech is. So we also look for very deep spikes across people, right? Folks at Cognition, uh, deployed engineers at Cognition, we will hire them from, like, product management backgrounds if they have a really good sense of, like, how products are supposed to

fit into each other, right? Because if the s- cost of software engineering is going to zero, you actually need to know how to, like, design a product that makes sense 'cause you can just prompt it.

But we also hire folks that are, like, founders, uh, software engineers, but specifically, like, very spiky in the technology domains, right? It's fine if you don't have, like, the strongest business sense.

That can be learned, but it's also very hard to teach, like, technicality and being the expert in the room while you're on the job. So there's a couple pr- personas that we specifically hire for.

Good customer engineers or deployed engineers do everything, right? Like, they can map the product back to the roadmap. Like, they can solve the customer's problems. Great customer engineers are able to actually have that relentless curiosity for why.

So at Cognition we always ask ourselves, "Why are we solving this problem? Does this problem matter to the, to the business as a whole? And can I communicate this back to the roadmap in a way that, like, improves the problem for everybody else?"

But we also are-- The second m- mantra that we subscribe to is that you have to be relentlessly tied into the customer, right? They are our lifeblood at the end of the day.

Making them successful is the only way that you can survive as a business and become the obvious choice for an enterprise partnership. So these are the two mantras that we specifically subscribe to as, uh, deployed engineers at Cognition.

So previously, um, [chuckles] the first approach to deployed engineering was just, you know, token maxing, right? Uh, the next era that I'll talk about is intelligent orchestration, but with, like, outcomes that you can actually measure and I'll give you guys some examples.

So previously, like, a year, maybe a year and a half, two years ago, uh, the target or KPI for whatever deployed engineers were trying to do is maximize token usage, right?

It was, it was like the perfect time. You didn't have to worry about budgets. Everything was subsidized. You could just, like, run anything you wanted. But now the problem has really shifted into the delivery space, right?

A lot of organizations that we work with, some of the largest and most regulated enterprises in the world, they really care about are we getting true value out of this solution or are we just burning tokens for no reason, right?

And that is one core differentiator that I need to call out between us and some of the other platforms. You can make engineers, like, ten X faster. That's fine.

That's still valuable. But can you make an organization ten X faster, including every single person that might be technical or non-technical, uh, across the company? That's when you unlock the true value of being in the partnership and that's why, like, single point tools that are just like CLIs or just IDEs, they fail to do that.

So let me just give you some proof points of how we've operated across the enterprise.

So at Cognition, um, when we run the agent and the agent has, like, a specific trace or trajectory or, like, the agent does something, we call that a session.

A session itself, right, we have metrics that allow you to derive how many engineering hours you can actually generate and how mu- how many engineering hours are actually productive that users are running.

Right, so one of these examples is a case study where we embedded ourselves within a customer for three months. We brought them on board, and functionally, over the course of those three months, we delivered about 150 per-- like, plus headcount.

So if you thought about, like, a project that you're trying to ship or something that you're trying to develop or a migration that you're trying to go through, imagine having 150 extra coworkers doing that with you by your side.

You might just say, "Hey, this actually just kind of looks like token maxing," right? Like, you're just giving me a metric that says it's just, you know, engineering hours.

You're, like, running a bunch of different sessions. Like, how do we know that these sessions are true, meaningful and valuable? So the second part of that is, okay, we can think about how we've reduced timelines for delivery projects and an order of magnitude, so about, like, eighty-two percent reduction across, like, delivery.

So if you subscribe to the agile deliver-- the, the agile development methodology, obviously, like, you have tickets, you have sprints, like, you have things that need to be built.

If you look at every single metric that you measure before you bring in Devin and after you bring in Devin, you can take a look at that. We can compress this timeline by a factor of, like, eighty-two percent.

So across the board, whenever we get developed, whenever we get deployed and fully activated within the customer environment, not only do we deliver, uh, massive scale in terms of, like, engineering capacity, but we also reduce the time to value in terms of bringing things to market.

Now, the third part of that is, hey, but I actually really care about the numbers, right? I actually really wanna see how many PRs are you actually shipping. Like, is this meaningful?

Does this actually make sense? So if you think about s- dissecting the numbers a l-- like one dimension further, and you wanna just look at, like, the raw PRs that people are ripping across the enterprise, we deliver almost double the amount of PRs that engineers were able to do with single point tools and before you brought in

an agent harness like Devin. So there's three proof points of anonymized case studies in which we are able to deliver value at scale and across, like, various different problem domains.

What I'll say is these aren't, like, you know, private case studies. We have a bunch of different public case studies as well. So we partner with companies like Nubank, right?

If you folks have ever gone to Latin America, uh, you can understand that, you know, the-- there's a lot of developers there. There's a lot of projects that are tangentially related.

So specifically, like, we can say that there was an ETL migration. They had fifty engineers staffing this migration. We were able to deliver this within, um, I think, like, one-third of the timeline

just with Devin autonomously. We have another bank, right? We have another use case where we work with one of the largest banks in Latin America. They were trying to migrate, like, the tax identification system.

I know, like rocket science, right? Um, but the idea is that we were able to deliver this with half of the amount of effort actually required. So if you think about, like, legacy languages like COBOL, if you think about things like JCLs, you think about things like, like people don't learn anymore just because it's, like, not fun

and not interesting, we're able to operate across some of the most complicated code bases in the world and deliver results that actually matter.

And the last but not least, obviously, like, if we think about the Built card, um, specifically, we're able to actually merge, like, an order of magnitude more, um, in terms of, like, PR acceptance rate.

We deliver, like, ten X person-- like, worth of engineering talent, like, every single week. And then we're actually able to, you know, generate the weekly output of, like, over ten engineers at the organization.

Built has great engineers, by the way, right? These guys are so cracked. So if you take one of these engineers and multiply them by ten, you just imagine the amount of returns.

So what I'll say is I'll, I'll probably, like, round off this talk by just saying that at Cognition, we don't just, you know, embed ourselves in the customers. We don't just, you know, propagate feedback back to everybody else.

But the core values of the company are things and principles that we subscribe to, not internally to the company, but external to the company as well, right? It's really fun being on the winning team.

It's really fun when you come into an organization and say, "We can actually deliver so much cool stuff," and, like, make people our champions, right? Whoever deploys Devin within the organization, they can show results that are essentially unmatched across the board.

And we go for it all, right? We leave nothing on the table. We've deployed somebody in Brazil for, like, two months [laughs]

to, to live next to one of the customers to just make them successful. So we're down for the mission, and it's, it's more about, like, correctness, right? Like, if, if there are engineering practices that we wanna fix, if there are things that we wanna flag and raise, like, these are all things that we take back to product,

and there's no ego involved. At the end of the day, we are all in the same boat, we're on the same mission, and we're just shipping.

And everybody essentially is go-to-market. I know forward deployed engineering is kinda like this fuzzy thing where it's like, "Am I part of sales? Am I part of post-sales? Like, what do I actually do as an FDE?"

But everybody is go-to-market because the target is to make the customer successful at all costs.

And at the end of the day, we just do things because every second counts. So if you're interested in, you know, forward deployed engineering at Cognition, being the intersection between some of the hardest problems in this world, being part of, like, all of the software disruption at scale, and then being on the other side of these problems,

we should talk. Thank you. [audience applauding] [upbeat music]