AI tools for Forward Deployed Engineering

Talk transcript

1. 0:00

   [upbeat music] All right. First and foremost, thanks so much for being here. Um, it's, uh, been a great experience, you know, obviously chatting amongst, uh, other industry giants like Cursor and Factory and Anthropic, and I'm sure you guys are mostly here for them, but [laughs] thanks for

2. 0:25

   sticking around for this talk. Uh, my name is Vas. I'm the CEO of Varick Agents. We work with some of the largest companies on the planet, transforming them from the inside out with, uh, AI and agents.

3. 0:34

   Um, and because of the nature of our work, which is highly bespoke, we go very deep into our clients. It requires a lot of forward deployed engineering, and this conversation is around why that's so important, how we approach it [REDACTED:username], uh, and some of the internal tooling that we've created internally to allow us to scale that

4. 0:51

   forward deployed motion without, you know, increasing headcount exponentially. And it's titled The Next Bottleneck because I fundamentally believe the next bottleneck is: how deep can you go into a customer without scaling headcount, uh, exponentially?

5. 1:04

   How can AI do that job for you?

6. 1:07

   So, as stated previously, AI is solving the execution of work. Um, if you were to look back a couple of years ago before, uh, you know, thinking agents and reasoning agents were a widespread phenomenon, uh, and I had asked you, "How many of you have used AI to solve an end-to-end task?"

7. 1:24

   The answer would be slim to none. But if I ask that same question to everyone today saying, "Has AI solved an end-to-end task for you today?" I'm sure every single one of you would raise your hands.

8. 1:33

   So clearly, execution is no longer the core bottleneck. The models are improving to the point where intelligence is no longer the constraint, and harnesses are being built in a way that allow us to use, whether it's browser-use tooling or API tooling, uh, with very robust MCPs that allow us to execute work, uh, with near perfection.

9. 1:52

   The difference, and the bottleneck that is still here, is how much can you understand the business? Because every business, every consumer is different. Uh, one sales department for a healthcare company, for example, operates completely differently than the sales department for a SaaS company, and this is something that we see with our work today [REDACTED:username] Agents.

10. 2:11

    And if there are any business operators in the room, you know exactly how hard it is to wrangle, uh, the latest models to solve for your specific use cases.

11. 2:20

    It's very difficult to extract that context from your, you know, employees and from your team, and it's very difficult to feed that into an API call or a simple model call, uh, that doesn't break down very quickly.

12. 2:32

    So the bottleneck is how much can you process re-engineer, how much can you process understand? And that's the job that we do here [REDACTED:username]. So right now, operations are fundamentally centered around the human.

13. 2:45

    Uh, right now, the work that you do today is done by humans, whether it's on top of software or completely agnostic to software. Uh, but in the future, operations will be centered around AI.

14. 2:53

    And this means not only, you know, providing companies with AI tooling like a Cursor, Claude Code, Codex, like a Factory, like any of the other brilliant AI tools that you-- I'm sure you're experiencing here today, but also changing the operations and the processes themselves.

15. 3:10

    And fundamentally, that is the role of a forward deployed engineer. It's going into the company, understanding how things run today, and re-envisioning what it could look like tomorrow. And we believe that is our job [REDACTED:username] and why forward deployed engineering is such a core part of what we do.

16. 3:27

    So a forward-deployed agent, what does that really mean? So why do we need FDEs? As stated previously, I'm not gonna, you know, go into this too much. I'm sure you've been hearing a lot of this today.

17. 3:36

    Uh, FDEs are responsible for a few different things. One is they map the way that humans are doing their work today. So how we do that [REDACTED:username] is several forward deployed engineers will be embedded directly with a customer.

18. 3:45

    You can imagine it's a enterprise company with thousands of employees, but we'll scope it down to a single department. In a finance department, for example, we'll have them sit down with the process leads for AP, AR, card reconciliation, banking, billing, FP&A, et cetera.

19. 4:01

    So interviewing every single one of these process leads to understand not only how are things running today, but more importantly, when things go wrong, what happens. You know, a lot of the documentation that you have at companies about the golden path and maybe an edge case or two, but this is still fundamentally not the reality.

20. 4:18

    Where when we talk to customers, it's, it's a lot of, you know, Sarah in AP handles the workflow today in this way, but when things go wrong, she actually sends it over to Chris, who then takes four days of cycle time to handle reconciliations between a purchase order and an invoice.

21. 4:34

    Those are the realities that are, one, unique to every single company. The way that they handle things is different from one company to the next, and two, the real bottleneck for why AI can't just run amok and handle end-to-end processes without the handholding that you see today in the enterprise.

22. 4:49

    Uh, so this is the first section, which is mapping how humans do the work. The second is really re-engineering the process around AI. So what does this mean? You know, there's a lot of talk being given today in terms of slapping AI onto broken processes, and that's fundamentally why you don't see the ROI across the industry today.

23. 5:07

    There's a lot of, you know, semi-outdated but still very relevant statistics like the MIT review saying that ninety-five percent of generative AI pilots fail to reach production, or the other statistic which was eighty-seven percent, very similar thing, that most AI pilots don't produce measurable ROI or they don't ship to production, period.

24. 5:27

    And the reason for that is a lot of the time, AI is being slapped on top of broken processes in a way that the AI doesn't actually understand how to do things.

25. 5:37

    Um, you see th-this at the simplest level with coding, where as an engineer, it's very difficult, even with goal loops and the, the latest technology there, uh, to just say, "Go and solve this for me," and have it run off and refactor entire code bases without some degree of human input.

26. 5:55

    Now, if you extrapolate that to a business context, these are very non-technical operators in finance, sales, marketing, procurement, logistics, uh, et cetera. So giving them this AI tooling will not allow them to receive the same ROI that a software engineer might be able to, to, uh, produce or create.

27. 6:13

    Um, so what this means is you need forward-deployed engineers to help them re-engineer their current process around AI. It needs to be not too different to where they don't understand, you know, how to operate the system.

28. 6:24

    For example, if they're used to an eleven-step workflow and you come in and change that with a one-step, they might be taken aback, the adoption rates might suffer, et cetera, as was alluded to in previous presentations.

29. 6:34

    Um, but at the same time, it needs to be different enough to where you're actually capturing the ROI, meaning you do say, "All right, four out of these eight steps will be handled completely autonomously.

30. 6:45

    The other three will be handled with some human-in-the-loop intervention, and one step of that process will be handled by a human, period," either because the risk is too high or because, you know, it's, it's not unique enough for an agent to produce measurable value in that specific step of the process.

31. 7:02

    So that's the second major step of a forward-deployed engineer. That's why we need them. And third and finally, and this is what I want to, uh, bring one of my heads of engineering to discuss in just a moment, is actually deploying these agents on top of existing systems.

32. 7:15

    So fundamentally [REDACTED:username], we believe that the AI wave left a lot of enterprise behind. A lot of enterprise is married to their systems of record. Not everybody, but most of them are.

33. 7:26

    Uh, they've migrated to NetSuite, they've migrated to Dynamics, uh, they've migrated to SAP and Salesforce, and when you pitch them AI solutions that live completely disparate from these systems, uh, you're ignoring the reality of enterprise.

34. 7:39

    One of the quotes from our clients said that they spent five million dollars and five years migrating to, uh, NetSuite. That's a real quote. So if you're telling them, "Hey, I have this fancy AI tooling, but by the way, you have to migrate off of NetSuite," they're gonna tell you to get out.

35. 7:54

    They don't have any appetite for that. So what we believe in [REDACTED:username]-- believe in [REDACTED:username] is we'll build the agents on top of your systems of record, and the way that we do that is quite, uh, unique.

36. 8:04

    We have our own Varick OS platform that allows us to spin up agents, monitor them, uh, et cetera, with the full governance and ESL suite baked in. But at the same time, it lives on top of your systems of record.

37. 8:14

    So if you are on a Salesforce or a NetSuite or a Dynamics or an SAP, we will not ask you to migrate off of that. And that is where enterprise needs AI the most, uh, because they're too large to move up.

38. 8:28

    So why build the FDE agent in the first place? As mentioned previously, I think everyone is saying that twenty twenty-six and onwards is the year of the forward-deployed engineer, and to some extent, we believe that's completely correct.

39. 8:40

    There has never been more of a need to go deep into customers and understand their business c- use cases and help them adopt the latest in AI tooling. But at the same time, we realize that it's actually very difficult to find forward-deployed engineers who are both the technical, like, top one percent, who are really able to understand

40. 8:57

    and speak AI, uh, ten thousand times better than the average, you know, enterprise customer, but also have the communication and human skills needed to be, you know, as alluded to previously, very high IQ, high EQ, extracting the information from the customer and meeting them where they are in real time.

41. 9:15

    You know, typically you'll have consultants that you then train on the technical side or engineers that you then kind of train on the soft skill side, but it's very hard to find people who are, you know, the best of both.

42. 9:26

    Um, so the FDE agent is our effort to bolster the existing forward-deployed engineers that we do have. So, for example, allowing one forward-deployed engineer or forward-deployed strategist to manage and maintain several client communications.

43. 9:41

    I know that most of the folks in the room are technical, but it's very easy to misunderstand how deeply involved you have to be with the client. They're emailing you twenty-four/seven.

44. 9:51

    They're sending you hundreds of pages of documentation, and every single process lead will pull you in a different direction. AP relies on AR, relies on reconciliation, relies on FP&A, and they each have their own version of what they think is the most important.

45. 10:05

    So being able to manage that context and being able to serve them all equally while also not hiring fifty people to do so is fundamentally very important. It's how we [REDACTED:username] avoid being, you know, a traditional consultancy while also offering that handheld-- hand-holding and, like, very human experience, human-centered approach of consulting that we thi- we do

46. 10:24

    think is very valuable. Um, so in the past, in twenty twenty-four and around that time, execution work was still the bottleneck. This was before the models gained the intelligence and the harnesses gained the integration abilities that allowed them to move past the execution bottleneck.

47. 10:39

    Um, now the AI models are trained to solve the execution of knowledge work. I will go as so far as to say that knowledge work is almost entirely solved.

48. 10:48

    The difference is, and what we're realizing now, is that designing how work gets completed around AI is the next bottleneck. It's the ability to go deep within the customer, redesign their workflows, deciding what should be automated versus shouldn't, and building this in a robust and scalable way on a platform that moves the needle for our clients.

49. 11:08

    You know, as opposed to doing a point solution, which promises to transform just one part of your sales process, for example, uh, maybe it's prospecting. Uh, that ROI might deliver five to ten percent ROI for you as a sales function.

50. 11:20

    Same thing on finance. If you're just doing AP and no other part of your department, you might have a five, ten percent ROI. But [REDACTED:username], we deliver department-wide transformations, holistically transforming the entire department at a time, and that's how we get the ROI that we see for our clients, which is twenty-five percent, fifty percent, seventy-five percent,

51. 11:40

    truly giving them back, you know, the three things, which is revenue uplift, cost savings, and risk mitigation, as was so eloquently stated previously. And an AI FDE is trained to re-engineer these necessary tasks around AI.

52. 11:53

    So I want to invite my head of engineering, uh, JD Pruitt, to come up and, and share some more of the deep technical stuff on our FDE agent, uh, because I haven't written a line of production code in a while.

53. 12:05

    So here's JD.

54. 12:06

    Okay. Thanks, Paul.

55. 12:09

    And maybe if we can get his mic going.

56. 12:15

    Great. Thank you. Um, thanks, Vas. So, uh, this project to give tools to our FDEs, um, basically started with me. I lead the platform team, and we're over on one side of the office.

57. 12:25

    We're hanging out, we're chilling, we're having a great time. Uh, Codex, Claude, we're all hanging out, and then I look over at the FDE side of the room. They look stressed, they are sleep-deprived, they're extremely miserable.

58. 12:35

    They've got clients emailing them twenty-four/seven. I'm like, "Oh, my God, you guys haven't slept at all." So I go, and I start talking to them and I'm like, "Okay, what is your guys' process like right now?

59. 12:42

    How are you actually engaging with these clients?" And like, "Well, we, you know, upload about a hundred and fifty pages of documentation to Claude, and then we prompt Claude, and then we wait, like, two minutes, and then we get analysis, and then it's verbose and incorrect, and it kind of sucks."

60. 12:54

    And I was like, "All right. We gotta fix this." So we have been working on an FDE agent, which is the Codex for our FDEs, basically. And there's three stages of it, um, the last of which is certainly still in, in development.

61. 13:07

    The first is what we call the engagement-- It, it-- The f- first function is the eng- is the engagement agent. And essentially, this is a better version of Claude built just for our FDEs.

62. 13:18

    It's their assistant. They have... It pulls in their Granola notes. It synthesizes documentation, it reads PowerPoint slides. It allows them to query and say, "Who is responsible for this process?

63. 13:27

    Uh, I got an email that mentioned Sarah spelled, you know, this way, and I have a, you know, a Slack message with a different way. Are these the same people?"

64. 13:35

    'Cause these are the questions that our FDEs are asking all day, every day, and they waste a ton of time just waiting on, on Claude to respond. So the engagement agent is their way of-- it's their, it's their assistant to build, um, to build the workflow.

65. 13:47

    Then there's the workflow agent. And what we did is we took our engagement agent, and we embedded it inside of our platform, so that when our FDEs go and they actually build the workflow, the FDE agent is right there saying, "Oh, you forgot about this edge case.

66. 14:02

    Um, you should probably ask me, uh, you know, who owns this process so I make sure the email goes to the right place." And it lives-- It's basically a-- I can talk a little more of the details on the next slide, but...

67. 14:17

    Whoops. It lives, uh, it lives onside of our-- inside of our platform. It works next to Claude or Codex, whatever model you're using, um, and makes sure that the workflow that the FDE is constructing is actually, uh-- it correctly shadows the process that we want to engineer.

68. 14:35

    And then there's the final stage, which we're not at yet, which is a, um, an autonomous assistant for FDEs, where it's receiving emails from clients who say, "Actually, I wanna change, you know, where my QC report goes to.

69. 14:49

    I wanna, you know, change it to a different email," et cetera. And our agent is able to process that information, query the understanding of the company that we currently have, ship an autonomous change to the workflow on top of our platform, and then our FDE never has to get involved, saving their time for the much more high-value

70. 15:07

    work of sitting down, interviewing with the clients, really understanding what their process is, um, and not dealing with all of the small, little minutiae that anyone who has been an FDE can tell you, uh, takes up a lot of their time.

71. 15:22

    So how do we, how do we build this? Um, the first thing is we need some single source of truth, some representation of a company's, uh, functioning. There's a lot of different ways to do this.

72. 15:32

    If you were at the booths downstairs this morning, there was, you know, five companies trying to sell you a GraphDB, and you can just use Postgres, whatever it is.

73. 15:40

    Yes, I'm looking at you. Um, [clears throat] doesn't really matter what you use, but the point is, we use a dependency graph. Uh, most of these workflows inside of enterprise are remarkably linear.

74. 15:49

    They just have a lot of cycles in them. But at the, at the end of the day, the process owners want things to be as dependency-driven as possible. They don't want person C in the process to have to deal with something before A and B have approved it.

75. 16:02

    So a dependency graph is a very nice representation of this.

76. 16:06

    Um, then we do our own model training. Um, and there's really two parts to this problem that we're trying to solve. The first is, given extracted context for the FDE, do we get a good, high-quality output?

77. 16:20

    And the answer is, with Claude, honestly, no, which is kind of surprising. But the really, um-- I'm sure you guys have experienced this. When you are trying to do a long analysis, frontier models are extremely verbose, and they lack the, um,

78. 16:35

    you know, I would say the, um... I only started believing in consultants once we started hiring them [REDACTED:username], and the reason is, they are so good at figuring out what is the part of the detail the client actually cares about and what is the part that can get glossed over.

79. 16:50

    And frontier models have absolutely no concept of this. So we started post-training our own models on top of, on top of open source models. We're a fan of Kimi K two six, but a lot of these would do diff-- a lot of these would do fine to really get that nice balance between details and, uh, clarity that

80. 17:07

    the frontier models often, often lack. So that's a, that's a bit about writing a good normalized process flow from extracted context. But there's the second half of the challenge, which is

81. 17:20

    getting good at traversing-- at extracting the right context. So we might have this huge knowledge graph, but it's remarkably difficult to traverse this knowledge graph in a reliable way that finds us the right context.

82. 17:33

    So once we have our post-trained model, we create an RL environment where we have exposed our own custom tools specifically designed to traverse our knowledge graph. These tools are things like make sure person A and person B are actually the same person because a lot of-- you know, there's a lot of Mikes in every company we work

83. 17:50

    with, and Claude gets very confused by this. Um, a second thing might be something like, um,

84. 17:58

    uh, identifying, identifying redundancy cycles or, uh, like, violations of your DAG inside of your knowledge graph. And so in our RL environment, we train really good tools to do a good job of traversing this graph to extract the right context.

85. 18:11

    So that's how we solve the two problems, writing good analysis from the context and extracting the correct context in the first place. And then the third part, which we are, um- Still building towards is an agent that operates autonomously, um, to do the kind of, uh, workflow management on the small things that the FDE doesn't have to

86. 18:31

    waste their time on. I think that's all I've got. I'll hand it back over to Vas.

87. 18:35

    Thanks, JD. So where does that leave us? And I want to share a little more about Varick, uh, because obviously we're not the Cursor, Anthropic, or OpenAI of the world.

88. 18:44

    Um, when we started this company, we fundamentally believed that the way the puck was moving, you had to get ahead of it, and you had to start learning how a business runs and building with that in mind.

89. 18:57

    I think a lot of Silicon Valley starts to go product, product, product, but what we're building for cannot be solved for with just a product. We start off every single engagement with an audit where we actually send our forward-deployed engineers, strategists into a company to learn how it works from the inside out.

90. 19:13

    That is what I believe is the biggest bottleneck. And after that, we go into implementation. We build agents on top of our platform. And yes, you do still need all the bells and whistles and the fancy technology that allows us to, you know, really automate work, uh, in the future.

91. 19:27

    But again, the bottleneck is the forward deployed motion, which is why we are so bullish here [REDACTED:username] on our AI FDE. Um, and if you're interested in learning more about it or if you're interested in joining, uh, one of the fastest-growing startups in Silicon Valley, working with some of the largest clients on the planet, uh, come

92. 19:45

    find us after and we'll have a chat because we are aggressively hiring. Um, and if you are a company looking to understand how AI can really move the needle for you internally instead of just slapping a frontier model on top of everything and watching it break in production, come find me after as well.

93. 20:00

    Thank you all so much for the time. I really appreciate it and, uh, cheers. [audience applauding] [upbeat music]