How Forward Deployed Engineering is done at Ramp

Talk transcript

1. 0:00

   [upbeat music] Awesome. Thank you, guys. Awesome. It's great to, to meet everyone.

2. 0:16

   I mean, I hope that, uh, after the talk, you know, if you wanna come out and we can chat, um, would love to. Um, cool. So yeah. Today, my goal is to share with you guys the two most important principles from what we learned doing FDE at Ramp.

3. 0:32

   So just, yeah, briefly a little bit about myself. Yeah. I'm a director of engineering at Ramp. Uh, I joined the company two and a half years ago when it was just, you know, FDE was just two engineers at the time, and today my org is about 30 engineers across four deployed developer API and our new AI services,

4. 0:54

   um, business. So I know this is kind of a running theme, but like, no one knows what FDE is, so I'm just gonna spend a moment on that. [laughs]

5. 1:06

   So yeah. I- I- I actually kinda like this meme. It's g- To me, it's kinda funny. Um, I, um, but I- I actually think it's, like, totally wrong. I- I don't see this as the actual, like, true form of what FDE is.

6. 1:19

   Um, I don't see it as, like, the final evolution or, like, boss mode of technical go-to-market roles. Um, now this might be true at some companies, but at least at Ramp, it's a little bit different.

7. 1:35

   So FDE at Ramp, uh, we live within the engineering organization, and our goal is to help Ramp win up market. So with that in mind, what we do is we basically work on the core product and our new agentic features and make them work really well for our largest enterprise customers.

8. 1:59

   So that's just a little bit of intro context. I wanna dig in and, and today, like I said, there's just two things I'm gonna share with you. Literally two things.

9. 2:07

   Very easy talk. And these are the principles that I would say have really guided us, and I would say probably the two most important things that we have: always be scoping and scale with tokens.

10. 2:21

    So let's get-- Let's start with the first one on scoping.

11. 2:25

    So I would say there's this thing where, like, peop-- many, many people think that as an FDE, your job is to just say yes to the customer, but that's wrong.

12. 2:36

    If you were just to say yes, you know, instead of, like, beautiful Waymos that we have driving us around in San Francisco, you'd have something like this, you know?

13. 2:45

    Yeah, hor-horses with, like, rockets strapped to their legs. And the point is, you, you want to help the customer be successful. You want to try to figure out a way to say yes, but you actually wanna deliver good software.

14. 2:59

    You need to build the right thing, so you don't just endlessly say yes to people. And I, I, I do wanna share, uh, an example of, um, something that I would say happens somewhat regularly in one form or another at Ramp.

15. 3:15

    So it's Friday night, and an enterprise sales rep comes to us with an urgent request that this super important strategic logo is only gonna close if we build out an SAP S/4HANA integration.

16. 3:32

    And I think that the default engineering reflex is like, "Shit, like, what are the SAP API docs? Like, where do I find them, and how do I build this integration?"

17. 3:43

    But what an-- what a well-trained FDE would do is, like, pause for a second and say, "Okay, first of all, like, what's driving the urgency here?" Like, one thing I've seen is I've seen sales reps who, like, go kinda crazy because it's, like, the end of the quarter and they're trying to hit their quota and close the

18. 4:00

    deal and not because the customer is the one driving the urgency. So that's, like, one example. [clears throat]

19. 4:06

    But, you know, as an FDE, you're asking tons of questions to gather context about what's important, um, and what actually is the right thing to build. And so you might ask, like, "Who's using this integration?

20. 4:21

    Have we exhausted all the different workarounds? Is there something manual that we can do in the meantime? Does the customer have, like, technical resources? Can they hit our API such that we don't have to build this thing?"

21. 4:32

    But I'd say the most important thing that an FDE does is also looks beyond this one request and looks at the other prospects that are coming down the pipeline and other customers to see if anyone else would benefit from this as well.

22. 4:48

    And the point is that by gaining all this context, you can do a better job of building the right thing.

23. 4:57

    So I wanna share another story that was really, really painful for us in the early days.

24. 5:02

    We had this large enterprise customer, and they needed this reimbursement feature on mobile.

25. 5:10

    Unfortunately, our mobile team was totally swamped. Like, w- we basically just had to roll up our sleeves as FDEs and just get-- find out a way to get things done, and we had two of the engineers on the team just, like, learn how to do iOS and Android development, and it was awesome.

26. 5:27

    We were super excited. We're like, "Okay, we're gonna ship this feature. It's gonna be so good. Like, hell yeah." So we grinded for a couple weeks, got the feature done on both platforms, and we go to the customer, and we're like, "Awesome."

27. 5:39

    Like, "Can you send us your list of, you know, beta users for Android?"

28. 5:44

    And that's when they told us they only-- They, like, they require, they mandate all of their employees to use iOS devices.

29. 5:55

    So you're like, "What the fuck?" Like, uh, not, not to the customer, you know. [laughs] Just internally. [laughs]

30. 6:00

    But like, obviously it was super disappointing for us because we'd put all this effort in. And so it was a, a big lesson for us to remember the importance of scoping.

31. 6:09

    Even some of the most basic assumptions like which, you know, mobile platform you build on, it's, it's super important, um, to validate them and, and thus kind of emphasizes the importance of scoping upfront.

32. 6:20

    Now, okay, so let's say that you and your team have become masters of scoping. You know, you're, you're amazing. In today's world, this is not enough.

33. 6:33

    So unless you are scaling with model capabilities, you are going to fall behind.

34. 6:40

    Now, I'm not gonna belabor this point too much. I think like every talk in this, uh, in, in this conference is probably some flavor of this. But like,

35. 6:47

    the point is that we basically have to reinvent our jobs constantly now. So whatever work we are doing today, you know, for the most part it's knowledge work, we have to figure out how to have models and agents do it for us.

36. 7:01

    And so that brings me to the second half of this talk and the, the other point that I wanna convey today, which is all of us have to figure out how to scale with tokens.

37. 7:13

    And the way that I interpret scaling with tokens for FDE is take a look at the whole life cycle of what an FDE does. From gathering context, to scoping out a request, to writing out a spec and then implementing the feature, each stage of that pipeline can be replaced with agents.

38. 7:33

    And at first it seems kinda daunting. You're like, "How..." like, "How are you gonna go and approach and like solve that?" But if you break the problem down and then make progress on it, it's, it's actually pretty tractable.

39. 7:44

    And so I'll share, share with you guys one example of something-- oops, something that we, um, that we've done at Ramp. So we have this internal Slack channel called FDE Requests, and this is where account managers, solutions, uh, sales reps will post whenever there is a blocker for a prospect or customer that's large enough, basically.

40. 8:09

    And so we get these requests. In this case actually, uh, one of the CSMs on our team, Greg, posted here. And, um, if you were to-- It's actually a Notion workflow.

41. 8:20

    If any of you work at Notion, by the way, thank you. We like use Notion so much. Um, if you were to click open in Notion, you'd see like a pretty long request that has all the details of what, what exactly it is.

42. 8:31

    And the problem is there's a super high variance. Like some people will submit like really detailed, good requests from the customer, and others are just gonna submit like one line like, "We need, uh, you know, we need this SAP integration." [chuckles]

43. 8:46

    And before what would happen is we would have FDEs manually kind of go through this request. We'd read the whole thing, understand it, figure out what exists in the product, do a bunch of back and forth with the customer, and this is like exactly what the first half of the talk was about, always be scoping.

44. 9:04

    You know, we would spend a lot of time really digging in and validating what exactly was, uh, you know, absolutely necessary. And so you can see here, what we, what we did then was we basically, um, used Notion, uh, Notion agents to build a V1, which literally just took the request and asked a couple of questions.

45. 9:24

    That was it. And, um, after... It was kind of astonishing. Literally, after a couple of weeks, we found that it was like saving us a lot of time because first of all, immedi-- like the latency of replies went from like hours or days to like, you know, seconds.

46. 9:39

    And immediately, like the account reps, the, uh, the, the account managers, the reps would start kind of engaging with this agent. And one of the things that we did was because it went so well, this, this is actually, um, a more recent iteration of it.

47. 9:51

    It's very cute, you know. The little penguin actually helps make it seem a little more friendly and approachable. Um, and what it does is it actually goes and does several rounds of back and forth questioning with the submitter until it deems that it's ready to create a li- uh, a spec basically.

48. 10:09

    And it's actually been incredible how helpful this has been for us. I, I would say it's probably saved us like a large percentage, I don't know, twenty percent of the time that we'd spend on scoping out these requests.

49. 10:22

    So, you know, this is, this is a great example. Or for us, this has been really helpful. It's-- I'm super excited about this. It's gonna help automate a lot of the work that we've been doing manually.

50. 10:32

    But, um, it's really just the first stage of this pipeline that I was alluding to. So if you look at the first part here, like we've been able to make some progress on it.

51. 10:44

    The last step as well, going from a, a well-shaped sc- uh, spec to like a working product, obviously like frontier models can like one-shot medium-sized features, and so the last part is also e- is, is a lot easier for us.

52. 10:56

    It's this middle part that I would say is super like gnarly and like unformed and difficult. And I'm, I'm really excited about our team kind of investing a lot more and spending a lot more of our time just like building out this factory, building out agents to replace each one of these steps.

53. 11:14

    And the thing is, if you look at-- if I were to say six to twelve months from now, like what does FDE at Ramp look like?

54. 11:24

    Like these are the sorts of applied AI problems that we're gonna be spending all of our time on, I think. Like, you know, making sure that the agent harness that's running each of those steps is running super smoothly.

55. 11:36

    Um, making sure that the, the output quality of each o- of the outputs of the pi- the pipeline is, is actually good, that, you know, with, with evals, with rubrics, with human feedback.

56. 11:48

    Um, and there's of course like one of the biggest challenges, which is getting your agent the right context, you know, when you're making the LLM call, ensuring that it has the right context.

57. 11:57

    So there's like a lot of historical data, data about the p- the product. Imagine like all the knowledge that a product manager has in their head about their product, like how do you get that into an agent?

58. 12:08

    Like Notion docs and all your existing knowledge base and help articles only give you so much of that. Um, yeah, skills, memories, tools, I could go on for a bit, but ultimately- The most important thing here is that as an FDE, we, we still have the responsibility of taste and judgment over the final output.

59. 12:28

    So that's gonna be, like, the underlying kind of through line.

60. 12:33

    Okay, so let's say that you've done an amazing job building out this factory, but the problem is, and to tie this to the first half of the talk, if you don't do a good job of scoping out requests or, or building upon the principles of scoping things well,

61. 12:53

    you're gonna get a token maxing slop cannon.

62. 12:56

    And so the whole point is that you have to do these both because the other way around is actually quite bad as well. If you are, you know, amazing at scoping, but don't invest in building out this, you know, agent factory, you know, it's gonna be over for you. [laughs]

63. 13:15

    Like, uh, your, your agent native competitors are just gonna overtake you and out-compete.

64. 13:22

    And so that's why, um, in the end here, I wanna close with the, the-- The most important thing is that if you have both of these, it can set you up for success in the future, always be scoping and scaling with tokens.

65. 13:41

    The future of FDE needs both. That's all. Thank you, guys. [clapping] [outro music]