CI/CD Is Dead, Agents Need Continuous Compute and Computers — Hugo Santos and Madison Faulkner

Talk transcript

1. 0:00

   [upbeat music] All right.

2. 0:16

   Can you all hear me?

3. 0:17

   Yes.

4. 0:18

   Great. There we go. All right. Well, we're only thirty-five to forty minutes late, but thank you for sticking around. [chuckles] Um, we're gonna talk about why CI/CD is dead, and we're gonna propose that continuous compute is gonna be the next thing.

5. 0:38

   Maybe. All right. So just quick introduction. We're gonna have two speakers. One's already-- one's getting mic'd up. Um, my name's Madison, and I'm a partner at NEA investing in technology.

6. 0:52

   Uh, I do focus in infra and dev tools, and I formerly used to be a Meta AI researcher. Uh, so I used to lead data and AI teams. I got really frustrated by the state of infrastructure, and so I jumped into venture to do something about it from the top down.

7. 1:07

   And then I'm also gonna, uh, introduce on behalf of my, uh, partner here, Hugo Santos. So he's the CEO of Namespace, which is building high-performance compute infrastructure, and at this point, what we believe is going to eclipse the new CI/CD wave.

8. 1:22

   He also formerly led microservices at Google.

9. 1:26

   Yeah. Uh, great to be here with you folks.

10. 1:28

    So we're gonna talk about why agentic software is breaking traditional CI/CD.

11. 1:34

    Obviously, we're not gonna get through this today, but [chuckles] the point is, on the left side, what started off in, in agentic software was really monolithic agents. Uh, we were really using the LLM as one engine.

12. 1:48

    But now we're moving into the right side, which is microservices, uh, with agents, and that's how we really need to think about software development in an agentic world.

13. 1:59

    So the life cycle, um, is very fragmented. This is quite a mess, right? We've, we've really kind of brought together all these traditional CI/CD systems, build, test, deploy, DevOps, um, but we also now have new IDEs, we have autonomous agentic engineering solutions, um, and then we have our traditional DevOps in the middle, which we believe is really

14. 2:20

    going to innovate in the next year. So let's explain why we think it's dead. So first, how does CI/CD pipelines work today? Well, we all know, human developers are currently submitting one, maybe a, maybe a couple of diffs when they're just writing it themselves, and those PRs then take your colleagues a bunch of time to review.

15. 2:41

    Then you have to go through GitHub Actions and run, build, test, and deploy steps. And then finally, you're addressing this failed test case, and maybe you're iterating on the diff.

16. 2:49

    So in that, in that scenario, it was really, uh, just one or two a week. So now, how do we think of this at agent scale? You've got agents using the exact same systems, but they have, you know, N number of PRs, maybe N number of repos.

17. 3:05

    Still takes a similar amount of time to verify, unless you're using review bots, which gets a little crazy. Um, and then we correct those failed cases just like we did in the p-past scenario.

18. 3:16

    So what ends up happening? With a human, pretty predictable. Um, and you've got local caches, which are often warm. With an agent, this starts to get really complicated. You have thousands of short-lived branches.

19. 3:29

    It's all trying to pull the same code base in a few different directions. You start to get to a point where merging all these different versions together is really impossible, and that is where we start to have a huge problem.

20. 3:43

    So let's look at, in real time, GitHub activity has gotten absolutely crazy. The white line here is the actual number of commits in the last couple of months, and then the number of, uh, lines added versus deleted.

21. 3:56

    I mean, this is just an unbelievable spike.

22. 4:01

    So how do we start with replacing CI/CD? Well, the starting point should be at the acceleration. So obviously, right now, I know a lot of you are struggling with very slow build, test, and deploy times for your CI/CD solutions.

23. 4:16

    This is a very common problem. Um, but where we're headed is being able to first speed that up by inserting over the existing GitHub Actions and other, um, underlying infrastructure for CI/CD.

24. 4:30

    So that cache is really gonna become the orchestration layer in this scenario, and this is really critical to do through a hardware and software co-design.

25. 4:41

    So what does this start to look like, and how does this start to e-eclipse previous CI/CD? So first, we have our intake, um, which requires ingress shaping and rate limiting.

26. 4:54

    Then we move to our cache, and this is the next big step. Um, how do we think about orchestrating and making sure we're routing to the right infrastructure? Uh, from there we can even move into agentic identity for software and thinking about, uh, retries at scale.

27. 5:13

    And then if you don't believe me, let's ask the experts. So Mitchell Hashimoto, one of, you know, the, the, uh, coolest DevRel at scale, he's also the former founder of HashiCorp, uh, wrote exactly what he would do to fix GitHub today.

28. 5:29

    And a lot of this, uh, has to do with even shutting down Copilot, thinking about how do you actually just evolve GitHub to be first in the cloud era, but second, um, actually really enabling, um, inference at scale.

29. 5:43

    And then we've got a number of other data points on the left-hand side, um, that we need to be able to serve AI and agentic users first, um, or we die. [chuckles]

30. 5:54

    And thinking about friendly code storage, uh, solutions that may also help. So there, there's a lot of frustration around existing CI/CD, but this is really just the starting point.

31. 6:04

    We've only just started to see agentic software take over. So now I'm gonna pass it to Hugo to talk more about what a real solution could look like.

32. 6:11

    Yeah. So, um, I'm fortunate, and me and my team, we, we spend a lot of time with companies today that are going from how traditional CI/CD look like into how we think it's going to look at-- into the future.

33. 6:25

    And, uh, giving a little bit of a hint, it's, it's agents all the way down.

34. 6:30

    So we work with, with companies like Fall and, uh, Zed and Ramp and many others that are really at the forefront of, uh, everything around development. And you, you probably recognize yourselves, uh, in between these two, um, bits where, uh, up to six months, humans were writing all the code very slowly, uh, and some of them actually

35. 6:51

    fairly quickly, but in hindsight, fairly s- slowly. We package, uh, all these changes in PRs. We do validation as part of those PRs. And, uh, behind the scenes, um, the machines are a little bit slow, but all of that is hidden behind the human latency.

36. 7:07

    And, uh, many of you might already be seeing a bit of what's happening today, where code generation is very cheap, work is much more continuous, and, and that kind of forces the validation to go into the inner loop.

37. 7:19

    It's okay. Um, so what you might not realize is up to this point, uh, you as a human, you are the agent. Uh, you have a stop of mind, "Here's what I'm trying to accomplish, and then I'm going through all of these phases."

38. 7:36

    Uh, okay, I start the pull request. Uh, the pull request within your team says, "Well, you didn't quite follow the right format, so go back to the beginning." You're in a loop.

39. 7:47

    Uh, now your changes, uh, are in the PR. The tests are running, they fail. You need to go and change something in the code. You're back to the loop.

40. 7:56

    Uh, a human reviewer comes back and says, "Well, you know, you didn't quite use the right API. Please go and change it." You're back in the loop. And then when you go and get your code, you're, you're finally done, and you go and merge it, uh, the merge queue says, "Well, you know, some- another colleague managed to

41. 8:12

    get some code ahead of you," and you have to go back in the loop.

42. 8:17

    And, uh, when you're at your human scale, this opportunity to merge, the time that you go from when you're working the code until the code goes into, into the repository, um, can be large, because there's only so many changes that you're doing at the same time.

43. 8:32

    But as you accelerate, this opportunity to merge is really, really important because the rate of change increases dramatically.

44. 8:43

    So, uh, we talked a, a little bit about this. Like, the, a PR, um, is, is kind of used as the unit of work, and, uh, that's what really designed for human review.

45. 8:53

    It's, it's, it's, uh, it, it expects a bit of delayed feedback. It's, it's expected to, to go into kind of discrete handoffs, where you send it over to the reviewer, and then it comes back.

46. 9:04

    Uh, CI matters because it's kind of validating the work that you're doing. Uh, it's doing things like, well, are you introducing a regression? Uh, are you compiling y- and building your code from a well-known source?

47. 9:17

    Uh, are there other changes that are going on that would be conflicting with this change? Uh, is this change allowed? So all of that is kind of part of this validation process that is automated.

48. 9:27

    Um, human reviewers are overwhelmed. Um, you've heard this many times. I don't have to repeat it. And the interesting thing is that this, the, the act of merging, um, is starting to look a lot like, um, high-performance, uh, uh, database problems, where you have serialization, and you have a single ledger where every single change needs to go

49. 9:47

    in, and you need to lock the, the database in or- in order to be able to commit. And the time that you have to lock when there are humans is large, but when there's machines, it's short.

50. 10:00

    So the time to merge really matters. We need a new architecture. Uh, this is already how our team is working today and how we see a lot of the companies working today already, uh, that are at the forefront.

51. 10:13

    Uh, there are no PRs. Uh, we start with intent and plan. This is what we want to achieve, and we codify it. That's a spec. Someone writes it down.

52. 10:25

    It might be in a linear ticket. It might be on Slack. It's somewhere. Somewhere you have written down what is the goal, what are you trying to achieve. That goes into a loop, and this loop is a typical agent harness.

53. 10:39

    So it might be your, might be your cloud code, might be... We're, we're a big Amped fl- fan, so in our case, it's often Amp. Uh, it might be Cursor.

54. 10:48

    It might be Factory. Uh, you, you go into a loop, and here, uh, the agent will check out your code, and we'll start kind of moving towards the im- and implementing your plan.

55. 10:59

    Uh, very importantly, already makes use of some of these invariants. Well, it checks out a well-known commit, so it doesn't just start from, from anything, for example. Then intern- what is internal validation?

56. 11:12

    Well, it goes and uses the assets that exist in the repository to actually validate that the change is correct. So it builds it. It test- tests it. Then it comes back and tells you as a human, uh, "Hey, I've just finished.

57. 11:25

    Does it look good? Should I change something else?" And you say yes, or you say continue, like continue is probably the word that we use the most nowadays, and, and then it just goes back and continues through the plan.

58. 11:37

    Eventually, you're done, and you go into the merge queue, and, and then it goes into the ledger. So your repository, your Git repository is, is kind of like a ledger.

59. 11:47

    Uh, this is fast, but it's not fast enough because in this external validation, you still have a human in the loop.

60. 11:59

    Uh, so where do we think we're kind of moving towards? And this is in the span of weeks to months, not years. W- It's, it's a world where

61. 12:11

    generating code becomes much faster. It's already fast, but inference will only get faster. Uh, internal validation, so running your builds and tests need to be extremely fast as well, and that's where you cannot go and spend fifteen minutes running your tests or forty-five minutes or any sort of minutes because you are delaying the whole loop.

62. 12:32

    And external validation no longer has humans. We have other agents that are evaluating the changes. So you may have, um, a security, uh, focused LLM. You may have an, um, uh, API conformance, uh, based LLM that is providing feedback within the loop to the changes that your main harness is then, uh, incorporating

63. 12:57

    back into the code, and it's doing this very quickly.

64. 13:03

    Um, when it's done... And in order to do it very quickly, it actually needs to be running in a stateful environment. Memory is important, state is important because if you're starting things from scratch all the time, you're just going to delay things even further.

65. 13:17

    So the statefulness of it is really important within this loop. Um, you are getting world signals from time to time, things like, well, the plan changed, or someone else got a, a change in.

66. 13:28

    So the harness is also adapting its intent and plan, which then creates a new loop. And then when you're done, because there are so many changes going on, uh, and you haven't yet really, as a human, the team hasn't accepted this change, you don't go directly into the repository.

67. 13:47

    You go into a pre-queue, which we're starting to call a pre-merge, where there's a queue of changes that are done, that it would have been merged, uh, if w- if the process of merging was fast enough.

68. 14:01

    But the reality is that you will have so many of these running in parallel and operating on the same parts of the code base that you need a process that reconciles them so that you can have serializ- serializability, so that you actually can guarantee that all the changes go back to back into the, into, into your ledger,

69. 14:20

    into your repository. And that's the point where you get external approval. That's where the human comes in, where looks at not the code, but that the, this was the intent and this was the result.

70. 14:34

    And the result might be, here's the video of the feature working. It might be, uh, here's the o- uh, the output of the security focus LLM on, on this particular change.

71. 14:47

    And it's not on one commit or one PR. It might actually be on multiple of them. So you may even have multiple agents, uh, independently working on features that go into this pre-merge queue and semantically get grouped into something that you as a human can manage because there's going to be way too many.

72. 15:06

    We already see that today where within our team, where our, our volume of what we would call PRs from in, from the past is four times as big as before.

73. 15:16

    It's impossible for a human reviewer to look at every single PR.

74. 15:21

    Um, and if we think a little bit more into the future after this,

75. 15:28

    if this process is extremely quick, one thing that may end up happening is that you may have to step into the multiverse where, uh, the starting point where the intent and plan gets applied is not the tip of the ledger.

76. 15:46

    It's not the latest commit, uh, that... of your repository because that is moving. There's many candidates. So the agents may actually be working on multiple commits at the same time to address the same plan.

77. 16:00

    And in order to get that, uh, to get there, this inner loop needs to be extremely quickly, uh, ex-extremely quick, and, um, it adds up in terms of capacity.

78. 16:09

    So resource usage will also blow up because of all of the candidates that you're going to be exploring at the same time. This is the world that we think that we're moving towards.

79. 16:19

    Uh, we're, uh, obsessed about performance and efficiency, so we're, uh, spending a lot of energy finding ways to maintain efficiency within this loop, and part of it is, uh, well, don't do work that is not necessary.

80. 16:33

    Don't start things from scratch all the time. Uh, have agents work a lot more as we did as engineers that in our own work sessions that were much more incremental, and, and that's kind of the world that we're moving towards.

81. 16:49

    Uh, did CI go away? Well, CI still matters, but it's just shifted because the principles of, uh, well, for example, does, does the code actually work, no longer is a separate phase, but it's just part of this loop.

82. 17:06

    Every single iteration is going through validation now. It's still going through enforcing those invariants as well. Like you still ha- want to have, for example, for compliance reasons, you still want to have guarantees that you're starting from a well-known, uh, checkout, that you don't have someone in the, in the, in your company that came in and added

83. 17:26

    other code that was never vetted, and you're starting from there. So those invariants need to still be enforced, but they're enforced on a continuous basis. Coordination moves away from CI.

84. 17:37

    So CI no longer has to, uh, kind of guide different changes and making sure that different tests are passing in order for changes to be committed. That needs to be part of the overall loop.

85. 17:48

    And governance is still important, uh, but it also gets much more lifted into the harness and how the harness is, uh, uh, coercing the change towards following everything that your team has codified, um, within these processes.

86. 18:03

    And that's it. This is where we, we, we believe that the world is moving towards. Um, if you're interested about this topic, uh, us at Namespace, um, uh, spend a lot of time thinking about it.

87. 18:12

    Uh, there's other folks in the industry as well. Uh, it's a crazy world, and we need to be ready for it. Uh, thank you, and, uh, yeah, let's go for lunch. [audience applauding] [upbeat music]