Software Engineering Is Becoming Plan and Review

Talk transcript

1. 0:00

   [upbeat music] Is this mic on?

2. 0:16

   Yes, this mic is on. How are we doing? [cheering] Woo! Fucking fantastic. Yeah. Let's go. [clapping] All right. Um, this, I've ... This is ... So the, the title of this is, is about planning and review, but I think the real point behind this is, like, basically what are we all gonna do after AI continues to get really, really

3. 0:37

   good. Uh, I'm Louis. I'm the founder of a startup called Vibe Kanban, and I also started the London chapter of AI Tinkerers, uh, which is a great community, um, if you're in London looking for events.

4. 0:52

   And you should listen to me because I have done some stuff, like get on the SWE-bench Verified leaderboard ahead of OpenAI. This is a couple of months old now, but anyway, you know, it's always nice to know that the people talking have done some research in the space.

5. 1:10

   Um, the agenda for today class is we're going to ... Uh, I'm gonna walk you through why I have arrived at the conclusion that basically all software engineers are gonna do all day is plan and review stuff.

6. 1:23

   And I'm gonna talk about how to think about balancing that if that is what you do all day. Uh, I'm gonna talk about time horizon and how agents are getting, uh, running for longer, and how that changes the behavior of the job.

7. 1:39

   And then at the end, we're gonna shut my company down, and we'll get onto that later on. So let's get started. Everything is plan and review. So work that we software engineers do.

8. 1:51

   Who's ... Everybody in here is a software engineer, right? Yes? Okay. Most people. Um, we plan stuff, we write code, we review code, and we review other people's code.

9. 2:01

   Roughly the work that we do. The ratios until GitHub Copilot hit the scene were roughly this for me. I know it depends on whether you work in a big company or a small company and things like that, but a lot of it was writing code, and not very much of it compared to that was planning and reviewing

10. 2:20

    code. And what we see is over time, with things like the first version of GitHub Copilot, that basically the writing code part starts to shrink. So, you know, ChatGPT arrived.

11. 2:32

    Suddenly, you know, you can like generate functions and paste them in. And then Cursor, not Cursor today, but like the original version of Cursor arrives, and then it's like able to complete a whole page of code.

12. 2:44

    Um, and then you get Claude Code, and it's like, wow, you know, I actually am not really doing much code writing anymore. Um, so it kind of poses an interesting question though, which is what ...

13. 2:55

    You know, say we were spending four hours a day coding before, does that mean I now get four hours back if I'm not doing any actual coding? The answer, of course, is no.

14. 3:05

    It has displaced work. Work that you ... Or time that was previously spent doing the coding has moved. It has moved to planning and reviewing. I think it is an accelerant.

15. 3:16

    You're probably getting more done in the day, but it's probably like, you know, you get, uh, 20 minutes back for every half an hour that you were, you know, spending coding, and some other time has gone to planning and reviewing.

16. 3:30

    So I wanna talk about that. Like, what is this new way of working? And I think there's, there's fundamentally two approaches that people take. I'm not gonna get too specific about, you know, I don't know, specs or, uh, Playwright MCPs, or things like that.

17. 3:44

    I think there's b- tons of fascinating talks about that at this event. I just wanna kind of conceptually ground what we're talking about here today. So the first way is the plan-based approach.

18. 3:56

    Um, this is where you spend a lot of time upfront planning the work that you want a coding agent to do. So the smells of whether you are doing this type of work would probably look like you're writing a very comprehensive plan doc, markdown file.

19. 4:12

    You're maybe using like one of these spec frameworks. You're, uh, interrogating. So, you know, I've seen some cool stuff where the model asks you questions repeatedly until it's like completely exhausted all possible questions it could have about what the work is that needs to be done.

20. 4:28

    The benefits of this course are that you basically spend less time reviewing that work. Because you have invested time upfront, uh, eliminating edge cases, giving the, the, the models as much information as possible about the work you're trying to do, the outcome of that will be that it ...

21. 4:48

    the model's less likely to fuck up, and you're gonna get like better, better outputs and probably, you know, fewer rounds of review. The downside is you have to spend more time planning, but, you know, that's just obvious, isn't it?

22. 5:00

    The other way of doing this, uh, the, the other big way of working with AI is you don't define a very detailed plan, but instead, uh, you let it, you know, run, and then you s- you end up spending more time reviewing that work.

23. 5:17

    So, you know, benefits of this, you can just YOLO something, be like, "Ah, let's add a contact form to the webpage." And then, you know, the, the, the payback you have to do is like you're gonna go back and forth a few times, correcting the styles, figuring things out.

24. 5:32

    Um, I would say if you think about the valuable thing being your human time and you have a choice, you always want to be doing the first of these behaviors, the planning, the planning approach, basically, because it will save you a lot of time.

25. 5:48

    It is very time-consuming to have to switch back and forth with an agent that is like giving you some half-delivered work, and you're constantly having to review it. I think another way of breaking down the modes of work, where one is plan and one is review, is to think about the type of thing that you're working on.

26. 6:08

    And I think feature development is actually very different from migrations and, uh, maintenance work. So, and front-end is very different from back-end. So I was tr- I was trying to kind of think about this before the talk, and this is the matrix I've come up with, where basically if you're working on, uh, the front end and you're

27. 6:24

    doing feature development, it's basically impossible to kind of really spec everything out. There's so many edge cases, ev- You know, front ends are very stateful. Uh, there's, like, interactions, animations, styles.

28. 6:35

    There's functionality. And so personally, like, I find it much better to kind of be in the loop with a coding agent, so the second, uh, one of those behaviors that we talked about.

29. 6:46

    Uh, but for everything else, I think it really is possible to be plan heavy. So back-end feature development, you can almost do test-driven development, and for anything like refactoring and migration based, you certainly, uh, you certainly can be doing that, and you shouldn't be in the loop with any of that work at all really.

30. 7:04

    That should all be kind of test-driven, um, development.

31. 7:09

    So the ... I guess if you had to distill that long, meandering spiel into a sentence, it would be, spending five minutes of planning saves you thirty minutes of reviewing AI-generated code, and that's basically the takeaway.

32. 7:26

    Uh, the other thing that I think is kind of interesting to consider is, is how things are running for longer. So as coding agents become more capable, so as models get better, as tool calling improves, you go from calling a very small set of tools to now, you know, the, the coding CLIs can call a, a huge

33. 7:47

    range of tools and do testing and things like that. The outcome of that is that every time you send off a prompt, you are waiting longer before it comes back to you and says, "Hey, Louis, time for you to do something."

34. 8:01

    So to illustrate this, I mean, you know, like, think back to GitHub Copilot. It completes a single line of code, and it takes seconds. Then you have, like, the original Cursor completing a single file, and that rou- runs for, you know, thirty seconds.

35. 8:15

    And then you have, uh, Claude Code, which, you know, last year would run for maybe a minute or two, and this year I've, I've been, you know, getting some pretty good results with five- or ten-minute executions.

36. 8:25

    And that is gonna continue, because basically we've gone from, uh, you asking the AI to do something and it just responds, to the AI running a type checker, to the AI testing its change.

37. 8:39

    I mean, this is, like, the frontier of things, and these things take increasing amounts of time. Just returning the code was really quick. Running the type checker's a bit slower than that.

38. 8:49

    Running Playwright MCP is an order of magnitude slower than any of those things. Um, but it's worth doing because, you know, what you're trying to maximize for, again, is how much time you are spend- or minimize, rather, is how much time you are spending working with the agent.

39. 9:05

    So, you know, if you can get higher accuracy by waiting longer, um, that is a, a, a worthwhile trade-off.

40. 9:14

    And this is, like, where the frontier probably is, is like, you know, if I had to forecast where we'd be at in nine months' time, I would say basically AI starts to be able to QA front-end work, and that's gonna be a huge breakthrough.

41. 9:27

    You see some cool demos of this on Twitter with, you know, Chrome or Playwright MCP clicking around on stuff. The reality is, I haven't met a single person who actually does this in their, in their mainstream development.

42. 9:35

    But I'm really excited for it, and I think it will be the next major breakthrough where essentially most of the back and forth that you do with a model is gonna just be done by the model itself, because it'll be able to actually run your project, click around and find the bugs, and make sure it's done it.

43. 9:51

    But this poses an interesting question, which is like, what happens when the average time that an agent is running for exceeds, say, five minutes? 'Cause I think five minutes is roughly the time when you can, like, sit there and wait for something, watch the logs.

44. 10:07

    Probably more realistically, like, browse Twitter, something like that. And when we cross that five-minute mark, you have to change your behavior. You know, imagine these things are gonna take twenty minutes to run.

45. 10:17

    You're not gonna sit there for twenty minutes watching agent logs. You're gonna have to think about coding and/or the job of being a software developer in a very, very different way.

46. 10:26

    Um, this is something I'm sure everybody's seen, which is, you know, this kind of terminal maxing thing where you ... You're basically parallelism, right? You run multiple of these things at once.

47. 10:37

    Say each of them take ten minutes to run. So the way you get around the waiting problem is you have multiple of them on the go at any given time.

48. 10:45

    So as soon as you've finished, say, reviewing one piece of work, another has finished and you can move on to that. And that's basically what we started working on.

49. 10:54

    So this is, uh, this is the, the, the project we started about a year ago called Vibe Kanban. And, uh, essentially it started as, uh, as an attempt to make it possible to parallelize agents very easily.

50. 11:06

    Um, we built some cool stuff. There is, uh, a sidebar where you can create multiple workspaces that run any coding agent, uh, like Codex, Claude Code, things like that.

51. 11:18

    Uh, when you wanna review the code, you get the diffs. If you wanna comment on something, you can do it, just kind of like how GitHub does. If you wanna preview something or, you know, click on something and kind of be like, "Ah, actually make this a bit bigger or that a bit smaller," you can do that

52. 11:33

    too. And you probably have seen all of this stuff before, but you may not have seen this stuff started as early as June 14th, 2025. We did it first, I swear.

53. 11:45

    Okay. So the considerations, I think, like, what ... So, so, so human behavior's gonna change because agents are gonna cross this, like, five-minute threshold, um, and who knows? That may continue.

54. 11:56

    You may end up crossing an hour threshold. So we need new interfaces to make this job awesome. 'Cause if you try and do it using the existing tools, it kind of sucks.

55. 12:08

    You have to jump around reviewing code in one thing, previewing things in another. Um, if I had a wishlist for what I would want the ultimate coding agent tool for software developers to look like, it'd basically embrace the fact that I have to be a manager of multiple streams of work at any given time, which is not

56. 12:27

    something most software developers have had to do. They've just been able to, like, lock in, in a, in a deep way to one piece of work. So it's all about kind of...

57. 12:37

    I, I've put focus maxing. I don't know if that's a word. I'm coining it. You heard it here first. Um, but it should embrace the fact that you can't pull humans out of something and back into something else every 30 seconds, 'cause it just fries their brain, and it's not, it's no way to live.

58. 12:52

    Um, so, you know, it, it should be built around, you know, getting the most out of the human so that an agent can run for as long as possible, and then yield back to the human, rather than encouraging patterns where you're constantly jumping in and out, and in and out of, uh, needing to get back into the

59. 13:10

    context of what a particular agent is doing. It should help you write tasks and plan things, obviously. It should help you QA work, because that's what a lot of the human's work is gonna be, and it should help you do code review.

60. 13:23

    I think, obviously, code review, y- you know, a lot of it's being done with AI, but very few companies with money on the line are actually gonna ship stuff that's fully Vibe coded without actually checking the code.

61. 13:32

    So reading code is probably something most people in this room are gonna still have to do. And then shepherding the change until it's deployed, which is kind of a new emerging one.

62. 13:40

    So, you know, at its simplest form, it's like monitor GitHub pull requests and just look for comments and kind of be reactionary to those automatically. A lot of the admin involved in getting something from, "I've finished the task," to, "I've deployed the task," is just literally, like, you know, following comments and, and reacting to them.

63. 14:01

    So [laughs] this... I wrote, I wrote this talk. I, I, I submitted this talk a few, a few weeks ago. And on Tuesday, I decided to shut the company down.

64. 14:14

    So I had a whole... Basically, there's a whole part of this talk which was just me telling you more about Vibe Kanban and trying to sell it to you, but that's not gonna happen anymore because now the company's shutting down.

65. 14:25

    So what I thought I'd do instead is we can actually shut the company down together. [laughs] I haven't, I haven't actually announced it yet. [clapping] So [laughs]

66. 14:37

    okay. And we're gonna do it using Vibe Kanban, of course. It has to be, you know. Okay, so, uh, please add a blog post to the website with this content.

67. 14:51

    Uh, and I've pre-written the, uh, you know, the weepy, the weeping note. Okay, so we've got Vibe Kanban website.

68. 15:01

    All right, that's gonna go and do it. I can give you a little tour of this thing as well. So it's running a setup script. Well, it's created a Git worktree.

69. 15:07

    It has run a setup script in the worktree to install the dependencies for our website. And once that's done, it proceeds to, uh, run whatever agent you've selected. I use, uh, Codex most of the time, but it supports eight of the most popular ones.

70. 15:24

    Um, and it's gonna go ahead and try and figure out how to do that. Um, what else is cool?

71. 15:31

    Are you sad?

72. 15:32

    Am I sad? I think I've just done so much thinking about it, I'm kind of relieved. [laughs] Like, you run a, you run a company for, for a few years, and you have this, like, enormous responsibility to kind of...

73. 15:45

    You know, you have staff and investors and all this stuff, and, and... But I don't know, I feel like kind of a weight has been lifted almost. Um, I...

74. 15:54

    We can talk through, like, why we're shutting it down as well. We have, we have lots of u- We have 30,000 monthly active users and, and 25,000 stars on GitHub, and actually the project will continue non-commercially.

75. 16:05

    Um, and we're already pushing changes even though we're, we're shutting things down. But it's actually very difficult, uh, to make money in the current environment. Uh, the... Everybody who is making money is doing two things.

76. 16:16

    They're selling to enterprise, and they're reselling tokens, and we were doing neither of those things. We're not a coding agent. We have a button that helps you run something in Codex or, or Claude Code.

77. 16:26

    And so people can... We, we have a subscription. People would spend, like, $30 with us and then press a button that helps them spend $3,000 with Codex. It's just, like, not sustainable.

78. 16:36

    Um, and all of our users are, like, individuals, startups, uh, smaller companies, and we could have done the work, I think, to address that and kind of move up into the enterprise, but, you know, I don't know.

79. 16:49

    It's a, it's a kind of... It's, it's a mature market at this point, and it's no fun playing for eighth place. So we decided to shut things down.

80. 17:00

    Uh, okay. Blog post is ready. Let's see if it works. So we've got the live preview feature, obviously. Just wait for it to compile.

81. 17:14

    Okay. That looks good. So we can go ahead, open a pull request.

82. 17:22

    Eh, uncommitted changes. Oh, uh, don't know what that's about.

83. 17:33

    Eh.

84. 17:36

    Are we finding this out before your investors? [laughs]

85. 17:40

    Oh, shit. [laughs] [laughs] Uh, you're not. Don't worry. [laughs] Uh, they most... Well, actually, some of them, I need to, I need to call them after this. That's a really good point. [laughs]

86. 17:56

    I'm fully committed to shutting this down live on stage. [laughs]

87. 18:02

    All right, it's done. That's gonna go through Cloudflare's CDN. [laughs] [clapping]

88. 18:11

    Thank you. Um, I think we've got time for one or two questions. I don't know if anybody in the audience has a... One, one minute 45.

89. 18:18

    What's next for you?

90. 18:20

    What's next? Uh, take some time off, start another company. My co-founders are gonna join a lab, and most of the team have already found good jobs at agent labs, things like that.

91. 18:31

    Yeah. Any other questions?

92. 18:33

    Overall feeling when you look back is positive or negative on-

93. 18:37

    Oh, yeah. No, I wouldn't, I wouldn't do anything differently. I think, like, uh,

94. 18:43

    it, it, it i- it was, like, the most interesting thing I've worked on, and, uh, it's right at the cutting edge of, like, agents and all of this stuff.

95. 18:51

    I think certainly... I don't know. It's increased my value as a human by doing this, so I would do it all again. Yeah. Go for it.

96. 19:01

    What are the most valuable things you've learned from these few years running a company?

97. 19:06

    Uh, most valuable things. I mean, just worked with great people. You know, we went through several rounds of the team, uh, and the team we ended up with was, like, phenomenally better.

98. 19:18

    No, no offense to previous team. But that's probably how to, you know, I think... And, and hard work as well. I think, like, it took us a while to learn, like, what hard work really was.

99. 19:29

    Um, and you get to a point where it's like you're sitting there at midnight with the, you know, the team in the office on a Saturday, and everybody's kind of motivated, and that's...

100. 19:40

     Yeah, it takes you a while to kind of figure out how to get to that point. And once you feel it, you know kind of what that's like. It's difficult until you get there, I think.

101. 19:48

     Yeah. Uh, 12 seconds. No? Okay.

102. 19:53

     If you go back to the past, uh, what would you change?

103. 19:57

     What would I change? I'd, I'd, uh, I'd hire somebody who's really good at selling to enterprise. [laughs]

104. 20:03

     All right. Thank you very much. [clapping] [upbeat music]