Don’t get one-shotted: Use AI to test, review, merge, and deploy code — Tomas Reimers, Graphite

Talk transcript

1. 0:00

   [upbeat music] Awesome.

2. 0:15

   Uh, hi, everyone. My name is Tomas. I'm one of the co-founders of Graphite. Graphite is an AI code review, uh, company. So to give some context on sort of where we see the industry right now and where we see it going.

3. 0:28

   Software development currently, and has always had, two loops. The inner loop, which is focused on development, and the outer loop that's focused on review. Developers spend time in the inner loop.

4. 0:39

   They get their code working. They get the feature the way they want it, and then they go ahead and they move it to the outer loop, where it's tested, reviewed, merged, deployed.

5. 0:48

   We're seeing the inner loop change right now more than we've ever seen it. More developers are using AI than ever. I think right here we have some statistics from the GitHub Developer Survey.

6. 0:56

   Nearly every developer surveyed used AI tools both inside and outside of work, and forty-six percent of GitHub is being written by Copi-- uh, code on GitHub is being written by Copilot.

7. 1:06

   We're seeing more and more code being written by AI. Here we have some statistics around how code has changed over time and how some people predict it will change.

8. 1:16

   And even if we take a more pessimistic view of that, we still see the way the world's going as just more and more and more code being written by AI.

9. 1:26

   The inner loop is changing, you know? AI is making more, uh, developers more productive. Developers are now producing higher volumes of code, but that code still needs to be reviewed.

10. 1:38

    When we first started looking at this, when we first started building, uh, Diamond, our AI code reviewer, about a year ago now, what we found was we read a lot of articles that scared us a lot.

11. 1:48

    We were seeing within our or-own organization a lot of developers adopting AI tools, but we were also seeing a problem. AI can hallucinate, it can make mistakes, and almost more scarily, it can make security vulnerabilities.

12. 2:00

    For us, what we saw was that while the inner loop was getting sped up by AI, the outer loop was rapidly becoming the bottleneck. Um, we were seeing tools like Cursor, Windsurf, Copilot, v0, Bull, all of those producing larger volumes of code than we were used to, than we'd ever seen before.

13. 2:17

    But we were also seeing our developers suddenly have to review higher volumes of code, test higher volumes of code, merge higher volumes of code, and deploy higher volumes of code.

14. 2:26

    That's what brought us to say, "There has to be a new outer loop here." This-- The way that things are going, this isn't going to work. We're going to break down.

15. 2:35

    We're watching the problems that used to only ail large companies start to ail all companies, where we were seeing companies deal with higher and higher and higher volumes of code.

16. 2:44

    The requirements for the new outer loop then look a lot like the problems that larger companies have always had to deal with. You need tools to better prioritize, track, and get notified about pull requests.

17. 2:54

    You need driver assist features to help reviewers focus and streamline the code review process. You need optimized CI pipelines and merge queues to be able to handle the sheer volume of code changes that are now happening, and you need better deployment tools.

18. 3:06

    So, um, when we first started looking at this through sort of an AI-first lens, we started to see that, well, the problems are being created by AI, they can also probably be solved by AI.

19. 3:19

    We can probably start to streamline a lot of these processes which have previously had to been manual, previously were parts of the process that developers did not enjoy, did not want to do.

20. 3:27

    Um, we wanted to see self-driving code review solutions where we no longer had to do those very manual and painful parts of review, but we could actually start to really focus on what matters most to the developers, making sure that your product can get out to users and that the features work as expected.

21. 3:41

    Um, we were seeing that AI-generated feedback wasn't perfect, and because of that, we were starting to think that bots weren't enough. I think an early, an early vision of ours was, well, can we solve this by just adding AI teammates, right?

22. 3:53

    Maybe it's background agents, maybe it's reviewers, maybe it's a whole lot of teammates to the workflow. And while we think that's part of the story, we don't think that's enough.

23. 4:00

    We think that, as we've built with Diamond, that your entire tool chain has to be AI native, not just your IDE. If you really are going to embrace AI in the age of development, if you're gonna accept the fact that developers are gonna be orders of magnitude more productive than they ever have before, you need tooling that

24. 4:15

    reflects that. We started by building Diamond, so the winning AI code review platform, with high signal, low noise, has a deep understanding of the code base and change history.

25. 4:26

    We summarize, prioritize, and review each change, and we integrate with your CI and your testing infrastructure to correct, uh, to summarize errors and correct failures. Um, our hope with it and what we've started to see as we've rolled it out to larger and larger customers and enterprises too, is we reve-- we reduce code review cycles, we enforce

26. 4:45

    quality and consistency, and we keep your code private and secure. Um, it's high signal, it's zero setup, it's actionable with one-click suggestions, and it's customizable. It's already being used by some of the fastest moving companies in the world.

27. 4:57

    It's expanding a lot more than we can even say publicly, um, a-and I hope that you all will embrace the idea that AI can change your entire developer workflow, not just your IDE.

28. 5:08

    Um, by the numbers, we see comments that our AI bot leaves to be downvoted at less than a four percent rate and to be accepted, meaning integrated into the pull request, um, that they were left on at a higher rate than human comments are.

29. 5:20

    Human comments are integrated about somewhere between forty-five and fifty percent. We're watching our Diamond comments be accepted about fifty-two percent. We've spent a lot of time tuning that. That's, that number is actually new as of March for us.

30. 5:30

    Um, that's, that's what I have to tell you around Graphite, um, what I have to tell you around Diamond. I hope you give it a shot, and, and thanks for having me. [upbeat music]