# Don’t get one-shotted: Use AI to test, review, merge, and deploy code — Tomas Reimers, Graphite

Tomas Reimers

AI Engineer World's Fair 2025

[Talk page](https://ai.engineer/talks/H6MrR5NbTZA-dont-get-one-shotted-use-ai-test) · [Video](https://www.youtube.com/watch?v=H6MrR5NbTZA)

## Don’t get one-shotted: Use AI to test, review, merge, and deploy code

Getting a feature working is only the first loop. AI-assisted development also demands faster review, testing, merging, and deployment without losing scrutiny.

### A working feature still has to ship

[0:15](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=15s)

A developer gets the code working and the feature behaving as intended. Then comes testing, review, merging, and deployment. Faster implementation helps with the first part, but it does not remove the second. [Tomas Reimers](https://www.tomasreimers.com/), co-founder of Graphite, an AI code review company, describes these as software development’s **inner loop** and **outer loop**.

| Loop | Work | Immediate objective |
|---|---|---|
| Inner | Code, build, debug | Get the feature working |
| Outer | Test, review, merge, deploy | Move the change through to release |

The distinction matters because producing a change and getting it safely to users are separate workloads. Accelerating one leaves the other to absorb its output.

![Infinity-shaped diagram showing Code, Build and Debug on the inner loop, and Author, Test, Review, Merge and Deploy on the outer loop.](https://ai.engineer/api/talk-essay-media/c3a9f19ea5f5764e012b75e68e3b8a2c3608d9c6a0c9db88a8013f2476ad0ec6)

Software development’s inner and outer loops.

### More generated code, more code to scrutinize

[0:48](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=48s)

AI adoption is already changing the inner loop. Reimers points to a GitHub survey showing widespread use among surveyed developers. That evidence concerns enterprise respondents having tried AI tools; it does not establish routine use by nearly every developer worldwide, or that every respondent used them both at work and outside it. The accompanying Copilot statistic also needs its historical scope: GitHub’s February 2023 announcement reported Copilot generating an average of 46% of code for developers using it, not 46% of all code hosted on GitHub.

The forecasts that follow reinforce a directional point: even under Reimers’s more pessimistic framing, AI writes an increasing share of code. Developers consequently produce more changes, and those changes still require review.

Graphite began developing [Diamond](https://www.graphite.com/blog/series-b-diamond-launch), its AI code reviewer, about a year before the talk, while its own developers were adopting AI tools. Diamond is the historical product name used here; its review capabilities later moved under Graphite Agent. The concern was not just volume. AI can hallucinate, make mistakes, and introduce security vulnerabilities, so generating code faster does not make scrutiny optional.

### The outer loop needs capacity of its own

[2:00](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=120s)

Tools such as Cursor, Windsurf, Copilot, v0, and Bolt were producing larger volumes of code than Graphite’s developers were accustomed to handling. That increased the work at every subsequent stage: reviewing, testing, merging, and deploying. **The outer loop was becoming the bottleneck.** Problems once associated with large engineering organizations were beginning to affect smaller companies as well.

The requirements follow the work that now has to scale:

- **Pull-request coordination:** Prioritize changes, track their progress, and notify people when attention is needed.
- **Reviewer assistance:** Help reviewers focus and streamline the review process.
- **Integration throughput:** Optimize CI pipelines and use merge queues to handle the increased volume of changes.
- **Deployment tooling:** Improve the final stage that gets accepted changes to users.

These are requirements for the wider workflow, not just for an AI reviewer. Deployment is part of that scope, though Reimers does not describe a deployment implementation.

### Adding AI teammates is only part of the solution

[3:06](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=186s)

The same technology creating pressure on review could also reduce its manual workload. Reimers’s goal for self-driving code review is to remove painful, repetitive parts of the process, leaving developers more attention for getting products to users and ensuring that features behave as expected.

But AI-generated feedback is imperfect. Graphite’s early idea—add AI teammates to the existing workflow—therefore became only part of the answer. Background agents and automated reviewers can contribute, but the tools around them must also support the increased workload. If developers become orders of magnitude more productive, as Reimers anticipates rather than measures here, **the entire toolchain needs to accommodate AI-assisted development, not only the IDE**.

![Large white statement about making the entire toolchain AI-native on a black and blue slide, with the speaker inset below.](https://ai.engineer/api/talk-essay-media/19079cd6f6c2e44b071468d6d6a9440340333e55031cf98aa18730485e5033fa)

“Your entire toolchain should be AI-native—not just your IDE.”

### Diamond’s place in the review workflow

[4:15](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=255s)

Diamond is Graphite’s starting point for that broader toolchain. Reimers describes a review platform designed for high signal and low noise, with an understanding of both the codebase and its change history. Its proposed workflow combines several responsibilities:

- **Change context:** Summarize, prioritize, and review each change.
- **Review relevance:** Use codebase and change-history understanding to produce useful feedback.
- **CI and testing:** Integrate with existing infrastructure to summarize errors and correct failures.

The intended scope extends from understanding a pull request to helping resolve problems exposed by its checks.

![Slide listing high signal and low noise, codebase and change-history understanding, change review and prioritization, and CI and testing integration, beside a code-review suggestion screenshot.](https://ai.engineer/api/talk-essay-media/e9d9d28b0f0d5be9028b1275cdae69dfa6291ec2238eb3894efe253a6fdbc5d6)

Proposed capabilities of an AI code review platform.

As Diamond reached larger customers and enterprises, Reimers describes shorter review cycles and more consistent quality as hoped-for benefits that Graphite was beginning to observe. He also presents private and secure code handling, zero setup, one-click suggestions, and customization as product attributes. Some further expansion was not yet public. The product case returns to the broader workflow: AI should improve how changes get reviewed and shipped, as well as how they get written.

### Measure whether review comments lead to changes

[5:08](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=308s)

Reimers closes with reported comment-level results. Here, **acceptance** means that feedback was integrated into the pull request where the comment appeared. A downvote is explicit feedback on a comment, not a measured code-error rate.

| Measure | Reimers’s reported result |
|---|---|
| Diamond comments downvoted | Less than 4% |
| Human comments integrated into the pull request | Approximately 45–50% |
| Diamond comments integrated into the pull request | Approximately 52% |

The Diamond acceptance figure was new as of March and followed substantial tuning. Reimers does not supply sample sizes, measurement windows, or details establishing comparability between the human and Diamond comment groups. The useful operational distinction is between a reviewer producing feedback and that feedback resulting in a code change: the latter is the behavior Graphite was tuning for.

## Resources

- [Diamond's March 2025 launch](https://www.graphite.com/blog/series-b-diamond-launch) — Graphite's launch announcement explains Diamond's review features and the company's broader outer-loop product vision.
- [Customize Graphite AI reviews](https://www.graphite.com/docs/ai-review-customization) — Current instructions for configuring review rules, exclusions and filters, and monitoring their effectiveness.
- [Asleep at the Keyboard?](https://arxiv.org/abs/2108.09293) — A 2021 study of insecure Copilot outputs across 89 scenarios designed around high-risk security weaknesses.
- [Graphite's AI review evaluation workflow](https://www.braintrust.dev/customers/graphite) — A case study describing how Graphite uses developer actions, explicit feedback and evaluation datasets to improve review comments.

## Slides

### AI Engineer World’s Fair

[0:01](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=1s) [AI Engineer World’s Fair logo in white on black.](https://assets.ai.engineer/sites/aiecode2025/sha256/8a922a1cf431bb07fcb73741414531921d577962f991778d113f267119293a17.webp)

AI Engineer
World’s Fair

### Event sponsors

[0:11](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=11s) [AWS is named innovation partner above six platinum sponsor logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/5c01e126deb29a0326956049f95088e81f249f41969af6c5f5f435cc179018b6.webp)

INNOVATION PARTNER
aws

PLATINUM SPONSORS
Graphite
Windsurf
MongoDB®
daily™
augment code
WorkOS

### Software development has two loops

Developers code, build, and debug in the inner loop, then move working features to the outer loop for testing, review, merge, and deployment.

[0:30](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=30s) [An infinity diagram depicts an inner development loop and an outer review, merge and deployment loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/9484fec6b601314f0370f5b23e4c2b0b9642ea837cdf4eb8710bfbd6cb9145ad.webp)

Software development has 2 “loops”
The inner loop, focused on dev; the outer loop focused on review

Code
Build
Debug

Author
Test
Review
Merge
Deploy

AIE
Microsoft
a ws
smol ai
World’s Fair

### The inner loop is changing

The GitHub survey chart shows 70–82% of respondents in four countries used AI coding tools both at work and outside work. The slide also reports that Copilot wrote 46% of code on GitHub in 2023.

[0:56](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=56s) [Bullets discuss AI coding adoption and a GitHub survey chart compares reported use across four countries.](https://assets.ai.engineer/sites/aiecode2025/sha256/9d10617b4ffdc886b032f4642a65855cf233bfa021211b45a3cf67dd38dc214c.webp)

The inner loop is changing

• More developers are using AI
• Nearly every developer surveyed reported using AI tools both outside and inside of work
• In 2023 46% of code on GitHub was being written by Copilot

Almost every respondent has used AI coding tools at work

AI CODING TOOLS USAGE
Legend:
Yes, I have used both at work and outside of work
Yes, I have used at work but not outside of work
USA: 82%, 17%
Brazil: 82%, 17%
Germany: 70%, 27%
India: 79%, 20%
Axis ticks: 0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100
Have you used AI coding tools?

Source: 2024 GitHub

Interface text: AIE; Microsoft; aws; smol ai; World's Fair; AI Engineer

### AI-generated code: 2025 prediction

The chart plots the share of new GitHub code generated by AI at 27% and 46%, then extends it to Dario Amodei’s prediction of 90% by September 2025.

[1:05](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=65s) [A line chart shows points at 27% and 46%, then a prediction of 90% by September 2025.](https://assets.ai.engineer/sites/aiecode2025/sha256/48e147b9ed66fc5970a4bd12c0eb4a24394cc4b8eaa2067f8ff80f0711994fae.webp)

% of new code on GitHub generated by AI

Y-axis ticks: 25%, 50%, 75%, 100%
Timeline: [illegible] 2022, May 2022, Jul 2022, Sep 2022, Nov 2022, Jan 2023, Mar 2023, May 2023, Jul 2023, Sep 2023, Nov 2023, Jan 2024, Mar 2024, May 2024, Jul 2024, Sep 2024, Nov 2024, Jan 2025, Mar 2025, May 2025, Jul 2025, Sep 2025
Chart labels: 27%, 46%, 90%
Event labels: GitHub Copilot, ChatGPT, GPT 4, Claude 3, Claude 3.5, o1, Claude 3.7, GPT 4.5, o3, Claude 4, Dario Amodei prediction

AIE
Microsoft
aws
smol ai

### AI-generated code: 2030 prediction

The chart plots AI-generated shares of new code on GitHub at 27% and 46%, then rises toward Kevin Scott’s January 2030 prediction. Reimers calls this a more pessimistic view of AI-written code’s growth.

[1:16](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=76s) [A revised line chart extends the AI-generated code projection to January 2030, with a point near 95%.](https://assets.ai.engineer/sites/aiecode2025/sha256/ac84866e9c000a1271610864cfa2e2eeae2941b3de92cfde6202be3ad783c527.webp)

% of new code on GitHub generated by AI

Y-axis ticks: 25%, 50%, 75%, 100%
Chart labels: 27%, 46%
Timeline: Oct 2022, Jan 2023, Apr 2023, Jul 2023, Oct 2023, Jan 2024, Apr 2024, Jul 2024, Oct 2024, Jan 2025, Apr 2025, Jul 2025, Oct 2025, Jan 2026, Apr 2026, Jul 2026, Oct 2026, Jan 2027, Apr 2027, Jul 2027, Oct 2027, Jan 2028, Apr 2028, Jul 2028, Oct 2028, Jan 2029, Apr 2029, Jul 2029, Oct 2029, Jan 2030
Events: GitHub Copilot; ChatGPT; GPT 4; Claude 3; Claude 3.5; o1; Claude 3.7, GPT 4.5; o3; Claude 4; Kevin Scott prediction

AIE
Microsoft
aws
smol ai
AI Engineer World's Fair

### The inner loop is changing

McKinsey’s chart shows larger time savings for documentation and code generation than for complex tasks. As developers produce more code, that code still needs review.

[1:24](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=84s) [AI increases developer productivity for some tasks, producing more code that needs review; a chart compares task completion times.](https://assets.ai.engineer/sites/aiecode2025/sha256/138100cca4affa13db49269b446854ad48d654d55f771ff0a651dc1a0f832e3d.webp)

The inner loop is changing

• AI makes developers more productive (for some tasks)
• Developers are now producing a higher volume of code
• This code needs to be reviewed

Generative AI can increase developer speed, but less so for complex tasks.
Task completion time using generative AI, %
Legend: Without generative AI; With generative AI
Y-axis ticks: 0, 20, 40, 60, 80, 100
Categories: Code documentation; Code generation; Code refactoring; High-complexity tasks
Chart labels: 45–50; 35–45; 20–30; <10
McKinsey & Company

Source: McKinsey & Company

AIE
Microsoft
aws
smol ai

Interface text: AI Engineer World’s Fair

### AI makes review and testing more necessary

A study of ChatGPT answers to Stack Overflow questions found that 52% contained incorrect information, illustrating why Reimers sees review and testing as necessary.

[1:38](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=98s) [A research paper is shown with its finding that 52% of the ChatGPT answers studied contained incorrect information.](https://assets.ai.engineer/sites/aiecode2025/sha256/f2eae3c3209657067b98cb411a4133ea2a1fdf7a3638b51a920b033af2486477.webp)

AI makes review & testing more necessary

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions

Samia Kabir
Purdue University
West Lafayette, USA
[redacted]

David N. Udo-Imeh
Purdue University
West Lafayette, USA
[redacted]

Bonan Kou
Purdue University
West Lafayette, USA
[redacted]

Tianyi Zhang
Purdue University
West Lafayette, USA
[redacted]

ABSTRACT
Q&A platforms have been crucial for the online help-seeking behav- [illegible]

Conference on Human Factors in Computing Systems (CHI ’24), May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 17 pages. [illegible]

“Our analysis shows that 52% of ChatGPT answers contain incorrect information”

[illegible] implies the need to counter misinformation in ChatGPT answers to programming questions and raise awareness of the risks associated with seemingly correct answers.

CCS CONCEPTS
• Human-centered computing → Empirical studies in HCI;
• Software and its engineering; • General and reference → [illegible]

Despite the rising popularity of ChatGPT, there are also many increasing concerns. Previous studies show that LLMs can acquire factually incorrect knowledge during training and propagate the incorrect knowledge to generated content [9, 33, 35, 39, 56]. Besides, LLMs often generate fabricated texts that mimic truthful information and are hard to recognize, especially for users who lack the expertise [14, 21, 29]. Like other LLMs, ChatGPT is also plagued [illegible]

AIE
AI Engineer World’s Fair
Microsoft
t code
aws
smol ai

### AI makes review and testing more necessary

A study of GitHub Copilot’s code contributions found approximately 40% of 1,689 generated programs vulnerable, illustrating the security risks that make review and testing more necessary.

[1:55](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=115s) [A security study is shown with its finding that approximately 40% of the Copilot-generated programs studied were vulnerable.](https://assets.ai.engineer/sites/aiecode2025/sha256/b4da884e8eb00d63fa185d14b30554a02acc4a51b757f54ba3542cd9f11abcff.webp)

AI makes review & testing more necessary

Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions

Hammond Pearce — Department of ECE, New York University, Brooklyn, NY, USA
Baleegh Ahmad — Department of ECE, New York University, Brooklyn, NY, USA
Benjamin Tan — Department of ESE, University of Calgary, Calgary, Alberta, CA
Brendan Dolan-Gavitt — Department of CSE, New York University, Brooklyn, NY, USA
Ramesh Karri — Department of ECE, New York University, Brooklyn, NY, USA
[redacted]

Abstract—There is burgeoning interest in designing AI-based systems to assist humans in designing computing systems, [illegible] producing 1,689 programs. Of these, we found approximately 40% to be vulnerable.

Index Terms—Cybersecurity, Artificial Intelligence (AI), code generation, Common Weakness Enumerations (CWEs)

I. INTRODUCTION
With increasing pressure on software developers to produce [illegible]

[illegible] systematic examination of the security of ML-generated code. As GitHub Copilot is the largest and most capable such [illegible]

[illegible] together with testing practices and security tools, as well as your own judgment”. Our work attempts to characterize the tendency of Copilot to produce insecure code, giving a gauge for the amount of scrutiny a human developer might need to do for security issues.

We study Copilot’s behavior along three dimensions: (1) diversity of weakness, its propensity for generating code that [illegible]

“We found approximately 40% [of Co-pilot generated code] to be vulnerable”

AIE
AI Engineer World’s Fair
Microsoft
aws
smol ai

### With AI, the outer loop bottlenecks

An infinity loop places AI coding tools on one side. Reimers says their higher output leaves developers more code to review, test, merge and deploy.

[1:59](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=119s) [An infinity-loop graphic accompanies the claim that AI can produce code faster than teams can keep up.](https://assets.ai.engineer/sites/aiecode2025/sha256/3fab9cf6e44b91208d0dcb8a474310491c5ee59c19e11a4a0ef408ee4f8a7b89.webp)

With AI, the outer loop bottlenecks
AI can produce code instantly, but we can’t keep up

CURSOR
GitHub Copilot
v0
windsurf

AIE
AI Engineer World’s Fair
Microsoft
aws
smol ai

### The new outer loop

Reimers calls for a new outer loop as rising code volumes put more work on developers who review, test, merge, and deploy it.

[2:24](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=144s) [Graphite section slide titled The new outer loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/307151b70d1b556dad9d78ecc6e164fe818e08d06b85f2c9c293f15edf2d4a59.webp)

The new outer loop

Graphite

AIE
Microsoft
aws
smol ai

Stage backdrop: AI Engineer World’s Fair; aws

### Requirements for the new outer loop

As code volume grows, the outer loop needs tools to track pull requests, assist reviewers, run CI, queue merges, and deploy changes across the author-to-deploy cycle.

[2:42](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=162s) [Requirements include change tracking, review assistance, optimized CI pipelines, merge queues, and better deployment tools.](https://assets.ai.engineer/sites/aiecode2025/sha256/4d16814d44ce8d5d4dca2eeead8232c8455b81858ca33c34cdbfed0d93aab0dc.webp)

Requirements for the new outer loop

• Tooling to track, prioritize, and get notified about their in-flight changes
• “Driver assist” features for reviewers to focus and streamline the code review process
• Optimized CI pipelines
• Merge queues
• Better deployment tools

Diagram: Author, Test, Review, Merge, Deploy

Interface text:
AIE
AI Engineer World’s Fair
aws
augment [illegible]
Microsoft
smol ai

### AI for AI

Reimers hopes AI-assisted review can reduce manual work, freeing developers to focus on features and users. He says imperfect AI feedback means bots alone aren’t enough.

[3:06](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=186s) [Slide describing AI-assisted testing and code review, alongside an image of robotic hands shaking.](https://assets.ai.engineer/sites/aiecode2025/sha256/77e24a6e06d283b8bf313ad339333bfff39d7bdf6e5c847870359d696daf32f3.webp)

AI for AI

• AI streamlines testing and review
• “Level 2 self-driving” code review solutions are emerging
• AI-generated feedback is not perfect
• Bots aren’t enough

AIE
Microsoft
aws
smol ai

### An AI-native toolchain

Reimers argues that AI teammates alone are not enough: the whole development toolchain needs to be AI-native to support more productive developers.

[4:01](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=241s) [Slide stating that the entire toolchain should be AI-native, not just the IDE.](https://assets.ai.engineer/sites/aiecode2025/sha256/0c74ea414f1896eaac0b50da2c13309a9c0455e1687790bc6270b53887ec08d5.webp)

Your entire toolchain should be AI-native - not just your IDE

AIE
Microsoft
aws
smol ai

Interface text:
AI Engineer World's Fair

### The winning AI code review platform

Diamond reviews changes using codebase and history context and connects to CI and tests. The example flags a port mismatch and suggests changing 3000 to 8000.

[4:16](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=256s) [Slide listing desired code review capabilities beside an example review suggestion.](https://assets.ai.engineer/sites/aiecode2025/sha256/2f677725c340cbab5ddedb8db5d394a877d3f0f6f13cbbe0bc6ba6e15aa82a40.webp)

The winning AI code review platform

• High-signal, low noise
• Deep understanding of the codebase and change history
• Summarize, prioritize, and review each change
• Integrate with CI and testing infra to summarize errors and correct failures

Review example:
server.js
Javascript  +48
3 + const port = 3000;

Unresolved thread: line +3
Diamond  1h ago  Logic bug
Change the port from 3000 to 8000 to match the exported port in the tests. The CI test is failing because it expects the server to run on port 8000.

Suggested change
3 - const port = 3000;
3 + const port = 8000;
Commit suggestion

Interface text:
AIE
AI Engineer World's Fair
Microsoft
aws
smol ai

### Diamond

Diamond aims to shorten code review cycles, enforce quality and consistency, and keep code private and secure. Reimers says larger customer rollouts are starting to show those benefits.

[4:33](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=273s) [Slide presenting shorter review cycles, quality and consistency, and code privacy and security as Diamond benefits.](https://assets.ai.engineer/sites/aiecode2025/sha256/a1420c1695859b32b292f5d8a879ebd82de3623572758d28fffe1cdea01d286e.webp)

Diamond

Reduce code review cycles
Enforce quality and consistency
Keep your code private and secure

Interface text:
AIE
AI Engineer
World’s Fa[illegible]
aws
Microsoft
smol ai

### Diamond GitHub app

Diamond reviews GitHub pull requests and offers one-click suggestions. The example flags a possible null reference and suggests a null check or default value.

[4:51](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=291s) [Slide describing Diamond as a GitHub app that finds bugs, with feature highlights, customer logos, and product screenshots.](https://assets.ai.engineer/sites/aiecode2025/sha256/d219aaf40fb4a707795cd28d58ee299a19801e6342537e1372850560d1e56003.webp)

Diamond
A GitHub app that finds bugs

1. High-signal
2. Zero-setup: Connect to your GitHub
3. Actionable: one-click suggestions
4. Customizable: bring your own rules

replit
clay
duolingo
STATSIG
Watershed

Interface text:
Diamond - AI code review · Enabled in 4 repositories
Get immediate, high-signal feedback directly in your GitHub pull requests. Diamond automatically catches logic bugs, security issues, and performance problems before they reach production.
Your preferences
Workspace settings

2,950 issues found
Logic bug
Code quality/style
Accidentally committed code
Documentation issue
Potential edge case
Security issue
Performance issue

8,706 PRs reviewed
1,603 Issues led to fixes
3.5% Comments downvoted

Highlights
Feed
Null reference prevention
This comment prevents potential runtime crashes when stackJob is null, which could disrupt CI/CD pipelines and block development. Early detection saves debugging time and prevents potential production issues.

Diamond · 3d ago
Logic bug
463    mergingStacks: failureHandlingStacks,
464    mqRepository,
465    trunkSha,
466 +  failedStackJobId: stackJob.id,

This line is causing the test to fail because it's trying to access stackJob.id, but stackJob might be null in some test cases. Consider adding a null check or providing a default value, e.g., failedStackJobId: stackJob?.id || 'default-id'.

### Diamond by the numbers

Fewer than 4% of Diamond’s AI comments are downvoted. About 52% are accepted into pull requests, compared with roughly 45–50% of human comments.

[5:06](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=306s) [Diamond comments have a downvote rate below 4% and a 52% accept rate, higher than human comments.](https://assets.ai.engineer/sites/aiecode2025/sha256/32e930683264759139c95a28e0e81def01c8190874893f603d2fd8c272928b84.webp)

Diamond by the numbers

<4%
Downvote rate

52%
Accept rate
Higher than human comments

### Graphite — Thank you

[5:29](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=329s) [Graphite closing slide says thank you and displays graphite.dev.](https://assets.ai.engineer/sites/aiecode2025/sha256/acd58f5550b0351e3a58b093a693a56c64bd7f991b097d960271af55a3a0dca1.webp)

Graphite

Thank you

graphite.dev

AIE
Microsoft
aws
smol ai

AI Engineer
World’s Fai[illegible]

## Full transcript

[0:00](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=0s) [upbeat music] Awesome.

[0:15](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=15s) Uh, hi, everyone. My name is Tomas. I'm one of the co-founders of Graphite. Graphite is an AI code review, uh, company. So to give some context on sort of where we see the industry right now and where we see it going.

[0:28](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=28s) Software development currently, and has always had, two loops. The inner loop, which is focused on development, and the outer loop that's focused on review. Developers spend time in the inner loop.

[0:39](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=39s) They get their code working. They get the feature the way they want it, and then they go ahead and they move it to the outer loop, where it's tested, reviewed, merged, deployed.

[0:48](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=48s) We're seeing the inner loop change right now more than we've ever seen it. More developers are using AI than ever. I think right here we have some statistics from the GitHub Developer Survey.

[0:56](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=56s) Nearly every developer surveyed used AI tools both inside and outside of work, and forty-six percent of GitHub is being written by Copi-- uh, code on GitHub is being written by Copilot.

[1:06](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=66s) We're seeing more and more code being written by AI. Here we have some statistics around how code has changed over time and how some people predict it will change.

[1:16](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=76s) And even if we take a more pessimistic view of that, we still see the way the world's going as just more and more and more code being written by AI.

[1:26](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=86s) The inner loop is changing, you know? AI is making more, uh, developers more productive. Developers are now producing higher volumes of code, but that code still needs to be reviewed.

[1:38](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=98s) When we first started looking at this, when we first started building, uh, Diamond, our AI code reviewer, about a year ago now, what we found was we read a lot of articles that scared us a lot.

[1:48](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=108s) We were seeing within our or-own organization a lot of developers adopting AI tools, but we were also seeing a problem. AI can hallucinate, it can make mistakes, and almost more scarily, it can make security vulnerabilities.

[2:00](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=120s) For us, what we saw was that while the inner loop was getting sped up by AI, the outer loop was rapidly becoming the bottleneck. Um, we were seeing tools like Cursor, Windsurf, Copilot, v0, Bolt, all of those producing larger volumes of code than we were used to, than we'd ever seen before.

[2:17](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=137s) But we were also seeing our developers suddenly have to review higher volumes of code, test higher volumes of code, merge higher volumes of code, and deploy higher volumes of code.

[2:26](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=146s) That's what brought us to say, "There has to be a new outer loop here." This-- The way that things are going, this isn't going to work. We're going to break down.

[2:35](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=155s) We're watching the problems that used to only ail large companies start to ail all companies, where we were seeing companies deal with higher and higher and higher volumes of code.

[2:44](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=164s) The requirements for the new outer loop then look a lot like the problems that larger companies have always had to deal with. You need tools to better prioritize, track, and get notified about pull requests.

[2:54](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=174s) You need driver assist features to help reviewers focus and streamline the code review process. You need optimized CI pipelines and merge queues to be able to handle the sheer volume of code changes that are now happening, and you need better deployment tools.

[3:06](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=186s) So, um, when we first started looking at this through sort of an AI-first lens, we started to see that, well, the problems are being created by AI, they can also probably be solved by AI.

[3:19](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=199s) We can probably start to streamline a lot of these processes which have previously had to been manual, previously were parts of the process that developers did not enjoy, did not want to do.

[3:27](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=207s) Um, we wanted to see self-driving code review solutions where we no longer had to do those very manual and painful parts of review, but we could actually start to really focus on what matters most to the developers, making sure that your product can get out to users and that the features work as expected.

[3:41](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=221s) Um, we were seeing that AI-generated feedback wasn't perfect, and because of that, we were starting to think that bots weren't enough. I think an early, an early vision of ours was, well, can we solve this by just adding AI teammates, right?

[3:53](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=233s) Maybe it's background agents, maybe it's reviewers, maybe it's a whole lot of teammates to the workflow. And while we think that's part of the story, we don't think that's enough.

[4:00](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=240s) We think that, as we've built with Diamond, that your entire tool chain has to be AI native, not just your IDE. If you really are going to embrace AI in the age of development, if you're gonna accept the fact that developers are gonna be orders of magnitude more productive than they ever have before, you need tooling that

[4:15](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=255s) reflects that. We started by building Diamond, so the winning AI code review platform, with high signal, low noise, has a deep understanding of the code base and change history.

[4:26](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=266s) We summarize, prioritize, and review each change, and we integrate with your CI and your testing infrastructure to correct, uh, to summarize errors and correct failures. Um, our hope with it and what we've started to see as we've rolled it out to larger and larger customers and enterprises too, is we reve-- we reduce code review cycles, we enforce

[4:45](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=285s) quality and consistency, and we keep your code private and secure. Um, it's high signal, it's zero setup, it's actionable with one-click suggestions, and it's customizable. It's already being used by some of the fastest moving companies in the world.

[4:57](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=297s) It's expanding a lot more than we can even say publicly, um, a-and I hope that you all will embrace the idea that AI can change your entire developer workflow, not just your IDE.

[5:08](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=308s) Um, by the numbers, we see comments that our AI bot leaves to be downvoted at less than a four percent rate and to be accepted, meaning integrated into the pull request, um, that they were left on at a higher rate than human comments are.

[5:20](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=320s) Human comments are integrated about somewhere between forty-five and fifty percent. We're watching our Diamond comments be accepted about fifty-two percent. We've spent a lot of time tuning that. That's, that number is actually new as of March for us.

[5:30](https://www.youtube.com/watch?v=H6MrR5NbTZA&t=330s) Um, that's, that's what I have to tell you around Graphite, um, what I have to tell you around Diamond. I hope you give it a shot, and, and thanks for having me. [upbeat music]
