# How Coding Agents Change Software Development Forever - Hailong Zhang

Hailong Zhang

AI Engineer Summit 2025

[Talk page](https://ai.engineer/talks/EUHx5ooJHuQ-coding-agents-change-software-development-forever) · [Video](https://www.youtube.com/watch?v=EUHx5ooJHuQ)

## How Coding Agents Change Software Development Forever

Hailong Zhang’s Gru case study shows how an asynchronous agent can turn code changes into tested pull requests, and what evaluation, context, and shared infrastructure make that workflow possible.

Familiarity with Git pull requests and basic unit testing is helpful; no prior agent-building experience is required.

### Which work should an agent take on?

[0:32](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=32s)

If an AI agent takes over routine coding, debugging, testing, and operations, what remains the developer’s responsibility? Hailong Zhang begins with this division of work. His forecast leaves product design, architecture, and difficult issues with humans, while agents handle more of the routine implementation. **The resulting workflow depends on human–agent collaboration:** automating a task still leaves decisions about what to build and whether the result is right.

![Workflow from product design and architecture design, labeled creative work handled by humans, to coding, debugging, testing and operating, labeled routine work mostly handled by AI.](https://ai.engineer/api/talk-essay-media/2c324e6840a579939cd129068c55fdd7addeeaf02bdfb0f66f424982d2c484b2)

Human and AI roles across the software development workflow.

### Live assistance and asynchronous delivery

[1:11](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=71s)

The first collaboration pattern is familiar from GitHub Copilot and Cursor: an assistant lives inside the IDE and works while the developer types. Zhang places the broader category of synchronous assistance around 2020, with maturation in 2023 and rapid growth in 2024; that chronology is not a launch date for each named product. In the talk’s 2025 setting, he describes this pattern as widely adopted.

The second pattern moves the agent into the surrounding workflow. Zhang dates the emergence of this asynchronous category to 2024. A GitHub-style bot receives a manual or automatic trigger, completes a task without continuous human attention, and returns a deliverable.

| Collaboration | Where it works | Human involvement | Result |
| --- | --- | --- | --- |
| Synchronous | Inside the IDE | Developer and assistant work together | Assistance during editing |
| Asynchronous | Inside a repository workflow | Developer delegates and later reviews | A completed deliverable |

Both patterns remain useful because they serve different moments in development. An engineer may want immediate assistance while editing and separately delegate a task that can run to completion.

At repository scale, this becomes a collection of small agents operating at different stages: writing unit tests, fixing bugs, producing documentation, submitting code reviews, and handling releases. Each agent takes responsibility for a bounded piece of work, freeing human attention for the creative parts of the project.

### Fast generation makes tests more valuable

[2:55](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=175s)

Unit testing provides a concrete place to start. In Cursor, accepting a suggestion with Tab can generate multiple lines in multiple parts of a file. The developer can move quickly while overlooking individual changes. Zhang’s concern is the gap between generation speed and attention: code arrives faster, but bugs can arrive with it. Tests provide one way to catch those issues.

![Slide titled “Unit Tests are More Important in AI Era,” with a unit-test meme beside a code editor overlaid with three Tab labels and text warning about overlooked issues.](https://ai.engineer/api/talk-essay-media/132d18c91340c6084b9861c669f210245d33e7772a3bc8fe6ef45a878aefc930)

Unit testing concerns alongside repeated Tab-based code generation.

Writing those tests still requires effort. Zhang points to developers’ reluctance to do that work as the motivation for [Gru](https://github.com/gru-agent), an agent for generating and managing unit tests. The case study concerns the historical Test Gru workflow demonstrated in 2025.

### From a code change to a test pull request

[3:58](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=238s)

Gru’s unit of work begins with a repository change and ends with a reviewable pull request:

1. **Trigger the agent.** Invoke Gru manually or let a developer’s pull request trigger it automatically.
2. **Inspect the change.** Determine whether the changed code needs additional tests or changes to existing tests.
3. **Write and run the tests.** Generate the test code and execute it to check that it works.
4. **Submit a test pull request.** Include a summary of the tests and their coverage improvement.
5. **Ask for a human decision.** The reviewer judges the tests and decides whether to merge them.

The screenshot Zhang presents is the deliverable from this process: a test pull request with information a reviewer can use to assess it. **Autonomous task execution ends at a human merge decision.**

For a small TypeScript/Vitest illustration of that review boundary, suppose a change makes an order eligible for free shipping when its subtotal reaches 50. A proposed test can make the boundary explicit:

```typescript
import { describe, expect, it } from 'vitest';
import { qualifiesForFreeShipping } from './shipping';

describe('qualifiesForFreeShipping', () => {
  it.each([
    [49.99, false],
    [50, true],
    [50.01, true],
  ])('returns %s eligibility for subtotal %s', (subtotal, expected) => {
    expect(qualifiesForFreeShipping(subtotal)).toBe(expected);
  });
});
```

Running this proposed test checks the implementation against the encoded expectations. Reviewing it asks the further question: is 50 actually the intended inclusive threshold? That distinction explains why Gru’s workflow includes both test execution and human judgment before merging.

### What production use shows

[4:49](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=289s)

Zhang reports that over 50% of Gru’s pull requests in the displayed production GitHub list were accepted and merged by humans. He treats that level of acceptance as useful in practice while acknowledging substantial room for improvement. The talk does not establish a denominator, observation window, or model configuration for the figure, so its scope is the production example he presents.

Zhang also reports that Gru handles around 80% of the unit tests in its own repository, where the team uses it daily. This describes the agent’s share of unit-test work, not a code-coverage percentage or a pull-request acceptance rate.

By commit count, Zhang reports that Gru is the team’s leading contributor. Commit count measures repository activity separately from the quality or acceptance of the resulting tests. From that internal experience, he predicts that more agents will become repository contributors during 2025.

### Start with a task that can be evaluated

[5:50](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=350s)

Building such an agent begins with defining a clear, concrete, feasible problem. Unit testing gives the team a bounded task; software engineering as a whole is too broad to serve as the same kind of problem definition. The scope determines what the agent should do and what its evaluation must assess.

Zhang’s construction sequence puts evaluation before model and context work:

1. **Define the problem.** Choose a task specific enough to solve.
2. **Build evaluation datasets and a harness.** Establish the infrastructure for assessing candidate approaches.
3. **Work on models and context.** Develop the components that perform the task.
4. **Orchestrate the components.** Use a framework or Agent OS to bring them together.

This order gives model and context decisions an evaluation setting instead of leaving success defined only by a plausible-looking output.

### Select models and build context for the task

[6:36](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=396s)

Gru is designed to work across frontier models from OpenAI, Anthropic, DeepSeek, Google, and other providers. The team evaluates models in different scenarios to choose an appropriate model for each one. That choice can vary between stages of a single job: a job does not have to use one model throughout.

The team also adapts models. Zhang describes fine-tuning [**GPT-4o**](https://openai.com/index/gpt-4o-fine-tuning/) with human-labeled unit-test code to improve test generation. Scenario-based selection and fine-tuning address different decisions: which model to use for a stage, and how to adapt a model to the desired output.

![Three diagrams show Gru connected to OpenAI, Anthropic and DeepSeek; scenarios paired with different models; and human-labeled unit-test code flowing into GPT-4o.](https://ai.engineer/api/talk-essay-media/32ca2102055e8cd55a319f5ae2c1b5a3263d660e23dc085f73bfd38061a7b52a)

Gru’s model-agnostic, multi-model and fine-tuning approaches.

**Context is also task-specific.** For unit testing, Gru builds context around the programming language and testing framework, then gathers information from the repository environment. Relevant sources include GitHub issues, code reviews, commits, pull requests, README material, and the code itself. Gathering is followed by filtering and selection so the information fits into the model’s context. The mechanism is therefore more deliberate than simply passing the entire repository to an LLM.

### Share infrastructure across specialized agents

[7:57](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=477s)

The next targets extend beyond unit tests to refactoring and end-to-end testing. Building every agent from scratch would repeat much of the same engineering, so the team abstracts common infrastructure into a framework Zhang calls **Agent OS**. Different tasks can share runtime facilities, tools, and context infrastructure while retaining their own task-specific behavior.

The architecture slide places UnitTest Gru, Refactor Gru, and E2E Test Gru above shared layers. Those layers include runtime and tools; planning, decision making, context building, and environment grounding; and fine-tuning, RAG, and prompt engineering. The intended benefit is faster construction of additional agents within software engineering: each new task can build on an existing foundation instead of recreating the machinery needed to operate in a repository.

![Layered diagram places UnitTest Gru, Refactor Gru and E2E Test Gru above runtime and tools; planning, decision making, context building and environment grounding; and fine tuning, RAG and prompt engineering.](https://ai.engineer/api/talk-essay-media/e6b196f2aacacb7cb21099a413edf5b5098979a9842e3d4975e7238755f2476f)

Agent OS shares infrastructure across engineering agents.

## Resources

- [Test Gru TypeScript/Vitest example](https://github.com/gru-agent/testgru-example-ts-vitest) — Example repository documenting Test Gru's configuration, test execution, and automatic or manual pull-request workflow. Service setup instructions reflect an earlier product.
- [GPT-4o fine-tuning announcement](https://openai.com/index/gpt-4o-fine-tuning/) — Historical announcement of GPT-4o fine-tuning, now carrying a notice about the platform's 2026 wind-down.

## Slides

### Working with Coding Agents

Hailong Zhang introduces his presentation on working with coding agents.

[0:00](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=0s) [Title slide for Working with Coding Agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/9b24f0774e4ec0d7d3603965d3e965df6ec7c081d2e7a4a83e08403581f575f7.webp)

Working with Coding Agents

gru

### Working with Coding Agents: Agenda

The talk starts with the future software development workflow, then covers a coding agent for unit tests and building agents to solve real-life problems.

[0:08](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=8s) [Agenda with the future software development workflow highlighted, followed by unit testing and building agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/17f9eefe70df42289d8438e0e4bb841b23c2dbd9d123a7c7e5f8aa32123b737b.webp)

Working with Coding Agents

gru

1. The Future Software Development Workflow
2. How do We Boost Unit Test with Coding Agent
3. How do We Build Agents to Solve Real life Problems

### Human Role in Future Workflow

Zhang expects humans to handle product and architecture design and difficult issues, while AI handles much of the routine coding, debugging, testing and operations in a shared workflow.

[0:29](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=29s) [Diagram assigns product and architecture design to humans and depicts coding, debugging, testing, and operating as mostly handled by AI.](https://assets.ai.engineer/sites/aiecode2025/sha256/c640bae2b818006402647e96e1d858c7ac8f077ecbf49b7fb9eeb04f8ea46e9b.webp)

Human Role in Future Workflow

gru

Product Design → Arch Design → Coding → Debugging → Testing → Operating

Creative Works
Handled by Humans

Routine Works
Mostly Handled by AI

### How to Integrate AI

Synchronous AI works alongside a developer in the IDE; asynchronous agents work through tasks independently and submit a deliverable. The timeline places asynchronous work’s emergence in 2024.

[1:08](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=68s) [Examples of synchronous coding assistance and asynchronous agent work, with a timeline from 2020 to 2024.](https://assets.ai.engineer/sites/aiecode2025/sha256/c4e358780921354694eeecbbf11f3ca125991226a1bf0229a3fb6df89152bfea.webp)

How to Integrate AI

gru

Synchronous Work
Asynchronous Work

Timeline:
2020 — Synchronous Work Emergence
2023 — Synchronous Work PMF
2024 — Asynchronous Work Emergence; Synchronous Work Rapid Growth

Interface text:
TS button.ts

interface ButtonProps {
  onClick: () => void;
  text: string;
}

const Button: React.FC<Props> = ({ onClick, text }) => {
  return <button onClick={onClick}>{text}</button>;
};

export default Button;

Create a new button component
Accept
Discard

Gru Agent
committed 3 minutes ago
src/utils/formatBytes.test.ts

import { describe, it, expect } from 'vitest'
import { formatBytes } from './formatBytes.ts'

describe('formatBytes', () => {
  // Test zero bytes
  it('should return "0 Bytes" for 0', () => {
    expect(formatBytes(0)).toBe('0 Bytes')
  [illegible]

All tests passed
formatBytes (8 tests) 297ms
✓ should return "0 Bytes" for 0
✓ should format bytes correctly
✓ should format kilobytes correct[illegible]
✓ should format megabytes correct[illegible]
✓ should format gigabytes correct[illegible]
✓ should respect custom decimal p[illegible]
✓ should handle edge cases

### Agents in Your Workflow

Zhang imagines small agents across the development workflow, handling tasks from issue standardization and bug fixes to tests, code review, documentation, integration and monitoring.

[2:23](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=143s) [Agents assist across issue, code, pull request, merge, CI/CD, testing, and operating stages.](https://assets.ai.engineer/sites/aiecode2025/sha256/01abbba0a5b8cde872797d8ec6547c8fd16aa68dc5c8abb64f7e47fd038b1387.webp)

Agents in Your Workflow

gru

I will make issue more standardized
I will help you add unit test
I will help you add documents
I will test your software

Issue → Code → PR → Merge → CI/CD → Testing → Operating

I will fix the issue for you
I will review your code
I will help you do refactor
I will help you fix integration issue
I will monitor software operating

### Working with Coding Agents: Unit Testing

Zhang turns to a detailed example of a coding agent intended to boost unit testing.

[2:52](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=172s) [Agenda with boosting unit tests using a coding agent highlighted.](https://assets.ai.engineer/sites/aiecode2025/sha256/9ecd3f4a94b4c7e1a18aa8ff6dff9b8839a29cf5053eeb2001b3cee6654d4e5b.webp)

Working with Coding Agents

1. The Future Software Development Workflow
2. How do We Boost Unit Test with Coding Agent
3. How do We Build Agents to Solve Real life Problems

gru

### Unit Tests Are More Important in the AI Era

Zhang argues that repeated Tab completions can generate code faster than developers can inspect it, making unit tests more important—even though developers often avoid writing them.

[3:01](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=181s) [A meme about lacking time for unit tests sits beside an AI coding example with repeated Tab prompts.](https://assets.ai.engineer/sites/aiecode2025/sha256/59de191ffcec7be4484db85ae4ec91c7dd9023dd2fc6eb98114fa3c33d001b89.webp)

Unit Tests are More Important in AI Era

gru

UNIT TESTS?
WE DON'T HAVE TIME FOR THAT
imgflip.com

Everybody know how important unit test is, but everybody hate to write it

Tab
Tab
Tab

This way of writing code can easily overlook potential issues within the code

Interface text:
todo.tsx M

import React, { useState } from 'react';

const TodoComponent: React.FC = () => {
    const [todos, setTodos] = useState<string[]>([]);
    const [input, setInput] = useState<string>('');

    return (
        <>
            <button onClick={() => {
                setTodos([...todos, input][illegible]
                setInput('');
            }}>
                Add
            </button>

            <input
                value={input}
                onChange={(e) => setInput(e.target.value)}
            />

            <ul>
                {todos.map((todo, index) => (
                    <li key={index}>{todo}</li>
                ))}
            </ul>
        </>
    )
}

### How Test Gru Works

Triggered manually or by a pull request, Gru decides which tests need changes, writes and runs them, then submits a PR with a summary and coverage changes for human review.

[3:55](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=235s) [A workflow shows coding, pull requests, generated unit tests, testing, review and CI beside a pull request summary.](https://assets.ai.engineer/sites/aiecode2025/sha256/e371145735ac47429160651e11796d21ccb9130f6d8c4a433863dde91a15fd67.webp)

How Test Gru Works

gru

Coding → PR →
Gru Add Unit Test → Testing → PR →
Code Review → CI

Trigger Info
Trigger Type | Triggered By | Source Pull Request | Assignment
Auto Rebase | vangie | 2736 | Detail

Summary
This PR enhances the test coverage and functionality of the utils module in the sandbox core. Key updates include:

• Improved Test Coverage:
  • Added tests for k8sImage to ensure it handles cases where no custom image is provided.
  • Extended useGpuPod tests to cover scenarios where metadata is undefined.
  • Added comprehensive tests for the streamToString utility, including cases for empty streams, streams with string chunks, and streams with Buffer chunks.
• New Utility Functionality:
  • Introduced streamToString, a utility function to convert readable streams into strings.
• Code Cleanup:
  • Removed redundant comments in existing tests for better readability and maintainability.

These changes improve the robustness and reliability of the sandbox utilities by ensuring edge cases are handled and tested.

Coverage
The change in coverage value, such as: 0% -> 50%, indicates that the coverage was 0% before writing the tests and 50% after writing them.

https://web.dev/articles/ta-code-coverage

Type | Change
lines | 89.09% -> 100%
functions | 88.88% -> 100%
statements | 89.09% -> 100%
branches | 92.85% -> 100%

Interface text:
gru-agent bot commented 1 hour ago

### Test Gru in Production: Pull Requests

A GitHub pull request list accompanies the claim that 50% of Test Gru’s submitted pull requests have been merged. The speaker says humans review and accept them.

[4:47](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=287s) [A pull request list accompanies the claim that 50% of requests submitted by Test Gru have been merged.](https://assets.ai.engineer/sites/aiecode2025/sha256/1567d233084bd21f9bc3c7f768b94d4292ff42e9e3c9444b7e53ce0701b54605.webp)

Test Gru in Production

50% of the pull requests submitted by Test Gru have been merged.

gru

Interface text:
test: add unit test for src/providers/adaline.gateway.ts
#2834 by gru-agent bot was merged last week

test: add unit test for src/providers/portkey.ts
#2825 by gru-agent bot was merged last week · Approved

test: add unit test for src/validators/redteam.ts
#2819 by gru-agent bot was closed last week · Review required

test: add unit test for src/app/src/pages/redteam/report/components/RiskCategoryDrawer.tsx
#2813 by gru-agent bot was closed 2 weeks ago · Review required

test: add unit test for src/redteam/commands/generate.ts
#2808 by gru-agent bot was closed 2 weeks ago · Review required

test: add unit test for src/assertions/contextRelevance.ts
#2804 by gru-agent bot was merged 2 weeks ago · Review required

test: add unit test for src/assertions/geval.ts
#2803 by gru-agent bot [illegible]

test: add unit test for src[illegible]
#2801 by gru-agent bot [illegible]

test: add unit test for src[illegible]
#2800 by gru-agent bot [illegible]

test: add unit test for src[illegible]
#2799 by gru-agent bot [illegible]

test: add unit test for src/assertions/contextFaithfulness.ts
#2798 by gru-agent bot was merged 2 weeks ago · Review required

test: add unit test for src/providers/http.ts
#2786 by gru-agent bot was closed 2 weeks ago

test: add unit test for src/providers/http.ts
#2782 by gru-agent bot was closed 2 weeks ago

test: add unit test for src/fetch.ts
#2781 by gru-agent bot was merged 2 weeks ago · Review required

test: add unit test for src/redteam/plugins/harmful/graders.ts
#2771 by gru-agent bot was merged 2 weeks ago · Review required

test: add unit test for src/types/providers.ts
#2766 by gru-agent bot was merged 2 weeks ago

test: add unit test for src/redteam/plugins/competitors.ts
#2764 by gru-agent bot was merged 2 weeks ago

[illegible] add unit test for src/redteam/plugins/harmful/graders.ts
[illegible]763 by gru-agent bot was merged 2 weeks ago · Review required

[illegible] add unit test for src/redteam/graders.ts
[illegible]762 by gru-agent bot was merged 2 weeks ago · Review required

Popup:
promptfoo/promptfoo on Jan 23
test: add unit test for src/assertions/contextFaithfulness.ts #2798
Trigger Info Summary This PR introduces unit tests for the handleContextFaithfulness ...
main ← testgru-src-asser...
Review required

### Test Gru in Production: Contributions

In January 2025, Gru led its team with 105 commits, ahead of the next contributor’s 56. The speaker says Gru handles around 80% of unit tests in its own repo.

[5:17](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=317s) [A January 2025 commits chart places Gru first; the slide says Gru handles 80% of unit tests in its own repository.](https://assets.ai.engineer/sites/aiecode2025/sha256/065d5c142edf4f6da0195986ace764cd8a6a308f4e2fb4b770635e27ccabbeab.webp)

Test Gru in Production

gru

Commits in Jan. 2025
105, 56, 52, 43, 34, 33, 20, 10

More and more agents will become contributors in your repos in 2025.

Test Gru Handles 80% Unit Tests in it's Own Repo

### Working with Coding Agents: Building Agents

Zhang turns to how the team builds Gru to solve real-life problems.

[5:48](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=348s) [An agenda highlights the section on building agents to solve real-life problems.](https://assets.ai.engineer/sites/aiecode2025/sha256/b30d8f49e535708c934fee228dbb6bf3de8281a25c3cbfba6fdd9af50edf11dd.webp)

Working with Coding Agents

1. The Future Software Development Workflow
2. How do We Boost Unit Test with Coding Agent
3. How do We Build Agents to Solve Real life Problems

gru

### How to Build Agents to Solve Real-Life Problems

Start with a clear, doable problem, then build evaluation datasets and a harness. Work with LLMs and curated context, and use an Agent OS to orchestrate the system.

[5:56](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=356s) [Five icons represent problem definition, evaluation, working with LLMs, curated context and an agent operating system.](https://assets.ai.engineer/sites/aiecode2025/sha256/2e963e5c4a31d9f3993560cf8bec7617c3d8a83dc5e93f5660081c100768b7fd.webp)

How do We Build Agents to Solve Real life Problems

gru

Problem Defination
Evaluation
Work with LLMs
Curated Context
Agent OS

### Work with LLMs

Gru evaluates models for different scenarios and can use different LLMs within one job. It also fine-tunes GPT-4o on human-labeled unit-test code to improve test generation.

[6:33](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=393s) [Diagrams show model-agnostic integration, models for different scenarios, and fine-tuning GPT-4o with human-labeled unit test code.](https://assets.ai.engineer/sites/aiecode2025/sha256/7cc56927dbdc6fb6264812badec17dcc7d9e97e4a181be1273cf18deeac27ddf.webp)

Work with LLMs

gru

Gru System
OpenAI
Anthropic
DeepSeek
Model Agnostic

Scenario 1
o1
Sonnet 3.5

Scenario 2
GPT 4o
DeepSeek R1
Mult-Models

Human Labeled Unit Test Code
GPT 4o
FineTuned Models

### Agent OS

Gru proposes a shared Agent OS for unit testing, refactoring, and end-to-end testing agents, with common runtime, tools, and context-building infrastructure.

[7:54](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=474s) [A layered architecture shows engineering agents, workspace runtime and tools, operating system functions, and LLM techniques.](https://assets.ai.engineer/sites/aiecode2025/sha256/9ce41dc4fd5897893142957d2979541dfc51b3222120b0dfd05b3ba4e632548f.webp)

Agent OS

gru

Gru Manufacture
Different Grus for different engineering tasks, with the same fundamental technical architecture

Agents
UnitTest Gru
Refactor Gru
E2E Test Gru

Workspace
Runtime
Tools

Agent Operating System
Planning
Decision Making
Context Building
Env Grounding

LLM/Intelligence
Fine Tune
RAG
Prompt Engineering

### Embrace Agents in Workflow

Zhang closes with a call to embrace agents in workflows.

[8:40](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=520s) [Closing slide reads Embrace Agents in Workflow.](https://assets.ai.engineer/sites/aiecode2025/sha256/35412a0260ecc6123369aeeb11160f49a60d0eef57bc7d9cd35272212b6acffc.webp)

Embrace Agents in Workflow

## Full transcript

[0:01](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=1s) Hi, everyone. In this presentation, I'm going to talk about coding agents and how to work with coding agents.

[0:09](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=9s) This is my agenda. First, I'm going to talk about the future of software development workflow, and then I'm going to show you example of coding agent, and this agent will boost the unit test.

[0:21](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=21s) And finally, I'm going to dive deeper into how we solved real-life problems with agents, how we build the agents.

[0:32](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=32s) Generative AI has shifted a lot of things, and the people believe AI coding is going to be the future. And we believe a lot of the routine works will be handled by AI, including entry-level coding tasks, debugging, testing, and operating.

[0:49](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=49s) But still, I think the creative works, including product design, architecture design, and the difficult issues, still need to be handled by humans. So it must be a human and AI agents collaborate together to solve problems in same workflow in the future.

[1:11](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=71s) The collaboration has two types. First one is synchronous and another one is asynchronous. I think people may pretty familiar with the synchronous one, GitHub Copilot or Cursor. This is AI live inside your IDE, and this AI works simultaneously with human.

[1:29](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=89s) When you are typing, it's working, right? So

[1:33](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=93s) we have already had this kind of product for a long time, since twenty twenty, and it's getting mature since twenty twenty-three, and the rapidly grows twenty twenty-four, and it's widely adopted today.

[1:47](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=107s) But the asynchronous one is pretty new, and it just started twenty twenty-four. The asynchronous one is more like a bot inside your workflow. For example, GitHub bot, and this bot can be triggered either manually or automatically.

[2:02](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=122s) And this bot will complete the task without human's attention, fully autonomous, and will submit a deliverable once it's done. So it's totally different experience, and these two types of agents are all important.

[2:17](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=137s) And I think people need all these kind of agents to solve the real-life problems.

[2:26](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=146s) Imagine this is your workflow in the future,

[2:29](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=149s) and the workflow will have a lot of AI agents, small agents live inside your repos, workflows in different stages to solve different problems, including unit test, including fixing bugs, writing documents, submit code reviews, and releasing from these kind of tasks, and so human can focus on more creative part.

[2:55](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=175s) Next, I'm going to show you a detailed coding agent, which will boost unit test.

[3:02](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=182s) I think a unit test is more important today than before because in the AI era, a lot of code are generated by AI. As we just talked, if you are using Cursor, you must be familiar with Tab.

[3:15](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=195s) For each of the Tab, multiple lines of code in multiple parts of the file will be generated, and it's very hard to... for you to focus on every generation, and you may overlook something.

[3:26](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=206s) The coding speed is really, really fast, but it also easy to generate bugs. So one of the way to solve the issue or prevent the issue is to write a unit test.

[3:38](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=218s) People talk about unit test. People think unit test is important. But when writing unit test, people hates it. Yeah, developers, every developers hates to write unit test. So we build a Gru, an AI agent to help developers to write a unit test and managing unit test.

[3:58](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=238s) This is a screenshot that pull request of unit test generated by Gru. And the process is triggered either manually or automatically by a pull request. When human submit a pull request, Gru will detect the change of the code and determine if this change needs more unit test or change existing unit test.

[4:21](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=261s) And the Gru will do the coding work, writing the test, and to run the test to make sure everything works, and then prepare and submit a pull request, including all the information showing in this screenshot, the summary of the test and the coverage improvement of the test.

[4:37](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=277s) Now, human need to review this pull request and determine if this unit test is good or not, uh, to merge into the, to the repo.

[4:49](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=289s) Let's look at the, uh, performance of Gru in production. This is also a screenshot from GitHub. It's a list of pull requests. And, uh, as you can see, over fifty percent of the pull requests are merged and accepted by humans.

[5:04](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=304s) So fifty percent is not a very large figure, and there are a lot of rooms to... for improvement. But I think in real life, fifty percent is already meaningful in production and help solve a lot of problems.

[5:19](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=319s) And also, Gru handles around eighty percent of the unit test itself in its own repo. We are using Gru every day, and then most of our tests are generated by Gru.

[5:33](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=333s) From the commits, you can see Gru is already in, in the counts of commits, Gru is already the first contributor in our team, and we believe more and more agents will become contributors in people's repo this year, twenty twenty-five.

[5:50](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=350s) Next, I'm going to dive a little deeper into how we build Gru to solve real-life problems.

[5:58](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=358s) To build an agent, I think the first and the most important thing is to define the problem itself. A clear, concrete, and a doable problem is crucial For example, unit test is a problem, but software engineering is not a problem.

[6:16](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=376s) And once we have the problem, we need to build the datasets for the evaluation purpose and also the evaluation harness. And then we work on LLMs, building context. And the last, we need Agent OS or framework to orchestrate everything together.

[6:36](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=396s) As we all know, agents work on top of LLMs and Gru work on all frontier models, either it's from OpenAI, Anthropic, DeepSeek, Google, et cetera. And we evaluate the models on the different scenarios, try to find the best model for each of the scenario.

[6:53](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=413s) That means even within the same job, Gru may use different LLMs for different stages. And also, we fine-tune models to improve the performance. For example, we fine-tune GPT-4o with human-labeled unit-test code to improve the generation of the test code.

[7:18](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=438s) Building context in agents is really important, and, uh, we think it's, it's necessary to build the context for specific tasks. For example, here in unit test, we build the context for each of the languages and the frameworks, and we also gather all the information from environment.

[7:39](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=459s) For example, the GitHub issues, uh, code reviews, commits, pull requests, README code itself, and all this information need to be gathered together and to be filtered and to be selected to fit into the context.

[7:57](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=477s) We talk about unit test a lot, but our vision is not only unit test. We want to build agents for different software engineering tasks. For example, the, the refactor task, the E2E test task.

[8:10](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=490s) It is almost import... impossible for us to build every of the agents from scratch, so we build the framework, or we can call it agent operating system, Agent OS.

[8:22](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=502s) Different tasks may share similar runtime, similar tools, similar context, so we abstract the common infrastructure, and this operating system will enable us to build the agents in this domain in a really fast pace.

[8:42](https://www.youtube.com/watch?v=EUHx5ooJHuQ&t=522s) The agent era is coming. Let's embrace agents in workflow. Thanks for watching.
