Automating Large-Scale Refactors with Parallel Agents

How do you automate a maintenance project that is too large for one coding agent to finish? Technical debt, dependency maintenance, and code modernization contain plenty of repeatable work, but their scope stretches beyond a single prompt and a single development loop. Agent orchestration makes that work manageable by dividing it among agents and coordinating their results.

Robert Brennan introduces himself as OpenHands’ co-founder and CEO, with a background in open-source developer tools and natural language processing. OpenHands is an MIT-licensed coding agent that grew out of OpenDevin after Devin’s autonomous-development demonstration. Its founding motivation was to give the software community a say in how agents change engineering, rather than leave that change inside a proprietary black box.

Brennan says nearly every line he writes now passes through an agent. The critical thinking remains, while the immediate activity shifts from typing code to describing work and evaluating results. He expects operational adoption to keep changing engineering even if model capabilities stop improving: organizations still have to learn how to delegate, review, and trust this work. His forecast is substantial change over the following two to three years, driven partly by overcoming psychological and organizational barriers.

The progression starts with context-unaware snippets: ask for bubble sort or SQL access code, receive an answer in chat, and paste it into a project the model cannot see. IDE autocomplete adds local context. GitHub Copilot can use the variable names, surrounding code, and database table names already present in the project. In Brennan’s illustration, the robot gains eyes.

Autonomous agents add the ability to act. With Devin and OpenDevin in early 2024, the loop extends from writing code to running it, searching an error message, consulting Stack Overflow, adding debug statements, and trying again. The robot gains arms: it can work through the inner development loop until it has running code and passing tests. Parallel orchestration extends that capability to multiple agents, including agents that communicate or create additional agents. Brennan describes this last stage as an emerging practice.

The corresponding product progression runs from IDE plugins through AI-enhanced IDEs and local agents such as Claude Code, then into cloud agents and orchestration. A cloud agent gets its own execution environment, making concurrent work easier and reducing exposure of the developer’s laptop to destructive commands or unwanted installations. Container isolation is a relative safety improvement; its practical boundary still depends on what credentials, mounts, and network access the environment receives. With an appropriate boundary, the developer can permit more autonomous execution instead of approving every command.

Orchestration adds coordination between those environments. OpenHands began with cloud agents and later added a local CLI to fit familiar developer habits. Brennan compares delegating to an agent fleet with his transition from individual contributor to manager: the work increasingly consists of defining tasks, supplying context, and judging outcomes. He characterizes orchestration adopters as a very small leading edge of engineers, tackling backlogs their teams had previously left untouched.

CVE remediation provides a concrete unit: one vulnerability in one repository. Brennan describes a client with tens of thousands of developers and thousands of repositories. A newly announced vulnerability triggers repository inspection, a dependency update where necessary, repairs for breaking API changes, and a pull request for the responsible team. Brennan reports a 30× improvement in CVE time to resolution for that client. He does not provide the baseline duration or measurement method. The workflow leaves the downstream team with a proposed change to review and merge.

Other suitable workloads share that repeated structure:

• Maintenance: generate documentation and release notes.
• Modernization: add Python 3 type annotations or divide a Java monolith into microservices.
• Migration: upgrade Java versions, move Spark 2 jobs to Spark 3, or replace Redux with Zustand in a React frontend.
• Technical debt: remove unused code or respond to new error patterns in logs by adding error handling and fixing the underlying defect.

Brennan cites the Redux-to-Zustand migration as work performed on OpenHands itself and describes the log-driven repair workflow as another client’s SDK application. The repeated edits are automatable, but the boundaries and architectural decisions still require engineering judgment.

A long task can exceed the agent’s useful context even before it exceeds its nominal context window. Compaction may discard details the agent needs later. Agents can also stop prematurely: Brennan recounts an attempt in which an agent migrated three of a hundred services and suggested hiring six people to finish. Missing domain knowledge compounds the problem. An early mistaken assumption can become the template for every subsequent edit.

The human side has corresponding failure modes. A request to split a monolith into microservices may omit the architecture already present in the engineer’s head. The agent then chooses boundaries from familiar patterns without understanding the intended design. Effective delegation therefore needs tasks sized to the agent’s capabilities, intermediate review, and a clear definition of done. Without that definition, neither the agent nor its reviewer has a reliable stopping condition.

This is a specialized workflow, not a replacement for every local coding session. Brennan expects ordinary feature development and bug fixing to remain common single-agent tasks. He contrasts a roughly 20% general productivity lift with exceptional gains on selected maintenance workloads, while explicitly rejecting a universal 3,000% improvement. Those general figures are not tied to a named study here; his narrower claim is that carefully selected CVE and modernization projects can compress engineer-years of work into weeks.

The familiar single-agent loop is prompt, execute, inspect the output and tests, then either prompt again or commit and push. An orchestrated project adds decomposition before execution and integration afterward. A human, optionally working with an agent, defines individual tasks; agents carry them out; their outputs are collected into a coherent change. Review belongs at both levels: each intermediate result and the final integration.

Brennan proposes approximately 90% automation as a target, describing it as an opportunity for an order-of-magnitude productivity improvement rather than a measured result. The remaining work is consequential: choose boundaries that make each step verifiable and prevent a large accumulation of poorly understood AI-written code.

His Git workflow gives the project a shared integration point:

1. Create a dedicated branch for the migration.
2. Add project instructions in AGENTS.md or an OpenHands microagent, explaining the intended transformation.
3. Add any scaffolding needed to keep intermediate states usable.
4. Start agents from that prepared branch and accumulate their reviewed changes into it.
5. Remove the scaffolding when the migration is complete, then merge into main.

For a first attempt, Brennan recommends three to five concurrent agents so the review load stays manageable. He reports much larger deployments with hundreds or thousands running concurrently, but those distribute review—for example, by sending pull requests to the teams responsible for individual repositories.

Calvin’s recorded demonstration applies this approach to code smells in OpenHands. The Refactor SDK is the orchestration interface; the underlying OpenHands Agent SDK supplies the coding agents. Focused work such as fixing failing CI or adding and debugging endpoints fits an individual agent well. Repository-wide typing, dependency upgrades, and framework migrations require a larger structure.

Calvin describes the core agent code under examination as about 380 files spanning 60,000 lines. A dependency graph makes its structure visible: each node is a file, and an edge represents an import relationship. The next step is to divide that graph into PR-sized batches that both an agent and a reviewer can understand. Graph algorithms are one option; the demonstration uses directory structure to keep semantically related files together. Colors then identify which files belong to the same batch.

The interface collapses the file graph into a batch graph. Each new node represents a batch, and its edges inherit dependencies from the files inside it. This smaller graph exposes which batches can be processed first. Calvin inspects a 16-file batch and an apparently empty initialization file; a human can move a file, leave a note, or record that the batch has been reviewed. A later four-file batch contains substantially more complex code. File count alone does not determine review difficulty, so the complexity indicators help direct attention.

Before changing code, the system needs to identify what is wrong. Its verifier can be programmatic: run a Bash command that invokes tests, a linter, or a type checker. For this code-smell exercise, Calvin instead uses an LLM with supplied rules. The resulting report explains which smells it found and which it did not. A trivial batch passes and turns green, with that status reflected in the graph.

Verification proceeds in dependency order, choosing batches without unresolved prerequisites. A second simple batch passes. The more complex batch fails, producing a red node and a longer, file-by-file report. The graph now contains two green batches and one red batch. Turning the whole graph green means satisfying the configured verifier; the red node identifies work for the next stage.

The fixer has several possible implementations:

The Agent SDK implementation can inspect code, run tests, consult internet documentation, and iterate. While it works, Calvin continues verifying and launching fixes for independent batches. The sped-up demonstration stops advancing along a dependency path when an upstream batch is still incomplete. A completed fixer resets the batch’s status: its output still needs verification.

Each fixer produces a pull request for human approval. Its description records the code smells, the changes made, and changes deliberately omitted. Calvin describes the inspected PR as changing a couple hundred lines, mostly by extracting a deeply nested block into a separate function. Narrow instructions and batching keep the change understandable even when other PRs are larger. Review and merge unblock downstream work; verification then establishes whether the revised batch passes. Calvin reports that the tool has also been used for strict typing and improved test coverage. The workshop next builds a smaller version of the same discovery-and-fix pattern.

A good unit of delegated work fits into one commit or pull request and has a reasonable chance of finishing without repeated prompting. It should be separable from concurrent work and quick to inspect: green CI, a short application walkthrough, or a focused command should provide meaningful evidence. Dependencies must be explicit. Brennan uses the same planning pattern as an engineering team: A unlocks B, C, and D; completing those unlocks E.

The simplest decomposition is piece by piece: enumerate files, directories, functions, or classes and assign each unit independently. Adding Python type annotations can fit this approach when coupling is limited, with results collected into a final PR. Where changes affect dependents, add an ordering constraint. Start with leaf utilities, incorporate their fixes, then work toward the application entry point so later agents receive the updated dependencies.

The Redux-to-Zustand migration adds another technique: temporary scaffolding that keeps both implementations usable. An agent first creates support for both state-management systems. Parallel agents then migrate individual React components, and each completed component can be checked while other components still use Redux. Once everything uses Zustand, the bridge and remaining Redux references are removed. The bridge is intentionally temporary; its purpose is to make intermediate states testable instead of postponing feedback until the entire migration is finished.

Decomposition does not eliminate shared discoveries. Several agents may hit the same obstacle, revealing that the original plan missed something. Broadcasting every agent’s full context is a poor response: it rapidly consumes context windows and recreates the burden of one long agent trajectory. The useful information is the discovery that changes another agent’s next decision.

Brennan compares three more selective approaches:

• Human updates: paste a correction into each conversation—for example, use library version 1.2.3 instead of 1.2.2—or maintain shared instructions in AGENTS.md or a microagent. This is direct but demands ongoing human effort.
• Agent-proposed shared context: let agents update an instructions file, potentially through pull requests. Human review helps filter excessive or unimportant lessons before they become instructions for everyone.
• Message-passing tools: allow agents to broadcast discoveries or communicate point to point. This makes coordination more autonomous but introduces additional nondeterminism.

The last approach can produce interactions unrelated to the task. Brennan’s memorable example is a pair of agents repeatedly wishing each other Zen perfection. Communication needs a purpose and a stopping condition just as coding does.

The hands-on exercise takes a GitHub repository as input and builds a CVE-remediation workflow. Brennan directs attendees to the workshop presentation, with copyable prompts and links around slide 29. The first agent scans the repository, choosing tools appropriate to what it finds: Trivy for a Docker image, for example, or npm audit for a project with package.json.

Each discovered vulnerability then becomes a separate remediation task. Its agent researches whether a fix exists, updates the relevant dependency, repairs breaking API changes, and opens a pull request. Independent PRs let useful work land even when another vulnerability is unfixable or an agent gets stuck. Brennan uses 90–95% remediation as a hypothetical example of valuable partial completion, not an observed success rate.

The SDK construction follows a small set of objects: configure an LLM, give it to an Agent with terminal, file-editor, and optional task-tracking tools, supply a workspace, send a task, and run it. After the scanner finishes, the host program iterates over its findings and launches a remediation agent for each one. That structure separates repository-level discovery from vulnerability-level execution.

The exercise starts with a new GitHub repository, a GitHub token, and an LLM token. Brennan offers workshop participants $10 in introductory LLM credits and offers to top up existing accounts. A Docker agent-server container houses the agents’ execution environment. He describes Kubernetes as a possible way to provision many workspaces for thousands of CVEs; the workshop itself uses one container. OpenHands CLI is his authoring interface, but attendees can use Cursor, Claude Code, or another familiar coding agent.

After creating the repository, Brennan adds a repository microagent describing the CVE-remediation project and pointing to SDK documentation. He then walks through GitHub’s Settings → Developer settings → Personal access tokens, choosing a classic token with repo scope so agents can open pull requests. The OpenHands profile’s API Keys page supplies credentials for its LLM proxy. He keeps the tokens off screen, clones the repository, and starts the agent-server container using the presentation’s command.

The spoken CLI installation command is:

uv tool install openhands

Brennan then starts the CLI, while reiterating that the authoring tool is interchangeable. The package installation becomes a live troubleshooting issue later in the exercise.

The first prompt asks the coding agent to consult the SDK documentation and build a minimal LLM call. Brennan chooses Claude Sonnet 4 through the OpenHands proxy. Provider-specific model identifiers differ, so he directs attendees using Anthropic or OpenAI credentials to the LiteLLM documentation for the appropriate model string. The first milestone is simply a successful response from the configured model.

For repository context, he shows .openhands/microagents/repo.md, containing the project description and SDK documentation links; a compatible tool can instead use AGENTS.md. These are the workshop’s historical setup conventions. Current Agent Skills & Context documentation also covers AGENTS.md, legacy context formats, and SKILL.md; it does not make the recording’s imports, package behavior, or model identifiers current API guarantees. The Python script reads credentials from environment variables, which Brennan takes care not to commit. Its first run exposes an API-use error, which he pastes back into the coding agent for correction. The next attempt encounters an authentication error.

Meanwhile, an attendee using uv 0.9.6 reports that installation finds no executables in the OpenHands package and fails to install entry points. Brennan suggests trying uv 0.9.11, his own version, without establishing that as the fix. Another attendee reports creating a PR through the hosted web interface. Brennan says that is normally his preferred workflow: let the web agent push changes, then pull locally if needed. He chose the CLI here to make the locally running script easier to show and hand off.

The connectivity test eventually returns Hello, OpenHands. Brennan inspects the small construction behind it: initialize the LLM with a model and API key, then send a test message. Prompt two adds actual agent work. It accepts a GitHub repository, connects a remote workspace to the Docker agent server at localhost:8000, clones the repository into that environment, and asks an agent to scan for vulnerabilities. Asked how to interrupt the CLI, Brennan suggests Control-P to pause, followed by a correction message or continue.

The installation discussion continues separately. An attendee gets the -ai package to install, but others report a crash and a deprecation warning. Brennan believes openhands-ai is deprecated and says he will raise the issue with the team. He suggests a release-page executable, Docker, or uv run as alternatives. The discussion eventually turns to specifying a version of regular openhands, but no complete, confirmed working command emerges.

Brennan runs the scanner against a repository expected to contain vulnerabilities. The SDK displays activity much like the CLI: task-list updates, repository cloning, tool installation, and command execution. Trivy is missing, so the agent installs it and then runs the scan. This is the autonomous inner loop in concrete form—the agent resolves an environmental prerequisite as part of carrying out the task.

The generated Python code passes the previously configured LLM to an agent with terminal and file-editor tools. A remote workspace connects that agent to the Docker container. A Conversation holds the context for this unit of work and receives explicit task instructions. These objects have different responsibilities: the model supplies decisions, tools enable actions, the workspace supplies the execution environment, and the conversation maintains the task’s context. Brennan reports that this initial scanner run completes successfully.

The next prompt addresses a different problem: getting useful data out of the agent’s environment. Brennan asks the scanner to save only the vulnerabilities to a JSON file. The host can retrieve it through the workspace’s command-execution or file-manipulation functions. Before launching remediation agents, he first wants the script to iterate over the retrieved list and print it. This isolates the handoff from the more complex fan-out that will consume it.

For a JSON-array contract, the Python side can keep that boundary small and explicit:

import json
from pathlib import Path


def load_vulnerabilities(path: Path) -> list[dict[str, object]]:
    payload = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(payload, list):
        raise ValueError("Expected a JSON array of vulnerabilities")
    if not all(isinstance(item, dict) for item in payload):
        raise ValueError("Each vulnerability must be a JSON object")
    return payload


if __name__ == "__main__":
    for vulnerability in load_vulnerabilities(Path("vulnerabilities.json")):
        print(json.dumps(vulnerability, ensure_ascii=False))

Here vulnerabilities.json is the retrieved artifact; scanner-specific fields can be checked before a record becomes a remediation task. In the live run, vulnerability results appear, but the script then fails while parsing JSON. Successful scanning and successful structured output transfer are separate milestones.

An audience question about observation events clarifies what the SDK is exposing during execution. Running a command is an action; the command’s returned output is an observation. More generally, an LLM tool call produces an action, and the tool result produces an observation. Brennan describes these as core components of the agent’s trajectory, rather than merely optional hooks attached to it.

Near the end, Brennan says he has completed prompt three and observes a CVE list printing while an attendee discusses prompt four. He explains that the five successive prompts are meant to demonstrate the experience of building with the SDK, exposing each intermediate step instead of starting from a prebuilt Kubernetes package. The visible progress reaches scanning and work on the JSON handoff; the session does not show his planned parallel remediation fan-out completing or its resulting PRs.

An attendee asks whether the growing script should be divided into files or sections. Brennan agrees: keeping everything in one script is a presentation convenience. The terminal view during this discussion still shows JSON parsing and base64 decoding failures alongside an attempted output-transfer fix. Those failures make the boundary between agent execution and host-readable results especially concrete.

Brennan points to OpenHands/cve-demo as a more organized version, describing a CVE agent subclass that reuses the same LLM. That is the next structural step: preserve the working pieces while separating agent construction, scanning, result retrieval, and remediation into maintainable units. The script’s growth should follow verified capabilities—first a model response, then a scanner, then a readable vulnerability list—so each added layer rests on a result the developer can inspect.