The Agents Were Fine. Everything Around Them Broke. - Joey Wang, Harvey

In the month before Joey Wang’s talk, Spectre, Harvey’s internal cloud agent platform, started more than one agent per minute on average, day and night. About 460 people used it that month. Agents had written 10,000 pull requests, and people were finding other uses for the same setup. They asked data questions in Slack and got answers back. Someone in marketing dropped a prototype and a talk track into a channel and asked for help turning them into a story for a product video.

Why build Spectre when engineers already have coding agents they like? Spectre runs those agents. Wang put the practical limit on a laptop at roughly five to ten at a time. Moving work to the cloud buys capacity, but a remote agent still needs a repository checkout, credentials and access to Harvey’s stack. It needs a way to pick up an alert or ticket and get a change through review. Spectre supplies that harness. Sharing it across the company adds another responsibility. Even when one engineer has plenty of work to hand off, the next person needs room to run.

Engineers adopting Cursor had two immediate questions. How could they get access to Harvey’s secrets, and how could they test against the real stack? Wiring both into Cursor would leave the team with the same job the next time engineers switched tools. Harvey let engineers ask their preferred agent to use Spectre. A session could start on a laptop, move to the cloud and come back. The team could keep its test environment as engineers changed agents.

Take Wang’s example of an alert arriving in Slack at 2 a.m. An agent picks it up without anyone typing a prompt, writes a fix, spins up a copy of Harvey’s stack and posts evidence on the pull request. If continuous integration (CI) fails, the failure itself becomes input to the next attempt. A follow-up loop reads what failed and pushes another fix while the owner is asleep. By morning the PR is in the shared review queue under the same rules as any other change. Spectre waits until the owner’s working day begins to send a nudge. Once the change is approved, an agent can merge it automatically.

The follow-up loop was the stage Wang would build first. Harvey enabled it for everyone in early October. Of the agent-written PRs that merged into Harvey’s backend that month, four out of five had gone through the loop.

Testing a tricky service can depend on something one engineer knows, right down to which flag to leave alone. Spectre gave that knowledge a home in projects. People could record what mattered, agents could record what they learned, and a team could pin the skills it trusted. Any agent working in the project could draw on that knowledge. The project also kept a record of who ran what. Wang’s view was that whichever agent an engineer picked, the checks and the audit trail should be the same.

In May, people usually opened a browser tab or tagged Spectre in Slack. Exactly one task that month came from a terminal. The team adopted a rule that any action a person could click should also be callable by an agent. The command-line interface could start a task, turn on the follow-up loop, enable auto-merge or publish a page. An agent on a laptop, an agent already inside a sandbox and a CI job could all use those commands.

Where did the new work come from? By October the command line was starting about 6,500 tasks a month, while web-app usage held up. Some people started hundreds of tasks a month without ever opening the app. One conversation with an agent could now set a whole batch of work in motion.

Wang used that access in two directions over the course of October.

• Hand work out. Wang asked a coding session to coordinate a large ticket. It divided the work into ten milestones, kept one for Wang and handed the rest to Spectre as hundreds of tasks. Each task got its own sandbox for end-to-end validation.
• Bring decisions back. Later that month, another agent watched Spectre for questions that were holding up work. It listed the questions, drafted Wang’s replies and sent them back once Wang approved them. Wang could answer workers without visiting each task to find out what it needed.

Each sandbox also needed code to work on. When the team measured where task time went, the slowest step was Git checkout. Checking out all of Harvey’s repositories took more than a minute on average. The team kept a copy of every repository on every machine and let each task borrow the local copy. A burst of tasks on the same machine could reuse repositories the machine already had.

A mid-August release shipped a bug that fired an alert once per workspace. Hundreds of alerts landed in one Slack channel. A workflow tagged the agent on every alert, and every tag started a task. On an ordinary day that workflow started about 50 tasks. During the incident it started around 900 in four hours, all chasing the same bug until the cluster filled up.

What should happen when the next workspace reports the bug? Under the original workflow, that report was an instruction to launch a worker. With the fix, a repeat alert joined the investigation already open. The same investigation could collect reports from many workspaces.

Two months later, an engineer did exactly what the team had encouraged and handed work to Spectre. One laptop submitted about 950 command-line tasks in about three hours. The cluster filled, and everyone else’s work sat behind that laptop. Routing repeat alerts to an open investigation addressed the August failure. This time Spectre needed a way to accept a large batch and still let other people work.

Four days later the team had a fair launch queue. Submitting a task gave its owner a visible place in line. A dispatcher started work when the cluster had room, giving priority to people waiting at a keyboard. Each user had a fair share of capacity and could borrow more while the cluster was quiet. A large batch could use idle machines while the dispatcher retained control over who got the next opening.

What happened to completion as volume rose? On the busy August day Wang used for comparison, about one task in ten failed to finish. On November 4, Spectre handled more than twice that day’s volume, and a larger share of tasks finished. Several changes landed together, so Wang couldn’t say which fix contributed most.

At the end of October, Harvey’s developer platform team raised a different alarm. Company-wide PR volume had jumped around 60% in one month. Merging a change used to take 15 to 30 minutes. Now it took over an hour, and the team repeatedly had to switch off the merge queue.

With the queue off, a PR rebased onto main, reran CI and merged if the checks passed. Every PR depended on that shared starting point. If main broke, everyone’s CI broke with it. The fair launch queue could make room for people to start work. How to keep their finished changes moving through merge was still open.

Spectre’s fastest-growing source of work was automation. A schedule or an event could wake an agent to triage an alert, read a PR or write a post. About 110 people had automations running in the month before the talk. An unattended agent could start thousands of tasks, so each automation switched itself off after five consecutive failures. Its owner could also give it a budget or an expiry date.

Wang wondered what would happen if every Slack channel had a persistent agent that could follow a conversation and recognize when to begin work on its own. For reviewers, the question was whether every PR could arrive with evidence that the change worked before a human even looked at it. At the time of the talk, only one merge in 20 carried that kind of proof. And even a change that came with proof would face the question Harvey had yet to answer. How do you merge when agents keep opening PRs?