How One Founder Manages 20 AI Coding Agents at Once - Ryan Carson, Untangle

Untangle began as a consumer app for people going through a divorce. It soon shifted to selling an AI paralegal to family law firms. Founder and CEO Ryan Carson had seen his previous startup grow to about 110 people, and one part of the management job felt familiar. Someone still had to talk to customers, decide what mattered, and turn that decision into work. At Untangle, people kept the first two responsibilities and gave a cloud agent session the job of engineering manager.

The manager leaves all the coding to child sessions called builders. Carson mostly talks to the manager by voice because he finds a spoken update easier to absorb than a written one. He can follow one conversation while several changes are underway.

A Linear ticket board tells the manager what needs building. Untangle aims to size each ticket to one pull request (PR). That gives a builder a defined change to make and the team a single PR to review. The manager uses dependencies between tickets to decide which work can start next.

About 20 builders can work concurrently under one manager. Carson puts the working range at roughly five to 25. The upper end depends partly on the person listening to the updates. Even if the manager can reach every builder, it can have more changes underway than its human can follow.

Carson stays out of the individual builder sessions. A builder with a technical question sends it to the manager, which can often answer it or clear a small obstacle. Those ordinary questions are precisely the ones worth delegating. If each required Carson to open a session and recover its context, he would still have 20 conversations to manage.

What makes the manager check on a builder that has gone quiet? A scheduled automation tells the manager to read every child session and move stalled work along. The interval shown in the talk is 10 minutes. By the time he presented, Carson had settled on 15. The script makes the rounds happen on schedule, something he found unreliable when left to the agent. The manager still decides what each builder needs.

Each builder produces code and tests. Where does it send the change for a check, and how does a finding get back to Carson? Two kinds of session branch off the builder.

• Code review. The builder calls a built-in reviewer and can revise the PR in response to its findings. Untangle usually caps the exchange at three rounds. At that point the builder can raise important findings with the engineering manager, which can bring in Carson if needed.
• Quality assurance. A separate process starts a child of the builder to run QA. This gives the change another check outside the exchange between the builder and its code reviewer.

The cost Carson watches is the delay at the merge queue. He considers a QA session cheap enough to start wherever it helps. But every extra round spent picking at a PR keeps that change waiting to land. The cap gives the builder a point at which to take an important finding to someone who can decide what to do about it.

What did the user actually see while the tests passed? Every builder records the feature in use. The built-in testing tool creates test cases, records the interaction, and gives each case a green or red result. For a UI change, the product owner can compare the two outputs. The results identify which checks passed. The recording shows what the feature did and how it looked while passing them.

The manager alerts Carson when a video is ready. Coming back to one PR after directing many builders, he first reads a short summary of what the change is supposed to do. That gives him a specific expectation to check as he watches. The decision to ship belongs to a person, and this is where the product owner has to give one change their full attention.

Just before the talk, Carson was speaking to the manager on his phone when it told him a video was ready. He could carry on the conversation just fine. When he played the recording, though, he could barely make out the change. Tempted to approve it anyway, he held off until he could watch on his machine. After hundreds of adjustments, Carson credited the process with helping Untangle ship a lot of high-quality PRs. It worked for his team at the time, and he expected it to change quickly.

Carson’s advice to move development to the cloud follows from how he runs a team. Take his example of ten people working at once. Each would typically have an engineering manager session, and one person can have more than one. Carson himself had two running. The sessions need to know which other sessions exist and be able to message them directly. When one has information another needs, they can unblock each other without asking a person to relay the question.

Database migration numbers illustrate what still needs coordination across the team. Two builders working on different PRs can each reach for the same next migration number. Each number looks reasonable in its own PR. Bring the changes together and the numbers collide. Untangle assigns another session, the merge queue manager, to coordinate details like these across incoming PRs.

Like the engineering manager, the merge queue manager is an ordinary cloud session. Carson names it for its job and gives it latitude to work out how to coordinate the wave of changes. GitHub provides the merge queue itself.

Running engineering this way changes the first question Untangle asks about a candidate. Can this person direct an agent through shipping a feature? Carson had found it easy to come away from a screening call impressed by someone’s hobbies or the way they spoke. Untangle delays that conversation until the team has seen the person work.

The first submission is an unpaid, roughly 45-minute recording of the candidate’s whole screen. The assignment is to ship a feature PR with an agent, typically on an application the candidate already maintains. Untangle leaves the method up to them. The PR shows what they shipped. The recording lets a reviewer follow the directions they gave the agent and what they did with its work.

A ranking agent takes the videos and transcripts and puts the candidates in order, saving someone the time it would take to watch every submission. Carson distrusted its first ranking. In hindsight, he thought he should have hired its first choice and attributed his resistance to his own bias. That was his judgment about one hiring decision. The recordings also vary in what they reveal. Some candidates narrate freely. When someone mumbles or works silently, a reviewer may see what they did and still have little idea why.

The five highest-ranked candidates receive an invitation to a paid trial. Untangle pays a fixed fee for 16 hours, asks each candidate to sign a nondisclosure agreement, and provides a seat in its cloud coding environment. This time the task is to get a real Untangle PR ready for merge. The team gets to see how someone who could ship a feature in a familiar app handles the work they would actually be hired to do.

Loom captures the trial, and Untangle gives the recordings to an agent for a recommendation. In the last round, Carson also watched 16 hours of footage himself. He found it grueling, but it gave him a feel for the candidates. He recommends choosing finalists before calling them, so an easy conversation has less chance to sway the assessment of their work. The call can then begin with the team already inclined to hire. It still serves two purposes Carson values: a final human judgment and a chance for the two people to connect.