Prototyping as Leadership: How a CTO Ships with AI Agents

Fifteen-plus recurring meetings, seven direct reports, stand-ups, reviews and recruiting: Hursh Agrawal opens with his actual calendar. As CTO and co-founder of The Browser Company, makers of Arc and Dia, he has an organization to run. Yet something has changed in the gaps between appointments. Agrawal reports shipping two to ten PRs a week with coding agents while working a 40–50-hour week. With a toddler at home, extending the workday indefinitely is not the solution.

Traditionally, growing as a leader meant influencing more work through roadmaps, documents and meetings. You supplied context to engineers who could spend sustained time implementing it. As coding agents become capable of longer autonomous tasks, the fragmented day described in Paul Graham’s Maker’s Schedule, Manager’s Schedule becomes usable for building, too. The manager does not need to remain at the keyboard throughout the implementation.

Building becomes a way to acquire the judgment needed to lead. Agrawal describes frontier model families changing on roughly three-month cycles: their capabilities shift, their responses to prompting change, and previously impractical product ideas become plausible. Opinions about each release are plentiful; firsthand intuition is harder to acquire. Using the models while building helps a leader set engineering expectations and assess implications for the product, business and strategy. It also makes questions such as what might happen if open-source models catch up in three to six months concrete enough to explore, rather than treating that possible timeline as a certainty.

That intuition is difficult to transfer through enthusiasm alone. An engineer who has not tried the new model may reasonably remain unconvinced that it changes what the team can build. A working prototype moves the conversation forward: the capability is now inside the product, and colleagues can try it themselves. The artifact carries information that another persuasive meeting cannot.

Leaders also bring useful inputs to agent work: business priorities, strategic trade-offs, past decisions and upcoming imperatives. Agrawal argues that this wider context makes their steering especially consequential. Delegation skills transfer directly—set a goal, supply context, check progress and coach when necessary. In his experience with the models of the period, execution is stronger than judgment. When an agent declares an algorithm impossible or says it cannot finish a task, suggesting another approach can restart useful work. That assessment is explicitly time-dependent; it is a reason to keep testing the models’ limits.

Agrawal attributes four useful categories to Julie Zhuo’s poll of technical leaders:

• Internal efficiency: tools that make the organization work better.
• Quality of life: improvements to the codebase or product—the gardening that otherwise waits behind larger commitments.
• Celebration: artifacts that recognize people and their contributions.
• Vision: prototypes that connect new model capabilities to opportunities for the product and business.

The vision category is especially valuable because it combines learning with demonstration: discover what a model can do, then put that possibility in colleagues’ hands.

Keep this work off the critical path. Fires, recruiting calls and meetings still interrupt a leader’s day. A prototype can wait while those responsibilities take precedence; another engineer’s blocked delivery should not have to. The choice of task makes the schedule workable before any prompting technique does.

Agrawal’s example routine allocates two to three hours a day to coding, distributed around management responsibilities.

The final block is the crucial handoff. It turns a short period of attention into a task that can continue after the workday ends.

The loop starts by gathering context and answering clarifying questions. The coding agent then works unattended; Agrawal describes runs lasting four, six or eight hours. In the morning, its report gives him something concrete to assess and decide whether to ship. He applies this same structure to three kinds of work: implementing features, improving AI features through evaluations, and training custom machine-learning models.

For feature implementation, the useful question changes from how to break the work into individually prompted steps to what the model needs to know to make the decisions you would make. An agent working overnight cannot rely on continuous steering. It needs the business goals and the reasons behind the request, not just an implementation checklist.

Agrawal uses an agent connected to organizational information—Slack, Jira, Confluence, Notion and the repository—to prepare that context. Before a meeting, he asks it to research the proposed work and produce a prompt for the coding agent. The request covers prior attempts, trade-offs, what worked, what failed and why the feature or model matters to the business. His example is a roughly thirty-second dictation through Wispr Flow into Dia, requesting about twenty minutes of research. The result is a substantial prompt that can be pasted into Claude Code, Cursor or Codex.

The evening instruction can then be broad: implement the whole feature, using the gathered context. The research pass supplies the decision background that would otherwise remain in the leader’s head or scattered across internal systems.

An unattended implementation needs ways to check itself while it proceeds. Agrawal’s instructions establish a sequence:

1. Write tests first. In his experience, agents write less rigorous tests when they add them after the implementation.
2. Exercise the complete flow with computer use. Click through the interface and check whether the feature solves the business problem described in the context.
3. Prepare reviewer-friendly PRs. Split the work into readable changes with clear descriptions, monitor CI and get the whole stack passing.

The distinction between tests and end-to-end interaction matters: passing checks and a coherent product flow answer different questions.

The run should also include AI code review. The Browser Company has internal reviewers; Agrawal mentions Qodo as an external example and suggests running a review skill in a clean sub-agent when appropriate. The coding agent should then fix findings, watch PRs for bot comments and failing checks, and resolve review threads. This expands the assignment from producing code to preparing a reviewable stack.

Once setup questions are settled, he asks the agent to continue without overnight interruptions and leave a morning report. He also adds encouragement as a personal habit, while saying he does not know whether it helps. The required output is substantive: the completed stack, results and an explanation of trade-offs.

Agrawal says models such as Claude Opus 4.8 and newer GPT models have handled work he previously regarded as taking weeks in a single overnight run. That is his experience, not a controlled productivity benchmark. The practical benefit is the package waiting in the morning: something to inspect and improve, rather than a task still waiting to begin.

The first version of an LLM feature usually leaves quality, latency and cost to optimize. To create material for that work, Agrawal adds a feedback button and text box after the LLM or agent run. During a thirty- or sixty-minute usage block, he tries the feature and records qualitative judgments: good, bad, or what should have been better. Each run produces a JSON dump in his downloads folder containing the system prompt, inputs and feedback.

For a summarization feature, the same record structure could look like this:

{
  "system_prompt": "Summarize the meeting notes, preserving decisions and unresolved questions.",
  "inputs": {
    "notes": "We agreed to test the prototype with employees. Public release remains undecided."
  },
  "feedback": "The summary treated public release as approved. Preserve the distinction between an agreed employee test and an undecided public release."
}

The useful information is the connection between the instructions, a concrete input and the user’s judgment. That gives the next agent something more specific to work with than a request to improve quality.

Agrawal starts small: four, five or ten examples can begin the process, with coworkers helping collect twenty or thirty. At evening setup, he gives the agent those files and asks it to turn them into a local eval set, using something as simple as SQLite or Markdown. Before handing over execution, he discusses the evaluation design and scoring functions interactively. The human judgment in the feedback has to become an agreed criterion the harness can evaluate.

With scoring agreed, the agent builds a harness that runs the target call against the evals and iterates to improve the score. This is hill climbing: evaluate a candidate, make changes and evaluate again. A small dataset creates an obvious overfitting risk. Agrawal reports explicitly asking the agent to avoid overfitting and keep improvements general; that instruction expresses the goal, but does not itself establish generalization. The overnight assignment ends with a report describing the work and results.

He also asks the agent to save the workflow as a reusable skill. Subsequent runs can improve both the feature and the process used to optimize it. Agrawal reports that improvements persisted as features moved into employee dogfooding and production. The accompanying slide reports an evaluation result moving from 2/7 to 7/7 and code-review recall rising from 10% to 55%. These are internal reported results; the supplied explanation does not specify enough evaluation conditions to treat them as general benchmarks or evidence that overfitting is impossible.

An internal AI code reviewer is one concrete application. The team repeatedly used overnight runs to improve its quality, applying the same feedback, evaluation and iteration loop to a tool that supports future development work.

The third application extends the overnight loop to custom model training. Agrawal describes a ModernBERT classifier for personally identifiable information, or PII. Earlier attempts using Opus and Haiku had unsatisfactory cost, latency and precision-recall for the task. The starting point for a different approach was collected training data plus business context.

The agent’s assignment covered the training workflow, not just a training command:

1. Prepare the data. Clean the collected examples and augment them with synthetic data as needed, using access to OpenAI and Anthropic models as an ensemble to improve data quality.
2. Choose candidate models. Propose suitable model classes and train two candidates rather than committing to one immediately.
3. Use isolated infrastructure. Work within an AWS sandbox, explicitly without production access.
4. Complete the resource lifecycle. Select GPU and EC2 resources, train the candidates, test them against evaluation examples and deprovision the infrastructure.
5. Return deployable artifacts and a plan. Deliver the trained models and a report, including how inference could be hosted within the existing codebase.

The sandbox boundary is part of the task definition. The agent needs enough access to carry out the experiment without gaining the ability to disrupt production.

After answering clarifying questions, Agrawal leaves the run to proceed. He reports receiving two trained models and a full report the next morning, and says the team has moved multiple models from this kind of workflow into production. He supplies no numerical precision, recall, latency or cost measurements for this classifier, so the example demonstrates the scope of delegated work rather than a measured comparison between model families.

Across the three examples, the experiment is to keep increasing task scope. Agrawal cites Notion co-founder Simon Last’s observation that modern models can do more than people tend to assume. Testing those boundaries teaches a leader both how to direct engineering work and what new utility the product might offer. His invitation to try work on the scale of weeks or months in an overnight run is an ambition to explore, not an established universal speedup.

That ambition depends on existing organizational support. AI code reviewers, maintained CLAUDE.md and AGENTS.md instructions, and CI that the team can trust give autonomous work a framework for checking correctness. Feature flags and staged distribution limit where an experiment can have an effect. The Browser Company also has a prototype branch that distributes changes to employees without releasing them to production. This lets colleagues try the artifacts while preserving a separate production boundary.

The safeguards do not make agents infallible. Agrawal acknowledges that his code has annoyed engineers and caused severity incidents, or SEVs. He still considers building worthwhile for what it teaches and for the example it sets, but that example includes engineering hygiene as well as ambitious delegation.

The morning block therefore includes personal testing of what the agent produced. Agrawal recommends testing everything before putting the PR up; automated computer interaction does not replace trying the feature yourself. Small, readable PRs matter for the same reason: a leader establishes what others see as acceptable. His negative example is submitting three 5,000-line PRs and encouraging the team to imitate that behavior.

Read and review the generated code before requesting a colleague’s review. A review-ready stack from an agent is still your responsibility. Handing unread code to another engineer transfers the burden of discovering obvious mistakes—and undermines the standard you are trying to establish. Agrawal’s warning comes from having made that mistake himself.

His closing recommendation is to make building part of leadership, even with only one or two hours a day—less than the earlier example routine. Repeated practice develops a feel for each new model family and for the amount of scope it can handle. Working artifacts then show the organization what is possible, while the experience of making them generates ideas for the next overnight run.