Don't Build Slop (4 Levels of AI Agent Maturity)

Everyone else seems to have a room full of robots working for them. Should you have fifteen agents running while you supervise, or should you still read every line of code? That is the uncomfortable choice Ara Khan opens with: the apparent productivity of other people's agents makes it difficult to judge how much automation your own work actually needs.

The useful response is to slow down long enough to identify a problem worth solving. Sometimes that means building quickly to find out whether an idea works; sometimes it means deliberately slowing down to make it suitable for production. Khan illustrates the confusion with three similar-looking interfaces from Factory, Codex and Cursor. If the products look interchangeable, copying their appearance offers little guidance about what your agent should do.

His four maturity levels separate the questions that otherwise get bundled into “build an agent.”

These are working heuristics for balancing experimentation, implementation, interaction and deployment. Each level addresses a different constraint.

Suppose you want an agent to aggregate emails. At this stage, finding the best model or designing a bespoke runtime may matter less than discovering whether the workflow is useful. LangChain and LangGraph are examples of frameworks that can get that experiment moving. Khan is explicit about his perspective: he builds Cline's agent experience himself and does not use these frameworks, so he is not offering a detailed comparison of them.

Khan estimates that a framework can produce a working prototype in half an hour for a rudimentary task where choosing the best model is not the priority. The immediate payoff is something concrete to try: you can see whether the agent helps and whether the proposed product has a reason to exist. The accompanying slide broadens the framework menu to LangChain, LangGraph, CrewAI, AutoGen and LlamaIndex.

The tradeoff appears when the experiment becomes a serious implementation. Khan's objection is that the customization, modularity and ability to adapt to future models that he wants are difficult to obtain through a framework. That is his opinionated reason to move to the second level, rather than a demonstrated limitation shared by every framework: once controlling the agent becomes the central problem, he prefers to own its implementation.

Building the agent yourself is too large a subject for one recipe, so Khan gives five rules. The first is to treat the agent as a state machine: a loop with conditions, transitions and terminal states. At any moment, you should be able to explain where execution is and what can happen next. That mental model remains useful even when the product surrounding the loop is sophisticated.

His example is a request to read a few files and explain them through Claude Code. The user supplies the task; the agent chooses a file-reading action; the tool returns the contents; the agent determines that it has enough information and calls a completion tool. The task ends at that explicit terminal state.

A small TypeScript implementation can make that control flow visible. Here, decide represents the model-facing decision step, while the runtime owns tool execution and termination:

import { readFile } from "node:fs/promises";

type FileResult = { path: string; contents: string };
type Action =
  | { kind: "read_files"; paths: string[] }
  | { kind: "complete"; explanation: string };

type State = {
  task: string;
  phase: "deciding" | "reading" | "complete";
  files: FileResult[];
};

type Decide = (state: Readonly<State>) => Promise<Action>;

export async function runAgent(
  task: string,
  decide: Decide,
): Promise<string> {
  const state: State = { task, phase: "deciding", files: [] };

  while (true) {
    state.phase = "deciding";
    const action = await decide(state);

    if (action.kind === "complete") {
      state.phase = "complete";
      return action.explanation;
    }

    state.phase = "reading";
    for (const path of action.paths) {
      const contents = await readFile(path, "utf8");
      state.files.push({ path, contents });
    }
  }
}

The file results become inputs to the next decision, rather than ending the task by themselves. Khan notes that a more complex task can keep traversing the same state machine for eight to ten minutes, or even hours. Longer execution makes knowing the current state more valuable; it does not change the underlying loop.

More instructions do not automatically produce better behavior. Large system prompts, accumulated edge cases and elaborate if/else logic can interfere with a frontier model's ability to do its job. Khan describes this as a bitter lesson from agent development: instructions introduced to compensate for earlier limitations may become unnecessary constraints as models improve.

He cites the Codex repository's prompts for GPT-5 and GPT-5.3, invoking a one-third size comparison to illustrate his view that newer models need shorter prompts. The comparison is not precise enough to establish a measured reduction: he does not identify the prompt files, revisions or counting method. The engineering point is to reconsider accumulated instructions instead of assuming each one remains beneficial. Khan describes excessive instruction load as making it harder for the model to determine which direction to follow.

Pruning can extend beyond the prompt. Khan reports that his team rewrote Cline in full after finding too much material inherited from older versions. He also tentatively recalls at least seven complete rewrites of Claude Code, while explicitly saying that its team would know better. Both examples serve his preference for removing obsolete scaffolding rather than continually building around it.

An agent implementation should be easy to operate from a CLI. Khan calls the resulting development workflow a “pseudo RL pipeline”: another coding agent can change the implementation, run it, inspect the result and try again. The feedback here comes from building and testing the software; he does not describe a model-weight training procedure.

This changes how humans prepare a repository. Instead of only giving an AI instructions, developers also shape the environment so an AI can work through it effectively. Useful AGENTS.md instructions, appropriate skills and accessible CI/CD commands make the build-and-test loop easier to follow. End-to-end tests matter because the coding agent needs a way to check the resulting behavior after making changes.

With that loop in place, a long-running coding agent can work in a parallel thread, modify the implementation and test the result while you focus elsewhere. If building or testing requires a difficult sequence of manual interventions, that same friction limits agent-assisted maintenance. Ease of verification becomes part of the agent's architecture.

High token throughput makes it easy to generate a large implementation before deciding whether its design makes sense. Khan's fourth rule—don't build slop—is a request to spend time on the intended behavior, architecture and design outline before letting generation carry the project forward.

Read the code even if you did not write it by hand. In Khan's experience, the architectural work needs thoughtful human ownership: what the agent is supposed to do, which states it can enter, and how those states connect. An agent can help you reason through that design in conversation, but the conversation should produce decisions you understand. Simply allowing models to keep generating code does not supply that understanding.

Switching models is not always as simple as changing an endpoint and a model name. Khan characterizes provider-specific API behavior as a source of lock-in because it makes models harder to interchange. His examples are Opus 4.6, Gemini 3.1 Pro and GPT-5.3 Codex, whose reasoning-related conversation state requires careful handling.

He uses the broad term reasoning traces for information that participates in caching and the model's reasoning loop. Across conversational turns, the application needs to preserve and return that information in the format the provider expects. Flattening a conversation into ordinary text can therefore lose behavior that matters even when the visible messages look intact.

Khan warns that incorrect handling can leave responses apparently working while degrading performance. Failure behavior, however, depends on the provider and request structure. Current Gemini thought-signature documentation specifies a 400 error when a required signature is omitted from the first function call in a Gemini 3 current-turn step. Current Claude extended-thinking guidance also distinguishes modes: Opus 4.6 supports interleaved thinking through adaptive mode, not manual extended thinking. These current contracts qualify the talk's broad warning; they are not a single shared reasoning protocol.

A routing layer does not remove the need to understand those differences. Khan specifically argues that using OpenRouter alone is insufficient assurance that every provider integration behaves correctly. The practical requirement is to test the conversation and tool-use path for each integration, including the reasoning-related state it must carry between requests.

Once the agent works, the next question is how to interact with several agents at once. Returning to the similar interfaces from the opening, Khan proposes Kanban as a useful form factor, familiar from products such as Linear. He dates his own advocacy to a March 26 tweet.

The reason starts with waiting. Khan describes Codex or Opus tasks running for eight to ten minutes, motivating users to keep two or three agents working concurrently. When inference is the limiting factor, starting another useful task can fill that waiting time. But those agents may be changing the same source, so parallel execution needs isolation of mutable state. A board can display the work; the underlying execution arrangement still has to prevent tasks from interfering with one another.

Kanban supplies a headline-level view of each task and makes dependencies easier to reason about: when these two tasks finish, start the next one. The user's role begins to resemble an engineering manager coordinating individual contributors. Instead of following every agent's output continuously, you can inspect the work that currently needs attention.

In this talk, scheduled for April 10, 2026, Khan says Claude Code had introduced a similar interface about ten hours earlier; that relative release timing remains his account. He names Claude Code and Cline Kanban as workflow options. His emphasis is on the management pattern rather than requiring a particular coding agent.

An agent can be useful, tested and easy to inspect without being easy to distribute. Khan poses the next level as a scaling question: what would it take to support millions of tasks or users, or make the agent accessible throughout an 8,000-person company? Those are deployment scenarios, not scale results he demonstrates. Requiring every person to install software and maintain a complicated local workflow makes access harder.

His proposal is to do the environment work centrally and run the agents in the cloud. A separate machine for each agent provides a place for parallel execution without depending on each user's local setup. The agent can prepare its environment and perform interface tasks there. Cloud execution also creates room for longer runs, a capability Khan considers underused.

Khan reports sending cloud-agent tasks from his phone that run for fifteen to twenty minutes. His example is a VS Code extension change with an explicit interaction sequence:

1. Build the extension.
2. Sign in.
3. Open settings.
4. Select the requested theme.
5. Test the result in the terminal.

The task includes both implementation and exercising the resulting interface.

In Khan's account, the cloud agent performs the QA clicks, checks whether they worked and keeps iterating when they fail. He says an extended implementation-and-QA cycle could take fifty to sixty minutes. The point of remote execution is that the task can continue through that cycle without occupying the user's machine or demanding continuous attention.

Several such tasks can be dispatched from a phone or laptop. Later, back at the laptop, the user can pull down the resulting PR. The useful handoff is a change that has gone through the requested interaction checks, ready for the next stage of human work.

A cloud environment also gives collaborators a common setup they can share and modify. Combined with a board for coordinating tasks, that leads to Khan's forecast: much of agent interaction will move toward Kanban, while much of the actual execution will move to cloud compute. It is a prediction about the division between the work interface and the machines doing the work.

That forecast does not make the last two levels easy or mandatory. Khan calls Kanban coordination and cloud deployment intricate problems, then returns to the simplest starting point: build the bare minimum that helps. Move up or down the levels according to the effort the problem warrants. He closes the main talk by acknowledging that these are opinionated heuristics with open questions, rather than a complete prescription for every agent product.

The audience question exposes an important requirement for the Kanban interface: where does back-and-forth planning happen? Requirements gathering can be the most useful part of working with an agent, especially when the agent asks questions that clarify what the user wants. A task card must leave room for that conversation.

Khan answers by opening a task and explaining that it exposes the entire task trace. He identifies the interface inside it as the actual Codex CLI. His workflow is to converse there until the task is clear enough for autonomous work, then leave it running and focus elsewhere. The board provides the overview while the task retains the detailed interaction.

A follow-up asks whether the task changes state when it needs review or has a question. Khan says yes: in his file-reading example, the task begins in in progress and moves to review when it needs human input. That transition is the return path from delegation to conversation. The agent can work independently while the requirements are sufficient, then make the need for human attention visible on the board.