AI Engineer Europe 2026
Don't Build Slop (4 Levels of AI Agent Maturity)
Read the talk
Don't Build Slop: Four Levels of AI Agent Maturity
Build a useful agent by testing the idea quickly, making its execution understandable, coordinating concurrent work, and moving long-running tasks into shared cloud environments.
From a talk by Ara Khan
Before you start: Familiarity with coding agents, tool calls and basic application development will help; the state-machine and Kanban concepts are explained as they arise.
Should you run fifteen agents or read every line?
Everyone else seems to have a room full of robots working for them. Should you have fifteen agents running while you supervise, or should you still read every line of code? That is the uncomfortable choice Ara Khan opens with: the apparent productivity of other people's agents makes it difficult to judge how much automation your own work actually needs.
The useful response is to slow down long enough to identify a problem worth solving. Sometimes that means building quickly to find out whether an idea works; sometimes it means deliberately slowing down to make it suitable for production. Khan illustrates the confusion with three similar-looking interfaces from Factory, Codex and Cursor. If the products look interchangeable, copying their appearance offers little guidance about what your agent should do.
His four maturity levels separate the questions that otherwise get bundled into “build an agent.”
| Level | Immediate question | Approach |
|---|---|---|
| 1 | Does the idea work? | Prototype with a framework |
| 2 | Can we control its behavior? | Build an explicit state machine |
| 3 | Can we coordinate the work? | Organize agents through Kanban |
| 4 | Can others run it easily? | Move execution to the cloud |
These are working heuristics for balancing experimentation, implementation, interaction and deployment. Each level addresses a different constraint.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use a framework to test the idea
Suppose you want an agent to aggregate emails. At this stage, finding the best model or designing a bespoke runtime may matter less than discovering whether the workflow is useful. LangChain and LangGraph are examples of frameworks that can get that experiment moving. Khan is explicit about his perspective: he builds Cline's agent experience himself and does not use these frameworks, so he is not offering a detailed comparison of them.
Khan estimates that a framework can produce a working prototype in half an hour for a rudimentary task where choosing the best model is not the priority. The immediate payoff is something concrete to try: you can see whether the agent helps and whether the proposed product has a reason to exist. The accompanying slide broadens the framework menu to LangChain, LangGraph, CrewAI, AutoGen and LlamaIndex.
The tradeoff appears when the experiment becomes a serious implementation. Khan's objection is that the customization, modularity and ability to adapt to future models that he wants are difficult to obtain through a framework. That is his opinionated reason to move to the second level, rather than a demonstrated limitation shared by every framework: once controlling the agent becomes the central problem, he prefers to own its implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rule one: know which state the agent is in
Building the agent yourself is too large a subject for one recipe, so Khan gives five rules. The first is to treat the agent as a state machine: a loop with conditions, transitions and terminal states. At any moment, you should be able to explain where execution is and what can happen next. That mental model remains useful even when the product surrounding the loop is sophisticated.
His example is a request to read a few files and explain them through Claude Code. The user supplies the task; the agent chooses a file-reading action; the tool returns the contents; the agent determines that it has enough information and calls a completion tool. The task ends at that explicit terminal state.
A small TypeScript implementation can make that control flow visible. Here, decide represents the model-facing decision step, while the runtime owns tool execution and termination:
typescript
import { readFile } from "node:fs/promises";
type FileResult = { path: string; contents: string };
type Action =
| { kind: "read_files"; paths: string[] }
| { kind: "complete"; explanation: string };
type State = {
task: string;
phase: "deciding" | "reading" | "complete";
files: FileResult[];
};
type Decide = (state: Readonly<State>) => Promise<Action>;
export async function runAgent(
task: string,
decide: Decide,
): Promise<string> {
const state: State = { task, phase: "deciding", files: [] };
while (true) {
state.phase = "deciding";
const action = await decide(state);
if (action.kind === "complete") {
state.phase = "complete";
return action.explanation;
}
state.phase = "reading";
for (const path of action.paths) {
const contents = await readFile(path, "utf8");
state.files.push({ path, contents });
}
}
}
The file results become inputs to the next decision, rather than ending the task by themselves. Khan notes that a more complex task can keep traversing the same state machine for eight to ten minutes, or even hours. Longer execution makes knowing the current state more valuable; it does not change the underlying loop.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rule two: every addition can make the agent worse
More instructions do not automatically produce better behavior. Large system prompts, accumulated edge cases and elaborate if/else logic can interfere with a frontier model's ability to do its job. Khan describes this as a bitter lesson from agent development: instructions introduced to compensate for earlier limitations may become unnecessary constraints as models improve.
He cites the Codex repository's prompts for GPT-5 and GPT-5.3, invoking a one-third size comparison to illustrate his view that newer models need shorter prompts. The comparison is not precise enough to establish a measured reduction: he does not identify the prompt files, revisions or counting method. The engineering point is to reconsider accumulated instructions instead of assuming each one remains beneficial. Khan describes excessive instruction load as making it harder for the model to determine which direction to follow.
Pruning can extend beyond the prompt. Khan reports that his team rewrote Cline in full after finding too much material inherited from older versions. He also tentatively recalls at least seven complete rewrites of Claude Code, while explicitly saying that its team would know better. Both examples serve his preference for removing obsolete scaffolding rather than continually building around it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rule three: let other agents build and test yours
An agent implementation should be easy to operate from a CLI. Khan calls the resulting development workflow a “pseudo RL pipeline”: another coding agent can change the implementation, run it, inspect the result and try again. The feedback here comes from building and testing the software; he does not describe a model-weight training procedure.
This changes how humans prepare a repository. Instead of only giving an AI instructions, developers also shape the environment so an AI can work through it effectively. Useful AGENTS.md instructions, appropriate skills and accessible CI/CD commands make the build-and-test loop easier to follow. End-to-end tests matter because the coding agent needs a way to check the resulting behavior after making changes.
With that loop in place, a long-running coding agent can work in a parallel thread, modify the implementation and test the result while you focus elsewhere. If building or testing requires a difficult sequence of manual interventions, that same friction limits agent-assisted maintenance. Ease of verification becomes part of the agent's architecture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rule four: spend human attention on architecture
High token throughput makes it easy to generate a large implementation before deciding whether its design makes sense. Khan's fourth rule—don't build slop—is a request to spend time on the intended behavior, architecture and design outline before letting generation carry the project forward.
Read the code even if you did not write it by hand. In Khan's experience, the architectural work needs thoughtful human ownership: what the agent is supposed to do, which states it can enter, and how those states connect. An agent can help you reason through that design in conversation, but the conversation should produce decisions you understand. Simply allowing models to keep generating code does not supply that understanding.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Rule five: preserve each provider's API semantics
Switching models is not always as simple as changing an endpoint and a model name. Khan characterizes provider-specific API behavior as a source of lock-in because it makes models harder to interchange. His examples are Opus 4.6, Gemini 3.1 Pro and GPT-5.3 Codex, whose reasoning-related conversation state requires careful handling.
He uses the broad term reasoning traces for information that participates in caching and the model's reasoning loop. Across conversational turns, the application needs to preserve and return that information in the format the provider expects. Flattening a conversation into ordinary text can therefore lose behavior that matters even when the visible messages look intact.
Khan warns that incorrect handling can leave responses apparently working while degrading performance. Failure behavior, however, depends on the provider and request structure. Current Gemini thought-signature documentation specifies a 400 error when a required signature is omitted from the first function call in a Gemini 3 current-turn step. Current Claude extended-thinking guidance also distinguishes modes: Opus 4.6 supports interleaved thinking through adaptive mode, not manual extended thinking. These current contracts qualify the talk's broad warning; they are not a single shared reasoning protocol.
A routing layer does not remove the need to understand those differences. Khan specifically argues that using OpenRouter alone is insufficient assurance that every provider integration behaves correctly. The practical requirement is to test the conversation and tool-use path for each integration, including the reasoning-related state it must carry between requests.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Coordinate concurrent work through Kanban
Once the agent works, the next question is how to interact with several agents at once. Returning to the similar interfaces from the opening, Khan proposes Kanban as a useful form factor, familiar from products such as Linear. He dates his own advocacy to a March 26 tweet.
The reason starts with waiting. Khan describes Codex or Opus tasks running for eight to ten minutes, motivating users to keep two or three agents working concurrently. When inference is the limiting factor, starting another useful task can fill that waiting time. But those agents may be changing the same source, so parallel execution needs isolation of mutable state. A board can display the work; the underlying execution arrangement still has to prevent tasks from interfering with one another.
Kanban supplies a headline-level view of each task and makes dependencies easier to reason about: when these two tasks finish, start the next one. The user's role begins to resemble an engineering manager coordinating individual contributors. Instead of following every agent's output continuously, you can inspect the work that currently needs attention.
In this talk, scheduled for April 10, 2026, Khan says Claude Code had introduced a similar interface about ten hours earlier; that relative release timing remains his account. He names Claude Code and Cline Kanban as workflow options. His emphasis is on the management pattern rather than requiring a particular coding agent.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Move the environment off each user's machine
An agent can be useful, tested and easy to inspect without being easy to distribute. Khan poses the next level as a scaling question: what would it take to support millions of tasks or users, or make the agent accessible throughout an 8,000-person company? Those are deployment scenarios, not scale results he demonstrates. Requiring every person to install software and maintain a complicated local workflow makes access harder.
His proposal is to do the environment work centrally and run the agents in the cloud. A separate machine for each agent provides a place for parallel execution without depending on each user's local setup. The agent can prepare its environment and perform interface tasks there. Cloud execution also creates room for longer runs, a capability Khan considers underused.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Send a UX task, then let implementation and QA iterate
Khan reports sending cloud-agent tasks from his phone that run for fifteen to twenty minutes. His example is a VS Code extension change with an explicit interaction sequence:
- Build the extension.
- Sign in.
- Open settings.
- Select the requested theme.
- Test the result in the terminal.
The task includes both implementation and exercising the resulting interface.
In Khan's account, the cloud agent performs the QA clicks, checks whether they worked and keeps iterating when they fail. He says an extended implementation-and-QA cycle could take fifty to sixty minutes. The point of remote execution is that the task can continue through that cycle without occupying the user's machine or demanding continuous attention.
Several such tasks can be dispatched from a phone or laptop. Later, back at the laptop, the user can pull down the resulting PR. The useful handoff is a change that has gone through the requested interaction checks, ready for the next stage of human work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Share the setup, but start with the minimum
A cloud environment also gives collaborators a common setup they can share and modify. Combined with a board for coordinating tasks, that leads to Khan's forecast: much of agent interaction will move toward Kanban, while much of the actual execution will move to cloud compute. It is a prediction about the division between the work interface and the machines doing the work.
That forecast does not make the last two levels easy or mandatory. Khan calls Kanban coordination and cloud deployment intricate problems, then returns to the simplest starting point: build the bare minimum that helps. Move up or down the levels according to the effort the problem warrants. He closes the main talk by acknowledging that these are opinionated heuristics with open questions, rather than a complete prescription for every agent product.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Planning still happens inside the task
The audience question exposes an important requirement for the Kanban interface: where does back-and-forth planning happen? Requirements gathering can be the most useful part of working with an agent, especially when the agent asks questions that clarify what the user wants. A task card must leave room for that conversation.
Khan answers by opening a task and explaining that it exposes the entire task trace. He identifies the interface inside it as the actual Codex CLI. His workflow is to converse there until the task is clear enough for autonomous work, then leave it running and focus elsewhere. The board provides the overview while the task retains the detailed interaction.
A follow-up asks whether the task changes state when it needs review or has a question. Khan says yes: in his file-reading example, the task begins in in progress and moves to review when it needs human input. That transition is the return path from delegation to conversation. The agent can work independently while the requirements are sufficient, then make the need for human attention visible on the board.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
Further reading
- Introducing Cline KanbanArticle
Cline's launch article explains live agent task cards, task dependencies, and orchestration across Cline, Claude Code and Codex.
Updates since the talk
Provider documentation for preserving thought signatures across tool calls, including Gemini 3 validation errors.
Current guidance on manual thinking budgets, adaptive thinking, and model-specific support for interleaved reasoning.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hi there.
- 0:15
Um, very nice to see you guys. Um, it's, it's, like, it's so interesting to see, like, it's been such a long day, and you guys still showed up. And, uh, I'm always very flattered when, like, I'm trying to present something and, like, people are there.
- 0:29
It just, uh ... I- I- It f- it makes you feel like what you do matters. So today, the topic that I wanna talk about is, like, don't build slop: Four levels of AI agent maturity.
- 0:39
And what I'm trying to do here is that I wanna ... I'm gonna talk about something that, like, a lot of people have brought this up to me, and there's like a mass psychosis problem.
- 0:47
So basically, it's something like ... It's like every person feels like, um, there's, like, this, like, giant set of robots around all of you that are just, like, kind of like breezing through, doing a lot of things.
- 0:58
And, and you're, like, in the middle, and you're so confused. You're so confused of, like, what is this? What, what should I do? Should I have, like, 15 agents, like, ripping through all the time and I'm just, like, vibing?
- 1:10
Or, or should I just be like a peasant and just, like, read through every line of code? And it's, it's very hard to, like, make up your mind. I feel like you come to a place like this, and you're always ...
- 1:19
Like, the formula just, like, gets to you. You get, like, a panic attack or something. But, like, I think, I think ... I wanna ... I kinda wanna pull back out of this, and I wanna be like, "Guys, let's slow down.
- 1:28
Let's just slow down, and let's think through, like, what are the problems that we can, like, actually solve that will, like, actually help you build, like, really useful agents?"
- 1:36
And I wanna a- take care of, like, every end of the spectrum of, like, your necessity to build things really fast, but at the same time, like, your necessity to, at a certain point, build things slow and take things to production.
- 1:47
So that, that's basically, um, the goal. To give you an example of, like, mass psychosis, like there's a lot of things that are very similar. So I'll give, give you example of, like, three UIs.
- 1:56
And these are three frontier labs. And I guarantee you, not one of you can predict which one is which. Uh, one of them is Factory, one of them is Codex, and one of them is Cursor.
- 2:05
I, I am positive none of you know which one is which. Even I don't know which one [chuckles] is which. But yeah. So that's why I say the point. Like, everything is same.
- 2:12
You, you wanna do your own thing. So to build agents in particular, uh, here's what, here's, here's what I would, would present here, that I wanna break down the problem of building agents into, like, four different parts.
- 2:23
The first part is just, like, trying to somehow figure out a way to, like, see if this even makes sense, like this even works. That's, like, where you use a framework.
- 2:31
The second part is, like, when you actually are serious. Like, okay, I actually wanna do something here. Now you build things by yourself, like, as, like, a state machine, like building an actual agent.
- 2:40
The third part is, like, the UX workflow, where you use Kanban, which I suggest is, like, a great, uh, form factor to be able to work with agents. And the fourth part would be, like, shipping to cloud.
- 2:50
So this is, like, mostly out- outline read of, like, how we wanna be able to, uh, work with agents, but I think it will give you a good heuristics of, like, how you, how you would, uh, solve this problem.
- 3:00
So level one of building agents is, like, literally just, like, just, just use a framework. And I, I think there's, like, a couple different frameworks, uh, like LangChain, LangGraph.
- 3:08
Like, I, I wouldn't ... I don't use them. I, I, I work at, like, client. I've, I've ... I'm supposed to, like, write all the agentic experience stuff by myself, so why would I use a framework?
- 3:18
So I won't be the best person to give you advice. But I do think that if you're trying to find PMF, if you have a problem that's like, "Hey, I have this problem where I wanna be able to, I don't know, aggregate emails or do something, like, rudimentary, and I think that, like, an AI agent would probably
- 3:32
be helpful here. I probably don't care about the best model. I just, I just want something that works." Any of the agent frameworks can give you something that just works in, like, half an hour.
- 3:39
You can just vibe code this. It, it, it's, it's a great thing to get started. It's a great thing to see that agents actually do work and that you can build them yourself.
- 3:47
There are a lot of pitfalls with, uh, using frameworks, and one of the biggest ones that I think is that, like, if you really want to take things to production, if you really wanna build something serious, very quickly you'll learn that, like, the level of customizability, the level of, like, futuristicness, the level of, um, modularity that you
- 4:07
need, you just ... you won't find them in a framework. I know a lot of people will disagree with me, and a lot of those people are wrong. Anyway [chuckles], um, so level two is, like, building, building your agents yourself, right?
- 4:17
So here's how you, here's how you build. You, like ... I feel like, I feel like this is, like, a very intricate problem. Like, I can't sum this down.
- 4:22
So I'll, I'll just, like, give you, like, five rules of, like, when you actually write code to build your agents, like, there, there are five rules that you could use.
- 4:29
Um, and these rules will, like, give you a rough outline of, like, how to, how to build and write, write code for your own agent. So the first one is, like, you always wanna think of every agent as, like, a state machine.
- 4:40
Um, state machine, it's, like, a sophomore year or freshman year topic. But, like, state machine is basically, like, every agent is, at, at the end of the day, is like a recursive loop.
- 4:48
That's, that's basically ... Like, every, every, every hype cycle, whatever you think, it's at the end of the ... It's all a recursive while loop. It's a while loop with a few conditions.
- 4:55
And no matter what your agent's doing, it, it always ... It doesn't matter it's Cursor, Clock, or whatever, it is a while loop with a few conditions and a few end states.
- 5:02
And what you wanna be able to do is that at any point of time, you wanna be able to have, like, a mental model of which point in the state it is.
- 5:11
So let's say you wanna be able to, like, say, um ... Okay. So let's say you wanna be able to, uh, I guess, read a few files and explain them, uh, through Clock code.
- 5:22
So it starts from the user task at the top where you ask it, like, "Read a few files." It goes to the state of, like, reading a few files in, like, the action tool where it reads the file.
- 5:30
Then it realize, "Oh, I have read the file. It makes sense." And then it will call the completion tool and just, like, complete. And the, the right, right thing at the bottom, task complete, is when the state machine finishes.
- 5:39
Uh, you can take this in a very complex way where you can, like, rip through the whole state machine for, like, eight to 10 minutes or even, like, hours if you wanted to.
- 5:47
Uh, but essentially, every agent is a state machine. And if you can visualize that as a mental model, then every time you're building an agent, it will be so much easier for you to think through that.
- 5:55
Um, the second rule is that every single thing you add to an agent risks making it worse. I think this is the hardest thing that we've learned, and it has been a very bitter lesson that a lot of agent builders have learned, which is that large system prompts, lots of different edge cases, lots of different, like, fancy
- 6:12
if/else logic, all of that for frontier model just, just makes them worse Just like, just, just get out of the way of the model is the, is the lesson that we learned, which is that, like, frontier models are so good at their job that the less instructions you give them, they actually perform better.
- 6:29
A classic example is that if you go through the Codex repo, the prompt for GPT-5 versus the prompt for GPT-5.3 is one-third of the size. Uh, part of the reason for that is that the newer models are so good at their job that giving them too many instructions and longer system prompts, it leads to a sensory overload
- 6:45
where they get so many instructions that they get overwhelmed and can't figure out what the right thing to do. Um, so I think the simpler it is, it's always better, and it's like you have to think that, like, every single thing I'm adding, I, I'll be very careful that I'm-- I, I hope to God I'm not making
- 6:59
it worse, and just always try to prune it down. We, we took it so far where we literally rewrote the entirety of Cline because we realized there was so much, uh, junk from the older, older versions of Cline.
- 7:10
Claude code has been written, I think, at, at least seven times from scratch, I think. Um, but again, um, the, the, the people of the team might know better.
- 7:19
Um, the third rule is that you wanna be able to make agent an easy part of a pseudo RL pipeline. And this is a tricky one, but basically what this means is that anytime you're building an agent, you wanna be able to have some sort of like a CLI kind of a thing.
- 7:34
The reason for that is that as long as you have something that can build and test the agent really well in the form of a CLI, um, you wanna be able to build things that are very easy to build and test with, um, other coding agents.
- 7:49
So right now there's this thing, there's this, like, interactive dance that's happening between AI and humans where at-- in the back in the day, humans used to, like, guide AI, like, do this.
- 7:59
And I think at this point, we're at a point where humans are being guided by AI. And this is, I think, is a part of that where, like, if you're as a human, you wanna be able to build things in such a way that, like, the AI can work very easily through it.
- 8:10
So that might entail writing a- writing, you know, AGENTS.md the right way using the right skills, and building like a CL- CI or, or CD, um, such that like the agent, other coding agents can easily build your agent, test it, make changes to it, and then test it end-to-end is a very critical part.
- 8:27
Because that way, if you wanna make changes, you can just let, let a long-running agent run through in a parallel thread. It'll make all those changes, test it, and you have the whole thing running.
- 8:35
But if it's harder to build and test, it, it would also be harder for you to, like, use agents to work on your agent. Um, super meta. Um, rule number four, don't build slop.
- 8:45
Guys, don't. Please, for the love of God, don't, don't, don't build slop, please. I think that, I think that there's, there's like, there's so much like throughput that you can get and so much, uh, tokens that can go through really, really fast.
- 8:58
I think that the best lessons that we've learned as like real engineers is that, like, it's really worth spending some time just like thinking through the architecture, thinking through the design and the outline of what your agent's supposed to do, making sure it actually makes sense and, uh, and just like actually at least spend some time reading
- 9:16
the code, even if you don't write everything m-- by hand. I think that that is like, for us, we found that that to be super critical because I think that it, it's at least the architecture point of building an agent has to be done by a human and has to be done like very thoughtfully.
- 9:30
Even if you're using like an agent to have a conversation with it, spend a lot of time like trying to think through like what the architecture, what the state machine is going to be.
- 9:38
Don't, don't just let like, uh, other models like rip through the code. Um, and then the f-- rule fifth is Frontier labs kinda wanna lock you down, and this is a tricky one.
- 9:47
A lot of people will disagree, but basically what's happening is that a lot of times when you are working with the APIs of Frontier labs, the APIs are trying to lock you down, and they make the interchangeability harder.
- 9:58
So to give a very precise example, the new set of models that have come out, say, Opus 4.6, uh, Gemini 3.1 Pro, 5.3 Codex, they have this thing called reasoning traces.
- 10:08
And reasoning traces are, um, a part of the cache, and they're also the part of the reasoning, uh, uh, test-time compute loop that the model does. And when you have conversations and back-and-forth conversations with those models, you wanna send the reasoning traces in the exact precise format that is expected.
- 10:27
If you don't, the response would still work except that the performance would be degraded, and you would have no way of knowing. And a lot of people are kind of missing on the massive performance gains that are coming from the new models because they're just like not using the APIs in the exact precise way that they're supposed
- 10:42
to be used. Um, and there are asymmetries in the API. So some would argue that, "Hey, maybe you could use OpenRouter," but I don't think that's enough. I think you really wanna be careful that like if you're using different Frontier labs and different, uh, Frontier lab APIs, like they, they have-- you very carefully thought through, uh, if
- 10:58
the API is working correctly and you've actually tested it. Um, so the form factor is like the next step is like the form factor of like how do you like, how do you visualize the agents, right?
- 11:09
So I think originally I came back to like in the one of the previous slides, I tried to show you guys like the thing where like Codex and, uh, Cursor and others, they were all looking the same.
- 11:18
And I think I have a different claim. So for this, uh, uh, on March 26th, I made a tweet, uh, where I said we-- people should use Kanban. So Kanban boards, I think if anyone has used Linear, whatever, I'm sure all of you are familiar with Kanban boards.
- 11:32
So my argument is that Kanban boards are the thing, uh, to use. Um, Henzen in the audience was gracious enough to offer me his thoughts as well. So thank you so much.
- 11:41
So Kanban boards are, are, are this idea that like if you're working through an agent, you're always, uh, inference bound. A lot of you are working through like Codex or Opus, and it's working for eight to ten minutes at a time.
- 11:53
When one of the agent is working for eight to ten minutes, what do you do? You could doomscroll, but you can only doomscroll, scroll long. So then you run another agent, right?
- 12:03
So that way you have at least like two or three agents running in parallel at all times because you're inference bound and they're all like mutating the same source potentially.
- 12:12
So you wanna isolate the thing that they're mutating, uh, to take care of that, like the, the, the isolation of state and then the inference bounding. The best UX form factor to me is Kanban board, mainly because it takes care of-- it gives you like-- it makes you like an engineering manager which can look at all your
- 12:29
agents. And you ... Gives you, like, a headline-level view of what they are, and it also helps you build, like, flows with them that like, okay, if these two tasks fir- finish first, then I'll do the other task.
- 12:40
And that helps you, like, become, like ... You, you basically become like an engineering manager, and all your agents are your ICs and, and you can, uh, w- uh, look at them through this.
- 12:50
Um, so I was making this claim on March 26, and 10 hours ago Cloud Code came out with the same thing, so I believe I was right. Um, um, okay.
- 12:59
So yeah, you can use that through Cloud Code, Cline, wherever. You can use Cline as well. Whatever, uh, whatever works for you. Um, so yeah. So you wanna think of, like, ca- uh, Kanban as basically like an engineering manager, which y- you would be.
- 13:11
Then there's, like, a final step of, like, okay, you have, you have, like, an agent. You've tested it, you made it, and you, you have a good UX form factor to be able to, like, interface with it and look at it.
- 13:23
Uh, what do you, what do you do then? Like, how do you, how do you, how do you, how do you have an agent that's just, like, useful, it works well, but it, it scales?
- 13:32
It really scales to, like, millions of tasks, millions of users. If you're working for a company which has, say, 8,000 people, how do you, like, how do you make sure that, like, all of them can, like, very easily interface with this?
- 13:43
And I think that rather than, rather than, like, making people install and, like, you know, s- having these complex workflows in the machines and stuff, it's just, it's just so much junk and it's so hard.
- 13:53
Just, like, take it all, take it all out. Put, put the hard work once. Just take it all on the cloud. Um, so there are many benefits of cloud agents, but the primary one is that, like, you can completely paralyze and have, like, a separate machine for each of them.
- 14:08
There are, like, no local dependencies. Like, um, in the cloud, like, the agent can set up the environment, do all the UX tasks. Because right now, one of the most missing pieces, the thing that a lot of people are not using, that I use fairly extensively, is cloud agents, because they can really run, uh, really long.
- 14:26
So I, I on my phone often would send tasks that would run on a cloud agent for like 15 to 20 minutes. So let's say it's, like, a UX change of, like, um, go build this, uh, VS Code extension, and this VS Code extension I want you to sign in here, I want you to click on settings,
- 14:42
I wanna change the settings to, um, uh, pick this theme, and then I want you to test this thing in the terminal. And the cloud agents are so good that they will actually manually do all the clicks of the Q&A testing that I just described on their own, figure out if they worked, and if they didn't work,
- 14:59
it will just keep iterating, keep trying. And this thing could easily take, like, 50 to 60 minutes. But if you send, like, a lot of these tasks in parallel through your phone or through your laptop or whatever, whatever, um, I think you, you have, like, you have this, like, really easy, customizable, extensible thing, and it helps to
- 15:18
scale really fast. And then you could send the, all these tasks, like, running in, like, the, like the cloud machine. Like, you know, like a messiah. And you could come back to your laptop, and then it's like, oh, you could just pull down the PR, and then you've got the whole thing going.
- 15:32
Um, the other aspect of, like, cloud, cloud agents is that, like, if you're working with a lot of different people, um, I think that it's just, like, it helps you build, like, a common setup that, like, so many other people can share and so many people can mutate.
- 15:46
Um, so I think bringing that together, I think would be, like, the final form factor. So my f- my claim is that, like, there exists a future where, um, most of the UX of working with agents would be Kanban, and then most of the actual compute, uh, that's involved with, uh, with agents would be on the, on
- 16:04
the cloud. Um, I think these four levels of frameworks are just, like, some ... The last part of, like, you know, Kanban and shipping to cloud, like, those things are very difficult and very intricate problems, and I think that if I were you, I would just, like, use this, like, as, like, rough heuristics and start with, like,
- 16:19
okay, let me just, like, let me just do the bare minimum easy thing and then just, like, depending on how much effort I wanna put in, I will, like, slide up and down, um, up and down these levels.
- 16:29
So yeah. So that's it for me. And yeah. So, um, I, I made a lot of hot takes. I feel like I think I left a lot of, like, open-ended questions here.
- 16:38
If you have any questions for you, any thoughts, this is my Twitter. You're very welcome to reach out to me, send me any questions. And, uh, yeah. Um, it was very, very kind of you guys to give me your time.
- 16:49
Uh, thank you so much. [audience applauds] May I take a photo of you? All right.
- 17:00
Okay. All right. Thank you. And do you guys have any questions? It's, like, I've got a minute. [singing]
- 17:13
All right. This is it. [laughs] Oh. Yeah.
- 17:16
Um, how do you do planning inside of a Kanban? Uh, like, uh, I, I find the sort of, like, back and forth requirements gathering, you know, the, the, uh, getting the agent to figure out what it wants from me to be the most useful part of the-
- 17:30
Oh, yeah, yeah, yeah. Yeah, yeah.
- 17:32
So how do you handle that?
- 17:32
Oh, yeah. Let me, let me show you. Let me show you. So basically, like, I think my interpretation of, of, of Kanban is that, like, it's just, like ...
- 17:40
If you see the screen, like, you can go into any task, and it will give you, like, the entire trace of the task. Uh, and that point, it's like the ...
- 17:48
So this, this interface that you're looking at is basically, like, um, the actual CLI of Codex. And I think I would ... I, what I, what I usually do is just, like, have a conversation here, and then I know at a certain point that, like, "Bro, you can go out on your own, do your thing."
- 18:02
At that point I'll just pull out and just focus on other things.
- 18:05
And then d- does it transition state when it either needs your review or asks you a question?
- 18:09
Yes. Yes. Yes. So, so initially the phase, like ... It's like, let's say I would say, like, uh, if I say read a few files or whatever, right? Um, so it will be, like, initially it's in, in the in progress state, and then when it's, uh, when it's, like, it needs my input, it will go through review.
- 18:25
Um, yeah. All right. Thank you. [upbeat music]