The emerging skillset of wielding coding agents

How can experienced programmers try coding agents and reach opposite conclusions? Beyang Liu, Sourcegraph’s CTO and co-founder, opens with a disagreement among developers whose programming credentials are difficult to dismiss. In his retelling, Jonathan Blow—the creator of Braid—responds skeptically to Alex Albert’s enthusiasm about coding AI. Jessie Frazelle offers a distinction: perhaps someone in Blow’s rhetorical top 0.01% of programmers gains little, while the rest benefit substantially. Eric S. Raymond pushes back from another direction: an accomplished programmer can still find these tools useful. Thomas Ptacek’s My AI Skeptic Friends Are All Nuts makes the affirmative case especially directly.

Even among developers who find agents useful, the boundaries remain disputed. Are they good only for small edits, front-end work and weekend projects, or can they contribute to an existing production codebase? Liu connects this disagreement to Geoff Huntley’s interviews with developers at Canva using Cursor and other AI tools. Huntley found recurring misuse; Liu’s interpretation is that people were applying practices learned six months earlier to tools whose capabilities had already changed. Learning to use coding agents includes unlearning techniques that once worked.

Liu anchors this rapid evolution in ChatGPT’s November 2022 launch, then distinguishes successive application patterns. GPT-3 was a text-completion model: the developer typed, the model continued, and the developer resumed. Instruction-tuned GPT-3.5 made conversational questions natural. Developers then discovered that supplying examples from their own codebase helped the model follow local patterns and generate less invented code. Retrieval automated that context-gathering step, producing the chatbot-plus-RAG architecture Liu associates with 2023.

The transition to agents changes more than the input box. Once a model can gather context and act on it, the application can give it responsibility for a longer sequence of work.

Applications built around GPT-4 and Claude 3 inherited the constraints of chat-based interaction. Designing for tool-using models led Liu to three different choices:

• Apply edits directly. Asking for approval on every file change keeps the human inside the smallest execution loop. If the proposed change is wrong, the agent has already spent effort going in the wrong direction. Liu wants the developer steering and guiding the work without managing every edit.
• Keep the client light. A VS Code fork made sense when the interface needed extensive machinery for context selection and applying suggested changes. An agent that can locate code and edit it itself may need much less custom UI.
• Treat the model as part of the agent architecture. Replacing a model in a retrieval chatbot mainly changes how it responds to supplied context. Replacing the model inside an agent changes the decisions that drive tool calls and subsequent steps. Uneven tool-use ability makes that substitution consequential.

The same shift changes the economic comparison. Agents consume more tokens than chatbots, but Liu argues that the useful denominator is human time saved. His hypothetical $20-per-month subscription illustrates a conflicting incentive: like a gym membership, fixed pricing rewards the provider when the customer consumes less. Reducing inference expense by choosing a weaker model can transfer the cost back to the developer as additional work.

For tool design, Liu favors the Unix philosophy: small tools that compose with other tools, including command-line interfaces, over a vertically integrated application that owns every interaction. Sourcegraph had already built Cody, a retrieval-backed coding assistant. At the time of the talk, Liu said Cody remained supported and heavily used at Fortune 500 companies. The team nevertheless built a separate agent application so that the earlier product’s assumptions would not dictate the new one.

Early web portals exposed possibilities people might not yet know to ask for: celebrities, automobile shopping, movie reviews. As users learned what the web could do, navigating categories and hyperlinks became cumbersome. A single search box offered a more direct path. Liu uses that progression to explain Amp’s deliberately sparse interface: once users understand the agent’s capabilities, they should be able to express the work they want done.

The Amp version demonstrated here has two clients. A VS Code extension supplies the text interface while retaining useful editor features, especially diff viewing; Liu says he now spends more time in the diff view than the editor view. A CLI exposes the same tools and can be scripted or composed with other commands. The minimal interface moves complexity into the agent’s execution rather than requiring users to operate a large collection of UI controls.

The live task is a contribution to Amp itself, an application with real users and existing constraints. Liu opens its codebase in VS Code with Amp in the sidebar. Amp’s server provides the inference endpoint and team features, including shared usage information and a leaderboard. Connectors let it communicate with external services; the team’s issue tracker is Linear.

The problem is small but concrete: the Linear connector displays a generic network icon. When its MCP endpoint identifies it as Linear, Liu wants an icon appropriate to an issue tracker. He has already filed the request in Linear and asks Amp to find that issue and implement it. The task begins in the issue tracker, rather than with a manually selected source file.

Amp can read and edit files and run Bash commands. External tools can be attached through MCP, including Playwright, Postgres and the Linear integration. In this demonstration, the Linear MCP server mediates access to the issue API. Liu reports that the issue search returns 50 issues, with the requested connector-icon issue at the top. The agent selects tools and begins implementation without Liu specifying each call.

Routine API details stay out of the main conversation, but the work remains inspectable. Amp implements its search tool as a subagent that can use multiple search methods, including keyword and filename lookup. Expanding its result reveals the paths it followed, files it read and findings it collected. This gives the user a compact default view with a way to investigate the agent’s reasoning through the codebase when needed.

Amp supports concurrent threads and a keyboard shortcut for switching between them. That creates a new attention problem for developers accustomed to uninterrupted flow. Liu describes two useful secondary activities: choose something shallow enough to leave quickly, or learn more about the code involved in the primary task. He starts another thread asking how connectors and connections work in Amp, with a diagram to explain their relationship.

Meanwhile, the implementation thread reads relevant Svelte front-end files before changing them. A visible to-do list helps organize the task and discourages diving into code before understanding the surrounding system. Liu links this preparation to a more robust feedback cycle: without enough initial context, the developer ends up manually steering the agent through implementation details.

As edits arrive, the main human interface becomes the diff. Liu keeps VS Code’s diff view on a hotkey and prefers it to a GitHub PR or command-line git diff because he can inspect the whole file and use jump-to-definition. Asked whether Amp writes its own tests, he says it typically does, and can be prompted when it does not. During this run he describes incremental edits and diagnostic checks; his initial suggestion that it is probably running tests is tentative. The first visible result is an updated connector icon, achieved without step-by-step implementation instructions.

The change does not appear everywhere. Identifying a connector as Linear requires reading the MCP endpoint URL, which lives in its configuration. That configuration can also contain secrets, even though this particular example does not. Sending the entire configuration to a non-admin page would violate the application’s existing boundary. A seemingly cosmetic task therefore depends on how the server exposes data to different UI surfaces.

Liu’s follow-up identifies the concrete discrepancy: the icon changed on admin connections, but not on settings. He asks the agent to investigate why, then lets it continue while he returns to the slides. The intervention supplies an observed failure and a direction for investigation; it does not prescribe a patch or tell the agent to bypass the restriction.

At recording time, Amp’s public signup had been open for about two weeks, with little marketing. The team had deliberately cultivated users willing to experiment with how coding workflows might develop over the next six to twelve months. Liu reports rising average inference spending and considerable variation between users. Some Amp users were spending thousands of dollars per month on inference, Liu reports. The team initially suspected endpoint abuse, but conversations with those users revealed real development work. These interviews became a source of product decisions and operating practices.

One recurring behavior was writing long prompts. The most active users supplied detailed instructions and context rather than a few search keywords and an expectation that the model would infer the rest. To encourage that behavior, the demonstrated Amp interface makes Enter insert a newline and Command-Enter submit. That choice surprises users accustomed to Enter-to-submit in tools such as Cursor, but it makes room for treating a prompt as a substantive description of work.

Agents can search for context and invoke tests and linters, but an existing repository may have conventions that their default approach misses. A build may need to run from a particular subdirectory; a test suite may require a project-specific command. Liu observes that some users interpret this first failure as evidence that agents cannot handle back-end work. More effective users supply the missing instructions, allowing the agent to obtain the validation signal it needs to continue. Repository-specific feedback is part of the task context.

For front-end work, Liu describes a common combination: Playwright through MCP and Storybook for rendering components in isolation. The resulting cycle is straightforward:

1. Change the component’s code.
2. Use Playwright to open the relevant Storybook page.
3. Capture a snapshot of the result.
4. Use that feedback to revise the implementation and repeat.

Component isolation avoids loading the entire application for every iteration. The snapshot’s representation matters when configuring the tools: Playwright MCP distinguishes structured accessibility snapshots from screenshots, and Liu does not specify which he means here. The essential mechanism is the agent’s ability to observe the result of its change and act again.

This makes the human something like the agent’s developer-experience engineer: arrange the environment so that useful feedback arrives quickly. In response to an audience question, Liu identifies Playwright as a recommended tool in Amp’s tool panel. The workflow depends on providing that capability; the discussion does not imply that every agent session already has a browser connected.

Returning to the live task, Liu finds that the settings page now displays the customized icon too. He inspects the diff and judges that the agent appears to have found the right approach: the surrounding code already contains a mechanism for passing non-secret configuration fields to the UI. The agent reused that mechanism to expose the endpoint URL.

The important operation is explicit projection: construct the client-visible object from allowed fields, rather than forwarding the whole configuration. In a simplified TypeScript expression of that pattern, the projection remains unchanged when additional fields are added to the server-side configuration:

type ConnectorConfig = {
  endpoint: string;
  [field: string]: unknown;
};

type PublicConnectorConfig = {
  endpoint: string;
};

function toPublicConnectorConfig(
  config: ConnectorConfig,
): PublicConnectorConfig {
  return { endpoint: config.endpoint };
}

Here, only endpoint crosses the boundary. Any other configuration fields remain server-side because the function never copies them into its result. This illustrates the allowlist behavior Liu describes, rather than the exact names or types in Amp’s implementation.

That lets the settings UI distinguish the Linear connector without receiving the rest of its configuration. Liu describes the result as looking approximately right, supported by the visible icon and his diff inspection. The small visual change is useful precisely because it exercised a less visible constraint in a real codebase; the demonstration ends with that local result, not a shown merge or deployment.

Liu’s power-user observations challenge the expectation that agents necessarily make programmers less familiar with their code. He describes Tyler Bruno, a newly hired developer working full time while still in college, using Amp to learn how the codebase fits together. Diagrams and pointers to specific code make the tool useful for orientation. Code review presents a similar problem: the reviewer must understand unfamiliar code before deciding whether it is correct.

For a large diff, Liu uses the agent to prepare for review:

1. Ask it to read the diff and produce a high-level summary.
2. Ask where a capable senior developer should begin reviewing the change.
3. Use that entry point to inspect the code itself.

Finding a sensible starting point lowers the effort of beginning a review. Liu says this makes him less likely to postpone the work and helps him review more thoroughly. The summary is an orientation aid, not the review’s endpoint.

Long tasks introduce a different limit: accumulating context. Amp used a subagent for search from the beginning, and Liu sees additional uses for isolating complex subtasks. For the historical 200K-token context window of Sonnet 4, Liu reports degradation around 120–130K tokens and increasingly erratic behavior around 170K. These are his team’s observations, without a supplied task suite or quality metric. A subagent can contain the context consumed by a task such as implementing a small feature, keeping those working details from accumulating in the main agent’s context.

Micromanaging every interaction and under-specifying the task are opposite ways to limit an agent. Liu explains task knowledge as coming from training data or the context available during the work. A five-word request may suffice for a from-scratch 3D Flappy Bird game because the model has learned familiar patterns for that kind of project. A nuanced change in a large existing codebase needs the details a colleague would need: local constraints, relevant code and a way to establish correctness.

The human remains responsible for the code that ships. Liu rejects using agents merely to avoid reading code; he wants them to support more thorough review in less time. His language about becoming 10× or 100× more effective expresses that ambition, rather than a measured productivity result.

At the most intensive end of usage—what Liu rhetorically calls the top 1% of the 1%—developers orchestrate several agents at once. Liu describes Huntley’s roughly four-hour Twitter streams, in which three or four agents work on different parts of a compiler. Huntley reportedly constructs prompts and feedback loops carefully enough to leave the agents running while he sleeps. The example extends the same skill from one task to several: separate the work, communicate what each agent needs and provide checks that let execution continue without constant human intervention.

Liu sees composable tools as the route toward these agent fleets. Users can assemble interfaces around independent processes; he tentatively identifies Huntley’s setup as Tmux or another window manager. The value comes from enabling people to combine useful building blocks into workflows suited to their own engineering tasks.

Using coding agents well has a high ceiling, much like learning an editor or a programming language. Liu’s closing recommendation is to develop that skill through practice and share what works. Amp’s thread-sharing feature serves that purpose: a successful interaction can become an example for the rest of a team. He points attendees to an Amp owner’s manual and closes the conference invitation with a booth offer of $10 in free credits.

The final audience question brings the opening problem back to Liu’s own behavior. Why does he still type polite requests such as “can you” and stop to correct typos? He attributes the corrections partly to habit and partly to live-demo anxiety: a stray token might send the model off course. He associates that worry with his experience in 2023 and says newer models have become increasingly robust to typos. Even someone designing an agent product can carry yesterday’s interaction habits into a tool that no longer needs them.