How We Solved Agent Building — Andrew Qu, Vercel

Andrew Qu, Vercel’s Chief of Software, begins with the infrastructure surrounding agents. The AI SDK supplies a common interface across model providers; he contrasts changing one line with replacing 300–400 lines of provider-specific code. Model fallbacks, secure execution, durability, and resumability address other operational needs. His internal experiment asks what useful business work those capabilities can support.

The ambition was “an agent on every desk.” At 1:46, that ambition turns into a practical search: ask people in marketing, sales, finance, and legal what they dislike doing. The strongest candidate comes from the data team, which is lean and growing more slowly than the company’s demand for analysis.

A question about a customer or product interrupts a data scientist’s existing work. They must write a query, process its output, analyze the result, and return a recommendation. Meanwhile, customer data, analytics, metrics, and sales information still need to be aggregated and made usable. Qu and the VP of data therefore target the whole question-to-answer workflow: completing individual requests should stop consuming so much of the team’s time.

The first prototype, introduced around 3:46, tests a small capability before automating the surrounding work. A dump of the Snowflake schema goes into a large system prompt with a question. The model returns SQL, and Qu copies it into the database interface and runs it himself. The result gives him enough confidence to continue improving the context and harness, while execution still depends on a person.

A useful answer requires several kinds of work:

• Interpretation and exploration: Understand the question, inspect the semantic layer, and find the join patterns that connect the relevant entities.
• Execution and revision: Run SQL, then revisit it if execution fails or the query is too expensive.
• Reporting: Turn the results into visualizations, written analysis, or a retrospective.

Generated SQL is an intermediate artifact. Execution can reveal that the earlier interpretation needs to change.

At 4:48, the second version gives those phases separate agents. D0, the data science agent, becomes a chain covering query interpretation, planning, execution, and reporting. Each specialist has a dedicated system prompt and tools restricted to its job. Schema exploration, for example, can read entity YAML and search schemas before passing its findings onward.

This version removes the manual SQL copying and completes the loop from question to answer. The architecture expresses the workflow through explicit handoffs: each stage does its assigned work, then gives the next stage enough information to continue. That is a real improvement in automation, but the contents of those handoffs become the next design problem.

The chain passes only a summary and a small snippet of preceding work to each downstream agent. At 5:47, the redesign begins with a different requirement: let the system look back through its work and reconsider how it reached its current state. D0 becomes one agent that shifts among planning, building, executing, and reporting while retaining the broader working context. Qu describes a possible maximum of 100 steps for that run.

Follow the join-error case through this change. An execution attempt exposes a problem with how the query connects entities. The single agent can return to exploration, read more of the semantic layer, reconsider the relationship it chose, and try execution again. The important change is the direction of travel: a failure can send the same run backward to investigate an earlier decision, with its reasoning history still available.

How does retaining history change recovery? The diagram shows exploration and execution inside one run, with an error returning the agent to the information that can explain it. Reporting follows the result, rather than forcing every execution attempt directly into a reporting stage.

Confidence in that recovery loop leads to a small rollout among trusted users, avoiding very critical workloads. The verdict is “awful.” The team had been pleased to pass 30% of its evaluations—“we thought we were cooking”—but employees ask questions the team had not anticipated. Manually mapping each new scenario into the system does not look scalable.

At 7:48, Claude Code used with Opus 4.5 provides a different reference point. It answers many questions that trouble the custom agent. Qu attributes much of its advantage to an ordinary environment: a file system with a minimal set of operations for listing files, reading them, and running bash.

The proposed mechanism is familiarity and freedom to investigate. Models are well trained on these operations, so the harness can let them explore and write intermediate work instead of prescribing every discovery through a specialized tool. A few tools can support many sequences of work; the agent chooses the sequence as it learns what the question requires.

D0 is rebuilt in a sandbox containing the whole semantic layer. Bash, grep, file reads, and file writes let it inspect that material, while a few additional tools handle Vercel-specific capabilities. In the join-error example, the recovery loop now has a searchable working environment: the agent can return to the dumped definitions and investigate what it misunderstood. The semantic layer is available as files to explore, rather than only through the earlier narrow schema tools.

Qu reports that the evaluation score roughly doubles through the progression from the single agent to the Claude Code SDK and then the purpose-built file-system agent. The model and harness both changed, and the talk gives no evaluation methodology that separates their contributions, so this is a reported improvement in the combined system. The implementation he describes is small: attach a bash tool to a sandbox and supply files the agent can read, write, and execute.

The newly successful questions encourage broader access within Vercel and a public write-up. Qu calls it a “banger blog post” and says it accounted for 70% of Vercel’s website traffic during the week he wrote it. That is the moment the internal experiment becomes a pattern other builders want to adopt.

Broader use produces thousands of queries a day. At 10:17, the next opportunity comes from repetition: customer metrics, sales metrics, npm downloads, product lookups, and billing questions often share the same shape even when their subjects differ.

A recurring job takes recent queries and tries to distill them into skills. Qu reports roughly 100 skills covering aggregation and specific lookups. A new run previously began with the semantic layer and system prompt; a skill adds contextual knowledge from work already done. The persistence is in reusable files that later runs can use, rather than a described model-retraining process. The talk leaves skill validation and selection unspecified.

What crosses from one run to another? The diagram separates the stream of recent questions from the reusable skills folder. It makes the accumulation visible: the recurring job turns repeated work into additional starting context, so each new run need not rediscover every familiar procedure.

Qu also mentions skills.sh for discovering and running agent skills. Inside Vercel, however, the larger distribution problem is D0 itself. Employees keep forking it at different stages of development, inheriting whichever architecture existed at the time. A framework could let the next builder begin with the latest insight instead of repeating the whole journey.

At 12:09, the framework analogy is Next.js. File conventions let developers declare application structure while the framework determines how pages, functions, and caching fit into infrastructure. For agents, the proposed conventions are folders for skills, tools, and channels. Builders supply knowledge, capabilities, and ways to interact; the framework composes them.

Eve, released two weeks before the talk, packages that approach. A sample template supplies the starting agent, which builders extend with custom knowledge, tools, and familiar channels. The architectural distinction is between those channels and the runtime that performs the work.

The runtime has several parallel responsibilities:

• Durability: Support work that must continue or resume. The integrated Vercel path uses Vercel Workflows.
• Isolated execution: Give executable work an isolated environment. Vercel Sandbox supplies this part of the hosted setup.
• Model access: Allow different model interfaces, including an adapter for OpenAI’s Responses API.
• Connections: Reach other systems through adapters and connectors. Vercel Connect generates short-lived OAuth tokens.

The open adapter options named in the talk include Postgres, OpenAI’s Responses API, Docker, and other connectors. These examples are not all assigned to a particular runtime responsibility. The talk does not develop the underlying persistence, isolation, or token protocols.

D0 is rewritten in Eve while the framework is being built. The resulting agent is expressed through system instructions, skills, and tools, replacing more convoluted structures behind earlier versions. The maintainability benefit is that the pieces shaping behavior are easy to compose and iterate on.

One beta partner rebuilds a service-testing agent that visits websites, installs services, and attempts to use them. Qu reports fewer steps, better success, and better insights than the partner obtained with an off-the-shelf coding agent. This is a qualitative comparison without numerical results or test conditions, illustrating the intended value of a purpose-built agent rather than establishing a general performance advantage.

At 14:44, deployment on Vercel adds visibility into the work itself: agent runs, tool calls, individual steps, estimated costs, and possible optimizations. Those views expose the path to an answer, giving builders something more useful to inspect than the final response alone. Qu closes the product introduction with eve.dev, where builders can clone a template, deploy it, or self-host if needed.

The ending returns to the choice to build D0. At 15:30, database connectivity gives way to the harder question of business understanding. Vercel had tested vertical agents that could connect to Snowflake and run queries. Its own agent’s distinguishing ingredient was specific company knowledge: customers have websites and web properties, and useful analysis requires knowing how those entities relate and when to query particular information.

That explains why a semantic layer and reusable procedures matter so much. The agent needs guidance about what the company’s data means and how the business uses it. Qu regards packaged agents as useful to try, but recommends building a custom agent with as much company-specific knowledge as possible when seeking deeper value. This recommendation grows from Vercel’s experience rather than a comparison covering every organization.

Vercel has roughly 20 internal agents that Qu considers to have found useful roles:

• Marketing: Retrospectives and identifying whom to contact.
• Legal: An initial contract redline when a new negotiation arrives.
• Data: Answering data questions through the data science agent.

The scope of the legal example matters: the agent produces a first redline, one contribution to the negotiation.

The data-team example reaches its most useful outcome here. Individual questions no longer consume so much of the day, and the team can spend more time improving Snowflake performance, adding missing data sources, and filling gaps it previously could not address. Qu reports time savings and greater productivity without measured hours. The concrete organizational change is the shift from repeatedly writing requested queries toward improving the system that supports future analysis.

The invitation extends to companies of different sizes and to work in HR, finance, and sales. Start with tasks people dislike or spend too much time doing, then supply the knowledge needed for that particular job. Eve is Qu’s preferred way to assemble those agents; the lesson from D0 is why the assembly must include the company’s own relationships and procedures.