Computer-use models will agentify the web, not APIs

If agents are going to read, click and buy on our behalf, how will they operate on a web that is already hostile to automated traffic? The usual answer is that websites will expose APIs through protocols: MCP servers, WebMCP and a growing collection of payment interfaces. Dhruv Batra accepts the prospect of agents driving web activity, but challenges that implementation assumption. Popular services may provide APIs; the long tail cannot be assumed to follow.

Consider a request for business-class flights from Miami to Palma de Mallorca in August. Sending a computer-use agent to click through flights.google.com is an unnecessarily indirect route when flight aggregators already accept parameters and return JSON. An LLM can call a tool against that structured interface. The buttons exist to produce the request behind the scenes; when the appropriate API already exists, use it.

Now ask whether a restaurant has gluten-free menu items. The desired interface seems equally straightforward: supply a dietary filter and receive matching dishes. But that assumes the restaurant—or an aggregator—has already made its menu available as structured, queryable data. Batra walks through three restaurant sites to show what the agent actually encounters.

The progression matters. On the first site, the language is not a fundamental obstacle for a multilingual model. On the second, the information remains text, but the agent must discover and open the document. On the third, a photograph-heavy page offers no obvious text menu until the visitor recognizes Nuestra carta, opens it and then opens individual gallery images. Batra reports that ChatGPT struggles with OCR on the downloaded menu JPEGs. A simple dietary question has become a sequence of navigation and perception problems before any filtering can begin.

The missing endpoint is therefore not just an omitted convenience. It presupposes a clean data representation that the restaurant may never have created. The public interface is the collection of pages, documents and images a person can work through.

The enterprise version is a supplier asking whether a school district is currently procuring laptops. Given mydistrict.gov, the ideal interface would accept status=open and topic=laptop. Batra estimates a potential search space of 15,000–20,000 US school districts, though that range depends on whether regular districts or the broader category of education agencies is intended.

The demonstrations again become progressively harder. Ithaca Public Schools presents a portal that the agent must navigate. Another district places the relevant material under Finance → Purchasing → Current Solicitations, where the destination is a scanned PDF without extractable text. In the hardest example, obtaining information requires a Freedom of Information Act request. Staff scan the request email, upload it to Google Drive and attach the relevant PDFs. Expecting that institution to supply an MCP server overlooks the workflow it actually operates.

Batra uses roughly 200 million active websites and around a billion total websites as scale estimates. More consequential than the precise count is the ownership boundary. A coding agent might generate an API if given unrestricted access to an institution's software, but generating the code does not obtain that access, authorize deployment or change the institution's processes. Organizations still faxing documents will not all replace their infrastructure on an AI development schedule. The bottleneck is institutional adoption as well as software construction.

If the infrastructure will not change, why not point a coding agent at the HTML? A browser is software, so the information it displays must come from somewhere. The first complication is timing. In Batra's NBA.com example, a Minnesota Timberwolves–Brooklyn Nets page visibly shows a final score of 125–109. The initial HTML contains an empty placeholder. After a network-dependent delay, an asynchronous request returns JSON containing the score. Reading only the initial response misses the answer. Adding a wait seems like a reasonable repair—but the next example is not merely about waiting.

The next question is whether a 25mm osmium cube is in stock. A product dropdown shows three unavailable options and one available option. A human can interpret grayed-out, unclickable choices as unavailable, even when the interface does not explicitly label the remaining choice “in stock.” Yet the HTML option selector does not contain the availability text the visitor expects to find.

Batra describes product-count JSON containing a quantity value, followed by a separate rendering script that disables and grays out options whose quantity is zero. His account has the browser fetch that JSON; the companion explanation instead places it in the initial HTML and specifies a 25.4mm cube. Either placement leaves the central mechanism intact: availability is computed from data and expressed through rendered state, rather than supplied as a literal answer in the selector text.

A small JavaScript example captures that rendering rule. Here, a zero quantity makes the option unselectable without adding any “sold out” text:

const stock = { label: "25mm osmium cube", quantity: 0 };
const select = document.createElement("select");
const option = new Option(stock.label, stock.label);

option.disabled = stock.quantity === 0;
option.style.color = option.disabled ? "#888" : "#111";
select.append(option);
document.body.append(select);

Searching option.textContent would find the product name, not the stock answer. Recovering the visible meaning requires understanding the data, the rendering rule and the resulting interface. Batra compares the browser to a game engine: predicting its pixels from source code requires reconstructing the computation that produces them.

The web was built for human eyes. Its rendered interface is where text, images, scripts and interaction state come together for the intended consumer. Batra calls this a bitter lesson for web agents: accumulating website-specific extraction rules and scaffolding does not provide the same generality as interpreting the interface itself. Vision gives the agent access to the representation that website builders already maintain for people.

The first version of Navigator, released in November of the preceding year, used screenshots as inputs and button clicks and scrolls as outputs. Its discount-code demonstration turns a natural-language claim into a browser task. The code may apply only to a particular product, within certain dates or above a minimum cart value. Those conditions determine the interaction the agent must perform.

The validation procedure follows the same route a shopper would take:

1. Find the product described by the offer.
2. Add the eligible product to the cart, satisfying any minimum cart requirement.
3. Apply the discount code.
4. Inspect the resulting price to check whether the claimed 22% reduction was applied.

The percentage is the offer being tested, not a measured success rate. Batra describes these tasks as taking roughly 20–40 interaction steps, depending on complexity. The visual interface makes the task expressible even without a dedicated validation API; it does not eliminate practical accuracy gaps. His claim that the model can accomplish browser tasks is explicitly an in-principle claim.

Needing vision does not require imitating every human action. In the next model demonstration, a Chrome extension displays an Execute JS action while multiple form fields are filled together. The model has generated a function to perform the updates instead of clicking and typing into each field separately. The browser can execute code, so an agent can use that capability whenever it is the efficient way to act.

Choose the action method for the task, then inspect the rendered result. The model can read code, write JavaScript or click a control, and use the next screenshot to check what happened. Batra describes this as a “sort of formal verification” system, but the mechanism shown is visual outcome checking, not mathematical proof. Successful execution of a function is not itself evidence that the intended form state appeared; the screenshot supplies that feedback.

The same browser capability can be replicated. An orchestrator launches multiple Navigators, each in its own cloud sandbox, to work on different websites simultaneously. The advantage over a human operator comes from parallel execution across browser instances, not from requiring one agent to finish every interaction faster than a person.

Are computer-use models reliable enough to make this useful? Batra answers with Online-Mind2Web, showing model release dates against human-evaluated task success. Evaluators inspect the trajectories and judge whether the tasks were completed correctly. This measures outcomes across browser interactions, rather than whether an isolated click was plausible.

Batra reports approximately 97% human-evaluated success for Navigator n1.5, with eight incorrect trajectories out of 300 on the evaluated Online-Mind2Web version. He describes the tasks as involving roughly 30–50 interaction steps. For him, that result is a reason to retire this version of the benchmark and build harder, longer tasks. The result belongs to that evaluated task set; the benchmark's tasks change over time, so its version matters when comparing releases.

The evaluated model also reflects the hybrid design: pixels come in, while both button clicks and code can come out. The progress claim therefore concerns a visual browser agent with multiple ways to act, not a system restricted to reproducing human mouse movements.

Reliability does not remove the cost of repeatedly observing a screen and taking an action. Batra acknowledges that computer-use workflows can be slow and expensive, especially when repeated across many tasks. In comparisons with Opus 4.7 and GPT-5.5 on browser-use benchmarks, he treats Navigator's small accuracy advantage as within statistical noise. The stronger practical argument is latency per step and cost per completed task.

Batra attributes the speed advantage to a smaller model footprint. For tasks involving roughly 20–30 interaction steps on the compared datasets, he cites about $0.80 versus $2.30 per task. The spoken comparison does not assign each price to a specific baseline or establish billing assumptions, so it supports the cost contrast without a precise model-by-model price ranking. Reducing the repeated cost of each interaction is what makes launching browser agents at scale more plausible.

Agents can become a primary way of acting on the web without every institution rebuilding its interface for them. Batra doubts that decades of infrastructure assembled for human use will be reinvented within a few years. Yet the caller's desired abstraction remains sensible: submit a task, expressed in natural language or programmatic parameters, and receive its result.

His proposed path is to build that abstraction above the existing web. Improving models make browser interactions more reliable and less expensive. Behind the endpoint, many browser instances navigate pages, click controls and collect results; the service returns structured output to its caller. The underlying restaurant, school district or merchant need not have created a new API. Another layer of software absorbs the complexity of its existing interface.

Batra projects a future in which this work costs less than a penny per task, runs in less than 100 milliseconds and sometimes executes in the user's browser. These are forecasts, not the current measurements presented earlier. If the caller can issue a task and obtain structured data cheaply and quickly, the implementation behind that boundary becomes less important. From the caller's perspective, the browser automation service is the API.