Browser Agents Don't Need Better Models. They Need Better Eyes. - Kushan Raj, ARK

Kushan Raj begins with a gap between the appeal of browser agents and his own experience: he finds the idea compelling, but does not use them much. To explain why, he introduces a browser challenge involving a long sequence of tasks. Such a challenge tests whether an agent can keep making progress across many interactions, beyond recognizing a single button.

The first problem is latency before meaningful progress. Raj reports that the demonstrated agent takes roughly 10–20 seconds just to click Start. It then reaches step one of a 30-step challenge. Even without a final completion time, the example exposes a practical concern: slow individual interactions make a long workflow cumbersome before the agent encounters its harder decisions.

Raj next presents his own agent on the same website. He describes an interface that lets the user follow what the agent is thinking, alongside execution that he says is substantially faster despite using a cheaper model. His hypothesis is that model capability is already sufficient for more useful behavior, while the infrastructure around the model prevents it from acting effectively. The comparison is a demonstration; he does not identify the models or provide a controlled measurement here.

His diagnosis centers on an agent trying to click something without understanding what is happening on the page. A useful environment should let it plan a long sequence, identify where execution failed, and choose the next click correctly. Raj introduces a compressed page representation as the foundation: the model receives a view of the entire page in relatively few tokens. The intended benefit is enough context to connect an action to the surrounding page state and reason about what should happen next.

The first concrete workflow is downloading Raj’s Aadhaar. He expects the initial interaction to be straightforward: take a screenshot, recognize the visible button, and click it. In the Claude demonstration, however, he reports that the agent gets stuck. From 46 seconds into that example video, it takes a screenshot, scrolls, and takes another screenshot; Raj says the process lasts two minutes. Repeated observation is not enough to move the task forward.

Raj then presents his agent starting and reaching what he describes as completion almost immediately, again emphasizing that it uses a cheap model. This illustrates his intended user experience: a browser task should resolve quickly enough to feel convenient. He supplies no exact runtime for his agent or detailed account of the downloaded result, so the example supports his reported contrast in behavior without establishing a measured speedup or a general success rate.

The second example comes from planning a Sunday trek with friends. Raj describes difficulty navigating a website whose language he is not fluent in, then asks Claude to make the booking. The specific failure is date selection: the agent cannot pick a date and becomes stuck. This gives the comparison a concrete interaction to examine rather than treating the whole booking request as one undifferentiated task.

In his agent’s demonstration, Raj describes it making the selection and entering the date. That is the supported outcome of this example. Although the original request was to book the trek, the account does not establish a completed reservation or payment. Its narrower lesson is that successfully operating a troublesome date control can remove a point where another agent stops progressing.

Raj considers open-sourcing the project because he does not see the code itself as especially defensible. His proposed product could expose the commands used in the demonstrations through an API: a caller supplies a URL and an intent, and the service executes the task and returns a result. He also mentions a website or plugin as possible interfaces. These remain proposed directions; the common goal is to make browser agents faster, cheaper, and more reliable so that people use them routinely.

The closing technical explanation makes the representation concrete: Raj uses Markdown to represent a particular browser page. For one example, he estimates roughly 20,000 tokens for the full DOM, about 1,100 tokens for a screenshot, and about 1,800 tokens for his Markdown. The Markdown therefore costs more tokens than that individual screenshot, but he says it covers the entire page, whereas the screenshot shows only one portion. Against the full DOM, its advantage is substantial compression. These are estimates for one example, with no extraction algorithm or account of omitted information given.

The runtime also tells the agent how the page changes. Its feedback identifies newly appearing content, content that has disappeared, and obstructions that have been removed from something the agent wanted to click. It can report that an attempted click did not happen. These observations distinguish issuing an action from successfully carrying it out: the agent receives information about the result and the conditions that may now permit another attempt.

Raj attributes this feedback to tracking the browser page throughout execution. He closes by combining the pieces: the compact representation can accompany a screenshot, giving the model both the compressed page description and visual information. His claim is that this relatively inexpensive context helps the model reason well enough to construct long sequences of tasks. The explanation establishes the role of page representation and action feedback, but leaves the details of compression and failure detection unspecified.