The Web Browser Is All You Need

What happens when a user asks an agent to do something on a website that has no dedicated integration? Before the agent can navigate, someone has to run its browser. Paul Klein IV, founder of Browserbase, opens with that infrastructure problem: maintaining headless browsers yourself can be fragile. He describes Browserbase as supporting thousands of headless browsers in the cloud, with browser automation exposed through its MCP server. The focus here is what those browsers make possible inside agent applications.

Consider the DMV or a local barber. The DMV is unlikely to offer an MCP server soon, and Klein’s barber has better things to do than build a GraphQL API for appointment booking. Both already have an interface people know how to use: a website. The browser connects agents to services that were built for humans.

The useful opportunities include less glamorous work than booking flights or picking restaurants. Klein’s example is filing Delaware franchise tax: an administrative task founders need done, whether or not its website attracts much developer attention. When MCP, A2A, OpenAPI or a first-party integration is unavailable, the agent can still use the website. The browser is an integration of last resort, a route to take when a suitable primary integration does not exist.

A web agent needs to turn information about the current page into an action. One approach gives a model the page’s DOM and asks it to generate browser-control code. The ecosystem Klein describes includes WebVoyager, Adept’s Fuyu models, OpenAI’s Operator, Proxy by Convergence and H Company. These systems explore how models can operate interfaces rather than merely describe them.

WebVoyager is an early example of the visual approach: take a screenshot, use prompting to reason about the next action, then select a target on the page. Visual targets can be represented by coordinates or annotated labels. Text-based agents instead consume HTML and identify targets through structures such as XPath or Playwright locators.

Klein associates text-based control with repeatability and visual control with handling some complex pages, but neither is a universal winner. The right representation depends on the website being automated.

With set-of-mark prompting, the screenshot contains labeled boxes over possible targets. Instead of requiring the model to describe a button’s position precisely, the system can accept a selection such as box 25 and map it to the marked target. On the text side, an accessibility tree offers a more compact semantic representation of page elements, stripping away much of the surrounding div and class scaffolding. It exposes useful structure without needing to reproduce the page’s exact visual layout.

Computer-use models move beyond asking general image models to interpret individual screenshots. Training on web trajectories gives a model sequences of interactions: what it saw, what it did and what page followed. Reinforcement learning can use those sequences to teach decisions across multiple pages. The problem becomes choosing a path through a website, not merely choosing the right button on the current screen.

That capability raises a separate design choice: how much of the path should the model control? In Klein’s terminology, a web agent turns one prompt into many actions. Ask Operator to file Delaware franchise tax and it must decide how to reach the goal. Repeating the prompt may produce a different route. Its ability to keep looking for a way forward comes with nondeterminism because the agent owns the reasoning between steps.

A browser tool accepts a narrower instruction, such as clicking the sign-in button. Purchasing an item is a sequence of steps; clicking sign-in is one bounded action within a sequence. In Playwright, the latter can be expressed directly:

import type { Page } from "playwright";

export async function clickSignIn(page: Page): Promise<void> {
  await page.getByRole("button", { name: "Sign in", exact: true }).click();
}

The surrounding application still decides when to call this function and what comes next. That is the control boundary: selecting a target for one action versus planning the whole task.

Stagehand provides the browser-tool approach Klein recommends for workflows whose high-level steps are already known. The application supplies those steps, and the tool translates them into browser actions. If the incoming request is open-ended and the steps cannot be anticipated, a more general web agent may fit better. The Stagehand and MCP repositories linked here document current APIs; their installation examples and defaults should not be read as the historical implementation used in the recording.

The same distinction between a specific operation and a general capability appears in MCP server design. A vertical server exposes actions for an application or domain. A horizontal server exposes primitives that can work across applications.

The browser’s page can change while the primitive remains useful. That gives one server broad reach across websites.

Broad reach does not make direct integrations obsolete. If Salesforce has an appropriate MCP server, use that rather than driving its UI unnecessarily. A bespoke enterprise CRM without an MCP server is a different case: browser tools can operate its existing interface without first reverse engineering its APIs.

Approval is another part of the integration boundary. Klein expects security teams to scrutinize dynamic tool discovery—the prospect of an agent attaching itself to an effectively unlimited collection of MCP servers. A horizontal server can reduce the number of servers an organization has to onboard. That simplifies one part of approval; it does not grant blanket permission for every website or action reachable through the browser.

Klein is skeptical of public benchmarks, particularly rankings published by the company ranking its own system. The practical response is to build evaluations around the actual websites and tasks the application needs to handle. He names Braintrust as supporting Browserbase’s evaluations: the useful question is which model works best for a particular web task or site, rather than which model leads a general leaderboard.

Evaluation tells you whether the outcome was right. Observability helps explain how it happened. If an agent is supposed to buy an Xbox but buys AirPods, the final result alone cannot identify the mistake. You need the prompts, screenshots, browser actions and navigation history to reconstruct where the run diverged. Klein describes Browserbase as bundling session recordings and logs for this purpose.

These requirements qualify the browser’s role as the default MCP server for the rest of the internet. When a suitable API or dedicated integration is absent, browser access supplies a path forward—but that path still needs evaluation and a record of what the agent actually did.

The live demonstration puts a reasoning agent above browser primitives in Cursor. The request is to create a browser session, navigate to sfspca.org, close any modals, find a dog available for adoption in San Francisco and return its URL. This is a goal with several intermediate actions, not a single predetermined click.

The visible sequence makes the division of responsibility concrete:

1. Cursor calls the browser session tool to create a session.
2. The agent issues individual tool calls to interact with the page.
3. Klein opens the browser session beside the agent interface so its progress is visible.
4. The agent works on closing a modal and navigating the site.
5. The reasoning model chooses among the exposed tools and returns a dog’s URL.

The browser tools supply the actions; the model determines which action to request next.

Opening the result brings another interruption: a fundraising modal complicates the reveal. This is exactly the kind of page behavior that makes fixed expectations brittle. An agent needs to react to what is currently on the page, including overlays that were absent or already dismissed earlier. After the interruption, the visible adoption page shows two dog photographs and the heading “Meet Zoe” alongside the agent chat. The demonstration reaches an adoption profile; it does not complete an adoption transaction.

The broader application is bringing capable models to mundane work in organizations that are not AI-native. Klein describes a 55-year-old dairy trucking company that hired its first engineer that year and used Browserbase to automate a painful operational workflow. The example matters because the value comes from reaching an existing business process, not from requiring the business to rebuild itself around a new agent interface.

Klein closes the prepared portion by inviting the audience to try the available browser MCP server through a signup QR code. The adoption step is to give an agent browser access and apply it to a workflow that otherwise lacks an integration.

The first substantive audience question asks whether navigation requires a particular model. Browserbase’s boundary is model-agnostic: it supplies headless browser infrastructure, an MCP server and frameworks such as Stagehand. The application brings the model that decides how to use them.

Model selection also depends on operational requirements. Klein uses HIPAA and zero data retention as examples of considerations that may affect provider choice. Those are requirements to assess, not compliance guarantees conferred by a model name.

For the demonstration, he confirms that he used Cursor’s agent mode, then tentatively identifies the selected model as Claude Sonnet 4 after correcting himself. The key architectural point does not depend on that identification: Cursor’s agent supplied the reasoning while the browser infrastructure exposed the session and actions.

A financial-data question makes the distinction between visibility and control more precise. An advisor may need to review information before it enters a financial planning tool. Users may also need a clear account of what an agent did, even if they did not participate in the run.

Klein describes two capabilities:

• Session recordings: Embed a completed run in a user-facing application so someone can inspect what happened. Recording can be enabled or disabled according to data sensitivity.
• Live View: Embed the running browser through an iframe, allowing a person to watch and also click or type in the session.

Live View supports a browser copilot experience in which a human can take part in the interaction. These capabilities provide visibility and intervention; a mandatory approval gate before submitting financial data remains an application-level requirement.

CAPTCHAs expose another boundary: a website may resist automation even when the agent has a useful task. Klein distinguishes beneficial automation from malicious bots and describes Browserbase’s CAPTCHA-solving and proxy capabilities. His example includes customers encountering their own CAPTCHAs when their security operations process has not provided a suitable automation route.

His longer-term hope is agent authentication. Klein says most CAPTCHAs Browserbase encounters arise during login. If a site can identify both the agent and the person it represents, it has a stronger basis for deciding what that agent may do. He presents authentication as a potential route away from CAPTCHA solving, not as an established solution that every website already supports.

The final question asks whether automated navigation itself increases the likelihood of a challenge. Klein answers that detection often depends on behavior. He points to LinkedIn’s monitoring of actions per minute as an example: browser stealth features cannot guarantee access when an agent’s behavior crosses a site’s limits.

His operational advice is to be a responsible participant on the web: respect robots.txt, act carefully and recognize that infrastructure cannot prevent every block. Browser access does not erase a website’s constraints or the obligations of the person operating the agent. Klein ends by rejecting unlawful or unethical use of the platform.