MCP = Mega Context Problem - Matt Carey

An API already lets software interact with the world. How do you give an agent that same reach? Matt Carey, who works on MCP and agents at Cloudflare, starts with function calling: the model produces a call, the application executes it, and the result returns to the model so it can answer the user. The weather example on screen shows that whole round trip.

Initially, each agent bundled its own integrations. If one developer wrote tools for Gmail, the next developer building a different agent had to repeat that work. Remote MCP offered a shared surface: the service provider could implement standardized tools once, alongside its existing API, CLI or GraphQL interface, and multiple agents could consume them. With eight tools, that arrangement looked manageable.

Keep adding tools, however, and the interface itself becomes the problem. Before the agent has done any useful work, descriptions and parameter schemas consume its context. Carey reports that Cloudflare’s OpenAPI specification occupied 2.3 million tokens, and converting its endpoints into conventional tools still required roughly 1.1 million tokens. Exposing the entire API by loading one tool per endpoint was therefore impractical.

Cloudflare’s first response was to split the API into product-specific MCP servers. That reduced the context needed for any one integration, but transferred the selection problem to the user and left gaps in coverage. Carey gives a product-suite example with six exposed tools for roughly 30 API endpoints. A user could connect the right server and still lack the operation they needed.

Cloudflare quickly accumulated 16 MCP servers while its API had roughly 2,500 endpoints, approaching 2,600 at the time of the talk. More servers did not solve the underlying mismatch: users wanted access to a service, while the integration required them to choose subsets of that service in advance.

The missing mechanism is progressive discovery. MCP is a protocol; it does not require a client to dump every capability into the model’s context at once. Different discovery strategies can operate over it. The same principle applies to prompts, resources and skills: availability does not have to mean eager loading.

A CLI already provides a discovery tree. In Carey’s Wrangler demonstration, the agent can inspect the top-level commands, select d1, find the database-listing operation, and consult --help for parameters. The corresponding shell sequence is small:

wrangler
wrangler d1
wrangler d1 list --help
wrangler d1 list

Each step reveals enough information to choose the next one. Carey says this mostly works and points to OpenClaw as a popular example of CLI use. The requirement is shell access: an agent needs somewhere to run those commands.

Tool search provides a more structured alternative. For a request to create a Worker, Carey’s Claude Code example uses keyword matching to retrieve candidate tools. The model then selects the creation operation, but the other retrieved definitions remain in context. His illustrative retrieval size is eight tools, possibly six as implementations change. In Carey’s tool-search example, about 2,100 tokens enter context while only about 500 belong to the tool being used. Search still works quite well; it narrows the catalog without necessarily narrowing it to exactly one operation.

Cloudflare’s third approach, introduced in Code Mode: the better way to use MCP, lets the agent write code against an API. TypeScript types provide a concise description of inputs and outputs. Generate those types from an OpenAPI specification, give the relevant definitions to the model, and the model can compose operations into a program rather than select only an individual tool call.

The first demonstration is a request to list Workers. The model generates code against Cloudflare’s generated typed SDK; running that code returns the Workers on the account. Carey then moves to a workflow spanning multiple operations: deploy a Hello World Worker, protect it with Cloudflare Access, and apply a policy allowing only him to enter. Access is the application access-control service enforcing that policy.

The attraction is that the model supplies the composition. API authors maintain a specification, and generated types describe what the model can call. Better models can produce better programs, while a more accurate OpenAPI specification improves the contract those programs target. The specification remains the source of truth, rather than a separately maintained collection of agent-specific wrappers.

The approach did not immediately become standard in clients—here, “client” means the agent consuming MCP capabilities. Carey describes limited adoption in the months following the proposal. The obstacle was not simply generating code; it was accepting responsibility for running that code on a user’s behalf.

Unreviewed programs can read filesystem secrets, send those secrets in network requests, loop forever, exhaust resources or run a crypto miner. Giving an agent credentials makes it useful, but also makes the execution boundary consequential. A model’s ability to produce syntactically valid code does not establish what that code should be allowed to do.

Existing approaches make different compromises:

• Restricted languages: JSON-based DSLs and integration interpreters expose selected operations without accepting arbitrary programs.
• VM sandboxes: separate machines provide an execution environment with a larger infrastructure footprint.
• Code review: inspect programs before allowing them to run.

Carey introduces Cloudflare’s runtime primitive as another option for this problem, while explicitly expecting other implementations to emerge.

A dynamic Worker can be created from a string of source code and run in a V8 isolate. In the demonstration, the generated program executes on the backend, not in the browser displaying the slides. Carey then probes the environment: with Node compatibility enabled, printing process.env reveals no secrets in this example, while a Cloudflare global is visible. Turning Node compatibility off makes process.env unavailable and the probe errors.

These are configurable execution environments, lighter than the conventional VM sandbox Carey discussed earlier. He describes the hosted service as capable of handling billions of requests. That is a capacity claim about the platform, separate from what the small live demonstration establishes.

The next program attempts to call an external API. Initially, the runtime rejects internet access through global functions. Carey changes a server-side Boolean and permits the request; he then explains that a more selective function can restrict access to particular domains, which is the approach used by Cloudflare MCP.

The mechanism is explicit authority: the host decides which environment and network capabilities the generated program receives. The empty environment in this example should not be treated as a universal runtime default.

The next slide embeds an MCP client connected to the Cloudflare MCP server. Asking a question opens an authentication screen. After authorization, Carey has read-only access to his Cloudflare infrastructure through the API surface. The displayed account IDs identify accounts; Carey notes that they are not secrets.

The client lists a Worker. Carey then describes the broader range of operations the integration can expose: deploying Workers, adding Access, inspecting DNS and, prospectively, sending email. Its reach spans more than 2,000 API endpoints, but that does not grant the current session write permission. Coverage determines which operations can be represented; authorization determines which operations this user can execute. The live session remains read-only.

The landscape now contains several ways for agents to act: CLIs running locally or in VMs, tool search, remote isolated execution, and interfaces represented as JSON. Code offers a common advantage across those arrangements. A single tool accepting a program can carry a plan with more degrees of freedom than an individual function call. The model can express iteration and composition inside that program instead of negotiating every step separately through the tool interface.

Carey expects demand for these programs to produce more isolated runtimes. He names workerd, Deno and Pydantic Monty, then demonstrates the alternatives. The workerd example spawns a dynamic Worker and computes a Fibonacci sequence. A small TypeScript computation captures the kind of workload being sent into that environment:

function fibonacci(count: number): number[] {
  const values: number[] = [];
  let current = 0;
  let next = 1;

  for (let i = 0; i < count; i++) {
    values.push(current);
    [current, next] = [next, current + next];
  }

  return values;
}

console.log(fibonacci(10));

The host’s job is to execute the supplied computation under its chosen restrictions; the sequence calculation does not require broad filesystem or network access.

The Deno example uses deno run, although Carey says he does not understand the checking shown in that example. Monty is introduced as an interpreter for untrusted Python. Its demonstration pauses while Python downloads, then completes and returns to the terminal prompt. The examples illustrate several routes to running generated programs, without establishing equivalent security policies across the runtimes.

Accepting user-written programs recalls an older computing arrangement. Carey compares it to submitting punch cards to a computer operator: users provide instructions, and someone else’s infrastructure executes them. His forecast is that AI users will increasingly interact with services this way, whether the entry point is MCP, Bash or a CLI.

That changes the pressure on the service itself. A generated program can place API requests inside a loop, and multiple sandboxes can run those loops concurrently. Isolating each program does not protect the destination API from their combined traffic. Rate limiting must protect the service from concurrent generated workloads, not merely from one unusually active tool call.

On the client side, users experience outcomes rather than protocol machinery: their agent either retrieves Gmail messages or fails to do so; worse, it might delete the inbox. Yet much of the implementation effort has gone into stateful connections and resumability. Carey says that complexity pushed developers toward stripped-down clients built around bare-bones SDK behavior, leaving less room for distinctive user interfaces.

He expects programmatic tool calling to become a client capability as well as a server capability. Remote clients can execute generated code through primitives such as workerd, Deno or Monty. His aside about simply using eval locally is a joke, not a safety recommendation; local execution still needs an authority boundary.

Once an action exists as code, the client can save it. Instead of asking the model to regenerate the same workflow every time, a user could retain a small script and rerun it. Carey’s example is a web-scraping automation:

1. The user asks for a recurring scraping task without needing to know how scraping works.
2. The agent generates a script that runs daily or every two days.
3. When the brittle scraping logic breaks, the agent repairs the script and saves the replacement.

This is a proposed client experience: model reasoning creates and repairs the automation, while routine runs reuse the saved program.

Easier client development also suggests more agents deployed in the cloud. Carey contrasts a hypothetical world with one million agents against one with 100 agents per person. Maintaining a permanent sandbox for every agent becomes a different proposition at that scale. He expects more stateless agent loops, with state available when needed rather than an unavoidable property of every running client.

The final architectural prediction concerns the server SDK. Carey expects MCP to become middleware built into API frameworks: enabling it would feel like setting mcp: true. He predicts native integration in major TypeScript full-stack frameworks by the end of the talk’s year, 2026. The prerequisite is a lightweight SDK that expresses the protocol without imposing a large bundle or infrastructure burden.

In his hypothetical Next.js application, a thousand API operations could be exposed over MCP while clients use programmatic tool calling to work with them. The framework flag is a roadmap illustration, not a demonstrated configuration option. Carey identifies making the SDK small enough to fit application bundles as the remaining blocker to that experience.

For an implementation pointer, Carey closes with Code Mode: give agents an entire API in 1,000 tokens. The companion post’s approximately 1,000-token figure describes the discovery-and-execution tool surface, not total task context: discovery results and execution output still add tokens as the agent works. His adoption request is especially directed at providers with large APIs, including observability services whose data agents need to query.

The closing installation command is npm i agents, pointing readers toward the Cloudflare Agents SDK. For implementation today, that repository also lists @cloudflare/codemode as a separate package. The practical destination is an API that agents can discover and program against, with execution permissions and service protections strong enough to support that freedom.