AI Engineer Code 2025
Your MCP Server is Bad and You Should Feel Bad
Read the talk
Designing MCP Servers as Interfaces for Agents
A useful MCP server curates workflows, arguments and context for its agent user. Jeremiah Lowin shows how to move beyond API wrappers—and where client behavior changes the design.
From a talk by Jeremiah Lowin
Before you start: Familiarity with Python functions, API calls and basic LLM tool calling will help you follow the examples.
What makes a working MCP server a useful product?
Making tools available is only the beginning. What makes the resulting server useful to the agent that must discover and call them? Jeremiah Lowin approaches that question through years of building orchestration software: first as an Apache Airflow PMC member, then as Prefect’s founder and CEO, and later through the Marvin agent framework and FastMCP. Maintaining FastMCP gives him a view of how developers actually build these interfaces. Lowin reports 1.5 million FastMCP downloads on the day before the talk.
FastMCP appeared days after MCP’s introduction, and a version entered Anthropic’s official SDK. By the recording, Lowin was positioning his separately maintained framework as the high-level interface, with the SDK responsible for low-level primitives. He described removing the confusing shared FastMCP naming as a future change. That maintenance experience also exposed many poorly designed servers. The talk’s Futurama-inspired title is an invitation to improve them, rather than an insult.
An MCP server is a user interface for an agent. It deserves the same attention to user experience, capabilities and workflows as a human-facing product. The objection that an AI should be able to use any API assumes too much competence and overlooks how humans work: they usually put a website, SDK, client or mobile app between themselves and the API. Agents are powerful but fallible users, and their interfaces should account for that.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Discovery, iteration and context have different costs
Three differences explain why an interface that suits a developer may burden an agent.
| Dimension | Human developer | Agent |
|---|---|---|
| Discovery | Reads documentation and amortizes that work | Commonly receives tool names, schemas and descriptions on connection |
| Iteration | Writes a script that calls known routes | Spends additional model turns choosing and making calls |
| Context | Draws on accumulated experience | Works within a bounded active context |
A developer can inspect Swagger, identify the necessary routes and encode their sequence once. An agent may rediscover the interface whenever it connects. Each additional model turn adds latency and, depending on caching, processes the accumulated history again. Eagerly loading definitions is a common client behavior, not a requirement that every MCP definition enter model context.
Lowin uses a 200,000-token window to illustrate bounded working memory, alongside knowledge stored in model weights. His loose Apollo-computing analogy asks developers to treat that capacity as scarce even when the system’s behavior looks impressive. An agent may find the needle in a haystack, but the interface should not make it inspect every piece of hay first.
The design verb is curate: select and shape the information the agent needs. MCP supplies a standard connection between models and tools or data—the familiar USB-C analogy—and a way to control discovery and action. Actual clients constrain what servers can achieve. FastMCP makes the basic plumbing small enough that Lowin’s introductory example simply exposes a tool asking whether Washington, DC’s subway is on fire; the demonstration answers yes. Getting a callable function onto a server is easier than deciding what that function should do.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Turn order lookup into one agent story
An audience member describes the common shortcut: wrap an API, stringify its response and call the result an MCP server. Even a hand-built server can have the same product problem. Like a website with confusing navigation or an all-Flash interface, it may technically work while making ordinary tasks unnecessarily difficult. Lowin recommends Block’s Playbook for Designing MCP Servers and GitHub’s published guidance as examples of developing better conventions; he also discloses that the Block team had been a customer of his data business for six years.
The workshop’s order-status example makes the cost concrete. Separate decorated Python functions expose operations for discovering a user, retrieving and filtering orders, and checking status. A developer would call them in a known sequence and return a user-facing answer. The exposed order-status workflow requires at least three agent round trips in Lowin’s example. The agent must also infer the sequence and supply correctly formatted arguments at every step.
Start instead with the outcome: track the latest order given an email address. This is an agent story—a concrete objective pursued by an autonomous user with limited context. Deterministic orchestration belongs in code when the sequence is already known. An LLM remains useful as an orchestrator when the algorithm is unknown or cannot readily be expressed programmatically; routine order lookup is an expensive place to require that flexibility.
Name the tool so the agent can select it at the right moment. An explanatory name such as track_latest_order communicates more than a generic operation name. Block expresses the same principle as designing top-down from workflows rather than bottom-up from endpoints. In Lowin’s refactor, the three API calls still happen, but they sit inside one agent-facing tool. The server takes responsibility for a sequence it already knows how to perform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the valid arguments obvious
A single workflow tool can still be difficult to call if its input is config: dict. The agent has to reconstruct an object whose conventions may live elsewhere. Duplicating those conventions in a system prompt or subagent definition creates another problem: change the server and the two descriptions can disagree. A fully annotated Pydantic model improves on an untyped dictionary, but does not remove the burden of constructing a complex argument.
Client behavior can compound that burden. Lowin reports that Claude Desktop had sent structured object arguments as strings, prompting FastMCP to attempt deserialization when a parameter clearly expected an object. He was unsure whether the client issue remained at recording time. This was a compatibility workaround, not a reason to prefer string-encoded objects.
Prefer top-level primitives, descriptive names and strong defaults: email, include_canceled and a constrained format. When the valid choices are known, Python’s Literal or an enum conveys more than format: str = "basic". For the order example, the following small implementation shows the combined interface using local teaching records; the lookup, order selection and status retrieval remain internal.
python
from typing import Literal
from fastmcp import FastMCP
mcp = FastMCP("Orders")
USERS = {"alex@example.com": "user-1"}
ORDERS = {
"user-1": [
{"id": "order-42", "created": "2025-11-20", "canceled": False},
{"id": "order-43", "created": "2025-11-21", "canceled": True},
]
}
STATUSES = {"order-42": "shipped", "order-43": "canceled"}
def find_user(email: str) -> str:
return USERS[email]
def latest_order(user_id: str, include_canceled: bool) -> dict:
candidates = [
order for order in ORDERS[user_id]
if include_canceled or not order["canceled"]
]
return max(candidates, key=lambda order: order["created"])
def order_status(order_id: str) -> str:
return STATUSES[order_id]
The public function can then expose the workflow without exposing its internal operations. The default basic format returns the status alone; detailed includes the selected order’s ID and creation date:
python
@mcp.tool
def track_latest_order(
email: str,
include_canceled: bool = False,
format: Literal["basic", "detailed"] = "basic",
) -> str:
"""Track the latest order for an email, excluding canceled orders by default."""
user_id = find_user(email)
order = latest_order(user_id, include_canceled)
status = order_status(order["id"])
if format == "detailed":
return f"Order {order['id']} (created {order['created']}): {status}"
return status
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Documentation and errors shape the next action
Without instructions, an agent guesses how a server works, and those guesses accumulate in its history. Document both the server and its tools, but choose examples carefully: models may reproduce incidental properties as well as intended usage. Lowin’s tag-generation example repeatedly produced two tags despite instructions requesting at least ten, because a documentation example contained two. An audience member proposes out-of-distribution examples to weaken that pattern. Lowin considers the idea reasonable, without claiming it reliably overcomes the influence of implicit patterns.
Errors are prompts. An empty ValueError or a cryptic numeric error gives the model little information beyond the fact that its attempt failed. The response becomes input to its next decision, so explain what went wrong and how to recover. For the same order lookup, an informative missing-user error could be:
python
from fastmcp.exceptions import ToolError
def find_user(email: str) -> str:
user_id = USERS.get(email)
if user_id is None:
raise ToolError(
"No customer matches this email. Ask for the email used "
"at checkout before trying again."
)
return user_id
This gives the agent a next action instead of encouraging an unchanged retry.
For an unavoidably complex API, Lowin suggests an experiment: keep the initial description concise and return specific recovery instructions for common failures. That progressively discloses information according to the mistake, at the cost of accepting an unsuccessful first call. He does not recommend making failure the default discovery mechanism. Error wording also affects whether an agent continues using a tool; an alarming response may persuade it that the tool is permanently unusable. Full docstrings, useful examples and constructive errors work together.
Tool annotations provide another channel for communicating intent. The MCP Tools specification includes readOnlyHint, which clients may use when deciding how to present permissions. Lowin reports that ChatGPT developer mode requested extra permission when this annotation was absent, treating the tool as potentially having side effects. Mark a genuinely read-only lookup accordingly:
python
@mcp.tool(annotations={"readOnlyHint": True})
def get_order_status(order_id: str) -> str:
"""Return an existing order's status without changing it."""
return STATUSES[order_id]
The hint describes behavior; it does not grant authorization or enforce read-only access.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A handshake can leave no room to work
Context belongs to the whole agent session, not exclusively to one server. Lowin repeats a circulating estimate of roughly 200,000 tokens for a GitHub-server handshake, explicitly introducing it as a meme rather than a measurement. His broader point is that frameworks and clients must find ways to offer abundant functionality without loading all of it at once; simply doing less cannot be the only answer.
A customer wanted to expose 800 API endpoints. Under an assumed 200,000-token context budget, 800 tools leave just 250 tokens each for names, schemas and documentation, with nothing left for the task.
Even perfect adherence to that allowance would consume the entire window before the agent received useful working context or connected to another server. The calculation exposes a product constraint, not merely a documentation problem.
Set a budget even when the eventual client is unknown, and economize on the initial description. Moving occasional recovery instructions into error responses is one way to defer information. The larger opportunity is to change discovery: the common eager handshake loads descriptions together, whether or not the upcoming task needs them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reveal the relevant tools—and know your client
An audience implementation begins with a service catalog—Google, Notion and Slack—then exposes details through a separate describe step when a service is needed. The audience participant reports 95% less upfront context consumption across many services with this progressive-discovery approach. The discussion supplies no reproducible baseline or tokenizer, so the figure describes that participant’s experience.
Lowin welcomes the design but separates protocol support from client behavior. He reports that Claude Desktop cached initially discovered tools in SQLite and ignored later notifications that should have enabled changes. Discovery implemented through a tool call could get around that particular limitation. It also creates meta-tools: tools that describe other tools, and sometimes invoke them. Their schemas can reintroduce the complex argument construction that flattening was meant to avoid. Progressive discovery therefore needs its own careful interface design.
Credential-based masking addresses a different source of excess context. An audience member asks about hiding 28 Kubernetes tools that the caller cannot access. MCP leaves the implementation behind tools/list to the server; it does not prescribe a filtering policy. Lowin describes an overridable FastMCP middleware hook and says Prefect’s commercial products support per-tool masking on arbitrary criteria. Protocol, framework and product each have a different role in that decision.
Could the customer’s 800 endpoints simply be divided among several servers? Lowin’s answer is that all the desired information still would not fit—and the customer did not need all of it. The problem became deciding what product the agent actually needed. He anticipates a shift from talking about MCP transport to talking about context products. Owning a mobile app’s agent client, or choosing a mandatory internal client, enables more deliberate discovery and budget management. He offers no general way to fit the full interface into an arbitrary external client’s context.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Count tools from the agent’s perspective
Tool count is a useful warning signal, not a sufficient quality measure. Lowin puts GitHub’s server at roughly 170 tools in his account, while pointing to GitHub’s work on semantic routing as a reason a large interface deserves more careful consideration than automatic rejection. The cited routing work concerns selection in Copilot and VS Code, rather than an intrinsic capability of every GitHub MCP server. Before accepting a large inventory, ask whether admin and user workflows should be separated, whether namespaces would help, or whether different servers serve genuinely different users.
Lowin’s aspirational target is 5–15 tools, with roughly 50 tools per agent a warning threshold requiring care and evaluation. He explicitly corrects the slide’s per-server framing: two servers exposing 50 tools each give the same agent 100 tools. The aggregate interface is what matters, and these numbers are his design heuristics rather than measured universal limits.
Lowin illustrates the curation journey through two posts by Kelly Kohlleffel, at Fivetran, published about a month apart. The reported redesign reduced 188 tools across a data-stack MCP setup to five. The scope spans multiple data-stack servers, rather than solely a Fivetran server. Read together, the posts capture two distinct achievements: making the integration work, then making it work well for an agent.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Bootstrap from REST, then design the product
Lowin’s Stop Converting Your REST APIs to MCP comes with an apparent contradiction: he introduced FastMCP’s popular conversion feature and intends to keep it. Automatic conversion is useful for proving connectivity. It becomes a problem when the generated endpoint mirror is treated as the finished agent interface.
The practical sequence is incremental:
- Pick a few key endpoints and expose them through a converter or direct wrapper.
- Establish that an agent can discover and use a tool at all.
- Replace the raw endpoint mirror with curated workflows once that integration works.
- Ship the resulting product interface, rather than the unedited REST translation.
Lowin starts this way himself. Keeping the initial integration small avoids introducing several new failure modes before basic tool use works.
The same discipline applies after the first version. Avoid large payloads, unnecessary choices and tightly coupled arguments where possible. Treat instructions as deliberately designed context. Lowin regards context capacity as the hard boundary; the other guidelines admit practical tradeoffs. Experimental tools should be removed when they stop earning their place, rather than accumulating alongside old and V2 variants. A growing API inventory can be reasonable for developers while becoming an increasingly confusing interface for an agent. Treat the server as a user interface throughout its life.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Argument dependencies and documentation placement
The closing questions make argument coupling concrete. Suppose a tool accepts both a file type and a processing method. If the file type determines which processing methods are valid, the parameters are tightly coupled: individually valid values can form an invalid combination. That cross-parameter rule is another constraint the agent must remember.
Where should the instructions for such rules live? Client ownership changes the answer.
- Known client: Lowin sometimes places workflow documentation in files or Claude skills when he knows Claude Code will consume it. Infrequent instructions can then remain outside the initial context.
- Unknown client: Tool docstrings should carry the information needed to use the tool. Server authors cannot depend on a particular agent-side configuration.
- Server-wide guidance: The server’s
instructionsfield can hold concise high-level information. Lowin reports that most clients retrieve it on connection, but some ignore it; his team had filed bugs about that behavior.
The server-wide field is another opportunity to explain the product, not an invitation to include a novel.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Background execution changes who waits
Asked about forthcoming asynchronous operations, Lowin describes himself as closely connected to the MCP core committee, though not a member. His colleague Adam had presented on the topic the previous evening. Lowin introduces SEP-1686 as a proposal for background tasks across MCP operations, rather than a tool-only facility.
In his proposal-era explanation, the client opts into asynchronous execution and takes responsibility for checking progress, polling and collecting the result. Discovering the tool and expressing its inputs still require a usable interface. The server designer’s question is therefore whether the client should block waiting for the result—not whether Python can run work in the background. The subsequently released Tasks specification introduced experimental, negotiated support for tool calls, sampling and elicitation, rather than every operation; either peer can be the requestor.
A brief audience exchange then returns to model behavior after a code-analysis tool encounters an error in a person’s code. Lowin recognizes a recurring problem worth investigating further. No concrete remedy emerges from that exchange.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Ask for missing input during execution
Elicitation lets a running tool formally request additional structured input from its client. A common use is asking for approval before an irreversible side effect. That can produce a better interaction than demanding every possible argument before execution begins, but a server must account for whether the client supports the request. Lowin warns that unsupported elicitation can interrupt an otherwise usable workflow.
The client’s job is not always simple. A user-facing application can show a form; a background or automated client needs a policy for deciding who supplies the answer. Having an LLM fill the form does not necessarily satisfy the underlying need. With suitable client support, elicitation could request processing options after the file type is known, or obtain confirmation before a destructive action. Lowin contrasts this with a confirm=False argument that the model must explicitly change: model acknowledgment may affect its behavior, but it is not human approval.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Direct tools and code mode
An audience member already builds API-calling tools in LangGraph or the OpenAI SDK and controls the agent loop. Does that setup need MCP? Lowin says no. He anticipates that a more mature MCP ecosystem could offer useful observability and diagnosis of tool failures, especially when an agent framework returns failures to the model for recovery without making them easy for developers to inspect. He acknowledges framework differences and presents that benefit as a forecast. A developer controlling the whole loop can keep direct tools; the interface-design principles apply to ordinary Python functions too.
The final question explores another way to reduce orchestration overhead: code mode. Lowin credits Cloudflare’s Code Mode article, followed by Anthropic’s code-execution treatment. Instead of choosing every tool call in a separate model turn, the LLM writes code that invokes tools in sequence. This can sidestep some of the repeated interaction costs discussed earlier, but introduces the responsibilities of executing and sandboxing generated code.
A colleague had built a FastMCP extension for this approach and distributed it separately while the team considered how it fit the framework. Lowin describes plans to expose it through an experimental or optimization facility, but does not confirm its exact CLI name or whether it had shipped. Code mode broadens the available design choices: known workflows can remain deterministic server code, while generated orchestration can help when flexibility warrants its execution and sandboxing costs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Workflow-first tool design, context budgeting, documentation and permission guidance from Block's MCP deployments.
Python framework for building MCP servers and clients.
Jeremiah Lowin's companion argument for designing agent interfaces instead of exposing raw REST endpoints.
Cloudflare's explanation of using generated code to compose MCP tool calls.
On-demand tool discovery and code-based orchestration that keep intermediate results outside model context.
Further reading
- MCP Tools specification — June 2025Documentation
Historical protocol reference for tool discovery, invocation and tool metadata.
Updates since the talk
Experimental task support introduced in the November 2025 specification, including negotiated capabilities, polling and deferred results.
Read the complete timestamped transcript
- 0:11
[upbeat music] I really do appreciate that you're all here. I'm gonna try and make this as painless as possible. We're not going to do an interactive part.
- 0:27
We're gonna talk through stuff. I'm happy to go off-script. I'm happy to take questions if there's stuff we want to explore at any moment in this. My goal is I'd like to share with you a lot of things that I've learned.
- 0:38
Um, I'm gonna try and make them as actionable as possible, so there is real stuff to do here, um, more than we might in, like, a more high-level talk.
- 0:47
But let's be very honest, it is late. It is a lot. It is long. Let's, uh, let's talk about MCP. I'm hoping that folks here are interested in MCP, and that's why you came to this talk.
- 0:56
If you're here to learn about MCP, this might be a little bit of a, of a different bent. Just show of hands,
- 1:02
uh, heard of MCP? Used MCP? Written an MCP server?
- 1:10
Okay. Uh, anyone feel uncomfortable with MCP? Which is a hundred percent fine. We can tailor. Okay. Then I would say let's, let's just go. Let's dive in. Um, this is who I am.
- 1:22
Uh, I'm a founder and CEO of a company called Prefect Technologies. For the last seven or eight years, we've been building, um, data automation software and orchestration software. Before that, I was a member of the Apache Airflow PMC.
- 1:34
Um, I originally started Prefect to graduate those same orchestration ideas into data science. Today, we operate the full stack. And then, um, a few years ago, I, I developed an agent framework called Marvin, which I would not describe as wildly popular, but it was my leg into the world of AI, at least from a developer experience standpoint,
- 1:52
and learned a lot from that. And then more recently, I introduced a piece of software called FastMCP, which is, is, is wildly, wildly popular, maybe even too popular, and, um, hence my status today.
- 2:04
I'm a little overwhelmed. Uh, I find myself back in an open source maintenance seat, which I haven't been in in a few years, which has been a hell of a lot of fun.
- 2:11
Um, but the most important thing is that FastMCP has given me a very specific vantage point that is really the basis for this talk today. This is our downloads.
- 2:20
Uh, I've never seen anything like this. I've never worked on a project like this. It was downloaded a million and a half times yesterday. Um, there's a lot of MCP servers out there, and, um, FastMCP is just, it's, it's, it's become the de facto standard way to build MCP servers.
- 2:34
Um, I introduced it almost exactly a year ago. As many of you are probably aware, MCP itself was introduced almost exactly a year ago, and a few days later, I introduced the first version of FastMCP.
- 2:43
Uh, David at Anthropic, uh, called me up, said, "I think this is great. I think this is how people should build servers." We put a version of it into the official SDK, which was amazing.
- 2:52
And then as, um, as MCP has gone crazy in the last year, we found it actually to be constructive to position FastMCP, uh, as I'm maintaining it, as the high-level interface to the MCP ecosystem while the SDT-- SDK focuses on the low-level primitives, and actually we're gonna remove the FastMCP vocabulary from the low-level SDK, um, in a
- 3:14
couple of months. It's become a little bit of... It's, it's too confusing that there are these two things called FastMCP. So FastMCP will be a high-level interface to the world and, um, as a result, we see a lot of, um, not great MCP servers.
- 3:31
I, I named the talk after this meme, and then it occurred to me, like, do people even know what this meme is anymore? Like, this, this to me is very funny and very topical, and then it's from, like, a nineteen ninety-nine episode of Futurama.
- 3:42
So if you haven't seen this, my talk's title is not meant to be mean. [laughs] I'm sort of an optimist. I choose to interpret this as, "But you can do better."
- 3:51
And so we're going to find ways to do better. That is the goal of today's talk. In fact, to be more precise, what I wanna do today is I would really like to build an intuition for agentic product design.
- 4:02
Um, I don't see this talked about nearly as much as it should be given how many agents are using how many products today. And what I mean by this is the exact analog of what it would be if I were, if I were giving a talk on how to just build a good product for a user, for
- 4:16
a human. And we would talk about human interface guidelines, and we talk about user experience, and we talk about stories, and I found it really instructive to start talking about those things from an agentic perspective because what else is an MCP server but an interface, um, for an agent?
- 4:31
And we should design it for the strengths and weaknesses of those agents in the same way that we do everything else. Now, when I put this thought in the world, I very, very, very frequently get this pushback, which is, "But if a human can use an API, why can't an AI?"
- 4:47
And there are so many things wrong with this question. And the number one thing that's wrong with this question is that it has a assumption that I see in so much of AI product design, and it drives me nuts, which is that AIs are perfect or they're oracles or they're good at everything, and they are very, very,
- 5:03
very powerful tools. But I'm assuming based on your, uh, responses before, I think everyone in this room has some scars of the fact that they are fallible or they are limited or, you know, they're, they're imperfect.
- 5:15
And so I don't like this question because it presumes that they're, like, magically amazing at everything. But I really don't like this question. This is a literal question I've got, and I didn't paraphrase it.
- 5:23
I really don't like this question because humans don't use APIs. Very, very rarely do humans use APIs. Humans use products. We do anything we can to put something between us and an API.
- 5:34
We put a website. We put an SDK. We put a client. We put a mobile app. We, we do not like to use APIs unless we have to or we are the person responsible for building, um, that interface.
- 5:46
And so one of my core arguments, um, and why I love MCP so much is that I believe that agents deserve their own interface that is optimized for them and, uh, their own use case.
- 5:56
And in order to design that interface, which is what I wanna motivate today, uh, we have to think a little bit about- What is the difference between a human and an AI?
- 6:06
And it's one of these questions that, like, sounds really stupid when you say it out loud, but it's instructive to actually go through, and I, I'd like to make the argument to you that it exists on these three, um, dimensions of discovery, iteration, and context.
- 6:19
And so just to begin, humans, we find discovery really cheap. We tend to do it once. If you think, if, if any of you have had to implement something against a REST API, what do you do?
- 6:29
You call up the docs or you go in Swagger, whatever it is. You call it up, you look at it one time, you figure out what you need, you're never going to do that again.
- 6:36
And so while it may take you some time to do the discovery, it is cheap in the lifetime of the application you are building. AIs, not so much. Every single time that thing turns on, it shakes hands with a server, it learns about the server, it enumerates every single tool and every single description on that server.
- 6:53
So discovery is actually really expensive for agents. It consumes a lot of tokens. Um, next, iteration. Same idea. If you're a human developer and you're writing code against an API, you can iterate really quickly.
- 7:06
Why? Because you do your one-time discovery, you figure out the three routes you're gonna call, and then you write a script that calls them one after another as fast as your language allows.
- 7:16
So iteration is really cheap, and if that doesn't work, you just run it again until it does. Iteration is cheap, it's fast. Um, for agents, I think we all know iteration is slow.
- 7:26
Iteration is the enemy. Every additional call, um, subject to your caching setup also sends the entire history of all previous call- calls over the wire. Like, it is just, you do not want to iterate if you can avoid it, and so that's gonna be an important thing that we take into consideration.
- 7:40
And the last thing is on context, and this is a little bit hand-wavy, but it is important. As humans, in this conversation, I'm talking, you're hearing me, and you're comparing this to different memories you have and different experiences you have on different time scales, and it's all doing wonderful, amazing things in your brain.
- 7:55
And when you plug an LLM, uh, into any, um, given use case, it remembers the last two hundred thousand tokens it saw, and that's the extent of its, um, memory, plus whatever is, you know, embedded somewhere in its, in its weights, and that's it.
- 8:09
And so we need to be very, very, very conscious of the fact that it has a very small brain at this moment. I, I think it is a lot closer to when people talk about sending, you know, Apollo 11 to the moon and the, and with, like, one kilobyte of RAM, whatever it was.
- 8:23
I think that's actually how we need to think about these things that frankly feel quite magical because they go and, uh, open my PRs for me or whatever it is that they do.
- 8:32
Um, so these are the three key dimensions in my mind of what is different, and we should not build APIs that are good for humans on any of these dimensions and pretend that they are also good for agents.
- 8:45
And one way that I've kind of started talking about this is this idea, which is an agent can find a needle in a haystack. The problem is it's gonna look at every piece of hay and decide if it's a needle.
- 8:54
And that's, like, not literally true, but it is in an intuitive sense how we should think about what we're putting in front of the agents and how we're posing a problem, and an MCP server is nothing but an interface to that problem and/or solution.
- 9:08
And so finally, to go back to our product intuition statement, I argued to you that the most important word in the universe for MCP developers is curate. How do you curate from a huge amount of information which might be amenable for a human developer, a interface that is appropriate for one of these extremely limited AI agents, at
- 9:30
least on the dimensions that we just went through? Um, and that sort of brings us to this slide, why MCP? And I almost made this like the Derek Zoolander slide, like, but why MCP?
- 9:40
Like [laughs] but I just told you why MCP, Derek. It's because it does all of these things. It gives us a standard way of communicating, uh, information to agents in a way that's controllable, where we can control not only how it's discovered, but also how it is acted on.
- 9:53
There's a big asterisk on that because client implementations in the MCP space right now are not amazing, and they do some things that are themselves not compliant with the MCP spec.
- 10:03
Maybe at the end we'll get into that. It's not directly relevant to now, except that all we can do is try to build the best servers we can, subject to the limitations of the clients that will use them.
- 10:12
And again, I put this in here, I, I think we don't need to go through, uh, what MCP is for this audience, so we're gonna move quickly through this.
- 10:19
But it is, of course, for the, for the, for the sake of the transcript, the cliché is that it's USB-C, uh, for the internet. It is a standard way to connect LLMs and either tools or, um, data.
- 10:31
And if you haven't seen FastMCP, this is what it looks like to build a fully, fully functional MCP server. This one, I live in Washington, DC, the subway is often on fire there and so this checks whether or not the subway is on fire and, um, indeed it is.
- 10:48
Now, the question we are here to actually explore is why are there so many bad MCP servers?
- 10:57
Maybe a better question is, do you all agree with me that there are many bad MCP servers? [laughs] I sort of declare this as if it's true. I, I'm not trying to make a controversial statement.
- 11:05
There are many bad MCP servers in the world. I see a lot of them because people are using my framework to build them. It-- Does that surprise anyone that I'm sort of declaring that?
- 11:14
I'm [REDACTED:sexual_orientation] genuinely, I'm, I'm curious if that's a, if I'm, made an assumption. I don't-
- 11:19
In my experience, uh, I, I, I won't say every, e- every, every MCP I-
- 11:26
That's fine
- 11:26
... I came up to is like that, but a lot of them are like API wrappers. They just put a, like, stringify, uh, the content of the API and that's, um-
- 11:35
And that's it
- 11:36
... they call it an MCP.
- 11:37
Yeah. And I, and I think even, I'll, I'll make the argument, this is going a little off script here, but I'll make the argument that a lot of them, even when they're not wrappers, are just bad products because no thought was put into them.
- 11:47
And I mean, uh, one comparison that, that I talk about sometimes with my team is if you go to a, a bad website, you know it's a bad website.
- 11:56
We don't need to sit there and figure out why. It's, it's ugly or it's hard to use or it's hard to find what you're looking for or it's all Flash.
- 12:02
I don't know, I don't know what makes a bad website exactly, but you know what a bad website is when you go to one. Um, we don't like to- Point out all the things because there's an infinite number of them.
- 12:11
Instead, we try to find great examples of good websites. And so what I think we need more than anything else are MCP best practices. And so a big push of mine right now and part of where this talk came from is I wanna make sure that we have as many best practices in the world and documented.
- 12:25
And I do want to applaud, there are a few firms, um, these are screenshots from, uh, Block has an amazing playbook which if you hate this talk, read their, read their blog post.
- 12:34
It's, it's like a better version of what I'm doing right now. And GitHub recently put out one, and many other companies have done as well. I, I could've, I could've put a lot here, but, um, these are two that I've referred to, uh, quite frequently, and so I, I recommend them to you.
- 12:48
Um, the Block team in particular is just phenomenal what they're doing on MCP. I- by coincidence, the same team has been my customer for six years on the data side and they're...
- 12:57
I really love the work that they do, and, um, the blog posts they put out are very thoughtful and I highly, highly recommend them to you. Um, I wanna see more of this, and today is sort of one of my humble efforts to try and put some of that in the world.
- 13:10
And so what I thought we would do today, because I did not want to ask you to open your laptops up and set up environments and actually write code with me because it's four twenty-five on Saturday, [laughs] um, I thought that we would fix a server together sort of through slides, um, to make this again, as I said,
- 13:26
hopefully actionable but, um, but a gentle, a gentle approach to this. And so here is, here is the server that you were describing a moment ago, right? So someone wrote this server, um, I hope that the notation is, is clear enough to folks.
- 13:38
We have, we have a decorator that says that a function is a tool, and then we have the tool itself, and forgive me, I didn't bore you with the, with the details because we think this is a bad server to begin with.
- 13:48
Um, I think in this server, what's our example here? Right, we wanna, we wanna check an order status, and so in order to check an order status we need to learn a lot of things about the user and, uh, what their orders are, we need to filter it, we need to actually check the status.
- 14:00
And if this were a REST API, which presumably it is, we know exactly what we would do here. We would make one call to each of the functions in a sequence and return that as some user-facing output, and it would be easy, and it would be observable, and it would be fast, uh, and it would be testable.
- 14:18
Everything would be good. And instead, if we expose this to an agent,
- 14:22
what order is it gonna call these in? Does it know what the format of the arguments are? How long is it gonna take for the minimum three round trips this is gonna require?
- 14:31
These are all the problems that we're exposing just, just by looking at this. We're not, I haven't even solved them, but that's the problems I see if I were reviewing this as a product-facing, um, effort.
- 14:40
And so the first thing that we are gonna think about, and I think this is probably the most important thing when we think about an effective MCP server because it is product thinking, is outcomes, not operations.
- 14:52
What do we want to achieve? And this is a little bit annoying for engineers sometimes because it's forced product thinking. It's not someone coming along with a user story and, and mapping it all out and saying, "This is what we need to implement."
- 15:07
We cannot put something in this server unless we know for a fact it's going to be useful and have a good outcome. We have to start there. There's just not enough context for us to, uh, be frivolous.
- 15:18
And so here's kind of what this feels like so that we can get a sense for it. Um, the trap when you're falling into the trap, you have a whole bunch of atomic operations.
- 15:30
This is amazing if you're building a REST API. It is best practice if you're building a REST API. It is bad if you're building an MCP server. Instead, we want things like track latest order and give an email.
- 15:39
It's hard to screw up and you know what the outcome is when you call it. Um, with the, the other version of the trap is agent as glue or agent as orchestrator.
- 15:48
Um, please believe me since I've spent my career building orchestration software and automation software that there are things that are really good at doing orchestration, and there are things that are really bad at orchestration, and agents are right in the middle because they can do it, but it's expensive and slow and annoying and hard to debug and
- 16:04
stochastic. And so if you can avoid that, please do. If you can't, there are times when you don't know the algorithm and you don't know how to write the code and it's not programmatic, that's a perfect time to use an LLM as an orchestrator.
- 16:14
Finding out an order status, really bad time, really expensive time to choose to use an LLM as your orchestration service, so don't. Um, instead focus on this sort of one tool equals one agent story.
- 16:25
And again, even here we're trying to introduce a new vocabulary. It's not a user story because user story is everyone thinks human, even though it is a user. It's an agent story.
- 16:34
It's something that a programmatic autonomous agent with an objective and a limited context window is trying to achieve, and we need to satisfy that as much as we can.
- 16:42
And then this is one of those like little tips that feels obvious but I think is important. Name the tool for the agent. Don't name it for you. It's not a REST API.
- 16:50
It's not supposed to be clear to future developers who need to write, you know, you, you're not writing an API for change, you're writing an API so that the agent picks the right tool at the right time.
- 16:59
Don't be afraid about using silly but, um, explanatory names for your tools. I shouldn't say silly. Um, it might feel a little silly, but they're very user-facing in this moment, even though it feels like a, a deep, a deep, a deep API.
- 17:13
Um, this, uh, just in case any of you didn't go read the Block blog post, uh, I just found this section of it so, uh, important where they essentially say something very similar, "Design top-down from the workflow, not bottom-up from the API endpoints."
- 17:29
Two different ways to get to the same place, but they will result in very different forms of product thinking and very different MCP servers. So again, I just, I really encourage you to go and take a look at that, at that blog post.
- 17:40
And if we were to go back to that bad code example I showed you a moment ago and start rewriting this, and if we had our laptops, you're welcome to have your laptops out and follow along.
- 17:48
The code will essentially run, but there's no need. Um, here's what that could look like. We did the thing that you would do as a human. We made three calls in sequence that are configured, that are to our API, but we buried them in one agent-facing tool, and that's how we went from operations to outcomes.
- 18:07
The, the API calls still have to happen. There's no magic happening here. But the question is, are we gonna ask an agent to figure out the outcome and how to stitch them together to achieve it, or are we gonna just do it because we know how to, how to do it on its behalf?
- 18:21
So thing number one is outcomes over operations. Thing number two, another thing-- A, a lot of these, frankly, are gonna seem kinda silly actually when I say them out loud.
- 18:30
Please just trust me from the download graph that these are the most important things that I could offer as advice. And, uh, if-if none of them apply to you, think of yourself as in the top one percent of MCP developers.
- 18:40
Flatten your arguments. Um, I see this so often where I do this myself, I'll confess to you, where you say, uh, "Here's my tool, and one of the inputs is a configuration dictionary."
- 18:53
Hopefully, presumably, it's documented somewhere, in, maybe in the agent's instructions, maybe it's in the doc string. Um, you have a real problem when... By the way, I, I don't remember if I have a point for this later, so I'll say it now.
- 19:07
Uh, a very frequent trap that you can fall into with arguments that are complex is you'll put the explanation of how to use them in something like a system prompt or a sub-agent definition or something like that, and then you'll change the tool in the server, and now you-- it's almost worse than a poorly documented tool.
- 19:24
You have a doubly documented tool, and, and one is wrong and one is right, and only error messages will save you. Um, that's really bad. We're not-- This is a more gentle version of that.
- 19:35
Just don't ask your, um, LLM to invent complex arguments. Now, you could ask, what if it's a Pydantic model with every field annotated and... Fine, that's better than the dictionary, but it's still going to be hard.
- 19:51
There was, until very recently, there may still be a bug in-- Maybe it's not a bug because no one seems to fix it. But in Claude Desktop, all, um, all structured arguments, like object arguments, would be sent as a string.
- 20:06
And this created a real problem, um, because we do not want to support automatic string conversion to object. But Claude Desktop is one of the most popular MCP clients, and so we actually bowed to this and as a, as a matter of, like, necessity.
- 20:20
And so FastMCP will now try, if you are supplying a string argument to something that is very clearly a structured object, it will try to deserialize it. It will try to do the right thing.
- 20:30
I really hate that we have to do that. That feels very deeply wrong to me, that we have a, uh, a typed schema that said I need an object, and yet we're doing kludgy stuff like that.
- 20:38
And so this is an example of where this is an evolving ecosystem. It's a little, um, it's a little messy, but what does it look like when you do it right?
- 20:46
Top-level primitives. These are the arguments into the function. What's the limit? What is the status? What is the email? Clearly defined, just like naming your tool for the agent, name the arguments for the agent.
- 20:57
Um, and here's sort of what that looks like when we get that into code. Instead of having config:dict, we have an email, which is a string. We have include canceled, which is a, a flag.
- 21:09
And then I highly, highly recommend literals or enums whenever you can. Um, much better than a string if you know what the options are. Uh, at this time, very few LLMs know that this kind of syntax is supported, and so they would typically write this, if you had Claude Code or something write this, it would usually write
- 21:27
format:string=basic, which works. It just doesn't know to do this. And so it's one of those little, little actionable tips. Use literal or use enum equivalently. When you have a, a constrained choice, um, your, your agent will thank you.
- 21:44
And I do have instructions or context. I did get ahead of myself. I'm sorry, everybody. It is four thirty-five on a Saturday. Um, the next thing that I wanna talk about is the instructions that you give to the agent.
- 21:55
Um, this cuts both ways. Um, the most obvious way is when you have none. Uh, we mentioned that a moment ago. If you don't tell your agent how to use your MCP server,
- 22:08
it will guess, it will try, um, it will probably confuse itself, and all of those guesses will show up in its history, and that's not a great outcome. Um, please document your MCP server, document the server itself, document all the tools on it.
- 22:22
Um, uh, give examples. Examples are a little bit of a double-edged sword. Um, on the one hand, they're extremely helpful for showing the agent how it should use a tool.
- 22:32
On the other hand, it will almost always do whatever is in the example. Um, this is just one of those quirks. Perhaps as models improve, it will stop doing that.
- 22:41
But, uh, in my experience, if you have an example, let's say you have a field for tags, you wanna, you wanna collect tags for something. If your example has two tags, you will never get ten tags.
- 22:50
You will get two tags pretty much every time. They'll be accurate. It's not gonna do a bad job, but it really uses those examples, um, for a lot more dimensions than just the fact that they work, if that makes sense.
- 23:01
So, so, uh, use examples, but be careful with your examples. Yes, sir.
- 23:05
Uh, giving out of distribution examples is a way to solve for that. Have you seen that?
- 23:11
By out of distribution, do you mean-
- 23:12
Examples that, that are not, would not be representative of actual outputs that, like, have the same kind of-
- 23:17
It's so interesting. So, um, I don't have a strong opinion on that. That seems super reasonable to me. I don't have an opinion on it. I-- In my experience, the fact that an example has some implicit pattern, like the number of objects in array, it becomes such a strong signal that I almost gave this its own bullet
- 23:33
point called examples are contracts. Like, if you give one, expect to get something like it. Out of distribution is a really interesting way to sort of fight against, I guess, that inertia.
- 23:42
I would imagine it is better to do it that way.
- 23:46
I would just be careful of falling into this sort of more base layer trap, I think. So that's completely reasonable, and I would endorse it. I think this is just a more broad...
- 23:54
Whatever example you put out there, weird quirks of it will show up. I, I, on an MCP server that I'm building, I encountered this tag thing just, uh, yesterday, and it really confused me.
- 24:04
No matter how much I was like, "Use at least ten tags," it always was two, and I finally figured it was because one of my examples had, had two tags.
- 24:11
Um, so yes, good strategy. May or may not be enough to overcome these basic, these basic caveats. Um-
- 24:19
Oh, I do have examples or contracts. I'm sorry. It's [laughs] we're thirty-seven. Um, this one I think is one of the most interesting things on this slide. Uh, errors are prompts.
- 24:29
So, um, every response that comes out of the tool,
- 24:35
your, your LLM doesn't know that it's, it's, like, bad. It's not like it gets a four hundred or a five hundred or something like that. It gets what it sees as information about the fact that it didn't, uh, succeed in what it was attempting to do.
- 24:49
And so if you just allow Python in, in FastMCP's case, or whatever your tool of choice is, to raise, for example, an empty value error or a cryptic MCP error with an integer code, that's the information that goes back to your LLM, and does it know what to do with it or not?
- 25:04
Probably it knows at least to retry because it knows it was an error, but you actually have an opportunity to document your API through errors. And this leads to some interesting strategies that I don't want to wholeheartedly endorse, but I will mention, where, for example, if you do have a complex API because you can't get away from
- 25:20
that, then instead of documenting every possibility in the doc string that, that documents the entire tool, you might actually document how to recover from the most common failures. And so it's a very weird form of progressive disclosure of information, where you are acknowledging that it is likely that this agent will get its first call wrong, but based
- 25:41
on how it gets it wrong, you actually have an opportunity to send more information back in an error message. Um, as I said, this is a kind of a, not an amazing way to think about building software, but it is the ultimate version of what I'm recommending, which is be as helpful as possible in your error messages.
- 25:58
Do go overboard. They become part of, as far as the agent is concerned, its next prompt, and so they do matter. Um, if they are too aggressive or too scary, it may avoid the tool permanently.
- 26:08
It may decide the tool is inoperable. Um, so errors really matter, and I don't think this needs too much of an explanation, but this is what it looks like when you have a full doc string and an example, et cetera.
- 26:20
Um, uh, Block, uh, in their blog post makes a point which I haven't seen used too widely, although ChatGPT does take advantage of this in their developer mode, which is this read-only hint.
- 26:30
So the MCP spec has support for, um, annotations, which is a restricted subset of annotations that you can place on various components. One of them for tools is whether or not it's read-only.
- 26:42
And if you supply this optionally, clients can choose to treat that tool a little bit differently. And so the, uh, motivation behind the read-only hint was, uh,
- 26:52
basically to help with setting permissions and, uh, I don't know who here is a fan of dash, dash Yolo or dash, dash dangerous disable permissions or whatever, whatever they're called in different, in different terminals, but then you don't care about this.
- 27:05
But for example, ChatGPT will ask you for extra permission if a tool does not have this annotation set because it presumes that it can take a side effect and can, um, have an adverse effect.
- 27:15
So use those to your advantage. It is one other form of design that the client can choose to provide a better experience with. I've talked about this a bit now.
- 27:26
Respect the token budget. Um, I think the meme right now is that the GitHub server ships like two hundred thousand tokens when you handshake with it, something like that.
- 27:37
Um, this is a real thing, and I don't think it makes the GitHub server automatically bad. I think it's actually makes it endemic on folks like myself who build frameworks and folks who build clients to find ways to actually solve this problem because the answer can't always be do less.
- 27:51
In fact, right now we want to do more. We want an abundance of functionality, and so we'll talk about that maybe a little bit later. Um, but respect for the token budget really matters.
- 27:59
It is a very scarce resource, and your server is not the only one that the agent is going to talk to. So, uh, I was on a call with a customer of mine recently who is so excited that they're rolling out MCP.
- 28:11
And I met with the engineering team, and, and just to be clear, this is an incredibly forward-thinking, high-performing, um, massive company that I incredibly respect. I won't say who they are, but I really respect them.
- 28:24
And they got on the call, and they were so excited, and they were like, "We're in the process of converting our stuff to MCP so that we can use it."
- 28:30
And they had a, a strong argument why it actually had to be their API, so that's not even the punchline of this story, which is a whole other story in, in and of itself.
- 28:38
But it fundamentally came down to this. They had eight hundred endpoints that had to be exposed. To which I had this thought, which if by the time you finish reading this, this is the token budget for each of those eight hundred tools if you assume two hundred thousand, um, um, tokens in the context window.
- 28:55
So if each of those eight hundred tools had only this much space to document itself, not even document itself, share its schema, share its name, plus documentation, this is the amount of space you would get.
- 29:06
And when you were done taking up this space because you were so careful and each tool really fit in this, you would lobotomize the agent on handshake because it would have no room for anything else.
- 29:15
So the token budget really matters. Um, if this agent connected to a server with one more tool that had a one-word doc string, it would just fail. It would just have a over-- uh, effectively an overflow, right?
- 29:27
So the token budget matters. Um, there is probably a budget that's appropriate for whatever work you're doing. You may know what it is, you may not know what it is.
- 29:35
Pretend you know what it is and be mindful of it, um, in a worst case scenario. Try to be parsimonious, try to be as efficient as possible. That's why we do experiments like sending additional instructions in the error message.
- 29:47
It's one way to save on the token budget on a handshake, and the handshake is painful. Um, I'm not sure if folks know that, uh, when an, when an LLM connects to an MCP server, it typically does download all the descriptions in one go so that it knows what's available to it, and it's usually not done in
- 30:03
like a progressively disclosed way. That is done outright. Yes?
- 30:09
Uh, absolutely.
- 30:12
Actually facing that problem of... But it's not only a centralized configuration
- 30:20
So you have like Connect, Gmail, Notion, all of those in one place, and then you get one instance of it. Uh, but I also built this progressive disclosure mechanisms where when it first initializes the instance, it just comes back with a list of like Google Notion and stack services, and then there's an additional describe step for each
- 30:43
one of the connected services. So it's ninety-five percent less context window for a lot of services up front, and then
- 30:52
use whatever service. It doesn't actually expose that to the LLM unless it needs to do that.
- 30:58
That's-- Okay, so that, that's awesome. Let's, let's talk about this idea for one second because it's a really interesting design. Um,
- 31:05
there's a debate right now about what you can do that's compliant with the spec versus what you do that's not compliant with the spec. And as long as you do things that are compliant with the spec, then, then by all means, do them.
- 31:15
Who cares? One of the problems is that there are clients that are not compliant with the spec. Claude Desktop is one of them. I've mentioned it a few times. [laughs]
- 31:22
I have a history with Claude Desktop. Um, Claude Desktop caches all of the tools it receives on the first contact and puts them in a SQLite database, and it doesn't care what you do.
- 31:32
It doesn't care about the fact that the spec allows you to send more information. I think your solution would get around this because it's a tool call, but, um, many of the first attempts that people use to use spec compliant techniques for getting around this problem, such as notifications, fail in Claude Desktop.
- 31:49
Usually, you failed before this in Claude Desktop. I'm not a fan of Claude Desktop from MCP server. I think it's a real missed opportunity because it is such a flagship product of the company that has introduced MCP.
- 31:58
I think it's a real missed opportunity. Claude Code is great. Um, uh, it, it caches everything in SQLite database, so it, like, doesn't matter, uh, what you do. Um, techniques similar to what you've described where you provide mechanisms for learning more about a tool, that's a great idea.
- 32:12
I really like that. Um, there is a challenge where now you are back in a sort of flattened arguments world because you have meta tools now where I need to use tools to learn about tools and use to- tools to call tools in some extreme cases or beyond.
- 32:26
So you need to design this very carefully. That's why it usually does show up as a dedicated product. So thank you for sharing that. Um, uh, there are many really interesting techniques for trying to solve this problem.
- 32:36
Yes.
- 32:38
So you talk about, um, progressive disclosure. Do you support, um, tool masking? So for example, I connect to my Kubernetes server and my credentials only give me certain rights.
- 32:49
So therefore, there are twenty-eight tools that I don't have access to, so therefore you don't need to count it up.
- 32:58
So when you say do I, do I support that, do you mean does MCP support that or do I in my product support that?
- 33:03
Yeah, no, I was just asking because it's something I read halfway, but I don't think I would actually work for that, uh, uh, tool box.
- 33:10
Okay. So, so the spec makes no claim about this. The spec says when you call list tools, you get tools back, and how that happens is, is up to, up to implementation.
- 33:21
Um, FastMCP makes that an overridable hook through middleware, but again, makes no claim on how that is. Prefect commercial products, which I'm not here to pitch, allow per tool masking on any basis.
- 33:34
And we see that as like a place to have an opinionated in the commercial landscape as opposed to an opinion in the open source landscape as opposed to the protocol which should have no opinion at all.
- 33:42
So if that's interesting, we can chat about this.
- 33:45
So you might be getting to this, but like if you take this problem, the example, like the, like Jen mentioned, like kind of table of contents approach. I guess the approach is what you split up into four different chunks or maybe the eight hundred don't all justify having their own MCP server or what was the conclusion?
- 34:01
For them?
- 34:02
Yeah.
- 34:02
They can't do it. They, they, there's no solution that allowed them to have as much information as they wanted on the, on the context window they have.
- 34:10
They didn't need it?
- 34:10
They didn't need it. Um, and we-- and it became a design question. And, and frankly, it was, this call was probably four months ago now, and it was just call after call after call after call like this, um, which made me realize we need to have talks more like this and just talk about what it is to
- 34:24
design a product for an agent. My worry is MCP is viewed as infrastructure or a transport technology, and it is, and I'm very excited. I think by a year from now we will be talking about context products as opposed to MCP servers.
- 34:38
I'm very excited about that. We'll move past the transport. Um, but we need to figure out how to use it and so, so I think that's how we talk about it.
- 34:45
Um, the only other alternative that I have discussed with a few folks, a few companies when you have a problem like this is if you control the client,
- 34:53
much more interesting things become available to you. Um, if you can instruct your client to do things a certain way, for example, if you have a mobile app that presents an agentic interface to an end user, you control the client is what I mean by that.
- 35:05
Um, or if it's internal and you can dictate what, what client or what custom client a team uses, now you can do much more interesting things because you actually do know a lot more about that token budget and how to optimize it.
- 35:17
But for an external facing server, there's not a good, there's not a good solution.
- 35:25
I think by now we have talked through all of this, so I'll leave it for, uh, posterity, uh, in the interest of time.
- 35:34
Um, we talked about curate as a key verb earlier in this talk. Um, it is, I would argue, what we have been doing in each of these little vignettes that we've been working through with the code.
- 35:44
We are curating the same information set down to one that is more amenable and more recognizable for an agent. Um, fifty tools is where I draw the line where you're gonna have performance problems.
- 35:56
I think it seems really low to a lot of people. Some people will talk about it even lower than that. Some people might talk about it higher. If you have more than fifty tools on a server without knowing anything else about it, I'm gonna start to think that it's not a great server.
- 36:09
Um, the GitHub server has, I think, a hundred and seventy tools. Does that mean it's not a great server? No. There's a good argument there and the GitHub team has put out a lot of really interesting blog posts on semantic routing that they're doing.
- 36:20
They had one just yesterday actually on like some interesting techniques they're using. Um- Uh, there's software like, um, like the one you mentioned a moment ago, sir, which, which helps with this problem.
- 36:29
So having a lot of tools like that does not automatically make it a bad server, but it is a smell, and it does make me wonder, can we split them up?
- 36:36
Do you have admin tools mixed in with user tools? Could we namespace these tools differently? Would it be worthwhile having two servers instead of one? Um, that is a little bit of a smell.
- 36:46
If you can get down to five fifteen, that would be ideal. I know that's not achievable for most people, so it's one of those actionable but maybe not so actionable little tips.
- 36:54
It's an aspiration that you should have, and just be careful unless you are prepared to invest in a lot of care and evaluation, fifty tools per agent. I should have said per agent.
- 37:05
If I have a fifty-tool server and you have a fifty-tool server, that's a hundred tools to the agent. That's where the performance bottleneck is, not on the server. Sorry, uh, the slide should be corrected.
- 37:12
It's fifty tools to the agent is where you start to see performance degradation. Um, I love this. Um, Kelly Collefull is someone who I've known a long time. He's at Fivetran now.
- 37:21
And while I was putting this talk together, I happened to come across these two blog posts of his, which are a little bit of like a shot and a chaser.
- 37:27
They're written almost exactly a month apart. One's from October, one's from November. In the first one, he talks about building up a Fivetran server, and he goes from a couple of basic tools to, uh, I think a hundred and fifty-five?
- 37:39
A hundred and eighty-eight. And in the second blog post, he talks about how he curated that server from a hundred and eighty-eight down to five. You could read either of these blog posts.
- 37:46
You could view them independently as a success story on what his adventure was in learning MCP. I think taken together they tell a really interesting story about making something work and then making something work well, which is, of course, the product journey in some sense.
- 38:01
Um, and so where this, where this takes us is sort of the thing that
- 38:06
I-- Sorry, do you have a question? Oh, sorry. Um, where this takes us is sort of the thing that I have found is the most, like, obvious version of this.
- 38:14
I wrote a blog post that went a little bit viral on this, which is why I talk about it a lot, which is please, please just, if nothing else, stop converting REST APIs into MCP servers.
- 38:22
It is the fastest way to violate every single thing we've talked about today, every single one of the heuristics that we laid out about agents. Um, it really doesn't work and, comma, it's really complicated because this is the FastMCP documentation.
- 38:36
That's a blog post I had to write, and the blog post basically says, "I know I introduced the capability to do this. Please stop." And that's a really complicated thing.
- 38:43
That's a, that could be a workshop in and of itself. Um, I do bear a little bit of responsibility here. This is not just a feature of FastMCP, it's one of the most popular features of FastMCP, which is why, candidly, it's not going anywhere.
- 38:56
And instead, we're gonna document around that fact. Um, but here's the problem, right? Uh, you just, you ca-you just can't. You just can't convert a REST... [laughs] I'm not gonna explain it.
- 39:06
You just can't convert REST APIs into MCP server. But, comma, but it is an amazing way to bootstrap. Um, when you are trying to figure out if something is working, do not write a lot of code where you introduce new ways to figure out if you have failed.
- 39:21
Do start by picking a couple of key endpoints, mirroring them out with FastMCP's auto converter or any other tool you like, or even just write that code yourself. Make sure you solve one problem at a time and make the first problem being can you get an agent to use your tool at all?
- 39:36
Once it's using it, by all means, strip out the, the part of it that just regurgitates the REST API and start to curate it and start to apply some of what we've talked about today.
- 39:45
Um, this, this is just one of those candid things, right? It is the fastest way to get started. You don't have to do it this way. I start this way.
- 39:53
Um, just don't end up. Don't ship the REST API to prod as an MCP server. You will regret it. You will pay for it, um, a little bit later, even though there's a dopamine hit up front.
- 40:03
So, um, these are the five major things that we talked about today in our pseudo workshop, workshop that wasn't really a workshop, actionable talk. Um, outcomes, not operations. Focus on the workflow.
- 40:17
Focus on the top-down. Don't get caught up in all the little operations. Don't ask your agent to be an orchestrator unless you absolutely have to. Um, flatten your arguments.
- 40:25
Try not to ship large payloads. Try not to confuse the agent. Try not to give it too much choice. I don't think I said out loud when we talked about that, but try not to have tightly coupled arguments that really confuses the agent.
- 40:35
Um, see if you can, uh, design around that, uh, if possible. It's not always possible, but if you can. Um, instructions are context. Seems obvious to say out loud.
- 40:44
Of course they are. They're information for it. Use them as context. Design them as context. Really put thought into your instructions the same way as you would into your tool signature and schema.
- 40:54
Respect the token budget. Have to do it. It, it's, this is the only one on this list where if you don't actually do it, you will simply not have a usable server.
- 41:01
The other ones you can get away with, and frankly, the art of this intuition is start with these rules and then work backwards into practicality. But this is the only one where I think you can't actually cross the line.
- 41:10
And then curate ruthlessly. If you do nothing else, start with what works and then just tear it down to the essentials. Um, I, I have been writing MCP servers about as long as anyone at this point, um, a year, and I still find myself starting by putting too many tools in the world sometimes 'cause I'm not sure
- 41:29
which one it will use or, or I'm experimenting, and I have to do, I have to remind myself to go back and get rid of them, and it, and it's hard.
- 41:36
I think as an engineer, especially designing normal APIs, you're like, "Okay, like, here's my tool, here's V2, it's backwards compatible," right? Like, and you keep, you keep adding stuff, and that's a really natural way to work, and it can be a best practice, and, uh, it doesn't work here.
- 41:50
You are-- It would be like using a UI that just showed a REST API to a, to a user. Um, this is, this is a criticism I have offered of my own products at times when I'm like, "This looks a little bit too much like our REST API docs," right?
- 42:01
We're not doing our job to actually give this to our users in a, in a consumable way. Um, so if I can leave you with just one, just one thought, it's this: um, you are not building a tool, you are building a user interface.
- 42:15
And treat it like a user interface because it is the interface that your agent is going to use, and you can do a better job or you can do a worse job, and either you or your users will, will benefit from that.
- 42:27
Um, I think- I think we are at our time, so I'm gonna just open it up for questions or what's next or what, what other challenges we can solve.
- 42:37
Um, I hope that... I hope I found the... I hope I walked the tightrope between, uh, things that are useful to you all, but don't require you to write any code at four fifty-four on a Saturday now.
- 42:49
Um, but I, I hope, I hope, I hope I had some useful nuggets in there for you, more than you, more than you came in with, and happy to take any questions if there are any.
- 43:00
So, uh, what are examples of tightly coupled arguments?
- 43:03
Um, that would be where you have one argument that's like, um, what is the file type, and another argument that's like, how should we process the file? And your input to the file type argument determines the valid inputs for the other argument.
- 43:17
So they're, they're now tightly coupled. Some, some arguments on the second thing are invalid depending on what you said for the first thing, and it's just one extra thing to keep track of.
- 43:25
That's a good question. Sorry, I didn't define that.
- 43:28
Do you have a question?
- 43:29
Uh, I have two. I will start with the first one. Uh,
- 43:34
when you are given like an, an agent, an MCP server, you have to like document the tools or, or the, the capabilities of the server
- 43:43
in the server, uh, and in the agent, uh, and that is like, uh, not ideal.
- 43:49
Yes.
- 43:49
So what, what would you recommend? That or only in the MCP server?
- 43:53
So this, this comes down to do you control the client or not. If you control the client, then this is a real choice, and there are, uh, there are different ways to think about it.
- 44:01
So, um, for example, in some of my stuff that I write that I know I'm using, for example, Claude Code to access, um, I might actually document my MCP server as, um, files or Claude skills because I know what the workflows are gonna be.
- 44:17
I know that some of my workflows are infrequent, and I don't wanna pollute the context space with them. So if you, if you control the client, you, you have a real choice to make there.
- 44:26
If you don't control the client, then you don't have so much of a choice. You have to document it here because you have to assume you're, you're working with the worst possible client.
- 44:33
Um, honestly, many of the answers in MCP space boil down to do you control the client, then you can do really interesting things on both sides of the protocol.
- 44:42
From a server author perspective, you really do need to document everything in its docstring. The one escape hatch is that you can document a server itself. So every server has an instructions field.
- 44:54
Um, it is not respected by every client.
- 44:57
I believe my team has filed bugs where we have determined that to be the case. Um, so hopefully that's not a permanent thing. But most clients will on handshake download not only the tools and resources and everything, but a instructions blob for the server itself.
- 45:14
How much information you can put in there, I- I'd be careful. I don't think it wants to read a novel, but you do have this one other opportunity to document maybe the high level of your server.
- 45:25
That's one other one but-
- 45:26
Oh, yeah. Well, why don't we... Let's mix it up and we'll come back.
- 45:28
Yeah.
- 45:28
Did you have a question?
- 45:29
Yeah. How close are you to the MCP spec changes?
- 45:35
I'm pretty-- I'm not a member of the core committee, but I'm in very close contact with them, so maybe I can answer your question.
- 45:40
What about asynchronous, uh, the long-running-
- 45:43
I'm so excited about this.
- 45:44
Yeah.
- 45:45
Yes. This I know a lot about.
- 45:46
What do you think is gonna change?
- 45:50
It's gonna ch- it's, it's, it's going to expand. It's not actually gonna change so much because of the way it's implemented. Um, uh, what question could I answer? Like, what is it?
- 45:58
Oh, the-
- 45:58
Am I excited about it? I am excited about it. Um, so
- 46:05
all the rules still apply. That's a... That is a fantastic question. Let's talk about this for one second. Um, some of you, I don't know if any of you were at a meetup we hosted last night where my colleague actually gave a presentation on...
- 46:14
Oh, you were. Yes, that's right. [laughs] I was like, "I know at least somebody's coming." Um, uh, my colleague Adam gave a very good talk on this, which I can...
- 46:24
We'll chat after this. I'll, I'll send you a link to, um, to a recording of it. Um, but the nutshell version is this is, this is, uh, SEP-1686, uh, is the name of the proposal, and it adds asynchronous background tasks to the MCP protocol, not just for tools, but for every operation.
- 46:40
Um, and we don't need to talk about, too much about what that is. The reason it doesn't involve changes to any of these rules is, um, this is essentially an opt-in mode of operating in which the client is saying, "I want this to be run asynchronously."
- 46:55
And therefore, the client takes on new responsibilities about checking in on it and, and, and polling for the result and actually collecting the result. But the actual interface of learning about the tool or calling the tool, et cetera, is exactly the same as it is today.
- 47:09
So this is fully opt-in on the client side. Um, and that's why from a design standpoint, nothing changes. The only question from a server designer, um, standpoint is, is this an appropriate thing to be backgrounded as opposed to be done, you know, synchronously on the server?
- 47:27
Um, or sorry, let me take that back. You can background anything because it's a Python framework. So you can chuck anything in a Python framework. The question is should the client wait for it or not?
- 47:36
Should it be a blocking task is really the, is really the, the right vocabulary for this. Um, and that's a, that's just a design question for the server maintainer.
- 47:45
Is that... Am I in the-
- 47:47
Yeah
- 47:47
... the, the zone of what you were looking for?
- 47:50
Sure. I fixed it.
- 47:55
Oh, no kidding.
- 47:56
It's now on that one code issues tool, but when it gets an error in the person's code, the LLM is like impact.
- 48:07
Very, yes. This happens a lot actually and-
- 48:10
A lot
- 48:10
... but until you said this I didn't think of it as like a pattern, but I've seen this a lot. It's a real problem.
- 48:14
Yeah.
- 48:15
Maybe we'll write a, we'll write a blog post on that. That'd be fun. Um,
- 48:21
yes.
- 48:21
I was gonna say the rules still apply, but as far as, uh, elicitation is concerned, how do you view that in terms of-
- 48:28
Uh, elicitation is really interesting. So, um, now we're in advanced MCP Elicitation. Anyone not familiar with what that is? Yes. So elicitation is basically a way to ask the client for more input halfway through a tool execution.
- 48:45
So you take your initial arguments for the tool, you do an elicitation, it's a formal MCP request, and you say, "I need more information," and it's, uh, structured, is what's kinda cool about it.
- 48:53
So the most common use case of this in clients that support it is for approvals, where you say, "I need a yes or no of whether I can proceed," on maybe it's some
- 49:03
irreversible side effect or something like that. Um, when it works, it works amazingly. Again, it's one of those things that ha- doesn't have amazing client support, and therefore, a lot of people don't put it in their servers, 'cause it'll brick your server [chuckles] if you send out this thing and the client doesn't know how, what to do with
- 49:17
it. So you gotta be a little bit careful. Does it change the design? It's a fantastic question. I wish it were used more so I could say yes, and you should depend on it.
- 49:27
If all clients supported it and it was widely used. And the reason all clients don't support this one, by the way, I'm not trying to... it's not, like, a meme that clients are bad.
- 49:34
It's complicated to know how to handle elicitation because some clients are user-facing. Then it's super easy, just ask the user and give them a form. Some clients are automated, some are backgrounded, some are...
- 49:43
And so what you do with an elicitation is actually kinda complicated. If you just fill it in as an LLM,
- 49:49
uh, maybe you satisfied it, maybe you didn't. It's, it's a little tough to know. So if it were widely used, I would say absolutely it gives you an opportunity to put, in particular, tightly coupled arguments into an elicitation prompt, um, or confirmations.
- 50:04
Um, a lot of times you'll see for destructive tools, you'll see confirm, and it'll default to false, and you're forcing the LLM to acknowledge at least as a way of, you know, hopefully tipping it into a more sane operating mode.
- 50:16
Elicitation is a better way to design for that. I didn't, I don't think that made it into this, in any of these examples. So great question. Wish I could say yes.
- 50:23
I hope to say yes. How about that? You had a second question.
- 50:27
Yeah, uh, um, so, so in my, in my job, the main thing I do is, is build, uh, agents and I do, like, LangGraph, OpenAI SDK or something like that.
- 50:38
And I usually just, like, write the, the tools and the tools calling the APIs, and I don't, like, really see the, the need for the MCPs in, in the, in that, uh, that space.
- 50:49
Do, do you agree that the MCPs are, like-
- 50:52
I do
- 50:52
... not needed, uh, there, or do you have, like, a-
- 50:55
I do. I, I think, um-
- 50:58
Because-
- 51:00
I would not, I would not tell you to write an MCP server.
- 51:02
Yeah.
- 51:03
I think that within a year, the reason you would choose to write an MCP server is because you'll get better observability and, uh, understanding of what failed. Whereas the agent frameworks are not great because part of the whole agent framework's job is to not fail on tool call and actually surface it back to the LLM, similar to
- 51:20
what we were talking about a moment ago. So you often don't get good observability into tool call failures. Um, some do, but not all. Uh, and so one of the reasons to use an MCP server, even for a local case like that, is just because now you have an automatic infrastructure so you can actually di- debug and,
- 51:36
and diagnose and stuff. I don't think that's the strongest reason to do it. I think that's gonna be in a year when the ecosystem's more mature. I think if you are, if you fully control the client and you're doing client orchestration and you are writing, if you are writing the agentic loop and you're the only one, do
- 51:49
whatever you want.
- 51:50
Uh, I do think that all, all of the advice you gave today also applies when you are building tools, right?
- 51:54
It absolutely does. This is, this is... Yes, everything we said today applies to Py-
- 52:00
All the tools
- 52:00
... like, a Python tool. Absolutely. And that's, I mean, that's how FastMCP treats it. It's a good question.
- 52:05
Any last questions? I'm happy to... Yes. Yes.
- 52:18
Is there a way we could get on FastMCP client side?
- 52:24
What does that allow?
- 52:25
Yes. Um, so code mode is something that Anthropic, uh, bl- uh, Cloudflare actually blogged about, uh, first, and then Anthropic, uh, followed up, where you actually ask, you, you solve some of the problems I just described here.
- 52:36
You ask the LLM to write code that calls MCP tools in sequence, and it's a really interesting sidestep of a lot of what I just, uh, talked through here.
- 52:46
Um, the reason that I don't recommend it wholeheartedly is because it brings into other, other sandboxing and code ex- like, there's, there's other problems with it, but if you're in a position to do it, it can be super cool.
- 52:57
Um, I actually have a colleague who wrote, the day that came out, he wrote a FastMCP extension that supports it,
- 53:06
which we put in a package somewhere. We didn't, we at first didn't wanna put it in FastMCP main because we weren't sure. FastMCP tries to be opinionated, and we weren't sure how to fit that in.
- 53:16
And then actually it was so successful that we decided we're gonna add an experiments
- 53:22
flag to the CLI and have it, but I don't know if it's in yet.
- 53:26
I can probably check that.
- 53:29
Hmm?
- 53:40
The Fastforward.
- 53:45
Yeah. Th- this will go into this new, I forget if we called it experiments or optimize. It's, it's, it's on our roadmap right now, and this would, this would go in there.
- 53:53
Um, and then there's, like, a whole world right now of optimizing tool calls and stuff, but I, I would like to be respectful of your time and allow you all to go back to your, your lives.
- 54:01
You're very kind to spend an hour talking about MCPs with me. I'm more than happy to keep talking if anybody has, has questions, but I, I would like to free you all [laughs] from the conference.
- 54:11
I hope you all enjoyed the talk, and thank you very much for attending. [outro music]