Are MCPs Overhyped? A Rant about MCPs

Talk transcript

1. 0:02

   Hi, I'm Henry. Because MCP is such a new space, um, instead of going into technical details like most AI engineer videos that you're probably seeing, uh, today I'll be doing a high-level overview, uh, and a rant about all the problems within the MCP space and ecosystem.

2. 0:18

   A quick introduction, uh, about myself. Um, so I'm Henry. I'm the founder and CEO of Smithery. Uh, I'm also a member of the MCP steering committee. Before Smithery, um, I co-founded Jenny AI.

3. 0:30

   Um, let me tell you a little bit about the origin story for Smithery. Um, this is around November last year, uh, in twenty twenty-four. Uh, I went down this rabbit hole trying to figure out, uh, trying to tackle the ARC-AGI challenge.

4. 0:44

   And if you haven't heard of the ARC-AGI challenge, it's an IQ test made for LLMs. And in this challenge, um, you're given two or three different examples of, uh, a pattern, and you want your LLM to predict the missing, uh, last pattern here.

5. 0:59

   And it turns out that this challenge is very easy for humans to do. Humans get about, uh, eighty percent, uh, accuracy on this challenge, but it's pretty hard for an LLM to do.

6. 1:08

   And LLMs could not crack it for many, many years. So I, I kind of focused on tackling the challenge, uh, but very quickly, uh, OpenAI decided to release o3, and o3 completely dominated, uh, the ARC-AGI-1 challenge and basically got, uh, human level performance.

7. 1:24

   It scored very well on math benchmarks as well. So we're basically done, right? AGI achieved. Um, autonomous agents are going to be solving all our problems now. But fast-forward to twenty twenty-five, and I'm wondering, where are all the autonomous agents?

8. 1:40

   Why are they not solving all the tasks that we don't want to be doing? So what we're seeing here is sort of Claude's paradox. Uh, we have all these frontier labs creating, uh, all this intelligence, but all this intelligence is stuck in a box.

9. 1:54

   And in order to make AI agents practically useful, we have to start thinking about context and capability. What are the inputs and outputs, uh, to our models? Everyone from startups to major labs recognize this problem.

10. 2:08

    And so in November twenty twenty-four, Anthropic released the Model Context Protocol. Uh, it's an open standard to help LLMs connect to different services, uh, with a promise of standardizing, uh, this N times N problem.

11. 2:19

    So when I saw MCPs, uh, in around this time last year, I was pretty excited about it. Um, there was a small but vibrant developer community. The models are getting smarter, and MCPs are standardizing the way models talk to services.

12. 2:32

    So are we finally done? Well, not quite. With MCP and a new ecosystem of services trying to target AI agents, we have a bunch of new problems showing up.

13. 2:44

    So first, I want to go through the list of user problems that we have.

14. 2:48

    So the first one is fragmentation. There are more and more MCP servers being deployed every single day, and be-- it's becoming hard to find high-quality ones. The MCP committee is currently working on a better solution for this by creating a, an official registry, thanks to Tess and the Pulse MCP team.

15. 3:04

    Simply having a registry doesn't solve the problem. For example, how do we assign reputation to high-quality MCPs is still an open question.

16. 3:12

    Another problem that MCP users face is high friction install. If you go to any MCP's GitHub repository, you, you'll probably see this five-step installation process, which makes it very difficult to install.

17. 3:23

    And in addition to that, once you have your MCP installed, you might just have installed an insecure, uh, unsecure MCP. Finally, it's unclear how we're gonna create this new economy of AI-native services if we don't have a plan on how to handle agentic payments.

18. 3:37

    Open questions in this area include, "How do I get agents to pay on my behalf?" And, "How do I avoid subscribing to a hundred different services, uh, each charging ten dollars a month?"

19. 3:47

    Next, let's look at the developer side problems. So if you're building an MCP, you probably face hosting problems. Thanks to the Streamable HTTP transport, it's now much easier for developers to find platforms to host on.

20. 3:59

    But hosting is still a challenge. Uh, developers will have to deal with stateful sessions, resumability, and other issues.

21. 4:06

    The developer tooling is also, uh, quite lacking in the MCP space. We have a basic MCP Inspector provided by the official MCP repository, which allows you to test out different tools, uh, check out your prompts, but there are still open questions that developers are asking us.

22. 4:21

    For example, "How do I design the best MCPs? How do I know if my tool is gonna be called? How do I create the best agent experience?"

23. 4:30

    Next, we also have distribution as a problem. If you're a developer, you create an MCP, how does your MCP get discovered? Observability is also an issue. If you deploy your MCP and people are using it, how do you improve this MCP once it's launched?

24. 4:44

    And lastly, how do you make money out of your MCPs? So in summary, the MCP ecosystem is very exciting, but it has a bunch of different problems. On the left side, we have all the problems faced by MCP users.

25. 4:54

    On the right side, we have problems faced by all the MCP developers. That's why we're building Smithery. I started Smithery in around December twenty twenty-four to tackle these challenges.

26. 5:04

    Smithery is aiming to become the AI gateway to grow and orchestrate this new era of AI-native services for AI agents. So far, we've only scratched the surface of the problems I've described, uh, previously.

27. 5:16

    But we built this little demo to showcase what you can build when some of these problems are solved. So here what you're seeing is, uh, the Smithery playground. The main goal of the playground is for us to demonstrate what an AI agent can do when it has access to thousands of curated MCPs.

28. 5:32

    So let's start with a prompt. "Find the most pressing issue on my GitHub repository called [REDACTED:url] and create a new ticket on Linear."

29. 5:44

    Okay, so let's submit this prompt. You'll see the agent first start thinking about the issue. It does-- It then calls this search servers function, which, uh, looks up for all the servers within Smithery.

30. 5:57

    It connects to the best server. It also connects to Linear.

31. 6:04

    It's able to go on, uh, GitHub using the GitHub MCP to find, uh, different bugs.

32. 6:10

    It's trying to find the most high priority item.

33. 6:15

    So it found a team, and now it's creating, uh, a ticket on Linear.

34. 6:21

    Great. So it looks like our, our agent has successfully, uh, went on GitHub, looked for the most pressing issue, uh, on my GitHub issues, and created a Linear ticket.

35. 6:31

    So let's take a look at this ticket. Nice. Okay. So it has a lot of details. It has even a link to the original issue. Um, yeah. So we have sort of an end-to-end, uh, task, uh, being solved by an AI agent connected to two different MCPs.

36. 6:46

    So obviously, you know, this demo only scratched the surface, but you can imagine what we can do when more and more servers get deployed, uh, on Smithery. So this graph kind of shows you the number of servers being deployed, uh, over the last couple of months and the number of tool calls we've been getting.

37. 7:01

    With all this developer enthusiasm, I'm confident that we can solve all of the problems that I mentioned in this talk today.

38. 7:08

    It's clear to me that the future of the internet will be dominated by tool calls rather than clicks.

39. 7:14

    And in this new world, it's going to be the agent experience that matters more than the user experience. And this agent experience is not going to be built by just me or a few companies.

40. 7:24

    It's going to be built by all of you. Thank you.