Letting AI Interface with Your App with MCP

Hello, my name is Kent C. Dodds, and I teach people how to build excellent user experiences wherever they are. I've taught people how to build excellent user experiences on the web, and now guess what?

We're all moving over to AI because that's where our users are going. And so now I teach people how to build excellent user experiences with EpicAI.pro as my course platform.

Go take a look at it. Today, I wanna talk with you about how user interaction is changing, how Model Context Protocol and things like it are, uh, helping make that change possible, and what your role in this is as a product developer or as somebody who's trying to reach users where they want to be.

And where they want to be is inside of AI assistance. So let's talk about that. My talk is Letting AI Interf-Interface with your app with MCPs, or Model Context Protocol services.

Uh, and normally, when I start my talk, I actually invite people to stand up because... or, or at least get their body moving because our bodies need blood to flow for our brain to operate properly.

Um, this is kind of a-an awkward thing to do in a remote setting, so we're not gonna do that. But if you've been just sitting around for a while, I invite you to move your body, stand up if you can, and, uh, get blood flow because your brain does need that.

So make sure you're taking care of your body. [laughs] All right, so let's just get right into it. I'm going to show you a video of Tony Stark interacting with his AI assistant, Jarvis.

So if you're not familiar with Tony Stark, he's Iron Man, and, um, this is from one of the Iron Man movies. So, um, as we're watching this, I want you to, uh, consider what are the things that Tony is having his AI assistant, Jarvis, do that we are not able to do with the technology that we have

right now. And then we'll talk about it once the video is done. Here we go.

I've compiled a Mandarin database for you, sir, drawn from SHIELD, FBI, and CIA intercepts. Initiating virtual crime scene reconstruction.

Okay. What do we got here? Close.

The heat from the blast was in excess of three thousand degrees Celsius. Any subjects within twelve point five yards were vaporized instantly.

No bomb parts found in a three-mile radius of the Chinese Theater.

No, sir.

Any military victims?

Not according to public records, sir.

Bring up the thermogenic signatures again. Factor in three thousand degrees.

The Oracle Cloud has completed analysis. Accessing satellites and plotting the last twelve months of thermogenic occurrences now.

Take away everywhere that there's been a Mandarin attack.

That's two military guys. You ever been to Tennessee, Jarvis?

Creating a flight plan for Tennessee.

All right. Well, that's pretty cool. I would, I would like to have a Jarvis like that that can help me. Can you imagine Tony doing all of that research and everything without an AI assistant?

Uh, it would take longer. [laughs] Let's just put it that way. So what did Jarvis do? Compiled database, a generated UI on demand, accessed public records, brought up thermogenic signatures.

I, I think I know what that is. Uh, and Tony said, "Take away everywhere there's been a Mandarin attack," so, uh, joins across, uh, different datasets, uh, showed related news articles, uh, created a flight plan for Tennessee, and answered the doorbell, or at least showed Tony, um, the...

who was at the door with, like, all their... You know, don't go to Tony's house if, if you want your privacy. [laughs] But, uh, yeah, so this is really exciting, uh, exciting stuff, the, to think that, like, we could maybe have some of this.

And so the question is, can we? Like, what, what did Jarvis do that we can't do? Okay, so there were actually, uh, a couple of things. It's not just nothing.

Uh, we probably, uh, couldn't, uh, uh, create a database based off of feel, uh, FBI and, and CIA, and, uh, SHIELD database stuff, but not for technical reasons. And the holographic stuff, I think we're, we're still working out.

Uh, but generating UI, we absolutely can do that, and people do that all the time. Uh, so we have the technology to do this stuff. So my question is, why don't we each have our own Jarvis already?

Like, what, what's stopping us? And the, the interesting thing is that we've actually tried this. We've been trying this forever, and it's just not, it's not quite there. And so why is it not quite there?

What's, what's holding us back from being able to have Jarvis? Like, isn't that just the most amazing experience? And, and Jarvis isn't just like a... like something that you talk with.

You know, you know, if you're in the office with a bunch of other, other people, you probably don't wanna have a talking interface. It's also, like, he uses, uh, typing, and he also uses gestures and, and everything too.

And he te-- uh, generates dynamic UIs, and then he can interact with those as well. So, like, it's a really, really awesome user experience. So why don't we have a Jarvis, even though we've been working on it?

What, what is holding these different assistants back? And the answer is integrations. It's just really, really hard to build all the integrations for all the possible things. Here are a bunch of logos for things that you might recognize, s-some things that you don't.

Some are technical, some are, like, normal people things, and I want to have a robot that can interface with all of them. And, like, here's the, the real kicker: I wanna have a robot that interfaces with everything.

And the, uh, challenge of building an integration for every one of those things just means that Google is not going to build an integration for your local city's government website for reserving pavilions at the park, and I want that.

I want my robot to be able to do all of those things. And if it can't do everything, then why spend the time wiring everything up together so that it can do some things?

And there are some activities that we take or some activities that we do, some things that we do that don't really, um, like don't make it worth it to spend all the time wiring things up.

Like, we just don't do it enough. But it would be really cool if there was some standard mechanism for my AI assistant to interface with something that I've never used before and I won't ever use again, but I don't wanna have to like, click around the website and figure it out.

I want my LLM to figure that out. And that is what Mo- Model Context Protocol enables for us, is a standard mechanism whereby our AI assistants can communicate with these, uh, various, uh, tools and services.

So let's talk a little bit about the history and architecture of MCP, Model Context Protocol, and, and just like tool calling and that sort of thing in general.

So phase one, about three years ago, ChatGPT comes out, and, uh, this is a really, really exciting time because it answers our questions. It can... Like, we can say, like, tell it something and ask it something, and it can answer the questions.

And maybe the answer isn't always right because it's not intelligent, it's just generating tokens, but like lots of the times it is, and it's gotten may- way better. And so this is really exciting.

And what made it so pivotal is the fact, not, not the fact that we had an LLM. We've had LLMs for a long time. What made ChatGPT so awesome is the host application layer around the LLM, ChatGPT itself, not the LLM, but the application around it that brought a good user experience to interfacing with an LLM.

And then once that happened, a lot of investment went into LLMs, and they got better, better and better and better, much faster because we started to see a lot of value.

The problem was that you would have to provide the context yourself. You'd copy-paste your code and say, "Hey, what's wrong with this? Please refactor it," whatever. And then you'd have to copy and paste its result back into your code base, and then you find out it wasn't working for, uh, some other reason. [laughs]

Um, but you're-- there was a lot of context, like manual context, bring it in in the form of text, and then we added images, but, uh, having to manually bring in that stuff and then manually take it out, and then of course, you've got your context window and all that stuff.

So it, it couldn't do anything. It could answer questions, but like couldn't do anything, and, uh, managing context ourselves was kind of a pain.

So then we get into phase two, where the host application's like, "Hey, I'm like regular code. I, I interface with this LLM. I could tell the LLM, 'Hey, I need you to let me know if you need more context, and I can go get it for you.'" So now we have search engines, and, "Hey, if they want

to schedule a, a time to meet with somebody, I have a, a calendar integration," and, uh, "Oh, if you want to summarize your Slack messages or your notifications or something, I have a Slack integration," whatever.

So the host application is now enabling it to do stuff. I can tell it, "Hey, do this thing," and now it will do that thing. And, uh, the, the problem here is that you're kind of limited with the, the things that it can do and also the sources where it can get its, um, its context.

You're limited by the, uh, developer's time, uh, at OpenAI or Anthropic. How much time do they have to dedicate to building out these integrations? And this is really the same problem that we have with our existing assistants and the existing efforts that we've put into building integrations with all these different services.

You're not going to convince OpenAI or Anthropic developers to build an integration for your city government website or anything like that. That's just not going to work. And granted, OpenAI has their, uh, own GPT plugin system, but it's proprietary, and so it's, uh, difficult.

Even though ChatGPT has a lot of, uh, buy-in from, uh, the users that you have, um, it's difficult to justify building something special for ChatGPT and then something special for Anthropic and then some- something special for Google and whatever.

Uh, it'd be much better if we had some, like, standard mechanism. So yes, it can do stuff, and, and for a lot of people who couldn't get buy-in from the, um, the company to build a built-in integration, they would just build a wrapper, their own host application with their own tools.

And the thing is, users don't want to do that. Users want one Jarvis, just one, just one Jarvis that is able to augment itself with any capability in the world, any possible capability.

They don't want to have to load up your, uh, LLM wrapper with the context necessary and then grab that context and put it into the next tool and whatever.

They want one thing that can interface with all of them. And so, yeah, it can do stuff, but it can't do enough, and that is a big pain, and that's what pushes us into phase three, MCP.

Now, with MCP, it can do anything because MCP is a standard protocol that all of the, uh, different assistants support or will very soon support, and, um, now you can build to the MCP spec, and you're usable by any of these AI as- assistants because it's a standard.

So we can do anything. The clients aren't quite ready at the time of this recording just yet, but soon they will be, and it's going to be phenomenal. We are just one really good user experience application away from Jarvis for everybody, and that's very exciting.

So to expand out on this a little bit, uh, as far as the architecture is concerned, the host application is communicating with the LLM, and the host application tells the LLM what services are available to it, and those services can be dynamically added.

Uh, they, uh, can be, um, added and removed. So, like this context can be managed by the host application, and the LLM just knows what's available now and the user's query, and then it can select And the most appropriate tool.

The host application, uh, creates a client for every, uh, service that it wants to integra-- in-interface with, and this client is a standard, uh, client, so it doesn't need to have any special, uh, integration with anything.

There's a standard interface for this. And then the service provider creates the MCP servers, and, uh, those servers interface with tools, resources, prompts, sampling, uh, like lots of really, uh, cool features on that end of the spectrum.

And this is what makes it so great is that the service provider is in control of this stuff. This is the stuff that's gonna be unique to every, uh, service.

Uh, everything else like the, the server and the client, that interface, that communication is standard, and that's why this works. That's why it's so cool. This is how, uh, the way I say it is this is how we give Jarvis hands to be able to actually do stuff.

It's very exciting. So I wanna show you a little demo of, um, a, a couple of MCP servers that I put together, uh, to kind of explore what's possible here.

Like I said, the clients aren't quite ready, so the experience is not the, uh, what the Jarvis experience was, but I, I don't see any reason why we can't get there.

So let's get into that. So first, um, I made this, uh, in London, so it's gonna find out my current location here in a second [chuckles]. But, uh, here's the prompt.

This is in, in Claude Desktop, whic-which has been configured with my MCP servers, three of them. "Please write a journal entry for me." So one of the MCP servers is a, a journaling, uh, server.

Uh, "Here's my email address. About my trip with my daughter, I would like you to derive my location and weather conditions from my device location and make up a creative story with relevant text.

Thanks." So [clears throat] we execute this, and as part of the context, um, it, uh, has an MCP server called Locationator, which will determine the current location. And so with this, I'm going to allow it for this chat.

Now, you notice Tony didn't have to approve different tool calls and stuff for Jarvis because Tony has a lot of trust built up with Jarvis. We don't quite have that trust built up yet and-- or that, uh, capability, and so that's why we're not quite there yet, um, and we have to approve every tool call and stuff.

But we'll get there eventually once, uh, things get more reliable. So here it gets my current location. That was my location at the London hotel, so don't worry, this isn't my home [laughs].

Uh, and then I check the weather. This is from another tool, uh, another server, uh, Get Weather, for getting the current weather at, um, uh, given, uh, coordinates. And now it's gonna run another tool from Epic Me.

This is the authenticate tool because I haven't logged in yet. And so I'm going to provide my email address to the Epic Me server. And at the time of, uh, this demo, um, I didn't wanna actually open up my email and, uh, as part of this recording.

And so, um, I was just running it locally, and I logged it to the, uh, console. So let me just grab that token, and I provide that auth token.

And so now my MCP client has been authen-- or it's going to be authenticated with this validation token. So the MCP server has authentication built in. It's based on OAuth 2.1, and so it's, uh, as secure as anything else that you're using O-O-OAuth with.

And, um, my, uh, Jarvis client or, or Claude Desktop has been authenticated so I can perform authenticated tasks, which, uh, one of which is to create a journal entry.

Now, of course, the LLM's gonna write the journal entry for me. I didn't wanna take the time to do that, but you could write your own journal entry, uh, of course, and, uh, have it just insert that yourself.

And with the right kind of interface, you could, uh, speak your journal entry, and it could format it nicely for you and everything. That's what makes these LLMs so cool, and especially when we're talking about multi- multimodal stuff.

So, um, we're going to create a journal entry, um, with that information. Um, my MCP server configured the inputs and things, and now it's going to check, okay, what tags do you have available?

Okay, great. So you've got a travel tag. Uh, let's... Or let's create a travel tag because you don't have one for that, and that makes sense for this journal entry.

We'll add that, uh, tag to the entry and, uh, and then, like, it continues, uh, in this manner. And here's, uh, another really cool and interesting thing about MCP and the fact that it wor- working with an LLM, um, is I, I can actually ask it, "Hey,

could you show me that journal entry?" And remember, that journal entry, it had a title, and it had contents. Um, but here it's going to, uh, retrieve that journal entry, and then it decides based off of what it retrieves that there's a, a good way to format that that's better than just the JSON format that is provided.

So the server can communicate in a way that makes a lot of sense for the server, um, and then the client can take that as context to display whatever it wants to the user.

Right now, we don't have, um, very many, uh, clients that support any s- mechanism for displaying, like, a dynamic UI or, like, a card or, or something like that, but you can imagine that we could totally do that.

There's nothing stopping us from doing that. And I think it's actually also interesting that it rendered this as kind of markdown with a title and everything too, uh, which I think is pretty cool.

And additionally, uh, I could actually ask it to translate this into any language, and it would do a pretty good job of that too. So maybe the MCP server is, uh, sending responses in English, but you're talking to your LLM in Japanese, and the LLM can convert that English response into Japanese, which I just think is very,

very, uh, cool and interesting. So here now I'm telling it to delete the post and log me out because it's fake. And, um, again, all of the authentication stuff that you would expect, uh, all works here.

The Epic Me MCP server is completely only accessible via, uh, MCP a- and, uh, clients like this. You can't actually access it as a web application. And I think that is the transition that we're going to be experiencing very soon.

Users don't want to use a browser. They don't want to Google and have to figure out the right way to phrase their question. When I was teaching my parents how to Google, um, they just typed out a full-on question, and their results were not very good.

So we had to train them, "No, no, no. You have to write it in this special format, and then you'll find, like, with these keywords and stuff, and then you'll find what you're looking for."

Well, now we're going back to what's more natural. Just speak your question, and the AI will be able to figure out what you're trying to accomplish, and not just what you're trying to search for, but what you're actually trying to do, and it'll actually be able to do that thing for you.

That is what MCP enables, and that's why I'm so excited about it. So just to wrap this up, I've got a bunch of resources for you. There's the specification mo- for Model Context Protocol.

It's very good. You should definitely take a look at that. I am very active on Epic AI, and so you can go check out EpicAI.pro and learn all about not just MCP, but AI in general.

I've got a whole bunch of, uh, posts that you can take a look at, uh, all about, um, the, the future of user interaction and how AI is changing the game for us.

It's very exciting, and I hope that you join me there. I also have workshops and cohorts and things that you can go through, uh, join me on in the, the...

At whatever time you're watching this, there will be, uh, something to, to go and learn there. So go check out EpicAI.pro. With that, I just wanna say you're great.

Thank you so much for watching my talk.