MCP Tasks (async)/ Why the heck aren't any agents supporting MCP tasks/async?

Talk transcript

1. 0:00

   [on-hold jingle] I know it's one minute ahead, but these twenty-minute sessions are really short, so I'm gonna get started.

2. 0:18

   So title of my talk y'all have seen, 'cause you're all here, which is, "Why the heck aren't any agents supporting MCP Tasks?" If you don't know what tasks are, don't worry, you will know in just a moment.

3. 0:30

   But the first answer to that question is, well, 'cause they're smart. The people who are building those clients are smart. What I mean by that is that the MCP Tasks specification that came out in November was marked as experimental.

4. 0:44

   And so, well, you might shrug and say, "Well, gosh, those clients and servers, they're all supporting a whole bunch of experimental things. Why not MCP Tasks?" Well, again, you'll see the answer to that as we move forward.

5. 0:58

   Um, the next answer to that question is, well, they're pretty involved. Um, there's a lot of complexity in here, and that's what I wanna do over the next twenty minutes, is teach you some of that complexity.

6. 1:11

   Quick intro, my name is Cornelia Davis. I'm a technologist at, uh, Temporal. Uh, we're distributed systems stuff. I have a long history in distributed systems. Did a whole bunch of stuff in the microservices era, um, including Cloud Foundry, Kubernetes, GitOps, Weaveworks, all of that stuff, and I even wrote a book about that.

7. 1:31

   That's who I am. Today's agenda in the next nineteen minutes is that rather than just talking about things in the abstract, I'm gonna ground us in a very concrete example.

8. 1:42

   So I'm gonna give you the lay of the land of that concrete example. Then I'm gonna give you an overview of MCP Tasks. Quick question, who here is-- wants to do things with tasks?

9. 1:53

   Async MCP tools. Okay. So I'm gonna give you a little bit of an overview. Um, then we're gonna talk about MCP Tasks v1. That's the spec that came out in November, and spoiler alert, there's a new one coming out in July.

10. 2:07

    So that comment that I made about them being smart about not implementing it yet, well, there's some pretty radical changes. So I'm gonna show you, um, what's happening with v2, and I actually have some live demos to show all this working, and then we'll have some takeaways at the end.

11. 2:23

    So the use case that we're gonna talk about here is a simple purchase order use case. So the use case is you're gonna get in a purchase order, and then it's gonna go through a number of steps.

12. 2:35

    It's gonna record the fact that the goods were received, and then it's gonna do, in parallel, it's gonna do some back office stuff, updating inventory, sending out notifications. And then in parallel to that, it's going to pay some invoices.

13. 2:51

    Now, the invoicing is gonna happen via an MCP tool. Now, that MCP tool is-- has itself a number of steps. So it's gonna validate against an ERP, then it's gonna have a little human-in-the-loop to request approval, maybe.

14. 3:08

    Um, then it's gonna reconcile against the ERP again, do a little bit more human-in-the-loop, and so on. So you can see that on the right-hand side, that MCP server, that's gonna be-- it's a tool that's gonna be doing the invoice processing for us, it is long-running.

15. 3:25

    It's not going to work in a request-response style, and that's what MCP Tasks are all about. And what we're gonna do, and I'm-- today's talk is not about Temporal, but really what I did here was just showed you a couple of snippets of the code.

16. 3:41

    And yes, I will be sharing all the code for what I'm showing today. Couple of snippets here, and the real point that I want you to look at is that reject or approve.

17. 3:50

    That is showing you that there is a mechanism for signaling into a long-running process. And that's really the point, and that's what we need, is that this is all about asynchronous.

18. 4:02

    So you understand what MCP Tasks are now? MCP Tasks are allowing you to have an MCP tool that you can invoke, and then it is long-running in the background, and then eventually you can get back some response.

19. 4:17

    So let's talk about that MCP Tasks overview. This is a very simple sequence diagram. It's exactly what you all would expect when I tell you that MCP Tasks are long-running tasks.

20. 4:31

    You're going to invoke a tool, and instead of getting back a response, you're gonna get a handle, and you can interact with that handle, right? Obvious, right? This is-- this isn't rocket science.

21. 4:42

    Looks easy enough, right? Well, it turns out that if you actually want this to work over long horizons, it gets a little bit more complicated than that. So what are some of those complications?

22. 4:56

    Well, you can have all sorts of... The longer something runs, the more likely there's gonna be some kind of infrastructure blip that's gonna cause a problem in that long-running task.

23. 5:07

    So you could have network blips. You could have network challenges. You could have humans that you're waiting for their in-the-loop part, and they go away on vacation, like I'm about to, yay, um, day after tomorrow.

24. 5:22

    Um, or processes can crash, so your agent could go down, the agent that's processing the purchase order can go down, or your MCP server can go down as well.

25. 5:33

    So all of those problems you need to deal with, and those are the things that makes it a little bit more difficult. Now, in addition to what I've told you about MCP Tasks so far, that you're gonna get back a handle that you can interact with, by the specification, those MCP Tasks can't disappear.

26. 5:55

    This is verbiage from the spec itself that says, "Once you've lo-launched a task, it has to be durable." What that means is all of these things that I just showed you on the previous screen- Clients, humans going away f- on vacation, servers going down, clients going down, connections disconnecting.

27. 6:16

    The task needs to survive that, and you need to be able to interact with that task when the infrastructure comes back. And I'm gonna show you how all of that is done.

28. 6:28

    Now, on-- there's elements, there's server-side elements that talk about how you make the server side durable, and I did a talk at the MCP Dev Summit in March, and this is the QR code that will take you to that YouTube video, and that's where I go into a lot of detail about the server side and what you

29. 6:47

    need to do with the server side. Today, as you saw, is an extension of that work where I'm talking about the client side.

30. 6:57

    So without further ado, let me go into a demo. I-- for those of you who know me, I'm always doing demos. So what we have here is we have a dashboard.

31. 7:06

    Um, I am not doing this through a chat interface because it-- frankly, it's more efficient for me to click a couple of buttons here to show you this rather than trying to type things in.

32. 7:16

    So I have a user interface here that's showing you the number of purchase orders that have been submitted. I'm gonna submit a simple purchase order, so that's just a button that is kicking things off, and in a moment, if the dem-demo gods are with me, it says submitted, we should see the purchase order pop up here, and

33. 7:33

    it should show some... Ah, here's why it's not working, because I haven't started my servers. So remember I said it has to work even when the servers aren't running?

34. 7:44

    I forgot to show you here that what I'm doing in this, these two windows is in the upper window, I'm starting the back end, this is the MCP server, and in the, um, lower window, I am starting the MCP client, and you'll see what that client is in a moment.

35. 8:01

    You can see in the splash screen there that I am using FastMCP on the client side. So let's go back here, and notice that even though I submitted that, even though my servers weren't running, that submission did go through.

36. 8:15

    So it's captured that. So what you can see here, and you didn't see it cycle through, but on the far right-hand side, the invoice task is sh-- it initially showed you that it was wor- submitted, then it showed you that it was working, and now it's asking for input required.

37. 8:33

    I can come over here. Let me show you what's going on at the back end and at the front end. What I have here are some dashboards that are showing those running processes.

38. 8:43

    On the right-hand side, you have the back end. That's where the invoice processing is, and you can see the name here. Let me, uh, increase the font size there a little bit.

39. 8:53

    So you can see that this is running the invoice, and on the left-hand side, you can see that it's running the PO. I'll explain that task tracker thing in just a moment.

40. 9:03

    So if we go into the invoice, we can see that it has the process that we talked about earlier. It validated against the ERP, and now it's waiting for human input.

41. 9:12

    It's waiting for that approval. Over on the PO side, we can also see the process that I showed you earlier, which is to say... Let's go back here. It is...

42. 9:24

    So, ah, yes, so it-- did that record-- it recorded that the goods were received. Then in parallel, it's invoking the invoice processor, MCP task, and notice that there's this line item here that says, "Task tracker workflow."

43. 9:40

    Yes, indeed, that is my MCP client implementation. Remember I said nobody's implemented this on the client side? Well, I created my own implementation here. But in parallel with doing the invoice processing, we also had this back office stuff that was happening.

44. 9:56

    So if I come back over here, and I click on Input Required, I can approve this.

45. 10:02

    And I'll hit Submit, and we come over here, and you'll see in just a moment that the signal is gonna come into the back end. Ah, I need to refresh.

46. 10:12

    Oh, there it goes. So the approval came into the back end, and now the back end is going ahead with its additional process, paying the invoice, and you'll see a number of line items there.

47. 10:22

    There's some, um, some, uh, re- uh, retries that are-- have been programmed in here, but you can see here that it took a few tries before the r- the ERP went through.

48. 10:31

    We paid the line item, and now you can see that the task completed. So everything's completed. If I go back to the dashboard that you saw at the top,

49. 10:45

    you can see that all of those processes completed, okay? So that's the basic stuff, and I can run that again, but in the-- it-- I already gave you, inadvertently gave you the example of the infrastructure was down.

50. 11:00

    I could have killed that server halfway through, and it would have continued exactly as you saw here, okay? So you saw it at the very beginning. All right, let's go back to slides.

51. 11:12

    So that's the first demo. So let's talk about, um, Tasks v1. So in Tasks v1, there were a number of tool semantics, and again, I go over these tool semantics in a lot more detail in that MCP Dev Summit talk.

52. 11:28

    But there's one really interesting thing that I want to draw your attention to, which is that tasks come with-- it-- one of the things that the specification defines is a life cycle for tasks, and that's what you see here on the screen.

53. 11:43

    It has working. It can go into an input required. From input required, it can go back to working, and then eventually it'll complete or be canceled or fail. So that's one of the things that's super interesting about the task specification is that it's about the life cycle of the task.

54. 12:03

    There's a whole bunch of other semantics there as well around obtaining inputs and delivering results, and I'm gonna go through this fairly quickly because I already mentioned, uh, some of this is going away.

55. 12:15

    So this is what the tool semantics were before the task semantics. Notice that tools/call is exactly the same. There's some metadata that you pass in when you want it to be async, and then there's task get cancel list, as well as task result.

56. 12:35

    And so the top four are request response in style. The bottom one keeps a connection open. It keeps a connection alive. And the sequence diagram that you can see here is kind of the basic stuff.

57. 12:47

    Now, there's two hiccups with this, um, uh, two major challenges with this particular version of the protocol. The first one is right here, task list. This is a stateful protocol.

58. 13:03

    So what that means is that the-- Remember I said that the server was responsible for durability? Well, this particular endpoint allows me to go to the server and say, "Hey, what tasks do you have?"

59. 13:17

    So if I have had-- if the client has gone away, if the user took too long to respond, if my network dropped out and I had to reconnect, I can use this task list to go back to the server and say, "What have you got?"

60. 13:32

    And then you can continue on with that. That works fine if you have one task or two tasks, or maybe it be- it even works if you have ten tasks.

61. 13:46

    But what happens if you've got a whole slew of agents out there, and you've got a million tasks at the back end? Spoiler alert, there is no filter on that endpoint.

62. 13:56

    So you would have to go through a million tasks to find the one that you're looking for that you wanna interact with. This is going away, you'll see in just a moment, but that's one of the challenges.

63. 14:06

    Just because you can, doesn't mean you should. The other one is the task result because that is where we were tunneling the input required. So in the case of task result, this sequence diagram is really simple.

64. 14:20

    It doesn't have the, the interactivity. What we have, as soon as you do task res-- uh, as soon as you have input required, is the top and bottom are just fine, but this middle section has this weird protocol where you open a long-running connection, and then the server elicits a response from the client.

65. 14:40

    That gets super tricky. And I'm running short on time, so I'm not actually gonna show you this demo. Happy to show it to you. I'll be around all day tomorrow too, so I can-- happy to show it to you.

66. 14:52

    But I wanna show you instead, here's basically the architecture of what you need to build on the server side. This is-- Notice that this is using FastMCP. So FastMCP already has support for server side and some client side stuff as well.

67. 15:08

    But the interesting thing is notice that little box in the, on the left-hand side that, on the, on the lower part where it says MCP client protocol handler. That protocol handler with the ugliness that I just showed you around results actually looks like this.

68. 15:26

    And I can show this to you running, and it has all sorts of complexity in it. I gotta have the long-running connection. Well, what happens if my connection dies in the middle of that?

69. 15:35

    How do I pick up where I left off when I come back? You'll see that a big part of what the task specification does is it talks about durability.

70. 15:44

    So back to the question of why the heck aren't there any clients that are supporting this protocol? Uh, yeah, that's why. Super involved. It's still involved with v2, but it gets better.

71. 15:58

    So let me tell you about that. So in May, Angie Jones, who's responsible for developer experience at the Agentic AI Foundation, which is where MCP now lives, posted this blog.

72. 16:13

    And, um, one of the things that made me jump up and celebrate a little bit is that the, the protocol is going stateless. So as somebody who's been working in the microservices world for a long time, stateful protocols are the absolute worst thing in large-scale distributed systems.

73. 16:32

    So the protocol is going stateless. It's also doing a number of other things. So the first bullet is the stateless core. The second bullet is interesting 'cause it's-- they also have structured MCP so that there's a core and there's extensions.

74. 16:46

    If some of you were in the room for the previous two talks, they talked about MCP UI. Two talks ago, they mentioned extension. Well, that's what's happening here in the v2, um, uh, MCP protocol, is that they have extensions, and tasks have become an ins- extension.

75. 17:04

    So let me tell you a little bit about how tasks changed from v1 to v2, and I do wanna give you one more demo. So on the left-hand side, you can see what the protocol was before.

76. 17:16

    These are the RPC requests that you were doing over the wire. On the right-hand side, you can see a couple of things. Task list has gone away. Good. Wasn't particularly useful anyway, especially at large scale.

77. 17:33

    And instead of having this input required going over a long-running session, you now have an endpoint that allows you, from the client side, to say, "Here's an update." So if you remember a while ago, I showed you that screenshot that said Temporal has this notion of a signal.

78. 17:52

    That's effectively what this is. It's a way of signaling into this long-running task. The task result stays, but it changes because it no longer has this long session-based protocol.

79. 18:04

    But I put the picture on the right-hand side here to emphasize the fact that the lifecycle management of these tasks is unchanged. That's actually sound. Now- I go into this a lot into more detail in the talk that I keep referring to.

80. 18:23

    Um, on the server side, in invoice processing, I have my own state machine that the invoice is going through. And so part of what you're doing when you implement these server side, these tasks, is you're mapping from the lifecycle states of the task over to the domain state machine that's running the, the application, the, the, the MCP

81. 18:45

    server in the back end or the tool.

82. 18:48

    So list again goes away. Now, remember I said that the MCP Tasks specification has durability all over it? With this change, given that lists are gone, you now ha- are required on the client side, well, kind of required, there's a little anec- a little, uh, parenthetical remark here.

83. 19:10

    The, the spec right now says that clients should persist task IDs, but it also points out that if you don't persist task IDs, there is no way to get it back.

84. 19:22

    So I'm not quite sure why this doesn't have a, an all caps must. The other thing that I wanna point out is, and I already mentioned it, is that you're gonna have potentially a lot of agents that are processing POs, or a lot of agents that are doing a lot of things.

85. 19:40

    And so having multiple things running, I think is really, um, crucial as well. So with that,

86. 19:49

    um, I'm gonna go to the, the second demo, and I'm gonna go back to my purchase order now here. So what I'm gonna do now is I'm going to submit a number of things, and I'm actually still demoing here because I have 13 seconds left.

87. 20:07

    I'm not gonna switch over to my v2. You'll see that from the high level, it actually looks exactly the same. I am gonna show you what the client proto- client-server protocol looks like in the v1 case.

88. 20:20

    It's really quite ugly. But you'll notice here that we have, um, I've submitted a bunch of different ones. I can tell you with the v1 protocol, the reference implementation, if you had input required on multiple, even though you can see that there's many of them in flight, on the client side, they were FIFO, so you could only

89. 20:41

    respond to the first one, and part of the protocol that I implemented was to get around that gap. So let's come over here. We can refresh both of these, and you can see that there's gonna be a bunch of POs in flight.

90. 20:55

    And now I wanna show you the task tracker. So if we go into the task tracker, that's the MCP client, and now let me just expand this so we can see it in a little bit more detail.

91. 21:07

    What you can see here is that remember that that protocol, I showed you that big long sequence diagram, there's a lot of steps involved in that. And what I've done here is I've implemented it as a workflow, and you can see here that there's some elicitation handling that's going from the server side back to the client.

92. 21:27

    So I won't go into any more details 'cause I'm literally out of time now, but I wanna share two more things, and that is, um... So going from v1, remember this ugly picture, to v2 in the client-server protocol, much, much cleaner, much easier to implement.

93. 21:47

    So speaking of implementing, here's a summary of all the things that you need to do if you wanna implement tasks. Still relatively involved. Here's a picture. I'm gonna make these slides available in the Git repo that I'm about to show you.

94. 22:01

    And here's the Git repo that I'm about to show you. And while you're getting that screenshot, I'm gonna tell you about two pieces of work that I'm continuing with.

95. 22:10

    Number one, even though this is better, it still doesn't scale to the millions. Why? Because if I've got a million tasks running, I've got a million clients that are doing gets against each and every one of those tasks.

96. 22:27

    That does not scale. There is a part of the MCP c- task specification that is a notifications protocol, which I haven't gotten far enough yet, but it's showing promise, which is gonna allow you to, instead of having a million clients, uh, uh, to- um, polling their tasks, it's gonna have a single endpoint where they can say, "Has

97. 22:51

    something changed? And if it has, tell me which one, and now I'll go poll that task." So it's definitely from a scale perspective. The other thing that we're doing is in the very near future, in the next, uh, month or s- or two, we're gonna have a, an implementation of all of this where it's gonna be much

98. 23:12

    simpler for you. My goal is to actually, uh, implement it in, in FastMCP so that you can use the same protocol, uh, the same framework that you're using probably for your MCP servers today.

99. 23:24

    So without further ado, that is it. Thank you to the next speaker for letting me go a few minutes long, and I'll be around. I'll step out. If you have any questions, find me in the hallway. [audience applauding] [upbeat music]