Building an ACP-Compatible Agent Live — Bennet Fenner, Zed

Talk transcript

1. 0:00

   [upbeat music] I'm Bennet.

2. 0:15

   I work at Zed, and we build like an AI code editor, uh, all written in Rust. And last year was kind of, as you probably all know, the rise of the, uh, AI coding agent terminal user interfaces with every major model provider, like building Claude Code, Codex, Gemini CLI, and so on.

3. 0:34

   And so at Zed, we asked ourselves, like how can we let users bring their agent of choice to our tool and enjoy like a nice interface that is unified across all of them?

4. 0:47

   And so that's why we decided we need some kind of- type of protocol, uh, called Agent Client Protocol, uh, which is similar to like MCP or, uh, LSP. It's a JSON-RPC based protocol, and the idea is basically that, uh, agents and clients can talk to each other through, through a unified interface.

5. 1:06

   And, uh, yeah, the... It's, yeah, o-online. It's open source. Uh, you can contribute if you want. Um, at this point, we have a wide variety of agents already supporting this, either by like an adapter that kind of like translates,

6. 1:24

   um, the agent's native language to the ACP one. Uh, and then we have like, for example, OpenCode and co-- uh, and Cursor, uh, having like ACP mode built into their like CLI agents.

7. 1:36

   And we also have a bunch of, uh, clients at this point, up to forty that, uh, implement this, including OpenClaw for example. Like OpenClaw itself is a client and a server actually, like a client and an agent.

8. 1:49

   Um, and JetBrains and Obsidian and other people are supporting this. Um, great. So, um, I'm gonna do a live coding session. Let's see how well that goes. Um, so- [laughs]

9. 2:03

   Right. Um, basically we have some pre-existing code. So this is Zed. Uh, here I have some TypeScript code. Also bear with me, I'm, I'm a Rust developer. I have basically zero clue about TypeScript.

10. 2:15

    So if you see anything that you don't do in TypeScript, tell me afterwards. [laughs] Um, but yeah, like here's a very minimal coding agent that just doesn't support ACP, but it's kind of the bare minimum you need to like build a coding agent.

11. 2:29

    So all it has is really like two tools. One to read a file and one to edit an existing file. Um, yeah, this is like pretty basic. It provides...

12. 2:39

    The model has to provide a path. Then it has to provide an old text that is like then just replaced with new text, and that's kind of everything.

13. 2:48

    And then we have, uh, in this case, like I'm using Anthropic. Uh, there's a way to prompt the agent, um,

14. 2:55

    uh, with... Yeah, the user can prompt the agent, which is the function that we were gonna call here. And then we enter the agent loop and that is kind of like the way all agents basically work, is the model APIs are stateless, so you just attach some like conversation up to this point.

15. 3:12

    You call the endpoint. In this case it's like the Anthropic API. And then we, um, yeah, we get like a message from the model, and either it can be like a end turn.

16. 3:22

    That means like the model decided to, uh, yeah, just output some text and do nothing or it can call a tool. And in this case it would be like a read or a write, uh, tool call.

17. 3:33

    And then in the case of a tool call, we like run the, for example, the edit file tool call locally, um, collect the result, and then send it up to the model.

18. 3:42

    Again, that's why there's this loop. Um, and then we call the API again with the conversation up to that point, and that's kind of everything. And then we have like these-- this like handle tool call function here, which, yeah, handles like read and write, uh, tool calls, which does what you expect.

19. 4:00

    We get the path, we read it from the file system, and we return some, some, uh, result. Right. So now the question is: How do we make this thing ACP compatible?

20. 4:11

    And hopefully we can do it in ten minutes. So let's see. Um, so yeah, I have some boilerplate here. Uh, in this case, I'm using the TypeScript SDK, as I said.

21. 4:21

    Um, and the way this works, you implement, uh, the agent interface provided by this, uh, library. And then you have to, at minimum, implement these three, uh, four functions.

22. 4:33

    So the first one that we're looking at is kind of this initialize here. Um, all we really have to do here is like respond with the protocol version we support, which in this case is just the latest.

23. 4:44

    There's also like some capabilities like client and, uh, the agent can itself advertise capabilities of stuff it supports. But we're building a bare minimum coding agent, so we don't support anything outside that's necessary, I guess.

24. 4:57

    And then for us, like authentication is irrelevant since it's just using my API key from an env var. And then there's this concept of sessions in ACP. So basically every time you start a thread in Zed or in a different editor or client, uh, you, you call a new session.

25. 5:15

    And in a session you can prompt and then, yeah, from there on you can like get, get your output. So all we really do here is like we generate a random ID.

26. 5:27

    Um, then we, um, instantiate, uh, this coding agent with the current working directory, which we get from the client. Um, we store it in our internal map and then just return the ID to the client so that the client knows, um, yeah, knows the ID and that is kind of then used inside prompt.

27. 5:50

    Um, because what prompt does in this case, um, it like prompt-- the prompt request, if we go to the definition here, all this is kind of doing is, uh, it provides a prompt, which is an array of content blocks that can be like text, images, whatever.

28. 6:06

    The session ID for reference- And so all we kind of have to do here is, yeah, we look it up in our internal state. We get the relevant agent, uh, for that, uh, current session.

29. 6:18

    And then I have a hel- helper function here, which just in this case ignores everything that's not a text because we don't support images and stuff like this. Um, and then we call this like prompt function I showed earlier, which then, uh, runs the tool calling loop.

30. 6:34

    And so then just like as a nice-to-have, uh, feature, we also support cancellation. Uh, that's also pretty, pretty simple. So these are kind of like the four minimal things we have to implement, and I have this hooked up in Zed just by there's no special code.

31. 6:51

    All I have is kind of like I tell Zed there's some a- ACP-compatible agent, and you can run it with Node and then my path to that agent JS file.

32. 7:02

    Um, so when I... Uh, let's do that in here as well. So when I run, I'm gonna build, um,

33. 7:11

    I'm gonna build, right? And then I'm going to restart, uh, yeah, launch a new ACP demo agent. And you can see if I ask it for something, you can see a wait indicator,

34. 7:26

    and now it's done. So nothing. [laughs] Which is what we expect, right? Because, like what we can see here, if I go over... Oops. Oh.

35. 7:37

    If I go over here, we have some kind of like ACP debug view in Zed, so to make it easier for us to develop. And you can see, like Zed sends a session new request.

36. 7:47

    We respond with a session ID, and as soon as I type in something,

37. 7:53

    we get prompt, and now we get a stop reason, right? So these red errors come from the, uh, from the adapter that we are building from the agent, but it's not outputting tokens, right?

38. 8:06

    Like behind the scenes, it's obviously like Anthropic is giving us tokens, but how do we make them show up in Zed? So that's the next thing that we, uh, yeah, wanna focus on.

39. 8:16

    So the first thing that we kind of need to do, like our coding agent itself, uh, needs to have a way to, um, send something over the connection. So what we're gonna do here is like the coding agent is gonna take the ACP connection and then also the session ID, and then inside here we have to provide

40. 8:37

    this, uh, connection and the ID. And then if we go... Okay, I'm just gonna close this. Right. If we go back here, now inside of a prompt, Anthropic, like this is the Anthropic SDK.

41. 8:55

    So in this case, like in... Oops. You can just use, um, this like stream on, uh, yeah. That's like a Anthropic, uh, SDK. Like you can react to a text event.

42. 9:07

    So every time we basically get a chunk, uh, we send this kind of what ACP calls a session update. As I said, a session is like a single thread or a conversation, and then you can send updates, kind of like notifications to the client that are not like a usual request response, right?

43. 9:22

    It can happen at any time, and the client reacts to it. And so here we are associating with the session update with a session ID. And then this like, uh, there's a type of session updates.

44. 9:31

    There are multiple. We will see more in a second. But this agent message chunk is just like, okay, hey, here's some new output from the model.

45. 9:40

    And so if we build again, uh, we go back to Zed. I'm gonna have to restart the agent. I go here,

46. 9:51

    and I say hello. Hello, hello. Well, here's Opus talking, right? You can see like we're streaming in these individual, uh,

47. 10:07

    chunks, right? Right, that we get from Anthropic. So, so far, so good. Let's hope the demo gods, uh, stay with us. Um, right. So that's one piece. The next thing though is like I wanna, like it's a coding agent, right?

48. 10:21

    It's supposed to edit code. So I can actually tell it

49. 10:26

    already because we have tool support, right? I can tell it to read, for example, this like helpers.ts file,

50. 10:34

    and it's gonna give me, if the WiFi isn't too slow, it's giving me a description of what this file actually contains. Like if you look here,

51. 10:45

    that's basically the description of the file. But again, you don't see it in the UI, right?

52. 10:51

    So now we need to emit more session updates similar to like what we did for text. So if we actually go towards the read file,

53. 11:01

    kind of what we need to do here is like the way it works usually in ACP, like you emit an initial tool call update, uh, which I'm gonna paste in here.

54. 11:13

    So yeah, again, we're emitting a session update. In this ta- in this case, it's a type of tool call.

55. 11:21

    And then there are multiple properties we can specify, for example, like the title, kind of some kind of metadata, um, that Zed uses, for example, for like, yeah, using some icons and stuff like this.

56. 11:33

    And then we indicate that the status is in progress, and you can also like associate tool calls with like actual, uh,

57. 11:40

    locations. And so now we signaling over ACP that something is in progress, right? This tool call. But then we also want to, at the end, like once the tool call is finished, like at the bottom here, we kind of wanna send another update.

58. 11:56

    And in this case, it's not tool call itself, but it's a tool call update. So you have to emit on the agent. You have to emit like a tool call, like session update of type tool call.

59. 12:07

    And then once, once the client knows about it, uh, you can emit updates for that. Uh, and this is the way that we set the status, and we can also return the content.

60. 12:18

    Um, and then one additional thing that I'm gonna do here is the, um, instead of like you can see here, we're calling, I guess it's a bit hard to see, but like here we're calling FS read file, right?

61. 12:31

    So we're just using the native file system APIs. But MC, uh, ACP actually proxies the file system too. Like the client can provide a, uh, uh, a file system capability and if it does, um, we can like proxy those, uh, file system tool calls over ACP and it kind of makes sense, for example, for an editor, right?

62. 12:50

    You wanna, um, if I have unsaved changes in my buffer, they're not actually on the file system, but the agent should still see them. So that's why we call this like, uh, read text file, um, over ACP.

63. 13:04

    And so now if you go back here

64. 13:09

    and hit NPM run build and then go back here,

65. 13:15

    restart and say read, uh, this file. Oops, not Rust.

66. 13:29

    Then we should be seeing, um... Yeah, so now we see read tu- uh, tool calls, right? Okay, something is going wrong because everything is duplicated. [laughs]

67. 13:42

    Um, yeah, but like here you can see the output, um, of the actual file and because I think I'm low on time, um, we're, we're gonna basically gonna do the same for edit file.

68. 13:57

    Um, and then we're gonna compile again. And if I again restart

69. 14:08

    and say add a comment at the top of

70. 14:14

    agent, src/agent.typescript, then you're gonna see, um, uh, "Hello World" is good.

71. 14:31

    Hopefully Cla- uh, Opus is gonna decide to add Hello World at the top of the file and we actually get a diff here because the, over ACP we sent like a diff of the file.

72. 14:40

    Okay. Is it duplicating the first two tokens? Yeah, I think there's something going w- wrong with the connection. I'm, yeah, uh, uh, yeah, no time to debug right now. [laughs] [laughs]

73. 14:51

    Sorry. Uh, am I out of time or do I still have...? Um,

74. 14:57

    let's have a look.

75. 14:58

    There's more time.

76. 14:59

    Crack on. [laughs] Okay. May- maybe like we can see. So now that-- Okay, just another minute. Um, now that the coding agent supports reading and writing files, we can actually- You've got about five minutes.

77. 15:11

    Oh, great. Uh, okay, then I don't need to rush. Um, now we can kind of bootstrap the agent itself, right? Like the coding agent supports reading and writing files.

78. 15:22

    I can just ask it, I prepared a prompt here, to add a terminal tool to itself, right? Let's see how this works out with like the duplication we're seeing.

79. 15:32

    Uh, but basically, yeah, I'm telling it some of, something about the ACP, um, APIs and there's some, there's some, uh, APIs in ACP which also, that's another like capability of the client, where the client can, um,

80. 15:46

    advertise that it, uh, supports creating terminal and managing terminals for the agent. Like the agent can also do it itself of course. Um, but it's a nice way, uh, for us to add some more interactivity and so the agent here decided to add a new tool description.

81. 16:01

    Now it... Yeah. I'm vibe coding basically this terminal tool. [laughs] Um, and let's see if

82. 16:10

    it actually builds. It does. Well, that's good. Um, and then again, I'm gonna restart the agent. I'm gonna run our demo agent and I'm gonna ask it to run sleep five LS.

83. 16:26

    Let's see. There you go. You can see a terminal running.

84. 16:32

    Sleeping five seconds. There is the output. And there's... Yeah, and that's it basically. [clapping]

85. 16:42

    Thank you. Yeah. So that's how you kind of build an ACP compatible coding agent in 15 minutes or so. [laughs] Um, yeah, if you wanna, uh, check it out, just go to agentclientprotocol.com.

86. 16:58

    Um, in case anyone wants the demo code, but please don't use it in production, it's all agent generated. Um, it's not uploaded yet, but it, it's an empty repository.

87. 17:09

    I'm gonna upload it in a second. Uh, yeah. Thank you for listening. Happy to answer questions.

88. 17:17

    That's it. Yeah. [laughs] [clapping] Yeah. How does it handle the diffs? Is it only on the client side or? Uh, the diffs? Uh, yeah, we have like in, in ACP there are multiple content types and one content type is diff and so the agent sends old text, new text and then Zed does the diffing for you.

89. 17:38

    Yeah. Okay, great. Yeah. Awesome. Yeah. Cool. Do I need to go or can I? One more. Okay, one more question. Yes.

90. 17:47

    Um, so the connection, is it all localhost between the two? Yeah, the connection works over standard I/O. Uh, there are some folks, uh, I think from the JetBrains people are wor- working on like remote, uh, transport.

91. 17:59

    Uh, which we're gonna have soon I think so yeah. But right now it works over standard I/O. Yeah. [outro jingle]