Scaling Agents on Kubernetes with acpx and ACP

Talk transcript

1. 0:00

   [upbeat music] Welcome.

2. 0:16

   Um, so the talk is building an ACP on, uh, at OpenClaw. It's, uh, also about other things like how to, uh, put open source agents on open source agent frameworks on Kubernetes and stuff.

3. 0:32

   So I'm, I'm-- I hope, uh, t- I will h- today share i-in a nice way what I've been working on in the last two months. Um,

4. 0:41

   a little bit about me, very brief. My-- Ah. A little bit about me. I've been building harnesses since a few months before ChatGPT came out. I built a JupyterLab extension over o-OG Codex model, like DaVinci Code 2, um, back in the days, that eventually...

5. 0:58

   So I'm currently working for a startup, and that, that initial coding harness turned into this current harness over time, like a ship of Theseus. It got ripped apart and put back together so many times.

6. 1:12

   And, uh, yeah, I'm a founding engineer there. I've been, uh, i-in, in the industry since three and a half, four years. I'm using OpenClaw since Clawbot, uh, s- first dropped in Discord.

7. 1:24

   I went in there. I've been following Peter since he wrote, uh, "Claude Code is my computer." I was like, "This guy's crazy" [chuckles]

8. 1:34

   because, uh, Claude forced on it. Like, I wouldn't give my machine to it, but he, he vent- he ventured forth, and he's paved the way for us. And when I saw, uh, Clawbot in Discord, my mind was blown.

9. 1:46

   The next day, I installed it, uh, at the company, uh, at our, at our, like, uh, cluster. Uh, it was basically talking to people on Discord, and also everybody else's minds were blown.

10. 1:58

    And we are like, we have, uh, we're used to selling the enterprise for a few, uh, years now. So that's why I started by adding an MS Teams integration in case it might be useful at some point, and I became a maintainer a-along the way.

11. 2:12

    I was there when it was, uh, renamed, uh, two times. Um, and today I, I'm-- my focus is on agent interoperability and orchestration. Um, like one of my goals is accelerate enterprise adoption of OpenClaw and adjacent software and also address, you know, OpenClaw is not secure.

12. 2:30

    Well, this will be secure or... Peter talked a lot about that earlier, so I don't have anything else to add on top of it, like it's work in progress.

13. 2:47

    Sorry. I'll use this. So, uh, it started by, uh, I, I started on developer workflows right away, so I created a PR, uh, and then call it Discord-driven development.

14. 3:01

    Well, Telegram-driven development suits better because it's TDD, so... Um, and then, uh, s-

15. 3:09

    right after I set up personally and I realized, you know, Opus is not so reliable for complex-- Uh, it wasn't back in the days. It's now better. It's-- Agents are improving.

16. 3:20

    But Codex was main-- my main harness, and I wanted to use Codex in Discord. But it was basic-- I was telling-- I was playing a telephone game. I was telling Opus to tell Codex to do something.

17. 3:30

    God knows what it's saying, you know, because wording matters when prompting. But it was working somehow, and I would go, uh, [chuckles] and look at the Codex session, and then it paraphrased kind of what I was saying.

18. 3:42

    But eventually, you know, uh, I got some stuff done, but I knew this could be done much easier. Um,

19. 3:51

    and today I'm, uh, running a full ID on Discord. Like, uh, you know, we got parallel workloads. You have like one to f- like five channels. At any point, I'm working with one to five agents.

20. 4:01

    Um, so you see Codex one to five, and then Claw is for testing the OpenClaw's ACP feature. And then, yeah, uh, basically, that's how my ch-- that's how my channels look like.

21. 4:12

    And it's very good for coding on the go because I, I'm a guy who's addicted to side projects and, uh, I, I like to just... You know, uh, AI is making thing a l-- making things a lot easier to do these sort of things.

22. 4:27

    Like in parallel, you have an inspiration, you execute on a weekend, and you just get it ready, you ship it. acpx, which I'm gonna talk about, is o- is like similar.

23. 4:35

    I built it through, uh, Discord. And, uh, what you do is, uh, you bind the channel, uh, Discord channel to Codex through ACP. You can also use Codex App Server protocol.

24. 4:46

    Harold, another maintainer, uh, developed it. And here is me using it before I'm flying to London to create an PDF about, like, convert the docs of ACP into P-PDF.

25. 4:57

    And then I have to say, put it in temp because OpenClaw-- Codex doesn't know about the harness, and it cannot send me in Discord. So, uh, then I go to another channel and tell it to send it to me on that channel.

26. 5:09

    So we are developers and our tools, we don't have time to polish them very well, but we know the,

27. 5:16

    like, advantages and disadvantages. What is ACP? So it's not, uh, most people say, "Is it like MCP?" MCP for-- is for giving tools to the model. ACP is for, uh, like standardizing agent-to-client interaction.

28. 5:32

    It's, uh, shout out to Zed. I actually forgot to put Zed logo. Um, Zed is building a new editor in Rust, more efficient, like l-lower memory usage, not Electron.

29. 5:44

    So, uh, I'm using Zed also since like, uh, last fall. And, uh, you know, uh, if you use Codex on VS Code or Claude Code, they are all building different plugins.

30. 5:54

    It's not exactly-- It's like so much wasted work. If only you could just standardize them under one interface and you just build it once and then you, you ship it.

31. 6:02

    And that is what the i-- the, the idea-- that's the idea they had. It's also more-

32. 6:07

    It's, it, it's much less duplicate w- duplicated work. Um, there are competing standards, um, so agent, like agent to-- uh, Agent Protocol, that's for agent talking to agent. ACP, Agent Client Protocol, is for a- uh, human talking to agent, but agent can use the human one to talk to other agents as well.

33. 6:29

    I-in the long run, as, as these protocols get adopted, we will support all of them. We will w-weigh the, you know, the advantages, and then we will use them somehow.

34. 6:40

    But when I need... So I chose ACP because when I need it the most, I needed adapters for Codex and Claude Code, and only Zed had built them, and Google didn't, uh, have them back, back, back at the time, so that's why I chose it.

35. 6:55

    I, um, that-- So I-- When you're adding any functionality to OpenClaw, you do, uh, you use-- do it through a CLI. Um,

36. 7:06

    so I said, "Okay, let's create a CLI for ACP. Let's an agent, an agent call any other agent over the command line." So that's, that's how it started, uh,

37. 7:15

    and it's slowly turning into a Swiss Army knife for ACP, as I will show in a bit.

38. 7:26

    So in, uh, at OpenClaw, you have fire hoses. Uh, we have sixty KPRs, uh, over sixty KPRs total. Three hundred to five hundred, uh, per day on average, uh, are open.

39. 7:38

    And basically overnight, people woke up and decided they want-- like they like OpenClaw, so we have tens of thousands of stakeholders who want to add features to OpenClaw. And the biggest challenge of the project currently is how do you, uh, absorb all the needs and wants of these people, and how do you balance them?

40. 7:58

    How do you... Like, you can't please everyone. Well, how do you create an elegant system without, uh, creating AI slop, uh, that can cater everyone's needs? And

41. 8:12

    Peter's workflow, uh, gave me an idea, so this is also something I do. You go and you ask your clanker, "What is this?" So PR comes your way, and like

42. 8:22

    most of the time, it's AI-generated description. You're like, "What is this?" Uh, if the human put thought into it, great, but like, yeah, you need to ask what it's doing.

43. 8:32

    You ask what-- "Is this the best possible fix?" Most of the time, it's no. And then you either continue dis-- like, he, he wrote it like you, you do some back and forth with the agent.

44. 8:44

    The reason is people just, uh, run into an issue with their OpenClaw, and it, it can use GitHub. And then they just say, uh, "Please fix," and then they just send some slop your way.

45. 8:57

    You know, this is-- you can't merge it, but you can also fully discard it. You need to take this data point. You-- It's like crucial feedback from the user.

46. 9:04

    Uh, so you need to put it, categorize it, put it in a bin, you know. It, it tells you when some part of the code is broken, and Vincent also talked about that a bit.

47. 9:15

    Uh, and here is, uh, his, his Codex session doing that. On one side, there's the one that says it's, it's a good fix. Uh, and on this other side, it's just saying it's not bad.

48. 9:27

    And yeah, this is so mechanical, and you, you-- once you s- do this over and over, you realize you're repeating something. If only you could program something to automate it, so you're automating the automator.

49. 9:39

    Um, so I created-- I started, uh, to create like workflows, uh, in, in an abstract way. So, uh, item comes, PR, and then you find the intent. You're judging implementation.

50. 9:51

    You're like, uh, looking into con-- if a-- if there's conflicts, uh, i-if reviews gives you like some issues that need to be addressed. So you need to make the CI pass if it's not passing already.

51. 10:04

    Most people just don't care about that. All this mechanical work, by the time the main-- the PR ends up in front of you, should be resolved ideally and can be resolved, and that's what I'm working on.

52. 10:18

    Ah, in the workflow, we have the shameful, uh, RLHF review refactor loops. Uh, I, I'm a believer that, you know, when people say like give the--

53. 10:30

    Just, just run AI-- uh, running an agent in a loop doesn't-- not necessarily have to be, uh, uh, something that will create slop. As long as you're not making it design something but you're making it uncover shallow bugs that can be easily fixed, should be fine.

54. 10:46

    So in the w- abstract workflow that I created, um, which is actually code, like pro-- like turned into a program, you can, uh, tell it to, uh, do superficial refactors, and then you can tell if it's-- it needs a fundamental refactor, relay it to the human.

55. 11:03

    Um, resolving conflicts also doesn't need like no-- It was hard back in the days. I don't think anyone is like resolving conflicts by hand by now. Um, so we are basically creating, uh, standard operating procedures for agents.

56. 11:19

    That's a fancy word for workflow. Um, so that's what I built into acpx. Uh, it's a, an A10-like workflow engine, uh, but it's driving a Codex session you can see on the right.

57. 11:31

    Let me show you in action. So this was one PR.

58. 11:41

    It's loading. Uh, let's speed it up a little bit. So there's some programmatic parts. So this is just replaying, uh, what it's doing.

59. 11:51

    Uh, it's reproducing the bug. It's judging refactor. It's, it's reviewing now, doing a review loop. And then review didn't, uh, bring anything. It, it, it's, it's what I do, uh, but I make it output like JSON structured data, so I can put it in like an A-- an A10-like workflow.

60. 12:08

    And I will talk... You know, this is a general workflow eng- engine. Um- You can use it on other things as well.

61. 12:18

    Um, like you need to apply agents generously on problems. So I see it as like an ointment, uh, that you apply generously on any problem that can be solved with agents.

62. 12:33

    You need to take yourself out of the loop and solve it with agents. Um, and personal agents, I think, uh, are on a d-- Like, there, there's a spectrum enterprise and personal agent.

63. 12:45

    And normally you see on the, the work you use at the computer and then the PC-- The, the, the PC you use at work and the PC you use at, at, at, uh, home used to be relatively similar, but that will not be the case with agents because at, at work you will be using, consuming a lot

64. 13:03

    more inference. Uh, and that means in enterprise there will be a lot more money to be made. So that's why I'm a bit also excited about enterprise agents and OpenClaw's, uh, potential.

65. 13:15

    That's why, uh, I, I believe in, uh, on-demand disposable agents. If you use OpenClaw on Slack or Teams or Discord, it's just one instance. You create an app. The problem is you can't really talk to multiple instances of this.

66. 13:31

    Um, like you create a, a connection, you create a Slack app to connect another agent and another name and another, uh, profile picture, you need to create another app, and you have to create an app manifest.

67. 13:45

    And it's, it's, it's something that shouldn't be managed manually by clicking and the platforms like chat apps don't have this standard yet where you can, uh, mult-- do multi-agent provisioning, like cosmetically create different agents.

68. 14:00

    This is what-- This is on, on the screen you see I asked ChatGPT to gen-generate the idea. You run agents, uh, and then the name can be generated by un-an underlying app, and you can talk to them separately.

69. 14:12

    This is not supported, and this must be supported for this, uh, vision to work. Until it's supported, I'm using it on another UI because, like we are all gonna start one, uh, agent per task and there will be, uh, they will work on these tasks and they will be creating files, editing files, and it will all be

70. 14:34

    synchronized. It will be, it will be, uh, tad bit different than what you're used to with your, uh, personal agent. And to do-- to have that you need to have Ku-- like, uh, a few key components.

71. 14:46

    You need to have Kubernetes. You need to, uh, have an agent harness. OpenClaw could be one of them. Codex, Claude Code could be one of them. Could be ACP.

72. 14:55

    It could be, uh, GitHub. Uh, you give read, write access and you do like state data synchronization. Like you, maybe you do something that's use Rsync, some-something like whatev-whatever algorithm Dropbox is using.

73. 15:10

    And there are some projects that are, uh, taking this on. I've been working on this. Uh, this is outside of OpenClaw. This is my day job and, uh, it's an open source, uh, orchestrator.

74. 15:22

    Uh, it's a Go Operator that's, uh, basically handles the complicated parts. You know, there's a user experience. You wanna create a concierge on Slack, like concierge agent, and you're talking to it, but you get bottleneck because you're a hundred employees on Slack, so you need to, for, for some other task you may, you may need to create

75. 15:43

    a new one and then it creates and gives you like a website link. Um, I'm gonna skip the, like shout-outs to Cognition and Devin, um, because, uh, yeah, they invented the category, but I'm running low on time.

76. 15:58

    So the repo is at TextCortex Spritz. I'm just gonna, uh, demo it for our use case. We use it on error reporting, uh, currently. So if you're on Slack, you know, uh, you can ask it to dispatch an agent to debug it.

77. 16:12

    You know, you're like, "I'm, uh, asking any new bugs after prod release," and it's c-saying something and then you ask it, uh, to create an agent. Uh, if I could put that agent into Slack, I would, but I can't do that, so I have to put it in another UI.

78. 16:29

    And this is an open source project. You can take this, uh, UI's also, uh, like a React app hosted in the cluster that you're deploying this, these Helm charts to.

79. 16:40

    And, uh, yeah, you, you-- it starts the conversation there. It starts working on the, uh, problem. This is like Codex Web or Devin or anything, but it's actually using like a [clears throat] full Kubernetes pod.

80. 16:55

    Wasteful but I think it's the better abstraction because OpenClaw showed the power. You know, when you give a full computer to an agent, it's a lot more powerful. Um, and I, I believe that as well.

81. 17:08

    Um, I, uh, like, I think, uh, OpenHands uses Firecracker so I'm also, uh, not so, uh, well-versed on all the different virtualization frameworks. Uh,

82. 17:21

    so it's-- I'm also learning along the way but I have a working product that's, uh, running on Kubernetes.

83. 17:28

    Uh, and you can use this, uh, product, uh, you can-- If you're interested in deploying internally and, uh, using Codex on the web, uh, and then just spin things off if you're like, I don't know, a backend, uh...

84. 17:43

    If, if you're an open source project first of all, I can help you set this up. If you have a system like inlet of just hundred of, hundreds of issues per day, I can help you process it.

85. 17:53

    Um, this is, um, that does all the wiring around Slack and like keeping those agents al-- uh, like on and the user experience and then the, the interoperability. Like you're not locked in, in any agent.

86. 18:09

    You can switch. It's all abstract the way Bill-- uh, with ACP. Yeah. That was my talk. Uh, thank you for listening. Some social links in case you wanna- [audience clapping]

87. 18:20

    -uh, get in contact with me. I just wanna make c-clear like of my, uh, the OpenClaw, uh, side and, uh, TextCortex side. Um, so the, the last part was, uh, TextCo- about the work I do at TextCortex,

88. 18:35

    uh, just to give a disclaimer. Um, thank you for listening. I guess- [audience clapping]