AgentCraft: Putting the Orc in Agent Orchestration

Talk transcript

1. 0:00

   [upbeat music] So, uh, good morning, London.

2. 0:17

   Uh, my name is Ido Salomon. Uh, I'm the creator of AgentCraft. I am also the creator of MCP-UI, and creator and co-maintainer of MCP Apps. So I'm building some of the stuff that Davin has been talking about.

3. 0:30

   Um, as you've all heard in the past day, agents are amazing. Uh, but if one agent is so amazing, why don't we scale up to 10 or 20 or 100 different agents and be 100 times more amazing?

4. 0:43

   Uh, it is pretty simple. We just spin up a bunch of agents, we put them in this, like, nice, uh, uh, screen, and it looks really glorious. But it won't actually work, and the reason is that spinning in the map isn't the problem.

5. 0:58

   It's us. We are the bottleneck in orchestrating all of these agents. Now, if you think about it, the role of the engineer to actually go and manage dozens of reckless, uh, employees is not typically what we do in most companies.

6. 1:13

   Um, so it... We need to somehow find these new, potentially new skills to manage all of these agents.

7. 1:23

   Luckily, they're not really brand new. It's not something that we've never done before. It's just something that's been hiding in unexpected places. I mean, if you're a gamer or used to play games at any point, managing dozens of units probably sounds a little bit familiar.

8. 1:40

   Which is why I built AgentCraft, which is an orchestrator that aims to raise the ceiling of human agent collaboration by taking learnings from gaming and transferring them into productivity.

9. 1:54

   So let's see a quick walkthrough of that, and let's understand the journey to raise that ceiling.

10. 2:01

    So this is AgentCraft. There's a lot to unpack, uh, so we'll just start with the basics and go from there. Uh, this is an agent. Not a meta- metaphorical one.

11. 2:14

    This is actually a physical manifestation of a coding agent, like a live session. Um, it can be, uh, you know, Cursor, it can be Claude Code, Codex, OpenClau, whatever.

12. 2:25

    Uh, it's something that we can detect on the device and visualize it, but it's also something that we can spawn directly from here. At your command. [laughs]

13. 2:35

    So now [laughs] we have this agent, uh, and we can prompt it. We can use it like just any other agent that we have from our CLI or whatever. Uh, and what can we tell it to do?

14. 2:46

    It has all of these quirks, and we have voice, and we have text, and we have images, and so on, and we can just tell it to do stuff.

15. 2:52

    So for example, we can tell it to, um, develop some feature for us.

16. 2:58

    Prompt prompt. [laughs] And now the agent is working.

17. 3:03

    Whoo! [clapping]

18. 3:03

    So it's doing its work. [clapping] So it's doing work. Uh, and as you can see, uh, if you look at the, the UI, there's like a bunch of other stuff.

19. 3:17

    We have, uh, these buildings, and each building represents some functionality. So for example, you know, one of these buildings manages the skills and plugins and so on. Um, there's also, you know, like integrated, uh, terminal and Git just to, like, get that end-to-end workflow.

20. 3:34

    Uh, the second part of raising the ceiling, now that we have the basics, is visibility. We need to be able to quickly understand what each agent is doing. Uh, so we have this nice side panel here that really shows us, like, high level, uh, mission status, summary and so on, what they're actually doing.

21. 3:51

    But the cool thing about AgentCraft is that we don't just see a list of what they can do. We can actually see them working. So if we look at the map, you would notice that it's actually a projection of my file system.

22. 4:04

    Each part of my file system is actually on the map. So I have these directories here, and each one of these directories has files. These files are represented as rooms, as you can see here.

23. 4:15

    So I can actually track and see visually what the agent is working on, which file. I can see the entire change list of what happened there. And because we're orchestrating it, I also know which agent did what and when.

24. 4:28

    So we can have full lineage of what's going on. And we can take this one step further. If I know all of this stuff, why not just create a heat map?

25. 4:38

    I can actually try and see, visualize, uh, collisions, and I can even prevent them proactively.

26. 4:44

    Now, the, the cool thing here is that once we have this visibility, we're not exactly done yet because we still need to be able to react to the changes that are happening.

27. 4:54

    So we can lean into another cool mechanism from RTS games. We can simply use muscle memory to quickly cycle between the agents that need our help. They need, uh, us to approve the plan.

28. 5:05

    They need us to, uh, answer some questions, so on.

29. 5:09

    So now we have visibility, and we can react quickly. So we're done. We solved orchestration. Um, but not quite, uh, because that's really only the first step. Uh, I was able to use more agents in parallel, but only for a short amount of time.

30. 5:25

    Uh, there are a few reasons for that. The first one is that there's only a limit to how many ideas I can have in my head at any given time without being tired.

31. 5:34

    Uh, so what I did is basically tell the agent to do it. I tell them, "Okay, find missions for me to do." So I have quests now, and I can click a button, and they just do whatever.

32. 5:42

    They can refactor, test, uh, all the stuff that I don't wanna do. Uh, and the second one is that all of this babysitting takes a lot of time. Like I need...

33. 5:52

    I see what's going on. I can react to it very quickly by thinking to cycle through it. Uh, so what I did there is kind of say, "How do I take myself out of the equation as much as possible?"

34. 6:03

    So if agents are so amazing- Uh, why not just let them do it? Uh, I can just, like, give them some idea. I have this campaign feature. Uh, broadly say what I want to happen, and I will just spin up a container.

35. 6:18

    I would let the agents run there. They can decompose the task, they can plan it, they can present a plan to me. I don't care what they're doing because it's containered, so do whatever.

36. 6:28

    And the main thing here is that once it's decomposed, I'm not the one doing the babysitting. Now I have the campaign orchestrator, and that's his problem. Uh, so we're actually moving more of the effort only to the planning phase or the review phase.

37. 6:42

    Uh, and once we have that, we reach a point where we can just say, "Why is it my ideas? Why can't I tell it to have, like, run in a Chrome job, go to Twitter every day, scan cool ideas, and just implement them, and I'll just decide what I want?"

38. 6:57

    Which is actually how I implemented channels pretty quickly. Um, so we have that, and now we just have a lot of different PRs to review. So there's this nice capability of just review bundles, uh, and now I can see exactly what changes happened in each one, like why did they do stuff?

39. 7:14

    What are the tasks? And I also have visual evidence. So now I'm able to just look at screenshots. I can look at videos and really see what's going on without investing too much time in doing it.

40. 7:29

    And once we have that, we can actually shift more of the work from the planning to the review. How much time do I need to spend on the plan if I can just do it ten times and I'll just pick the one that is most fitting for me?

41. 7:43

    Uh, and the next part is we're still not done. I mean, if you think about it, this is only the first step because agents aren't that smart yet. Uh, so we need to offload it to someone else, uh, humans.

42. 7:56

    Uh, now, what I can do, and this is my favorite feature, is that we can actually create these workspaces. So I can collaborate with the product designer from my team, and they can do whatever they want, and you can-- I can just, uh, continue from where they left off.

43. 8:10

    So for example, let's say this is an agent actually from the product designer on their computer. So they can see my agents. I can see their agents. I can understand what they're doing, and we can just collaborate.

44. 8:23

    Um- Prompt. Prompt. Yeah, they just started working again. Uh, so I can see that they want to design this new page, uh, which is pretty cool. Uh, so I can wait for them to finish, or I can just go ahead now and just hand off from them to my agents.

45. 8:39

    Well, our agents. Uh, insert communist, uh, whatever. Uh, so we have our agents now, and I can just keep going from there. And the cool thing is that it's not just human-to-human collaboration.

46. 8:51

    Uh, we are also collaborating with the agents. So there's more direct stuff like this. I can just type stuff and prompt my agents or even their agents. Uh, but there's also a softer mechanism.

47. 9:03

    Uh, there's actually a chat that is, uh, between humans and humans, but also between the humans and the agents. You can see here that the agent said, "I'm starting to work on something," and then I can say, "I'm also working on it."

48. 9:15

    So the next time the agent does something, it knows someone else is working. They can also have soft collaboration, so they would know, uh, what files each one is changing.

49. 9:24

    So we've actually taken a bunch of stuff, uh, that were limiting us from really reaching our full potential with agents and kind of solved them one by one. There are a bunch of other features that I just didn't have time to go over, uh, but you can try them out and see for yourself if you can really,

50. 9:43

    uh, work better that way. So to sum up, uh- [laughing]

51. 9:52

    These are not exactly new skills. I mean, you're probably worried perhaps that we won't be able to get adapted to this future where we're not actually coding, we're just telling other people to code for us or other agents.

52. 10:05

    Uh, but these skills are there. They're just not something we used for work until now. Uh, so with games as one example, we can take these skills to the next level.

53. 10:18

    We need to somehow raise that ceiling. We need to somehow improve our collaboration with agents. A-and with AgentCraft, the goal is to take the learnings from games and really raise that to the next level with better visibility, uh, more autonomy to the agents, and human-to-agent collaboration.

54. 10:39

    So I invite you to go to, uh, the website. Uh, this is the QR code. You can-- It's free. You can just download it and play with it. Uh, it's still experimental.

55. 10:48

    It's still new. There's a bunch of stuff that need to change, uh, but it will only happen with great feedback. There's also a Discord. Uh, so please join, give us, uh, your feedback, and let's raise the ceiling together.

56. 11:01

    Thank you. [audience applauding] [upbeat music]