Arrakis: How To Build An AI Sandbox From Scratch

Talk transcript

1. 0:00

   Hello, everyone. My name is Abhishek. I'm the solo founder and developer of Arrakis, an open source code execution and computer use sandboxing service for AI agents. Today, we'll chat about why sandboxing is the next big unlock in intelligence and what goes into building an AI sandbox like Arrakis.

2. 0:20

   A little bit about my background. I went to school at Carnegie Mellon, where I studied distributed systems and operating systems. At Microsoft, I worked on Windows Subsystem for Linux and a smartwatch OS from scratch.

3. 0:33

   At Google, I worked on cutting-edge systems using containers and inventing something called microVMs, which we'll discuss in plenty detail today. The last couple of years, I was at Replit working on infra and RAG-based code chat before I founded Arrakis, right at the intersection of my background in operating systems, sandboxes, and AI agents.

4. 0:55

   So why do we need AI sandboxes? For one, the latest models like o3 all leverage tool calling, such as search or code exec during inference to give smarter replies to [REDACTED:username] queries.

5. 1:07

   These tool calls require AI sandboxes for execution. For reinforcement learning as well, during the training phase, you need sandboxes to run reward functions at scale.

6. 1:18

   Agents can also go a l-long way with the full Linux sandbox at their disposal. For example, during code generation, they can debug entire apps by using Linux commands like PS, lsof to see if the code is running and debugging it.

7. 1:33

   They can backtrack, replan, and work towards the goal again with a sandbox at their disposal. Security is paramount. Agent code is no different than you using any code from GitHub or Stack Overflow and running it on your host or production server.

8. 1:47

   This code could be buggy or malicious and can get root and can access your data or your client's data. So we need some amount of locking down, uh, in this sandbox.

9. 1:58

   Let's see a sandbox around us. I know we've seen, uh, Canvas by OpenAI, uh, Claude Artifacts. This is Manus AI. Uh, it uses a sandbox very heavily. Uh, you can see that I've asked it to create a ChatGPT clone, and it ru-- it's running all sorts of commands inside the Linux sandbox.

10. 2:15

    Uh, it will also try to run the app, see if it didn't work, and then try to fix it itself. You don't need a big prompt or a lot of like, uh, alignment and frameworks to make a coding agent when it has a Linux sandbox at its disposal.

11. 2:29

    Just because it has all of this, uh, like Linux knowledge, uh, in its pre-training data, so it could go a long way, uh, for code generation tasks inside the sandbox.

12. 2:40

    With that background, introducing Arrakis. Arrakis provides a secure, fully customizable, and self-hosted solution to spawn and manage AI sandboxes for code execution and computer use. It has out-of-the-box support for backtracking via snapshot and restore.

13. 2:56

    And what that means is agents can backtrack and don't have to start from scratch if they fail during multi-step workflows. Best of all, it is completely open source, and you can dive into the code, and we will do that today to see how to make an AI sandbox like Arrakis from scratch.

14. 3:14

    Now let's go over the features of Arrakis. Uh, let's start with microVM-based secure code execution. As discussed, security is paramount for AI sandboxes. Arrakis uses microVMs as a runtime, and we'll go into more detail there.

15. 3:29

    But like I mentioned, code generated by AI, AI agents would increase exponentially, uh, as people are using Windsurf, Cursor, and more and more coding agents. We don't want these malicious or potentially buggy code to get root, and we want to protect your data, your systems, your clients' data, um.

16. 3:45

    Second is speed. Speed is paramount for AI sandboxes. You want these things at scale to be able to call tools, generate code fast. Currently, Arrakis boots in less than seven seconds, which is way, way better than forty seconds for a traditional old VM on macOS.

17. 4:01

    We have a PR out right now to get that time below a second. Um, also snapshots are super fast, single-digit seconds and just getting, uh, lower and lower with work.

18. 4:13

    Uh, Arrakis handles all sorts of port forwarding for you, so code exec, browser use, you can access them easily, uh, via just a public URL and a port. You don't have to muck with IP tables, firewalls.

19. 4:25

    Arrakis takes care of all of that. Uh, easy computer use agent sort of workflows. Chrome is pre-installed, and there's a VNC server that's hooked up pre-installed, and so you can access the GUI of your browser very easily.

20. 4:39

    Uh, you can see a Chrome instance opened in an Arrakis sandbox here. Um,

21. 4:46

    backtracking is super inst-- important as well. Like I mentioned, Arrakis supports snapshot and restore, so agents can checkpoint progress by snapshotting the sandbox. If they fail in their multi-step flows, they can restore an old snapshot.

22. 5:01

    Uh, this way, we can get more reliable, higher order complex task execution via agents.

23. 5:08

    Arrakis has a dead simple and ubiquitous API. Uh, it has a Python API, a Golang client, an MCP server. It has an OpenAPI compatible YAML file, so you can generate any client in any language that you want.

24. 5:23

    Lastly, it's configurable with Docker tooling, so you can use your existing Docker commands, and there's a Docker file to customize what binaries and packages are installed in the sandbox.

25. 5:33

    So Arrakis gives you complete control and freedom on what you want to run inside your sandbox. So this is the high level architecture of Arrakis. Uh, you can see it has a REST server that spawns and manages microVM sandboxes.

26. 5:47

    Each of the sandboxes runs a VNC server and a code server. We do the port forwarding to expose a VNC server, and you can access the GUI and Chrome via VNC client.

27. 5:58

    We give Arrakis CLI, Golang based CLI called Arrakis client. There's a Python S-SDK. There's also an MCP server that's not shown here. Uh, so there's a lot going on here, but in the next half of the presentation, we'll work towards what all go inside each of these, uh, components and see how an AI sandbox, uh, is made.

28. 6:18

    Um, Arrakis is tied to Linux just because the microVM tech that we used, uh, is tied to, uh, Dev KVM, which is the Linux virtualization, uh, device. So it's tied to the Linux virtualization stack.

29. 6:34

    Let's move to the API of Arrakis, and, uh, you can see we have a very simple REST-based API. Uh, there's a key VMs, uh, uh, resource that you use to like, uh, like start, stop, delete a VM.

30. 6:48

    Inside that, you can-- you also have a snapshots resource, so you can like snapshot a VM very easily by the snapshots API. And within a VM, you get command execution with the command resource, and you can upload and download files from within the sandbox using the files API.

31. 7:04

    And finally, there's a health check endpoint as well if you want, uh, to have a distributed version and you want to see the health of your Arrakis, uh, REST server, uh, this is what the API exposes.

32. 7:18

    Okay, we discussed about the API that Arrakis exposes, and now we have an updated architecture. We can see that we have a REST server running on a Linux host or a Linux server, uh, exposing a REST API, uh, via the Python SDK or the Golang, uh, CLI.

33. 7:35

    We also have an f-- MCP server and an OpenAPI, uh, based YAML file that you can use to generate clients in any languages. Um, and we'll keep updating this architecture as we go inside the architecture of the sandbox.

34. 7:48

    Uh, so let's move forward. Arrakis uses something called microVMs to give secure sandboxes to, uh, models and AI agents. Uh, but what are microVMs and why do they or how do they provide a secure environment?

35. 8:04

    For that, let's take a quick tour of Linux sandboxing, and I'll build from scratch, uh, your knowledge about Linux sandboxing and what our different options are and why we chose microVMs.

36. 8:15

    Let's start with the Linux execution model. So this diagram kind of goes over the basic Linux execution model. Uh, a thread is the smallest unit of execution on Linux.

37. 8:26

    Each thread has a task struct in the kernel that's in the scheduler run queue. And what that means is there's a linked list or a data structure, uh, connecting these like task structs, and these are used to represent, uh, each thread or a unit of execution.

38. 8:42

    A process is a logical construct made up of multiple threads. Um, threads in a process have different tids but the same pid. They share page table and other resources.

39. 8:52

    Uh, and so they are logically bound and connected by shared resources. Uh, kernel provides privileged access to hardware. Um, if this wasn't the case, then any buggy or malicious, uh, uh, like code running in a thread can just crash our entire device or, or do malicious things.

40. 9:10

    Um, and special instructions are needed to switch to kernel mode, uh, or supervisor mode, um, and we need to invoke a system call for privileged access. Uh, and so you can see in the diagram when we do like int OX8O, that's us trying to get to privileged mode and, uh, get access to hardware or do like privileged

41. 9:28

    operations on our OS. Okay, we've discussed the basic Linux thread-based execution model. Now let's discuss about containers, uh, why do we have them and what they are. So first, uh, let me give you a programmer's perspective of, uh, what problem containers solve.

42. 9:47

    Uh, and so in this diagram, you can see a container, uh, and the concept of namespaces. But yeah, let's dive deeper. Um, so first, why? Like, let's say my app needs Python three point eight, uh, a version of a library foo with one point two, and my server has Python three point nine, three point six, three point

43. 10:05

    twelve, but doesn't have three point eight, and it doesn't even have the library foo. So how do I run my app on like an arbitrary server without these things, right?

44. 10:13

    And so containers from a programmer's perspective are a way to package an app's dependencies along with the core business logic. Um, uh, as a corollary, they let you run arbitrary [REDACTED:username] code on your machine, which as we've discussed is one of the core features required by an AI sandbox.

45. 10:30

    Um, getting more technical, on a Linux, a container is a collection of namespaces of different resources. So process, mount, and net are example of key resources that you can abstract in namespaces.

46. 10:44

    What that means is like if you have a process namespace, uh, for instance, in a container, um, it sees certain processes as pid one, pid two, and pid three within its namespace.

47. 10:55

    But outside, these are like arbitrary processes, uh, running in the [REDACTED:username] space, in the root namespace. So container has a very like, uh, abstracted or bound view of its own resources.

48. 11:07

    Uh, and the host can peek inside a child container namespace, but a container cannot like look upwards and see in its host namespace. So this is one way of like, um, telling processes or like contents running in the container, uh, that like you have your own, uh, controllable like resources.

49. 11:27

    Uh, uh, and it's like a little bit of a, a false boundary, so to speak.

50. 11:33

    Uh, so like I said, yeah, outside the container you can see everything inside your, inside your child's namespace, but it doesn't work the opposite way. Uh, we also use something called cgroups, uh, in tandem with namespaces.

51. 11:45

    So containers are collections of namespaces and cgroups like, uh, can control how many resource a cont-- resource and container can, uh, uh, access. So what percentage of memory, what percentage of CPU do you want to give, uh, to a specific container running on your machine?

52. 12:03

    Let's talk about the security story of containers.

53. 12:09

    Um, and so here you can see a container, uh, consisting of a paid namespace running on top of, uh, uh, a kernel, uh, on a server. Uh, and so you can see that after all containers run as native processes on top of the kernel, um, there's a logical binding of resources.

54. 12:27

    But at the end, these are just like processes running on top of the kernel. So you can imagine now that if you have a kernel vulnerability, um Uh, any malicious or buggy process can attack the kernel and gain root.

55. 12:39

    And then once it became-- becomes root, it can do basically anything it wants. It can dump memory, it can read your data, it can, like, [chuckles] pretend to be, like, uh, someone it's not.

56. 12:50

    So all sorts of attack vectors are on the cards.

57. 12:54

    Okay, so we now know what containers are and, like, what's-- what are one of the security flaws that containers have. Um, so what are, what are the alternatives? Like, uh, how can we, let's say, like, sandbox or jail container, uh, to kind of mitigate these things?

58. 13:08

    So let's look into that. Right. And so, like I said, like, if there's a kernel vulnerability, a container can attack it and get root. But let's see how we can reduce the attack surface by, uh, using some techniques that we have.

59. 13:24

    So one is, like, the fundamental principle is to jail the containers by restricting, like, the Linux capabilities and, like, the syscalls they can call. Uh, and so in effect, reducing the attack surface that's available to it.

60. 13:37

    And so the way this works is, right, like, Linux has a concept of, uh, caps or capabilities, and that kind of govern what syscalls or privileged operations a process can do.

61. 13:49

    And so you only give capabilities that are required by your container to do its work. You don't give it, like, a catchall. Um, and these indirectly control what system calls and what code paths they can take within the system call.

62. 14:01

    Another thing is seccomp, which, like, uh, uh, filters the arguments you can give to a system call or just block system calls altogether. So seccomp filters is another option.

63. 14:12

    Um, and so you can-- These are like gnarly or, uh, hard-to-use APIs with a lot of granular error checking. So there's a library called MiniJail that we used to use at Chrome OS that I highly recommend to check out that helps in, like, jailing and sandboxing of, uh, containers and processes.

64. 14:28

    Okay. Sandboxing can only go so far, and jails also have their limits. This can still bypass them. Uh, let's say you need a heavier hammer, right? Uh, what options do we have?

65. 14:40

    So let's discuss going from containers to virtualization on Linux.

66. 14:45

    So let's discuss a high-level view of virtualization. Um, virtualization provides another primitive to run untrusted or arbitrary code on your machines. Um, each VM has its own guest [REDACTED:username] space and guest kernel.

67. 14:58

    And so unlike the container model, where the processes run directly on top of the host kernel, uh, this-- in this way, the processes have their own isolated kernel and [REDACTED:username] space.

68. 15:08

    And, um, there's a very lit-- there's a smaller attack surface to get to the host kernel compared to a container. And so the question is in this diagram, like you need to access like hardware and other like resources.

69. 15:20

    So how do VMs then access the host resources in this virtualization model on Linux?

70. 15:26

    Okay, now let's dive deeper into Linux virtualization.

71. 15:35

    Yeah. As you can see that, uh, this is a more fleshed out diagram of Linux virtualization. Let me go over the main parts here. So on the right-hand side, there's some client that wants to spawn a virtual machine.

72. 15:47

    Um, and the main actor here is this process called the VMM, the virtual machine monitor. And you might have heard of QEMU, CrossVM, or Firecracker. The main process here is the VMM.

73. 15:59

    The VMM sits on top of dev KVM, which is like a device in the Linux kernel that exposes, uh, the processor's virtualization stack, uh, and provides a nice API for the VMM process, uh, to talk to in order to start a VM, uh, and, uh, give it access to privileged resources.

74. 16:19

    Uh, so let's discuss how this works. Um, the VMM talks to the KVM device to spawn VMs, and it also manages emulated devices inside. So you can see there's a block device and there's a net device.

75. 16:32

    Some VMMs spawn these in other, uh, sandbox processes, and some VMMs have them in the same process. Um, to the client process, it looks like the VMM thread is blocked, but really inside that thread, we are running all the guest code on the hardware.

76. 16:50

    So in a separate virtualization context on your processor, this hardware code is running. Uh, whenever the virtual machine needs to access disk or net or any other like privileged resource on the host, it returns back to the host.

77. 17:03

    So the VMM process gets a, a VM exit, uh, and then it figures out why the exit happened, like which device caused the exit, block, net, or something else.

78. 17:13

    It then talks to the host kernel to do-- talk to the block disk device or the net device, uh, gets a response and sends it back to the guest with a VM resume.

79. 17:22

    So the whole, the whole idea is that VM exits and resumes are very, very, like, performance, uh, uh, uh, affecting, and so you want to minimize this and want to keep the, uh, guest code running in the guest context as much as possible.

80. 17:37

    So if you're like CPU-bound inside the guest, you're t-technically paying no penalty in a, uh, hardware virtualization model because you're running directly on the processor. But if you need a lot of like disk and net and other device a-access, uh, then there are other ways to-- you have to think about performance because you might be exiting too

81. 17:55

    often. Uh, and there are other things you can do. You can cache more state in the guest memory and things. But, but yeah, containers run directly on top of your host OS, so performance isn't, uh, a big factor because they're like native processes, but the trade-off is of security.

82. 18:10

    Here you get better security, but in some loads the trade-off might be in performance.

83. 18:16

    Okay, we've now discussed Linux virtualization and how it works. So what are mi-microVMs and how do they differ from traditional VMs? Um, so let's take a look.

84. 18:29

    Okay. And so as you can see that, um, microVMs are slightly different from traditional VMs. So first of all, um This term came from the CrossVM project at Chrome OS.

85. 18:41

    CrossVM was the first Rust-based virtual machine monitor. And the reason we rewrote, uh, QEMU like VMMs into Rust is they provide, uh, a memory safe implementation of virtualization. And so one aspect is that, uh, technically untrusted code running in a guest can attack your host by attacking the devices written in C which have like memory safety related

86. 19:06

    bugs. Um, so writing Rust gives you some degree of, uh, sanity there. Separately, another feature of these VMMs is they jail their emulated devices separately. So, um, you can jail the block device to only have block related system calls.

87. 19:22

    So if you compromise block, you won't access network related things and vice versa. You jail the net device to just have network related system calls. So yeah, one way microVMs are different is they're written in Rust and they have this like more jailing, uh, based architecture for their emulated devices.

88. 19:39

    And so one aspect is security. The other aspect is why are they called microVMs? Where does the micro come from? Uh, and like they're known to boot really fast.

89. 19:48

    So why and how do they boot fast and also like take less memory? So old VMMs like QEMU support a lot of architectures and have many, many emulated devices supported.

90. 20:02

    Um, uh, microVMs like CrossVM, Firecracker, Cloud-Hypervisor don't do that. They only support like one or two architectures, Intel and ARM, and have support for the major devices and not like any like, uh, obscure device as well.

91. 20:17

    What this means is that there's less code, less code paths at boot, uh, and so they just boot up blazing fast and at runtime like take less memory. Uh, and so the microVM, the micro is actually a reflection of the VMM process, either CrossVM or Firecracker, uh, versus what's running inside the guest.

92. 20:36

    And so you can think of microVM as this new security first way of running virtual machines that are like lighter weight, boost, boot fast and consume less memory.

93. 20:47

    Okay, hopefully that background was useful on the different ways of sandboxing untrusted code on Linux. Oh, so with that, Arrakis chooses a microVM runtime as the final execution environment for these AI sandboxes.

94. 21:02

    Uh, and so security is one of the key design choices of Arrakis because it's written, it's written with coding agents in mind and coding agents might use, um, might have multi-tenant or like different clients, uh, running on the same server, uh, with LLM generated code accessing different clients' data.

95. 21:22

    So we don't want, uh, one piece of untrusted code getting root on your server and accessing some other, uh, client's data. And so multi-tenant code execution is, untrusted code execution is a very, very, uh, key use case and design factor behind, uh, choosing microVMs.

96. 21:39

    Uh, they also provide fast boot times as we discussed. Uh, snapshotting is super important for Arrakis and microVMs, uh, provide a way to fast snapshot by just dumping the entire guest memory.

97. 21:50

    This isn't as easy to do with containers and, uh, even with gVisor that we'll discuss. Uh, and with microVMs you can just allocate, uh, some virtual memory and then just dump it when you want to snapshot and restore it when you want to restore the VM.

98. 22:05

    Uh, and so I wanted to discuss like different VMMs that we have the option of choosing. So like I said, CrossVM started the whole microVM revolution and with that came a bunch of other Rust-based VMM.

99. 22:17

    So Firecracker and Cloud-Hypervisor are two VMMs forked from CrossVM. If you go over the code base, you'll see many remnants of CrossVM in both. Um, Firecracker is the underpinning behind AWS Lambda.

100. 22:29

     It's used for like serverlets, serverless loads, um, and, uh, it has a more fleshed out like REST API I would say and it also has a better jailing, uh, architecture.

101. 22:40

     Cloud-Hypervisor is a more general purpose enterprise VMM. Uh, when I chose it, it had hot plugging of devices which means you could add and remove RAM, uh, very easily as a PCI device when you're running a sandbox.

102. 22:53

     It had GPU support and it had snapshot support at the time. Uh, and from a software, uh, project point of view, it, it isn't controlled by one specific company.

103. 23:02

     There are different companies there and so it made a lot of sense to use Cloud-Hypervisor as the microVM VMM for Arrakis. Uh, another option we didn't discuss, uh, separate from microVMs is gVisor which I would say is closer to a container, uh, in performance and like s- uh, but slightly better in security.

104. 23:20

     Still you can attack the host kernel with untrusted code running in gVisor but based on your needs and security guarantees, um, it's a good in-between option. You do get GPU access, uh, more easily in gVisor and containers versus like, uh, microVMs.

105. 23:36

     So if that's a factor then, uh, you can choose one of those and decide what your security guarantees are.

106. 23:44

     Okay, so now we've discussed why we chose Cloud-Hypervisor and microVMs as the runtime for AI sandboxes in Arrakis. Uh, let's move forward.

107. 23:55

     Okay, so now we can see that, uh, there's the, uh, updated architecture from the one we started off before. So earlier we had just an API. Now we can see that, uh, the REST server is managing these, uh, microVM based sandboxes.

108. 24:11

     The dotted process is actually the Cloud-Hypervisor VMM process. Um, it runs, uh, the guest code in a thread in a virtual context and it's talking to dev KVM, um, to, uh, run these like microVM, uh, virtual machines.

109. 24:28

     Um, okay, so let's move forward. Okay, now let's discuss the storage of the file system within each sandbox. After all, it might need to create files, read from them, and write to them.

110. 24:41

     Um, and so one attack vector we haven't considered is like untrusted, uh, malicious or buggy code that can actually, uh, delete files inside your sandbox, and it could delete very important files on your file system, and it could make the sandbox like non-functional basically.

111. 24:57

     And so we need to kind of protect the rootfs running on inside the sandbox. So we have a s- have a shared base layer, uh, of our RO rootfs that's shared between sandboxes, and you can see that this is the yellow one with [REDACTED:username] [REDACTED:username] here.

112. 25:13

     Uh, but on top of that, every sandbox gets its own like read/write layer. So this is where all the new files it creates, uh, go. Uh, and so you have this very nice, uh, balance between the rootfs being protected and shared between sandboxes.

113. 25:28

     Um, but, uh, they get their own like read/write layer. Uh, and when we snapshot a sandbox, we also persist or back up just the read/write layer. We don't need to persist the read-only layer.

114. 25:38

     So we have this like nice sharing and like per sandbox, uh, semantics here.

115. 25:46

     Okay, and now let's go to the code and actually see how this happens, right? So, uh, the sandbox, when it boots up, already has this set up. All it sees is like a root mount path, and just like any other like Linux system, there's no difference.

116. 25:59

     But we do all the magic before the sandbox, uh, as soon as the sandbox boots, uh, and we start like the first, uh, PID1 inside the sandbox. Uh, and so you can see here we like, uh, uh, this is the init.sh script running inside, uh, the guest, which is our sandbox, and it's setting up this, uh, like

117. 26:18

     overlayfs here. Um, and as soon as it does that, we start like the booting process and like boot to PID1. Um, and every process, uh, within our sandbox just sees like a regular like file system, but we have done the magic underneath.

118. 26:37

     Yeah. And with that, like let's see our updated architecture. So we had the REST server, we have these like microVM sandboxes, and now we see like, uh, every disk device is, uh, mounted on top of this overlayfs, uh, and each sandbox gets its own like read/write layer.

119. 26:55

     Okay, with that, let's move on to networking.

120. 27:00

     Okay. Every sandbox needs to have networking. It might need to do other actions or call other tools or APIs. Um, and so we need to provide each sandbox with a network.

121. 27:11

     Um, and so each sandbox in Arrakis runs in a virtual machine like we discussed with its own isolated networking. The networking setup consists of a tap device, so it's, it's like a virtual networking interface on Linux.

122. 27:24

     Each sandbox gets a unique tap network device. So when we spawn the, uh, microVM sandbox, we are creating this tap device for it. Um, and we have a Linux bridge on the host, uh, which is the Linux server, where, um, all the tap div-- tap devices are connected to the Linux bridge on the host.

123. 27:43

     And the last part is, uh, we take care of port forwarding. So Arrakis like forwards ports, ports from your host into like the code server or the VNC server, so you don't have to worry about how to access, uh, these things, uh, on, on your sandbox.

124. 28:01

     Okay, uh, let's go over some code to set up networking here. So this is a function from Arrakis. It sets up the bridge device, and it also sets up the firewall rules to, uh, forward data back and forth, uh, from the host to the sandbox and the sandbox to the host.

125. 28:18

     Um, and so you can see that like here we come in and we create a bridge, uh, and we set it to up. Um, and then we set these like firewall or forwarding rules that are quite gnarly to set up, but we've taken, uh, care of them, uh, for you, and these decide how data flows from, uh,

126. 28:35

     from the host to the sandbox and back.

127. 28:41

     Uh, another function here that takes care of port forwarding, and you can see we use like Linux iptables command to do that and, uh, uh, like we have a destination port inside the guest, and we have this like, uh, DNAT argument here that does all the magic.

128. 28:58

     And, uh, we call this using like go's command.run. Um, uh, yeah.

129. 29:05

     Okay. With networking set up, our updated architecture looks like this. Uh, we have the REST server spawning these microVM sandboxes. Um, and now we see that we've taken care of port forwarding, so a VNC client running on your MacBook or your laptop can easily access the GUI via the VNC server, uh, running inside the sandbox.

130. 29:27

     Um, as mentioned before, uh, we use Docker tooling to customize Arrakis sandboxes, uh, so you can run whatever packages and binaries you want, uh, by just modifying a Dockerfile inside Arrakis.

131. 29:40

     Uh, so let's take over the-- Let's take a look over the Dockerfile that we have by default. Uh, and so you can see we base it on Ubuntu twenty-two point zero four, have these like standard packages inside for your agents to be productive without installing, uh, anything else at runtime.

132. 29:57

     Um, and you can see we also give you Chrome, uh, installed, and we boot it, uh, via systemd inside the sandbox. Uh, and also install like Node.js and npm so that you can make like Node apps.

133. 30:10

     Uh, Python's also already installed, so you can-- the agents can do a lot, uh, with these tools inside the sandbox, and you can configure them for, uh, in the future with whatever binaries you want inside.

134. 30:22

     Okay, let's put it all together and see how we create a VM in code. Um, so as you can see, like we create all the devices we talked about.

135. 30:30

     So there's a rootfs, there's the overlayfs writable path here. We allocate, uh, some vCPUs which are nothing but threads on your host OS. We calculate some memory and give it, uh, like give it as a fraction of the host memory available.

136. 30:47

     Uh, there's a networking device, so we created the tap device, uh, as well. And, uh, above in the snippet, we also did the port forwarding. Um, vsock is a way to cal- communicate between the guest and the host, uh, again.

137. 31:00

     Uh, and then, yeah, finally, we spawned a VMM process, um, and then call this create VM API on it with this configuration to start a guest VM.

138. 31:13

     As mentioned before, uh, code exec is one of the very important functionalities of an AI sandbox. Uh, Arrakis comes in bundled with a code execution server. Uh, here I show you the exact code, uh, that's running inside the code execution server.

139. 31:29

     There's a files API to upload and download files. We'll see in the Claude demo how this API is used liberally by Claude to send data back and forth. Uh, and there's of course a command, uh, uh, API which takes in a command, executes it, and returns either an output or error, uh, uh, in the form of JSON.

140. 31:49

     Um, we are running in a guest VM, so I'm much more confident about this code not escaping out, uh, and we can be relaxed with it versus if we were running directly on top of our host OS, then I'd be very, very scared exposing something like this.

141. 32:04

     Uh, on top of the code execution server, we also have Chrome pre-installed. And like I said, uh, we have-- we do port forwarding for you to the, to your VNC server inside the sandbox, so you can actually access Chrome directly and, and the GUI directly inside your sandbox very easily with Arrakis.

142. 32:25

     Okay. Now let's talk about one of the most exciting features of Arrax- Arrakis, snapshotting. Uh, so what's the motivation behind, um, saving state of a sandbox and snapshotting? So in general, um, agents fail when you're giving them a very big task, like make me a Google News-like app, um, because, uh, in- inevitably they fail

143. 32:50

     somewhere during this big, vague task. So they need like a step-by-step plan and as specific and as small the plan, the better the reliability. But even then, multi-stage plans can fail.

144. 33:01

     So imagine if you have a deep, like almost like a DAG of execution, you do all the work and you fail towards the very end. Um, you shouldn't have to start from scratch.

145. 33:11

     You should be able to backtrack to the last good checkpoint, replan and try again. So that's the motivation behind snapshotting. Uh, giving agents the capability to fail and backtrack, uh, and then at scale getting more reliable results by exploring multiple paths, uh, in parallel.

146. 33:28

     Um, so Arrakis lets agents save the entire running state of a sandbox. So this includes like just the guest memory and the file system, which is the RW part of the overlayfs.

147. 33:38

     So any files it created, any processes it spawned, all will be, uh, restored as is. So imagine on your MacBook, if you close the lid and you open it up, it's exactly the same semantic.

148. 33:50

     Um, and so yeah, any processes found, even like windows opened in the GUI, there's, that's nothing but, uh, uh, guest memory state, so they'll be restored as is. Um, and so agents can backtrack to a good snapshot if they fail and then replan and continue the workflow.

149. 34:06

     So this diagram shows like how we can do this at scale. Um, uh, if we fail, we can just like go back to, um, our path of execution to a last known good hash and like replan and try again.

150. 34:20

     Currently, we don't use BTRFS, but our plan is to use a, um, a file system that natively aware, uh, is aware of, uh, uh, incremental snapshots and is optimized for such.

151. 34:31

     Um, and so we are exploring using BTRFS instead of Ext4 for the file system as well.

152. 34:38

     Uh, now let's go over like how we actually do the snapshotting in code. And there are like four steps. Um, we actually first pause the VM by calling the pause API on the VMM.

153. 34:49

     Um, uh, then we call the snapshot API and, um, dump the guest, uh, memory. Uh, and then out of band, we manually like persist the thin, uh, read-write overlayfs to persist all the files, uh, that the sandbox or the agent, uh, created inside the sandbox.

154. 35:08

     And finally, we resume the VMM. So we checkpointed it, and then we just ran sandbox again, so it keeps doing what it has doing before. Uh, and you can see this in code, right?

155. 35:18

     Uh, uh, we go ahead and pause the VM here. Um, we make sure we resume b- before we exit the function here. Um, we create a copy of the stateful disk, which is the read-write layer of the overlayfs.

156. 35:32

     Uh, yeah, and finally call snapshot which, which dumps the guest memory, uh, for the VM and creates this magical experience when you resume it. Okay. We've gone through the entire architecture of Arrakis and how you would build a AI sandbox like Arrakis.

157. 35:49

     Uh, let's put it all together and how would we actually use it? So, um, Arrakis comes with a first-class support for a Python SDK. So you pip install the py-arrakis package, and it's very simple from then on.

158. 36:01

     You, uh, you self-host Arrakis on your own infrastructure, give it the IP here to the sandbox manager. Um, you figure out what all VMs are running by calling the list all API that gives you metadata about the VM, its IP, its ports.

159. 36:16

     Um, you start a sandbox with the start sandbox API, run a command and see the output or the error using the output or error JSON keys. Um, and here's how you can actually snapshot with just one command.

160. 36:28

     Like you give a snapshot ID, um, and then you destroy the VM if, uh, if when you're done. Um, and later on when you want to actually restore the checkpoint from your snapshot, it's very simple.

161. 36:40

     You just call restore with your VM name, and you give the snapshot ID that you got here from the snapshot call. So all in all, a very dead simple API to use, uh, spawn and use AI sandboxes, uh, for code exec and other things.

162. 36:56

     Okay, so we've seen the architecture of Arrakis. Uh, now let's see, uh, it in action. Um, let's see Claude Desktop make a Google Docs clone using Arrakis via its MCP server.

163. 37:09

     Um, and you will see that we don't have to do a whole lot of prompting or create a whole coding agent tool or framework. Uh, it really does a lot of heavy lifting if it-- once it realizes it has access to a full-blown like Linux sandbox.

164. 37:22

     Uh, so yeah, let's go over the demo real quick.

165. 37:27

     Uh, so you can see I prompted to create a Google Docs clone, uh, and that has collaboration built in, so multiple people can join. Um, we'll see that on the right-hand side, we are running Arrakis, um, and, uh, off the bat, Claude is just piping commands, uh, creates a sandbox and pipes commands directly in.

166. 37:46

     It creates this Google Docs clone, and we have networking set up so people can type in together. We can snapshot this version as well, so we created a snapshot of that copy, and now we try to add a feature on top of that, uh, clone.

167. 38:01

     Uh, and so yeah, we try to add dark mode inside the Google Docs clone.

168. 38:08

     Let's see Claude do this as well. Okay, let's see. So it added that dark mode feature inside, and you can see it works. Collaboration hasn't broken. Uh, but I changed my mind.

169. 38:20

     I wanna go back to the snapshot we had before without the dark mode. And so you can see we can restore the old checkpoint and go back to without dark mode here.

170. 38:36

     Right. So the cool part here is that, um, uh, it did the snapshotting and, and you could see it created end-to-end apps without me prompting it again and again.

171. 38:46

     But since we took care of networking, we could actually get a completely collaborative experience, like a full-on Google Docs thing, uh, versus just like fake collaboration that other client side, uh, uh, code gen, uh, tools might do.

172. 39:02

     Um, so that was like a one cool side effect, uh, of the demo. Uh, yeah.

173. 39:08

     Okay. Let's discuss, uh, some of the ongoing work in Arrakis. Um, top of mind is definitely getting boot time to be under one second, uh, and even lower if possible.

174. 39:18

     I know other sandboxes are also going, uh, on, uh, on this metric. Um, second is really, really have first-class support for snapshots and persistence. So what that means is moving to BTRFS, um, which is, uh, tailor-made for incremental snapshots.

175. 39:34

     Um, and then we want to bin pack as many sandboxes on one server as possible, so we need to do dynamic memory management and resource management. So ballooning or hot plugging of, uh, uh, or removal of memory at runtime is very, very important.

176. 39:50

     Um, yeah, that's it for today. I hope you liked, uh, the presentation and you got an inkling of why sandboxes are important, how they are made, uh, from scratch, and a taste of like cutting-edge Linux systems work.

177. 40:04

     Um, if any of this is interesting, please get in touch with me, uh, at those links. Uh, and I've, uh, left the links for the launch video, the GitHub repo.

178. 40:14

     Yeah, happy to help anyone.