Tinsley Shi

RL post-training for coding and agentic models turns every rollout into a sandbox: a fresh environment with a task image, a few commands, a reward, and teardown. A single training run can need Millions of these per hour across thousands of distinct images, with starts arriving in bursts of hundreds and lifetimes measured in seconds. This talk highlights what we learned running 1M sandboxes using an open-source harness on top of Kubernetes called Substrate, and all the various technological innovations in the stack - warm pooling, image acceleration, low latency scalable storage,microVMs, gVisor, golden snapshot and more. We cover the complete stack orchestration layers that make this work in state of the art Al labs including benchmarks and latency assessment.