Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes
Amazon runs agent reinforcement learning on shared Kubernetes with sandbox pods. Four SWE-bench Verified campaigns created 3.1 million sandboxes in October 2026.
The claim
Amazon runs agent training in sandboxes that scale to millions of environments.
The problem
Agent training needs thousands of short-lived tool environments in bursts.
Those bursts share clusters with other production jobs.
Slow setup and abandoned sandboxes can stall training.
How it works
Each sandbox is a Kubernetes pod with a small mounted daemon.
The daemon runs inside the unmodified task image.
A framework sends a task image and runs commands in the sandbox.
Files and processes persist until the framework releases the environment.
Sandman manages creation, readiness, cleanup, and reclaims abandoned sandboxes.
Figures
What changes
A scale test held 11,858 concurrent tiny sandboxes.
Four SWE-bench Verified campaigns created 3.1 million sandboxes in October 2026.
Limits
The paper reports design trade-offs and lessons, not baseline comparisons.
The daemon, provider, and pool will be open-sourced later.
The paper gives no cost or latency figures for sandboxes.
Source · arXiv