Topic

Sandboxing

All digests tagged Sandboxing

Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev) thumbnail

· 37:15

Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev)

The video argues that traditional methods of running AI agents—such as allocating a small corner of a local machine or relying solely on containers/CI/CD—create bottlenecks. To achieve true scale and autonomy in an 'AI developer workflow' (ADW), the entire software factory must be moved into dedicated, isolated agent sandboxes (e.g., using exe.dev). This architecture enables complex workflows like running 'Best of N' comparisons across multiple model configurations while maintaining zero blast radius via disposable keys.

Key takeaways

  1. The Bottleneck Problem

    If an engineer is 'in the loop,' they are the bottleneck. True scaling requires moving beyond local compute limitations by giving every agent its own isolated computer, achieving isolation, scale, and autonomy.

  2. Three-Tier Architecture for Scale 23:25

    The recommended architecture involves an Out-loop orchestrator (on the engineer's machine), an In-sandbox orchestrator (on each VM), and the core Software Factory/ADW agents running inside the sandbox. This allows the top-level agent to kick off work and then go quiet, only requiring human intervention at planning and reviewing stages.

  3. Best of N Pattern 17:05

    Sandboxes enable running 'Best of N' patterns by simultaneously executing the same prompt/workflow across multiple agent configurations (e.g., Default, Frontier, Deepest, Open Weights), allowing for comparison and selection of the optimal outcome.

  4. Security and Isolation 27:50

    Sandboxes provide critical security by ensuring a 'bounded blast radius.' Agents use ephemeral resources, such as OpenRouter provisioning keys with hard spend caps, which are revoked upon teardown, preventing unauthorized access to production systems (e.g., AWS).

Watch on YouTube Full article

Build a secure computer for your agent thumbnail

· 36:51

Build a secure computer for your agent

As agents evolve beyond simple Q&A to writing and executing complex code (e.g., data analysis, software engineering), they require an isolated, persistent computing environment. The session details the architecture of LangSmith Sandboxes, a solution designed to provide production-ready, secure execution by giving each agent its own disposable computer. This system addresses critical challenges like untrusted model-generated code, container escape vulnerabilities, and scaling limitations inherent in traditional local or vanilla container setups.

Key takeaways

  1. Production-Grade Isolation 20:45

    Sandboxes utilize hardware virtualized microVMs for kernel-level isolation, ensuring that malicious code cannot escape the environment (preventing issues like container escapes). This level of separation is necessary because agents run untrusted, model-generated code.

  2. Scalability and Performance 23:50

    The platform supports scaling from one to thousands of isolated sandboxes in parallel. Benchmarking shows a median spin-up time of approximately one second, making it suitable for high-volume, user-facing applications.

  3. Secure Credential Management 25:20

    The O proxy acts as a man-in-the-middle proxy controlling all egress from the VM. This ensures that credentials never touch the runtime, significantly mitigating risks associated with data exfiltration or malicious network calls.

  4. State Persistence and Resilience 27:10

    Sandboxes support persistent state across long-running, interruptible tasks. Users can snapshot and restore the entire environment (including file system and memory), allowing for rollbacks or forking to test multiple scenarios.

Watch on YouTube Full article