# Stop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI

## Executive summary

Code Mode represents an architectural shift for AI agents, moving beyond traditional tool-calling by providing a secure, dynamic programming sandbox. Instead of accumulating tool catalogs and raw outputs in the context window, Code Mode allows agents to search for, compose, and execute tools (e.g., Bash, Python, SQL) within a controlled environment. This approach significantly reduces context bloat, lowers token usage, and improves efficiency, enabling complex, multi-step, and emergent behaviors previously difficult to achieve with standard agent frameworks.

## Key takeaways

- Efficiency Gains via Code Mode: Code Mode significantly reduces context window bloat and overhead. By allowing agents to call multiple tools in a single turn and only returning necessary outputs, evaluations using the Harbor framework showed a 43% reduction in tokens and a 31% reduction in cost compared to traditional methods. (12:55)
- Architectural Shift from Tool Catalog to Sandbox: Traditional agents rely on passing a large catalog of tools, leading to context bloat and increased latency. Code Mode provides a programming environment (sandbox) where agents can dynamically search for and compose tools (e.g., sequential, parallel, conditional flows), minimizing context overhead. (3:03)
- Emergent Behavior and Composability: The sandbox enables emergent behavior, allowing agents to build their own tools or perform tasks (like PII removal) that require custom logic, rather than relying solely on pre-defined tool APIs. This composability is key to solving complex, multi-domain problems. (4:37)

## Technical details

- Code Mode Implementation (Phoenix): Arize built its agent harness within Phoenix, an open-source observability platform. This implementation supports multiple execution environments, including Bash, a Python sandbox (for REST CRUD endpoints), and SQL execution, enabling complex data manipulation and querying. (6:11)
- Operational Challenges and Risks: Implementing Code Mode introduces several operational complexities: managing CPU/memory in hosted sandboxes, handling network egress, securing secrets (e.g., using Vaulting strategies), managing permissions, and dealing with double sandboxing (where serialized JSON output must be parsed and put into another code mode). (8:19)
- Evaluation Framework (Harbor): Arize uses the Harbor framework to evaluate Code Mode agents. This process involves running agents in controlled, sandboxed environments with network access disabled, utilizing an RL framework (one or zero) to test task completion across multiple permutations and eliminate 'flake.' (11:44)

## Practical implications

- For build engineers, Code Mode offers a path to building highly efficient, complex agents that can execute multi-step logic (e.g., data aggregation via SQL, or complex file manipulation via Bash) without the context window limitations of traditional tool-calling.
- The focus on sandboxing and controlled execution environments is critical for productionizing AI agents, requiring careful consideration of resource management (CPU/memory) and security (secrets/permissions).
- The ability to measure agent performance using structured evaluation frameworks (like Harbor) provides quantifiable metrics (tokens, cost, pass rate) for iterative model improvement and reinforcement learning.

## Topics

AI Agents, Large Language Models (LLMs), Software Engineering, Sandboxing, Observability, Arize Phoenix, Harbor

Source: https://www.youtube.com/watch?v=dAoOYFx7ZYw