Stop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI

Published 2026-10-27 · Duration 14:38

Summary

Code Mode represents an architectural shift for AI agents, moving beyond traditional tool-calling by providing a secure, dynamic programming sandbox. Instead of accumulating tool catalogs and raw outputs in the context window, Code Mode allows agents to search for, compose, and execute tools (e.g., Bash, Python, SQL) within a controlled environment. This approach significantly reduces context bloat, lowers token usage, and improves efficiency, enabling complex, multi-step, and emergent behaviors previously difficult to achieve with standard agent frameworks.

Download summary

Key takeaways

Efficiency Gains via Code Mode
Code Mode significantly reduces context window bloat and overhead. By allowing agents to call multiple tools in a single turn and only returning necessary outputs, evaluations using the Harbor framework showed a 43% reduction in tokens and a 31% reduction in cost compared to traditional methods. (12:55)
Architectural Shift from Tool Catalog to Sandbox 5:03
Traditional agents rely on passing a large catalog of tools, leading to context bloat and increased latency. Code Mode provides a programming environment (sandbox) where agents can dynamically search for and compose tools (e.g., sequential, parallel, conditional flows), minimizing context overhead. (3:03)
Emergent Behavior and Composability 7:17
The sandbox enables emergent behavior, allowing agents to build their own tools or perform tasks (like PII removal) that require custom logic, rather than relying solely on pre-defined tool APIs. This composability is key to solving complex, multi-domain problems. (4:37)

Technical details

Code Mode Implementation (Phoenix) 611s
Arize built its agent harness within Phoenix, an open-source observability platform. This implementation supports multiple execution environments, including Bash, a Python sandbox (for REST CRUD endpoints), and SQL execution, enabling complex data manipulation and querying. (6:11)
Operational Challenges and Risks 819s
Implementing Code Mode introduces several operational complexities: managing CPU/memory in hosted sandboxes, handling network egress, securing secrets (e.g., using Vaulting strategies), managing permissions, and dealing with double sandboxing (where serialized JSON output must be parsed and put into another code mode). (8:19)
Evaluation Framework (Harbor)
Arize uses the Harbor framework to evaluate Code Mode agents. This process involves running agents in controlled, sandboxed environments with network access disabled, utilizing an RL framework (one or zero) to test task completion across multiple permutations and eliminate 'flake.' (11:44)

Mentioned resources

Arize Phoenix (Observability Platform)
Harbor (Evaluation Framework)

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.