# Codex, Behind the Harness — Dominik Kundel, OpenAI

## Executive summary

The Codex harness provides a comprehensive framework for building advanced, agentic AI systems. The system relies on two open protocols—the App Server (for UI-to-harness communication) and the Responses API (for harness-to-inference)—allowing developers to build custom agents regardless of their interface. Key features include sophisticated context management (using deferred tools and capping available skills), robust action capabilities (async tasks, code execution for computer use, and file system interaction via an 'apply patch tool'), and mandatory sandboxing layers (e.g., Seatbelt on macOS, Bubblewrap on Linux). Furthermore, the architecture addresses real-world enterprise concerns like security (Auto Review subagents) and performance (WebSocket mode and Auto Compaction), making it suitable for complex, long-running build processes.

## Key takeaways

- Open Ecosystem Protocols: The Codex harness is built on open standards: the App Server protocol (UI to harness) and the Responses API (harness to inference). These protocols allow developers to build custom UIs or integrate with different model providers, ensuring flexibility.
- Advanced Context Management: To manage context size and maintain cacheability, the system uses 'deferred tools,' which are not added directly to the context window but are surfaced via tool search. The available skills list is capped at 2% of the total context window.
- Secure and Robust Actions: Actions are handled through specialized tools: file edits use an 'apply patch tool,' while general navigation uses a shell tool (often defaulting to `ripgrep`). All interactions occur within mandatory sandboxes (e.g., Seatbelt on macOS, Bubblewrap on Linux).
- Mitigating Approval Fatigue: An 'auto review subagent' is implemented to automatically judge high-risk actions against user authorization and the transcript context, reducing reliance on manual full-access approvals.
- Optimized Communication: The system transitioned from Server-Sent Events (SSE) over HTTP to WebSocket mode. This persistent connection saves network overhead and provides stateful context, ensuring only changed data is transmitted.

## Technical details

- Protocols & Architecture: The Codex harness uses two primary protocols: the App Server (for UI interactions) and the Responses API. The Responses API was redesigned to support complex agentic capabilities like web search and image generation, and it is designed as an open schema for partners (e.g., Ollama, LM Studio).
- Context Construction & Efficiency: To balance context size, flexibility, and cacheability, the system implements deferred tools (available only via tool search) and limits the available skills list to a maximum of 2% of the total context window. This is possible since GPT-5.4.
- Agent Actions & Execution: Agents perform actions through specialized tools: Async tasks (e.g., sub-agents via `spawn agent` tool), Computer Use (now flexible enough to use code execution in JavaScript or Python, demonstrated with Playwright for browser interaction), and File System interactions (using an 'apply patch tool' for diffs).
- Security & Sandboxing: All file system interactions are routed through a sandbox layer. Specific sandboxes include Seatbelt (macOS), Bubblewrap (Linux), and a custom open-source solution built by the team for Windows.
- Long-Horizon Goal Setting: For complex, multi-step goals, the system injects a continuation prompt until the model explicitly calls an `update goal tool`, requiring concrete and verifiable objectives rather than general essays.

## Practical implications

- Build engineers can leverage the open-source Codex harness (Apache 2 licensed, written in Rust) as a blueprint for building their own secure and scalable agentic workflows.
- The ability to use code execution (JavaScript/Python) within an agent allows for highly flexible 'computer use' capabilities, moving beyond simple tool calls.
- Implementing auto-review subagents provides a critical security layer for production agents, mitigating risks associated with full access permissions during long-running tasks.

## Topics

AI Agents, Build Engineering, LLM APIs, Software Architecture, Security Sandboxing, Rust Programming, Codex harness, App Server Protocol, Responses API

Source: https://www.youtube.com/watch?v=shRR1e2HXMk