Codex, Behind the Harness — Dominik Kundel, OpenAI
The Codex harness provides a comprehensive framework for building advanced, agentic AI systems. The system relies on two open protocols—the App Server (for UI-to-harness communication) and the Responses API (for harness-to-inference)—allowing developers to build custom agents regardless of their interface. Key features include sophisticated context management (using deferred tools and capping available skills), robust action capabilities (async tasks, code execution for computer use, and file system interaction via an 'apply patch tool'), and mandatory sandboxing layers (e.g., Seatbelt on macOS, Bubblewrap on Linux). Furthermore, the architecture addresses real-world enterprise concerns like security (Auto Review subagents) and performance (WebSocket mode and Auto Compaction), making it suitable for complex, long-running build processes.
Key takeaways
-
Open Ecosystem Protocols
4:00
The Codex harness is built on open standards: the App Server protocol (UI to harness) and the Responses API (harness to inference). These protocols allow developers to build custom UIs or integrate with different model providers, ensuring flexibility.
-
Advanced Context Management
6:45
To manage context size and maintain cacheability, the system uses 'deferred tools,' which are not added directly to the context window but are surfaced via tool search. The available skills list is capped at 2% of the total context window.
-
Secure and Robust Actions
10:20
Actions are handled through specialized tools: file edits use an 'apply patch tool,' while general navigation uses a shell tool (often defaulting to `ripgrep`). All interactions occur within mandatory sandboxes (e.g., Seatbelt on macOS, Bubblewrap on Linux).
-
Mitigating Approval Fatigue
15:20
An 'auto review subagent' is implemented to automatically judge high-risk actions against user authorization and the transcript context, reducing reliance on manual full-access approvals.
-
Optimized Communication
17:15
The system transitioned from Server-Sent Events (SSE) over HTTP to WebSocket mode. This persistent connection saves network overhead and provides stateful context, ensuring only changed data is transmitted.
