# Inside OpenAI DevDay: Superhuman Computer Use, Decisions API, and the AI Cloud — Ari & Nikunj

## Executive summary

OpenAI announced significant advancements in AI agent capabilities, particularly in 'Computer Use,' aiming for superhuman performance in software operation. Key announcements include **Dots**, a personal assistant product giving agents access to private Linux virtual machines in the cloud, and **GPT-6.1**, a new model offering substantial cost and speed improvements. For developers, the **Agents API** provides a unified platform for building on Computer Use, while the new **Decisions API** offers ultra-fast, low-latency classification and structured output, inspired by the recent 'Jev' model. The overall trend is moving toward an 'AI cloud' with higher-level primitives for memory, compute, and state management.

## Key takeaways

- Computer Use is Superhumaning: Computer Use agents are dramatically improved, moving from human-level to potentially 'literally superhuman' performance in real-world software tasks. This is achieved by combining multiple modalities: screenshots, accessibility data, the DOM, Playwright, and generated JavaScript code, allowing agents to debug and recover from failures more effectively. (Transcript)
- Dots and Personal Computing: The 'Dots' product gives agents access to their own dedicated Linux virtual computer in the cloud, enabling them to run full desktop applications and interact with complex, non-API-exposed software, expanding agent utility beyond simple web browsing. (Transcript)
- New Developer APIs for Scale: New API features include **async tool calling**, **mid-turn steering**, and **WebSockets**, which enable bidirectional, real-time communication with the model. The **Agents API** centralizes Computer Use capabilities, allowing developers to build on a consistent, high-performance foundation. (Transcript)
- Decisions API for Speed: The Decisions API is designed for extremely fast classification and structured output. It operates in parallel and is a smaller model than those used for complex Computer Use tasks, making it ideal for high-throughput, low-latency applications. (Transcript)

## Technical details

- Model & Performance: The new **GPT-6.1** model offers significant cost and speed advantages for Computer Use, reportedly being five times cheaper than Astra and seven times cheaper for specific Computer Use tasks. (Transcript)
- Context & Input: The **App Shots** feature enhances context by capturing not just a screenshot, but the raw metadata, accessibility representation, and DOM structure of an application, providing the Language Model (LM) with full context. (Transcript)
- API Optimization: Performance gains are driven by techniques like **async tool calling** (allowing the model to continue reasoning while a tool runs), **mid-turn steering** (injecting messages during model reasoning), and **WebSockets** (enabling bidirectional communication). (Transcript)
- Context Management: Developers can manage large context windows using **compaction** techniques. Options include server-side compaction (automatic) or manual control via the `/compact` function, which is crucial for maintaining efficiency in long-running agent threads. (Transcript)

## Practical implications

- Build engineers can now design complex, multi-step workflows that automate tasks previously requiring human interaction (e.g., booking flights, complex ordering).
- The combination of the Agents API and Computer Use allows for the development of automated Software Development Life Cycles (SDLC), where the agent can not only build code but also test it. (Transcript)
- The focus on low-latency APIs (like Decisions API) and caching guarantees (12-hour cache window) is critical for building reliable, production-grade agent applications.
- Developers should prioritize building on the unified Agents API to benefit from the optimized Computer Use harness and model training.

## Topics

AI Agents, Computer Vision, LLM APIs, Software Automation, Build Engineering, Cloud Computing, OpenAI DevDay

Source: https://www.youtube.com/watch?v=z9OkBD2-MDU