Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang
Summary
Emulated focuses on creating high-fidelity training data environments that simulate entire companies and complex infrastructure operations, moving beyond simple code diffs or single-node sandboxes. The core argument is that for AI agents to achieve true autonomy in mission-critical systems (like cloud providers), they must be trained on long-horizon tasks involving distributed cluster failures, resource provisioning across VPCs/subnets, managing cost constraints, and reasoning through real-world operational incidents.
Key takeaways
-
The Data Gap in AI Agents
3:30
Current benchmarks (e.g., SweBench Pro, Terminal Bench) limit agents to operating within a codebase, failing to capture the complexity of real-world tasks like PM communication, performance testing, or owning underlying infrastructure over years.
-
Complexity Requires Full Simulation
6:10
Real infrastructure work is not a simple code diff; it involves managing failing nodes, stale deprecated components, live traffic serving, and operational blast radius across distributed clusters.
-
Limitations of Single-Node Sandboxes
10:40
Standard post-training pipelines often use homogeneous single-node sandboxes. However, real cloud services require simulating resource provisioning (EC2, Cloud Run), VPCs, subnets, and security groups, which necessitates a multi-node sandbox with access to real infrastructure.
Technical details
-
Distributed Systems Simulation
260s
Emulated simulates distributed clusters in single sandboxes, allowing agents to deal with operational issues like network failures between nodes, data corruption (e.g., MVCC issues), and clock skew.
-
Cloud Infrastructure Provisioning
740s
Simulating cloud services requires modeling resource provisioning beyond the code base, including VPCs, subnets, security groups, front-end APIs (for throttling/auth), and deployment components for managing versions and rollbacks.
-
Operational Constraints
937s
Agents must reason about real-world constraints such as cost management, gradual deployments to limit blast radius, and maintaining service availability while running live traffic through complex migrations (e.g., migrating off old hardware).
Mentioned resources
- SweBench Pro
- Terminal Bench
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.