Harness Engineering: Building an AI Software Factory
Summary
Harness engineering is a discipline focused on building automated loops of checks and maintenance agents that allow teams to delegate increasing amounts of codebase development to AI. The goal is not merely higher velocity, but achieving higher quality by shifting from manual code review (the primary bottleneck) to systematic process oversight. This involves tracking three key dimensions—autonomy, automation, and quality—and implementing layered validation systems: the Inner Loop (unit tests/linters), Outer Loop (agentic QA/UI testing), and Meta Loop (maintenance agents that analyze historical data for systemic improvements).
Key takeaways
-
The Three Dimensions of Agent Adoption
10:39
When adopting AI agents, track three metrics: Autonomy (how many human course corrections are needed); Automation (the level of oversight required, indicating trust); and Quality (ensuring the shipped product remains high quality). Progressing requires improving these dimensions sequentially.
-
The Three Loops of Harness Engineering
19:04
1. Inner Loop (Autonomy): Focuses on cheap, frequent checks like pedantic linting or unit tests to ensure agents get it right the first time. 2. Outer Loop (Automation): Involves slower, in-depth checks, such as agentic code review or running the product through a UI/CLI. 3. Meta Loop: Utilizes maintenance agents that analyze historical data (CI logs, PR comments) to propose systemic fixes and improvements to the entire process.
-
The Primary Barrier is Organizational
28:20
Harness engineering is fundamentally an organizational transformation, not just a technical one. Success requires changing workflows—for example, moving from monolithic PRs to smaller, low-risk chunks that can auto-merge, thereby shifting human behavior toward better practices.
Technical details
-
Software Factory Definition
450s
An end state where an agent system (a series of loops) builds the product, and human effort is focused on shaping tickets and improving the system's autonomy, automation, and quality.
-
Inner Loop Development
1240s
Improving unit tests and linters to be highly pedantic, ensuring that every agent mistake can be captured by a cheap, automated check (e.g., 'If you see an agent make a mistake, think: how do I put this into a check so that it never happens again?').
-
Outer Loop Development
1350s
Implementing deep validation processes like agentic code review or building tools to allow agents to click through the product UI and take screenshots for human review.
-
Verifiers (Advanced Linting)
2180s
A concept of very small, cheap, fast LLM-powered linting rules designed to move complex checks out of expensive agentic reviews and into the inner loop, significantly reducing cost and increasing speed.
Mentioned resources
- Tessl
- Claude Code / Codex / Cursor CLI
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.