Agents Write 95% of Our Code. Here's the Catch
As AI agents assume control over an estimated 95% of code production in advanced software factories, traditional code review processes are insufficient. The talk introduces the role of the 'harness engineer,' a new skill set focused on system-level controls: defining invariants, performing deep analytics on agent logs and PR data, and implementing fine-grained risk/operations policies (like auto-merge ladders). This shift requires engineers to move from writing code features to building robust guardrails that ensure consistency and quality across agent-driven pipelines.
Key takeaways
-
The Paradox of AI Adoption
25:24
While AI coding tool adoption is high, benchmarks are becoming saturated. Concurrently, the number of reported bugs and incidents is rising, indicating that agents may generate code that lacks maintainability or systemic health (00:15:24).
-
The Rise of the Harness Engineer
9:34
Engineering focus must shift from pure feature building to defining and enforcing system invariants. The three critical new skill sets are Systems Thinking, Analytics, and Risk/Operations (00:09:34).
-
Instruction Following Gap in Skills
8:23
Tessl's internal skills benchmark revealed that while agents achieved high task completion rates, they only followed approximately 70% of the total instructions defined within a skill (00:08:22).
-
Systemic Control through Invariants and CI Gates
12:56
Engineers must identify general principles (invariants)—such as design system rules or desired code structure—and encode them into deterministic checks, verifiers, or CI gates to ensure consistency across the codebase (00:12:56).