The State of AI in Software Development: Data from 400+ Orgs — Justin Reock, DX
This presentation analyzes the impact of AI on developer productivity using data from 200,000 engineers. While AI has increased deployment frequency and code maintainability, the data reveals critical tensions: Change Confidence has dropped 6%, and PR size has significantly increased (from ~44 to 72 lines). The core finding is that code generation was never the bottleneck; instead, non-AI factors like meeting overhead and context switching are the primary constraints on value generation. DX proposes a measurement framework focusing on Utilization, Impact, and Cost, and emphasizes that improving foundational Developer Experience (DX) metrics—such as reliable CI and modular code—is crucial for maximizing agent efficiency.
Key takeaways
-
Change Confidence vs. Maintainability
10:02
Code maintainability has risen by nearly 4%, but Change Confidence has dropped 6%. This suggests developers feel more capable of understanding AI-generated code but are more hesitant to trust it, indicating a psychological shift in risk perception.
-
PR Size and Incremental Delivery
11:42
Average Pull Request (PR) size has increased from approximately 44 to 72 lines. This trend, coupled with a 10% drop in the perception of incremental delivery, suggests developers are consolidating changes, which increases the risk of bugs and makes code less portable.
-
AI Efficiency by Role
14:02
While junior engineers use AI the most, staff+ engineers are achieving comparable time savings while consuming fewer tokens, suggesting that deep architectural understanding is key to efficient AI utilization.
-
The True Bottleneck
The median increase in PR throughput was only about 7.7%, far from the expected 2x gain. The speaker asserts that time savings from AI are often outweighed by non-AI factors like meeting overhead and context switching, which are the true constraints on value generation.
-
Agent Readiness Requires Good DX
The speaker argues that improving foundational developer experience (DX) metrics—such as clear documentation, modular code, and reliable, non-flaky test suites—is necessary to build effective AI agents.