Tokens Are the Cheap Part — Steve Jamison, Jellyfish
Summary
The cost efficiency of AI-driven development is not measured by token expenditure or raw throughput alone. By incorporating developer salaries into the cost model, the analysis shows that despite a 60x increase in AI spending, the total allocated cost per merged Pull Request (PR) fell 26% over the last year. This decline is primarily driven by increased developer productivity (43% more merged PRs per engineer), which spreads the fixed salary cost across more output. The core finding is that AI is currently a relatively inexpensive resource compared to human labor, making it a strong investment for increasing throughput.
Key takeaways
-
Cost per PR is the critical metric
4:08
The most accurate measure of efficiency is the total cost per merged PR, which must combine both AI costs and developer salaries. AI is currently only a small fraction (around 2.5%) of the total cost to produce a PR, meaning 'tokens are the cheap part.'
-
Throughput gains offset rising AI costs
5:30
Over the past year, the dollar cost of a merged PR fell 26%, even with a 60x increase in AI costs. This indicates that increased developer throughput (43% more merged PRs per engineer) is spreading the salary allocation thinner across more work.
-
Heavy AI spenders see the greatest gains
10:50
Companies in the top 20% of AI spend (the 'heaviest spend group') saw the largest improvements: a 67% increase in PR throughput and a 32% decrease in cost per PR, compared to 40% throughput gains and 25% cost decreases for lower spenders.
-
AI increases lines of code, not just PR count
The increase in PR throughput is not driven by smaller chunks of work. Instead, AI is contributing to an increase in lines of code generated per PR, suggesting more complex, larger changes.
Technical details
-
Cost Model
290s
The total cost to produce a PR is defined as (Developer Salary Cost + AI Spend). AI spend is estimated to be 2.5% of the total cost, while salary accounts for the remaining 97.5%.
-
AI Spending Metrics
330s
AI costs increased 60x over the last year. Token usage was analyzed using quintiles: the lightest group used about 0.3% of the PR cost (approx. 60 million tokens/dev/month), while the heaviest group used 8.5% (approx. 2.5 billion tokens/dev/month).
-
Productivity Metrics
790s
The analysis compared developer weeks with heavy agent skill use versus low use. Developers in the heavy use group spent twice as much on tokens, merged 27% more PRs, and saw an 18% reduction in cost per PR (when salary and tokens were combined).
Mentioned resources
- Jellyfish Platform
- Substack
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.