# Tokens Are the Cheap Part — Steve Jamison, Jellyfish

## Executive summary

The cost efficiency of AI-driven development is not measured by token expenditure or raw throughput alone. By incorporating developer salaries into the cost model, the analysis shows that despite a 60x increase in AI spending, the total allocated cost per merged Pull Request (PR) fell 26% over the last year. This decline is primarily driven by increased developer productivity (43% more merged PRs per engineer), which spreads the fixed salary cost across more output. The core finding is that AI is currently a relatively inexpensive resource compared to human labor, making it a strong investment for increasing throughput.

## Key takeaways

- Cost per PR is the critical metric: The most accurate measure of efficiency is the total cost per merged PR, which must combine both AI costs and developer salaries. AI is currently only a small fraction (around 2.5%) of the total cost to produce a PR, meaning 'tokens are the cheap part.'
- Throughput gains offset rising AI costs: Over the past year, the dollar cost of a merged PR fell 26%, even with a 60x increase in AI costs. This indicates that increased developer throughput (43% more merged PRs per engineer) is spreading the salary allocation thinner across more work.
- Heavy AI spenders see the greatest gains: Companies in the top 20% of AI spend (the 'heaviest spend group') saw the largest improvements: a 67% increase in PR throughput and a 32% decrease in cost per PR, compared to 40% throughput gains and 25% cost decreases for lower spenders.
- AI increases lines of code, not just PR count: The increase in PR throughput is not driven by smaller chunks of work. Instead, AI is contributing to an increase in lines of code generated per PR, suggesting more complex, larger changes.

## Technical details

- Cost Model: The total cost to produce a PR is defined as (Developer Salary Cost + AI Spend). AI spend is estimated to be 2.5% of the total cost, while salary accounts for the remaining 97.5%.
- AI Spending Metrics: AI costs increased 60x over the last year. Token usage was analyzed using quintiles: the lightest group used about 0.3% of the PR cost (approx. 60 million tokens/dev/month), while the heaviest group used 8.5% (approx. 2.5 billion tokens/dev/month).
- Productivity Metrics: The analysis compared developer weeks with heavy agent skill use versus low use. Developers in the heavy use group spent twice as much on tokens, merged 27% more PRs, and saw an 18% reduction in cost per PR (when salary and tokens were combined).

## Practical implications

- When assessing AI investment, track the total 'intelligence cost' (human salary + AI spend) per PR, rather than focusing solely on the AI bill or throughput.
- Use the cost per PR metric to determine if increased AI spending is generating genuine cost efficiencies, rather than just increasing the token bill.
- Management should track PR throughput, cost per PR, and quality metrics (like reverts) simultaneously to understand the full trade-off.
- Consider the total cost of ownership when evaluating AI tools, including the cost of integrating model routers or post-training custom models.

## Topics

AI Cost Management, Software Engineering Productivity, Build Metrics, Total Cost of Ownership (TCO), Developer Workflow Optimization, Jellyfish Platform, LinkedIn, Substack

Source: https://www.youtube.com/watch?v=U7ZYBCcdJPg