FinOps for AI Agents: Who Spent All the Tokens? — Tisha Chawla & Susheem Koul, Microsoft
The talk introduces TokenOps, a control plane designed to shift AI agent development from 'token maxing' (spending tokens) to 'value maxing' (maximizing value per token). It addresses the critical gap in current systems: the lack of cost governance between code execution and model calls. TokenOps operates out-of-band at the entire agent run level, utilizing a `boundary annotation` and `governor node` to implement sophisticated policies that can 'steer' an agent's behavior (e.g., instructing it to be more succinct) before hitting a budget cap, thereby preventing costly failures.
Key takeaways
-
Shift from Token Maxing to Value Maxing
The industry needs to move beyond simply spending tokens and focus on ensuring that every token spent has measurable business value. This requires proper attribution of costs back to specific agent runs.
-
Run-Level Cost Control is the Missing Piece
2:38
Existing tools (like model gateways) only control cost at the request level. TokenOps provides governance at the entire agent run layer, allowing control over complex loops and context growth.
-
Steering vs. Halting
Instead of simply halting an agent when a budget is exceeded (a circuit breaker), the 'steer' action uses a cost guard to predict overruns and injects instructions into the system prompt, guiding the agent toward more efficient outputs.
