# Give the Agent a Budget, Not a Token — Sachin Malhotra, Anthropic

## Executive summary

The presentation argues that granting AI agents 'unbounded power' via simple tokens is dangerous. Instead of narrowing the token scope (a boolean fix), engineers must implement a comprehensive 'budget' system with four dimensions: how much, how fast, what can be undone, and who notices. Key solutions include using asymmetric verbs (prioritizing loud failures), enforcing rate limits on writes, implementing trip wires over static allow lists, and utilizing an 'undo test' to size the overall safety policy.

## Key takeaways

- Budget vs. Token: A token is a boolean (yes/no) scope; a budget is multi-dimensional, considering volume, velocity, reversibility, and observability. The failure was giving the agent unbounded power, not the model itself.
- Asymmetric Verbs: Prioritize granting agents access to operations that fail loudly (e.g., unskipping a test, which causes CI to go red) and keep critical failure verbs (like skipping a test) reserved for human intervention with an audit trail.
- Rate Limits & Trip Wires: Implement rate limits on every write operation, ensuring the ceiling refills automatically. Use trip wires (monitoring aggregate behavior) instead of static allow lists, as trip wires adapt to real-world data.
- The Undo Test: This test asks if the agent can autonomously roll back its own changes and what the blast radius would be if it failed. If not, a second key (human involvement) and an audit record are required.

## Technical details

- Agent Safety Architecture: The core principle is that all agent actions must pass through a proxy layer. This proxy stamps every outbound call with the agent's identity, preventing the agent from changing headers to bypass rate limits or ownership checks.
- Proxy Functionality: The infrastructure layer (proxy) is responsible for enforcing deterministic rules (e.g., returning a 403 when a budget is crossed), regardless of the prompt's content or intent.
- CI/CD Workloads: The incident involved an agent deleting approximately 200 workloads in 90 seconds, impacting about 20 engineers. This highlights the risk of unconstrained cleanup actions.

## Practical implications

- Implement a proxy layer to enforce identity stamping and resource budgeting on all agent outbound calls.
- Shift focus from defining allowed resources (allow lists) to monitoring aggregate behavior (trip wires).
- Design failure verbs such that the most catastrophic failures are those requiring human intervention, not automated action.

## Topics

AI Agents, DevOps Automation, CI/CD Security, Rate Limiting, Policy Enforcement, Anthropic, CI team at Entropic

Source: https://www.youtube.com/watch?v=rbjWzZK2LU0