Tokens Should Have Jobs — Katelyn Lesse & Angela Jiang, Anthropic
The presentation challenges the assumption that all tokens used in agentic systems are fungible. Instead of relying solely on increasing the token budget (brute force), the speakers propose giving tokens specific 'jobs' through advanced strategies. These strategies—Advising, Grading, and Dreaming—allow agents to improve performance and reliability within a fixed budget, particularly when the task requires high precision (e.g., financial analysis). The analysis showed that while increasing the budget improves performance across all methods, specialized strategies significantly reduce the true cost (tokens) required to achieve a perfect, reliable outcome.
Key takeaways
-
Tokens are not fungible
The core thesis is that tokens should be assigned specific jobs (e.g., advising, grading, dreaming) rather than being used indiscriminately for simple execution. This allows for more efficient use of computational resources.
-
Three core agent strategies
3:45
Strategies include: 1) Advising (splitting an executor from an adviser that provides mid-task guidance); 2) Grading (defining a rubric and using a grader to iterate until a perfect outcome is achieved); and 3) Dreaming (allowing tokens to inspect transcripts and write findings to memory for future runs).
-
Cost vs. Accuracy in Financial Analysis
11:40
When evaluating financial tasks (like creating a P&L statement), 80% accuracy is considered useless if 100% accuracy is required. The analysis showed that the baseline 'Execute' strategy required an average of 1.8 million tokens to achieve a perfect answer, while 'Advise' and 'Grade' were significantly more token-efficient.
-
Optimization depends on the goal
The choice of strategy depends on the optimization goal: choose 'Advise' for maximum token efficiency, or 'Grade'/'Dream' if maximizing the percentage of perfect, reliable runs is the priority.