Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA
Summary
The presentation introduces reflective optimization, a method for self-improving AI agents and models using textual feedback, addressing the critical bottleneck of sample efficiency. The core techniques, GEPA (Generative Prompt Enhancement Algorithm) and Optimize Anything, allow systems to learn from the full trace of an agent's execution (including chains of thought, tool calls, and errors), rather than just a single reward score. This enables significant performance gains (e.g., lifting Gemini Flash on ARC-AGI from 32.5% to 89.5%) by optimizing prompts, agent architectures, and even code/policies, all while maintaining a Pareto pool of candidate solutions to avoid local optima.
Key takeaways
-
GEPA Outperforms Traditional RL
4:50
GEPA achieved twice the performance gains of GRPO (a leading RL technique) using only three data points and one round of reflection, compared to GRPO's 25,000 rollouts. This demonstrates superior sample efficiency. (4:50)
-
Optimization Beyond Prompts: Optimize Anything
10:35
The reflective optimization process can be applied to any system that can be expressed as text and scored, including entire agent harnesses (Python/JavaScript files), CUDA kernels, or cloud scheduling policies. (10:35)
-
Significant Performance Gains Demonstrated
13:35
GEPA was able to optimize an agent for ARC-AGI, boosting Gemini Flash's accuracy from 32.5% to 89.5%. It also improved GPT-5 mini's Go issue resolution from 24% to 93%. (13:35, 15:10)
-
Pareto Pool Strategy Beats Simple Loops
9:00
Maintaining a Pareto pool of candidates ensures a more balanced search process, preventing the model from getting stuck in local optima, which is a major advantage over simple iterative loops. (9:00)
Technical details
-
Sample Efficiency Bottleneck
60s
Traditional AI training (SFT, RL) requires massive amounts of data (trillions of tokens, hundreds of thousands of rollouts). Current challenges are limited domain-specific data and expensive, slow agentic rollouts. (1:00)
-
Reflective Optimization in Text Space
180s
Instead of relying solely on a binary reward signal (0 or 1), reflective optimization uses an LLM or agent to read the entire rollout trace (including chains of thought, tool calls, and error messages) and generate textual feedback to improve the system. (3:00)
-
GEPA Mechanism
265s
GEPA is a reflective prompt optimization technique that uses an evolutionary loop and a novel Pareto-based candidate selection. It treats prompt updates as a form of 'reinforcement learning in text space.' (4:25)
-
Optimize Anything API
635s
This universal API extends reflective optimization to any text parameter. It requires a fitness function that returns a score and any available domain-specific side information (e.g., compiler error messages, job traces, SLA violations). (10:35)
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.