# Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA

## Executive summary

The presentation introduces reflective optimization, a method for self-improving AI agents and models using textual feedback, addressing the critical bottleneck of sample efficiency. The core techniques, GEPA (Generative Prompt Enhancement Algorithm) and Optimize Anything, allow systems to learn from the full trace of an agent's execution (including chains of thought, tool calls, and errors), rather than just a single reward score. This enables significant performance gains (e.g., lifting Gemini Flash on ARC-AGI from 32.5% to 89.5%) by optimizing prompts, agent architectures, and even code/policies, all while maintaining a Pareto pool of candidate solutions to avoid local optima.

## Key takeaways

- GEPA Outperforms Traditional RL: GEPA achieved twice the performance gains of GRPO (a leading RL technique) using only three data points and one round of reflection, compared to GRPO's 25,000 rollouts. This demonstrates superior sample efficiency. (4:50)
- Optimization Beyond Prompts: Optimize Anything: The reflective optimization process can be applied to any system that can be expressed as text and scored, including entire agent harnesses (Python/JavaScript files), CUDA kernels, or cloud scheduling policies. (10:35)
- Significant Performance Gains Demonstrated: GEPA was able to optimize an agent for ARC-AGI, boosting Gemini Flash's accuracy from 32.5% to 89.5%. It also improved GPT-5 mini's Go issue resolution from 24% to 93%. (13:35, 15:10)
- Pareto Pool Strategy Beats Simple Loops: Maintaining a Pareto pool of candidates ensures a more balanced search process, preventing the model from getting stuck in local optima, which is a major advantage over simple iterative loops. (9:00)

## Technical details

- Sample Efficiency Bottleneck: Traditional AI training (SFT, RL) requires massive amounts of data (trillions of tokens, hundreds of thousands of rollouts). Current challenges are limited domain-specific data and expensive, slow agentic rollouts. (1:00)
- Reflective Optimization in Text Space: Instead of relying solely on a binary reward signal (0 or 1), reflective optimization uses an LLM or agent to read the entire rollout trace (including chains of thought, tool calls, and error messages) and generate textual feedback to improve the system. (3:00)
- GEPA Mechanism: GEPA is a reflective prompt optimization technique that uses an evolutionary loop and a novel Pareto-based candidate selection. It treats prompt updates as a form of 'reinforcement learning in text space.' (4:25)
- Optimize Anything API: This universal API extends reflective optimization to any text parameter. It requires a fitness function that returns a score and any available domain-specific side information (e.g., compiler error messages, job traces, SLA violations). (10:35)

## Practical implications

- Automating the discovery of optimal prompts and agent architectures, eliminating the need for manual, time-consuming human engineering.
- Improving complex, multi-step agents (e.g., solving ARC-AGI) by optimizing the entire agent harness, not just the initial prompt.
- Achieving massive cost reductions (e.g., Databricks tuning an open model to beat Claude Opus at 90x lower cost) by optimizing models on cheaper, open-source hardware.
- Applying optimization techniques to non-text domains like CUDA kernel code or cloud scheduling policies, provided they can be serialized and scored.

## Topics

AI Optimization, Reinforcement Learning, Large Language Models (LLMs), Agentic Systems, Prompt Engineering, Machine Learning Efficiency, GEPA on GitHub

Source: https://www.youtube.com/watch?v=OA-Mc60Rboo