Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA

Teaching an AI system a new task usually means updating its weights: during pretraining, supervised fine-tuning or reinforcement learning. Lakshya A. Agrawal opens with the practical obstacle to that approach. Most teams do not have the data and compute needed for large training runs, especially when the task involves unfamiliar domain knowledge or an agent that works for hours.

Two costs make sample efficiency—the amount learned from each example—central:

• Scarce domain examples. A specialized task may have too little existing knowledge or labeled data for supervised fine-tuning.
• Expensive attempts. Running the agent, waiting for its tools and measuring the result can all take substantial time. Repeating that process hundreds of thousands of times becomes impractical.

In the reinforcement-learning setup described here, a model produces several rollouts, an evaluator assigns rewards, and an algorithm such as GRPO converts those rewards into weight updates. The learning signal compresses a rich attempt into a scalar score. Yet the attempt also contains intermediate reasoning, tool calls, environment responses and error messages. A failure score tells the optimizer that something went wrong; an error message may tell it what went wrong.

Reflective optimization gives a language model the rollout itself. The model examines what worked and what failed, then proposes a change to the prompt. Reflection can use intermediate outputs and, potentially, retrieve supporting information from a company knowledge base or a guide. The update happens in text, where a short instruction can express a large behavioral change.

A summarizer makes the idea easy to see. Change its instruction from producing a one-line summary to producing a ten-line summary, and the requested output changes substantially through a tiny edit. The example illustrates the size of an update available in prompt space: reflection can identify a desired behavior and state it directly, rather than trying to obtain it through a sequence of small weight updates.

GEPA combines this reflection with an evolutionary search over candidate prompts. Each attempt supplies a score and textual feedback, including domain-specific information. In Agrawal’s presented comparison, one reflection round using three examples produced twice the performance gains that GRPO achieved after 25,000 rollouts. Further reflection increased the gap, and the model performed its own reflection without an external expert teacher. These and the later benchmark results are task-specific reported outcomes; the recording does not establish complete evaluation protocols or dataset splits, so they do not establish a universal advantage over reinforcement learning.

The useful output of reflection is often a clearer specification of the task: how to interpret the input, what a pipeline stage is supposed to accomplish, and which lessons the examples reveal. In a multihop question-answering system, the optimized second-hop prompt captured a division of work. First-hop documents often cover one entity or aspect of the question; the next retrieval should find documents related to that information. The prompt teaches the stage how to contribute to the whole pipeline.

This automates part of the work teams otherwise do by repeatedly editing prompts and discovering the task specification themselves. Agrawal gives a run-time estimate of half an hour to one hour, depending on the pipeline. The approach also works through proprietary model interfaces: an optimized GPT-4.1 mini prompt outperformed GPT-4.1 on the math task presented.

The AMD example shows what a concrete learned correction can do. A coding agent had to program a new NPU through an unfamiliar API with very little information available online. Its initial performance was roughly 4%. GEPA changed the prompt, leaving the rest of the agent unchanged, and the reported result rose to roughly 30%. One discovered instruction told the agent to avoid an AMD library header that existed for NPU programming but did not work with the hardware generation being tested.

The causal change is specific: an apparently relevant library was unsuitable for this target, reflection discovered that restriction, and the prompt carried the restriction into subsequent attempts. GEPA found it in one step. The overall gain belongs to the optimized prompt as a whole; the example does not isolate how much of that gain came from this single instruction.

GEPA’s basic loop runs the current pipeline on a few examples, collects feedback and asks a model or agent to propose a better prompt. The important search decision comes afterward: which candidates remain available for future improvement? The Pareto pool keeps candidates that win on even one training example, rather than retaining only the highest overall scorer.

Why preserve a candidate that is not the best overall? In the search example, a simple improvement loop reached a better prompt, then repeatedly generated changes that failed to improve it. Each rejected change returned the search to the same incumbent until the budget ran out. A pool preserves other starting points with useful strengths on particular examples. Those alternatives let the search develop several branches instead of spending everything around one local optimum.

What information travels through this search, and where does diversity survive? The diagram separates the diagnostic feedback used to write a candidate from the example-level results used to retain candidates. Reflection supplies the edit; the pool determines which lines of improvement remain available.

Across four benchmarks, Agrawal attributes more than half of GEPA’s gains to this candidate-selection strategy and reports almost twice the gains of a model-in-a-loop approach. He then shows results spanning question answering, instruction following, claim verification and math. The lesson from the search comparison is narrower and useful: improving prompts requires both informative edits and a way to avoid repeatedly editing the same dead end.

A prompt is one text artifact that controls behavior. An agent’s Python or JavaScript program is another. Optimize Anything extends the same search to an object that can be represented as text and evaluated with a score. Instead of editing only instructions inside an agent, the optimizer can edit the program that determines its tools and control flow.

The evaluator has two jobs: measure the candidate and return information that can help improve it. The examples make that distinction concrete:

• CUDA kernels. Compilation and profiling can produce both a score and diagnostics about the code.
• Numeric parameters. Numbers can be serialized as text, making them editable candidates too.
• Cloud scheduling policies. A policy or heuristic can be scored on cost or efficiency, while job traces and service-level agreement violations explain where it failed.

The API accepts the problems to solve and a fitness function that returns a score plus an open-ended dictionary of side information. Expert feedback, compiler errors, profiler messages, tool errors and documentation can all go into that dictionary. This is the integration work that makes reflection useful: expose what the environment knows about the candidate, rather than reducing everything to whether it passed.

A playful example optimizes a Python script that renders a 3D unicorn to a PNG. It demonstrates that the candidate can be an image-producing program, rather than a prompt that returns text. The recording compares the resulting renderings, but does not explain the visual fitness function; the example illustrates the range of editable artifacts rather than a reproducible measure of visual quality.

Agent development usually involves choosing tools, writing their descriptions and arranging control flow. In the ARC-AGI example, the starting point was a four-line Python program that called a model to reason through the problem. After 16 reflection rounds, Optimize Anything produced a six-step agent, raising the reported Gemini Flash accuracy from 32.5% to 89.5%. The editable object was the agent file itself, so search could introduce new steps.

The generated agent induced rule hypotheses, synthesized code, executed and traced that code, debugged it and proposed revisions. It then ran the resulting code on the test inputs and returned the output. This changes how an answer is produced: the system can inspect the consequences of an executable hypothesis and revise it before returning a result. A second harness example on MATH-500 produced a two-step agent and a reported accuracy improvement of 20% for GPT-4.1 nano; the recording does not specify whether that figure means a relative increase or percentage points.

Repository skills offer a different place to store what reflection learns. The objective is to learn a skill from an agent trajectory that will help when a similar problem appears. In the Go repository issue-resolution example, learned skills raised a budget-constrained GPT-5 mini coding agent from 24% to 93%. The skills contained practical repository knowledge: where features live, how the repository is organized, how to invoke tests and which build system it uses.

Those skills also transferred to Claude Sonnet, where Agrawal reports 100% issue resolution on the tested tasks and almost a 50% reduction in resolution time. The proposed explanation is practical: carrying repository knowledge into the next attempt reduces repeated discovery and token use. The skills learned with a cheaper model could therefore help a different model work more efficiently. The open-source skill-learning feature is available through the GEPA repository.

Optimize Anything distinguishes three optimization modes by what the result must serve:

• Single problem. Improve one artifact, such as a particular matrix-multiplication kernel.
• Multitask search. Work on related problems, such as matrix-multiplication and dot-product kernels, where information may transfer between them.
• Generalization. Optimize using a set of examples, then deploy the prompt, skill or agent architecture on new problems.

The production examples span different objectives. Agrawal reports almost 40% lower cloud scheduling costs compared with expert heuristics and almost 35% lower OCR error rates for leading models. He also describes Databricks tuning an open model to outperform Claude Opus on its agent task at 90 times lower cost. These examples put accuracy, error rate and operating cost on the same agenda: a successful optimization need not mean buying a larger model.

Does a better model make prompt optimization less useful? Agrawal’s answer is the opposite: stronger instruction following can increase the value of precise task instructions. In the presented comparison, Claude Opus gained more from optimization than the open model did. That supports his view that model capability and task specification can reinforce each other, although it does not establish how every future model will respond.

Subjective tasks complicate the requirement to score a candidate. The proposed starting point is production experience: collect agent trajectories and ask a human to annotate about 50 of them with detailed feedback. Comments can describe response length, quality, terminology or other task-specific preferences. GEPA then optimizes an LLM judge prompt using those annotations, and that judge evaluates candidates while the agent is optimized.

How does human feedback reach the next deployed agent? The cycle below makes the intermediate evaluator visible. Human annotations first shape the judge; the judge then guides agent optimization; deployment produces the next set of trajectories. The judge’s usefulness depends on how well the annotations capture the intended quality standard—this procedure makes that standard learnable, without making subjective evaluation objective.

The ending returns to weight training. In work called “Learning Fast and Slow,” Agrawal describes co-optimizing model weights and prompt harnesses for continual learning. He does not develop the update procedure in this recording. The proposal nevertheless changes the relationship established at the start: reflection in text can accompany weight updates, with both becoming parts of a learning system.

Agrawal closes with reports of production adoption and an integration claim: GEPA can work with different frameworks and models without hard dependencies. His practical advice is to expose as much actionable domain information as possible. A score selects the desired result; traces, errors, documentation and expert feedback give reflection something useful to change. That is the starting point for trying the approach through the open-source implementation.