How We Built an Agent That Improves Itself — Zubin Aysola, Weights & Biases

An agent harness changes what a model can accomplish: the tools it receives, the context it sees, and the way its work is packaged all affect performance. At Weights & Biases, Zubin Aysola’s team builds ARIA to do research inside the platform. Improving that agent creates a measurement problem almost immediately. Benchmarks, evaluations, and agent configurations all change together. A higher score can reflect a better agent, a different test, or a different way of judging the result.

The useful comparison spans two environments: the deployed agent handling real requests and the offline agent working through simulations. Those simulations need to reproduce enough of production to make improvements meaningful outside the benchmark. This is the familiar simulation-to-real-world problem applied to software agents: success in a constructed environment matters only insofar as it carries into actual use.

W&B Weave connects the two sides through tracing. Offline runs record ARIA’s behavior in simulated environments; production runs log the same kinds of information in the same format. That shared representation lets a production conversation become material for an offline experiment. Once ARIA can inspect traces and run those experiments itself, it can participate in the research loop used to improve its own behavior.

The live demonstration begins with a project for ARIA researching itself. ARIA receives a codebase stored as a Weights & Biases artifact and a request to work against the offline evaluation framework. Its job is to inspect production traces, add tasks, run experiments, and try a new variant. The evaluations take time, so the research continues while the presentation moves on.

A second conversation gives the loop a concrete input: take a trace already logged in Weave, put it into the offline evaluations, and run both the candidate and production agents on it. The original interaction now serves two purposes. It records what happened in production, and it supplies a task on which competing versions can be compared. As the evaluations run, their traces appear in Weave too.

This is also the team’s recurring workflow: inspect production traces, evaluate changes in the offline sandbox, and work together to deploy new agent versions. Nightly CI evaluates the production configuration alongside candidate variants, providing a history of relative performance. That infrastructure needs attention of its own; Aysola notes that CI had broken and ARIA had helped fix it the previous night.

The team mostly improves prompts and skills rather than training the agent with reinforcement learning. Strong underlying models already perform useful platform tasks. The research discipline still borrows from simulation-based learning: define environments carefully, run the agent through them, inspect its trajectory, and adjust behavior against measured results.

The baseline is a byte-for-byte identical agent in production and simulation. Research and production use the same code, with a synchronization job every four hours intended to prevent the research environment from drifting away as people create new variants and skills. This removes one avoidable source of confusion: an offline improvement should be tested against the agent actually being deployed.

Evaluation commands produce scored trajectories: records of the agent’s rollout over time, accompanied by performance metrics. These give the team something more useful than a final answer alone. A person can inspect a rollout, or ARIA can review its own or another rollout, identify what went well or badly, and propose prompting changes that encourage the desired behavior.

The harness makes experimentation cheap to express. A model-agnostic software stack handles context preparation, compaction, and UI payload assembly. YAML defines different agent configurations, allowing multiple variants to run in parallel. An intuition about a better prompt or model becomes a candidate that can be compared with alternatives. The preference is straightforward: make it easy to run more experiments.

ARIA’s sandbox gives it broad room to execute work. While preparing the presentation, Aysola asked it to create an environment for parallel executions of its own research loop—a capability he had not been certain it could perform. The attraction is that a general execution environment can support useful behavior that was not individually designed into a narrow tool interface.

A general sandbox still needs a repeatable evaluation setup. The pipeline begins with YAML, loads the live data required by the task, and constructs the environment. That setup can be expensive: tasks may involve full machine-learning training logs, substantial production data, or simulated GPU executions for research work. Parallelizing preparation can therefore matter before the agent even starts.

Why load configuration twice? Some runtime configuration cannot be encoded in the initial YAML specification. After creating the environment, the pipeline puts that runtime data back into the configuration, then runs the production-identical agent and scores the result. The diagram makes this dependency visible: execution waits for both the declared task and the environment’s runtime configuration.

Scoring has two distinct jobs:

• Task completion: A pass/fail judgment asks whether the agent satisfied the task.
• Relative behavior: A comparison asks which variant behaves better. For example, one variant may ask the user questions while another proceeds without asking; relative scoring helps compare those interaction styles.

The team also examines how evaluations drift from production. Identical agent code helps, but the environments and judgments still need scrutiny.

Finally, each run tears down its environment. Cleanup makes repeated parallel experiments possible without clobbering teammates’ work. The full unit of evaluation includes preparation and teardown, as well as the agent call and score.

An evaluation task describes a starting environment, user configuration, and desired ending condition. That makes the task a user flow with an outcome, rather than a prompt considered in isolation. Two concrete forms appear in the walkthrough:

• Single-request tasks: A text instruction asks ARIA to perform work, such as the research request being converted from the live production trace.
• Simulated conversations: A language model takes a user persona and asks questions in a specified order, creating multi-turn interactions with ARIA.

The team has 886 tasks, categorized by levels and exposed to the product team for review. Product judgment helps determine whether those tasks represent behavior worth measuring. Repeated runs then produce trajectories that can be examined across production and offline environments. Aysola sees conceptual guidance as an existing strength and project error analysis as an area to improve, so the suite includes difficult tasks that push those capabilities.

Failures supply new tasks, but successful production behavior does too. A good interaction is something to preserve and reinforce while changing the agent elsewhere. The evaluation flywheel therefore draws from both kinds of experience: capture behavior, turn it into a task, and use the task to guide the next variant.

Returning to the demo reveals the observable change: the production trace has become a new task, and ARIA has run and scored it. The research agent also generates a report linking its work. A conversation that previously existed as production history now has an offline evaluation counterpart, including a run of the production variant.

The regression task exposes a specific problem: improper use of the SDK call weave.log in the sandbox. ARIA reproduces the source trace, identifies a target for improvement, and runs agent variants against it. The closing inspection describes the candidate as a small prompt added to the system prompt or a skill to address that SDK error. The demonstrated change is therefore a targeted instruction change; the walkthrough does not establish a source-code patch or quantify a candidate-versus-production gain.

How does that error become something the agent can improve? The diagram follows the same interaction from a recorded failure to a repeatable task and competing executions. The important relationship is the shared test: the proposed instruction change is evaluated against the production variant on the regression task derived from the original trace.

This changes the day-to-day research workflow. Instead of repeatedly writing offline benchmarks by hand, Aysola moves between incoming production traces and the team’s offline evaluations, asking what changed in a skill or variant and inspecting the resulting runs. ARIA handles more of the task construction and execution; the researcher keeps choosing which behavior deserves attention.

“Auto mode” is tempting: let the agent write the implementation, run the evaluations, and do everything it can. Aysola describes relying on Claude to write his code, but the convenience does not remove the need to think through improvements. Automating execution frees time for the harder questions: what should become better, which behavior matters, and how should the system be constrained?

The same research capabilities extend to machine-learning work; Aysola mentions other demonstrations of ARIA training models on H200 GPUs. For this example, though, the practical gain is keeping production observation, offline experiments, and ongoing agent research together. ARIA runs evaluations on the researcher’s behalf while the researcher considers which guardrails will make the system more useful.

The final inspection returns to the actual execution: prediction and scoring work, tool calls, and the candidate’s small prompt change for the SDK error. That is the concrete meaning of an agent improving itself here. Production supplies an interaction to reproduce; simulation makes it testable; the agent proposes and evaluates a change. People still shape the improvement pattern that this machinery follows.