AI Engineer
← All speakers

Bio, Work & Ideas

Zubin Aysola

Conference affiliation: Senior Software Engineer Weave · Weights & Biases by CoreWeave · 2026

Zubin Aysola was listed in the 2026 AI Engineer World’s Fair presenter record as a Senior Software Engineer on Weave at Weights & Biases by CoreWeave. His presented work centers on building and evaluating ARIA, an agent for research within the Weights & Biases platform, with particular attention to reliable measurement and translating production behavior into offline tests.

In “How We Built an Agent That Improves Itself,” Aysola explained how his team uses Weave traces and simulation environments to compare agent variants while keeping research and production agent code identical. Their model-agnostic harness supports YAML-defined configurations, parallel experiments, and both pass/fail scoring and relative comparisons of behavior. He described 886 evaluation tasks, including simulated multi-turn user interactions, and a feedback loop that turns production failures and successful behaviors into further tests.

His live demonstration showed ARIA converting a production trace into a regression task, identifying improper weave.log SDK usage in a sandbox, and testing a targeted prompt or skill change against the production variant. Aysola emphasizes that automating implementation and evaluation leaves people responsible for deciding what improvements matter and which guardrails the system needs.

1 conference talk

▶ Watch ↗

AI Engineer World's Fair 202617:06

How We Built an Agent That Improves Itself — Zubin Aysola, Weights & Biases

Read the full talk →

Key ideas

Scroll to read ↓

Zubin Aysola explains how Weights & Biases turns ARIA’s production traces into offline evaluation tasks, compares candidate variants with the deployed agent, and uses the results to guide its next improvement.

  • Use the deployed agent as the offline baseline, and keep research code synchronized with production so experiments measure relevant behavior.
    5:43 ↗
  • Score both task completion and relative behavior: passing a task and choosing a better interaction style answer different questions.
    10:15 ↗
  • Turn successful production interactions into tasks alongside failures, so future changes can reinforce behavior worth keeping.
    12:21 ↗
  • ARIA’s demonstrated self-improvement loop reproduces a production SDK error, creates a regression task, and tests a targeted prompt or skill change.
    13:17 ↗
  • Automating task construction and evaluation leaves people responsible for deciding what improvement means and which guardrails help.
    14:54 ↗

References