▶ Watch ↗AI Engineer World's Fair 202621:16
An AI Research Agent That Runs Your Experiments — Tim Sweeney, Weights & Biases
Read the full talk →Key ideas
Scroll to read ↓ARIA turns a research request into GPU jobs, experiment analysis, and visual reports inside Weights & Biases. A live batch shows how that loop works; the team's own traces and nightly evaluations show how they decide whether the agent itself is improving.
- Keep long-running training outside the agent's main loop: ARIA starts experiments through Launch and polls while GPU jobs execute.5:19 ↗
- Evaluate useful research behavior along separate dimensions: the example task checks correctness, interesting insights, and a six-tool-call limit.16:47 ↗
- Production traces become more useful when human review and live judges turn observed behavior into repeatable tasks and candidate evaluations.18:13 ↗
- Domain context and available tools are practical places to improve an agent before adding elaborate harness or memory machinery.19:51 ↗
- The live batch completed 12 experiments and nearly matched the earlier best result; autonomous execution did not guarantee a better model.20:27 ↗