← All speakers

Tim Sweeney was identified in the 2026 AI Engineer World’s Fair biography as a Principal Engineer at Weights & Biases by CoreWeave. His main-stage session, “Closing the Loop: An Autonomous AI Research Agent,” explored autonomous AI research agents and the feedback loops behind them. In his introduction, Sweeney reported having a master’s in machine learning and reinforcement learning from Georgia Tech and previously serving as PM of Twitter’s ML stack.

Sweeney demonstrated ARIA iterating on code and hyperparameters, launching GPU experiments through W&B Launch, summarizing strong training runs, identifying experimental patterns, and creating reports and workspaces with built-in charts. He explained how the team improves the agent by logging traces to Weave, reviewing conversations with humans, and using live LLM judges to flag user frustration and low-quality responses. Those observations become YAML evaluation tasks that check correctness, insight quality, and tool-call efficiency; roughly 200 tasks run nightly to inform release decisions. His approach emphasizes observing agent behavior, treating evaluations as continuous integration, retaining human judgment, and improving domain context and tools before adding complexity to the agent harness.

1 conference talk

Key ideas

Scroll to read ↓

ARIA turns a research request into GPU jobs, experiment analysis, and visual reports inside Weights & Biases. A live batch shows how that loop works; the team's own traces and nightly evaluations show how they decide whether the agent itself is improving.

  • Keep long-running training outside the agent's main loop: ARIA starts experiments through Launch and polls while GPU jobs execute.
    5:19 ↗
  • Evaluate useful research behavior along separate dimensions: the example task checks correctness, interesting insights, and a six-tool-call limit.
    16:47 ↗
  • Production traces become more useful when human review and live judges turn observed behavior into repeatable tasks and candidate evaluations.
    18:13 ↗
  • Domain context and available tools are practical places to improve an agent before adding elaborate harness or memory machinery.
    19:51 ↗
  • The live batch completed 12 experiments and nearly matched the earlier best result; autonomous execution did not guarantee a better model.
    20:27 ↗

References