← All speakers

Fuad Ali is a senior product manager at Arize AI building tools that make AI agents easier to observe, evaluate, and improve in production. His work applies a background in distributed systems to a central problem of autonomous software: understanding what an agent actually did, where it failed, and how to make its next attempt better.

Ali studied electrical and biomedical engineering at McMaster University before engineering roles at Tesla, SpaceX, and Twitter. At Tesla, he worked on mobile infrastructure and energy products; at SpaceX, on flight software and autonomous systems; and at Twitter, on distributed systems supporting content moderation. He later joined Federato, progressing from senior engineering into product leadership, before moving to Arize in 2025. His professional portfolio traces that transition from infrastructure engineering to AI product development.

Building agents that improve with evidence

  • Coding-agent tracing and evaluation: Ali co-authored Arize’s introduction to coding-harness-tracing, an open-source project that instruments coding agents including Claude Code, Cursor, Codex, and GitHub Copilot. Capturing prompts, tool calls, file operations, failures, latency, and token usage turns opaque sessions into material for debugging, experiments, and better team workflows.
  • Evaluation-driven prompt improvement: Working with SallyAnn DeLucia on a prompt-learning workshop, Ali helped demonstrate how examples, human feedback, automated evaluations, and revised instructions can improve agent behavior iteratively. For subjective tasks such as judging property photographs, he recommends starting with a basic good-or-bad assessment, then introducing specific criteria such as lighting as evaluators accumulate useful signal.
  • Specialized agents and effective handoffs: Ali argues that agents in a larger workflow should refine instructions for their own responsibilities. Targeted improvement can strengthen handoffs and build expertise around a particular codebase without forcing every component into an overly generalized prompt.

Ali also co-hosts The Next Iteration with Damien Joseph, interviewing builders, founders, physicians, and researchers about technology, careers, health, and personal development.

Read the topics behind these talks

2 conference talks

Key ideas

Scroll to read ↓

SallyAnn DeLucia and Fuad Ali turn agent failures and evaluation explanations into revised prompts, then build a JSON webpage experiment that preserves each candidate and its results.

  • Find the missing guidance
    2:19 ↗
  • Give the optimizer more than a score
    4:39 ↗
  • Turn coding failures into reusable rules
    8:53 ↗
  • Reflection still needs a reliable evaluator
    13:16 ↗
  • Evaluate tasks without an obvious answer key
    16:29 ↗
  • Revise the agent and the judge in separate loops
    19:13 ↗
  • Start with a prototype and a bounded task
    23:07 ↗
  • Set the size of the notebook experiment
    26:35 ↗
  • Include the context of the decision you want to improve
    31:34 ↗
  • Evaluate overall correctness and individual rules
    36:48 ↗
  • Regenerate outputs for each candidate prompt
    39:59 ↗
  • Run the loop and retain every candidate's evidence
    42:57 ↗
  • Make the first run small enough to inspect
    47:35 ↗
  • Extract the selected prompt and keep its results
    49:52 ↗

Key ideas

Scroll to read ↓

Fuad Ali shows how a refund call can appear to recover in text while still failing in audio and execution—and how linked traces, audio evaluations and replayed experiments make those failures easier to find and fix.

  • A correction in the transcript does not prove that the action changed: the refund example still processed order forty instead of fourteen.
    2:42 ↗
  • Common audio conventions make provider events queryable; session IDs connect those events into a timeline of speech and tool activity.
    6:10 ↗
  • Evaluate audible behavior and executed outcomes together. Tone, response latency, interruptions and task success answer different questions.
    9:10 ↗
  • Attach evaluations to relevant spans so failures can be filtered, inspected and used to trigger investigations.
    12:40 ↗
  • The proposed self-healing workflow tests development fixes by replaying failed traces and gives a human reviewer comparison evidence before approval.
    14:39 ↗

References