▶ Watch ↗17:32
How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase
Read the full talk →Key ideas
Scroll to read ↓Evan Kormos presents a practical architecture for regulated workflows: deterministic policy and process boundaries outside, adaptive agent loops inside, and measured confidence gates that reduce human review only when the evidence supports it.
- Model the regulated outer process as a deterministic DAG, then allow bounded adaptive loops inside selected nodes.4:05 ↗
- A loop deserves no more trust than its acceptance check; repeated passes do not guarantee improvement.10:39 ↗
- Use deterministic tests for conventional software behavior, repeated evals for variable outputs, and traces to inspect how agent runs actually unfolded.11:36 ↗
- Automate nodes that are easy to check against policy, bounded, traceable, and able to fall back to a human.13:12 ↗
- Reduce oversight progressively—from full review to sampling to exception handling—only after measured confidence clears organization-defined policy thresholds.14:37 ↗
- Monitor the whole workflow: speeding up one node can move volume into design, security, or another downstream bottleneck.7:43 ↗