James Hush

AI now writes more than we can check: code, tests, support replies, translations. We used to trust work because a human read it. Nobody can read it all now, and it ships anyway.

I hit this in the smallest way possible: I'm building a voice agent that takes bubble tea orders in English and Mandarin. I live in Asia and speak Mandarin every day, like a kindergartener. I can't read or write it. So I can't read half of my own evals.

You may read both languages fine. It doesn't help. The volume of changes means nobody reads everything, in any language.

Picture what a bad judge costs: Light sugar becomes full sugar, the judge says "pass," and someone gets a bad drink. In a clinic, 十四 (fourteen) sounds like 四十 (forty). The bot hears the wrong dose, the judge says "pass," and someone gets hurt. Multiply that by every call nobody listens to.

So I'm building trust as a loop, with Pipecat Evals and open models on one laptop. An LLM writes evals in both languages. A simulated customer orders, switches languages, and changes their mind. A local judge grades the call. Then the loop checks the judge: pass/fail rules I can read, planted bugs the evals must catch, a second judge from a different model family to flag disagreements, and a native speaker for the calls that matter.

You'll leave with that loop, and one question for any eval: what would it take for this test to be wrong, and who would notice?