If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com

The SAT tells you something about a student's language and math skills. A college can use that evidence in an admissions decision, but the exam covers only part of what it takes to succeed there. Ramakrishnan Lokanathan sees the same boundary around AI benchmarks. SWE-bench gives a model a software engineering task. Humanity's Last Exam asks a different set of questions. Before either score can inform a deployment, someone has to explain which part of the product's promise it measures.

Lokanathan came to that problem as a founder. His team went full-time on GPT-3 applications in September 2022, about two months before ChatGPT arrived and upended its plans. After several pivots, the team built a pet care assistant, earned revenue through e-commerce and recommendations, and was acquired. He later red-teamed NVIDIA's Nemotron 3 models and worked on AI safety at Lawrence Berkeley Lab. Across those settings, his definition of an eval has three ingredients: a task, a standard for success, and evidence of how the system performed.

He expects better models may eventually need less evaluation, perhaps leaving evals largely as validation in a decade. Teams shipping today have a more immediate problem. He cites the MIT GenAI Divide report's figure of 95% of organizations seeing no return as a reason to take the gap between a demo and production seriously. The question for a team is whether its own system delivers what it promised its users.

Imagine the agent could do the job as well as the person you would trust with it. What must happen? What must never happen? When should the agent stop? The last question belongs in the specification as much as the desired outcome does. Together, the answers give the team a standard it can use even as the underlying model changes.

A score of 100 out of 100 troubles Lokanathan when the suite is supposed to reveal the limits of a system. If every case is within the agent's reach, the team learns little about where the promise starts to break down. He suggests revisiting the challenge cases as early as 85 or 90 out of 100. Those numbers are his rule of thumb for when to make the suite harder, rather than a deployment threshold.

The image-editing request contains one consequential word. Change only the background. The edited background is green, so a check that the background changed passes. Look at the object in the foreground, though, and it now has two handles. The user specified both a change and something to preserve. The color check measured only half the instruction.

Now imagine a workflow with a hundred steps. For each step, Lokanathan wants to know what went in, what came out, and what should have happened there. The check depends on the work. If a component repeatedly calls a tool to retrieve information, evaluate the accuracy of those calls at that point. Grading the final prose gives little evidence about whether the retrieval component did its job.

A writing score or a coding score could help select a model for a narrowly defined task. A persistent agent moves between tasks, and the team keeps changing the system after model selection. Lokanathan puts automated evals in continuous integration so the agent faces the questions on every production deployment. That gives the team repeated observations under the version it actually intends to ship.

Two other kinds of scrutiny answer different questions. Human reviewers inspect whether the results make sense, including cases where the automated grader accepted the output. An agent can speed up that work, but Lokanathan keeps people responsible for the judgment. Adversarial testing starts with a deliberate attempt to break the workflow. One way to create such a challenge is to give the agent an opponent who can respond to its last decision.

Suppose you're replacing an agent's model. Lokanathan proposes playing the old agent against the new one at chess and inspecting the moves. An early choice can become a liability several turns later. An opponent can upset a plan or expose a blunder, giving you a chance to see whether the agent adapts. The winner of one match has won a chess game. The sequence of decisions offers evidence you can examine alongside coding, writing, and other evaluations.

Lokanathan built eSlams, short for electronic slams, as an open-source Python package available through PyPI. He reports support for about 50 games and 90 providers. A team can bring its own agents and choose a game for the behavior it wants to observe. In chess and Go, both players can see the board and plan from it. Poker requires decisions with information hidden. Cooperative games ask whether agents can pursue a shared goal. The project was also trying arcade games.

In the eSlams preview, Fable 5.1 plays tic-tac-toe against GPT-6 Astra. The example lets a viewer follow each model's response to the other's turn, and the match ends in a tie. People can also play against the models themselves. Lokanathan positions these games as a way to investigate decisions over time. Their relevance still depends on the product. For an agent that needs current data, he points to the Parallel Search Model Index as another place to look for evidence about a capability the job requires.

Lokanathan used Google Assistant to set reminders. In his experience, when it gave way to Gemini, a request that had worked routinely drew an explanation that a language model couldn't set a reminder. From his perspective as a user, the upgrade had removed the feature he depended on.

That experience explains why making an eval harder still leaves a place for easy cases. Keep exercising known tasks and past failures when the system changes. A reminder that worked yesterday belongs in the regression suite even if every candidate model is expected to pass it. Fresh challenge cases serve a separate purpose. They keep probing the limits of what the next version can do.

A thumbs-down is a starting point. Lokanathan combines reports like that with observations from the running system, then applies a familiar software QA practice. Reproduce the incident. Locate the step that failed, whether it was the model's response, a tool call, or another part of the workflow. Then ask a second question about the eval itself. Which check let this version through?

Once a reviewer understands the failure, the team can turn the reproduced case into a test and run it in the deployment pipeline. Human review matters here because customers encounter combinations of circumstances the original test designer never anticipated. A green-background check can keep passing indefinitely. A user pointing out the extra handle gives the team the missing condition to test on every subsequent edit.