Malleable Evals: Why Are We Still Evaluating Adaptive Systems with Static Tests?

Around 2013, Vincent Koc tried VR goggles whose warning label limited use to five minutes. He recalls using them for three hours, then vomiting for three hours afterward. The device’s intended use and his actual behavior were very different—and that difference is where testing becomes interesting. Emerging technology invites experimentation, including uses its designers would rather people avoid.

Koc brings that experimental perspective to evaluation research at Comet. He describes working with universities and benchmarking systems for organizations including Uber, Netflix and UK banks. Against that background, the joke that evaluations are dead raises a practical question: what must evaluation become as the systems it measures grow more adaptive?

Software testing starts with an expectation: this thing is meant to do something. Examples become unit tests. A manual regression suite records that doing A and B sometimes produces an unwanted C. CI/CD checks help keep that behavior from shipping again. Beyond those known cases, chaos engineering deliberately stresses and breaks systems to discover their limits, while observability helps engineers understand what happened.

AI evaluation often stops earlier. Consider a risk-compliance check intended to prevent an assistant from selling financial services. A team handcrafts questions, tunes the system until it handles them correctly, and cycles through those questions offline before deployment. That establishes behavior on the selected cases, but leaves the exploratory layer underdeveloped.

Koc’s gap is the last row: how do teams discover where an agent breaks, rather than only checking the examples they already know?

Adding more datasets does not automatically close that gap. A benchmark about adding numbers may measure a capability, but its relevance to a particular application still needs explaining. Teams can accumulate a huge collection of evaluations and nevertheless return to diagnosis when an unexpected production failure arrives.

The application being evaluated is itself changing. Ordinary software releases can require revised tests, but adaptive agents make that relationship more immediate. Koc identifies himself as a core contributor to OpenClaw and describes harnesses that can shift their behavior and create skills. As those capabilities change, a benchmark tied to an earlier application can become less representative.

An adaptive-testing paper he introduces asks whether benchmarks can change with applications. He describes its approach as more selective, smarter testing: choosing what to test more intelligently. That is a step toward adaptive evaluation, but it is narrower than maintaining an evaluation system that follows changes in real usage. Selecting better questions from an existing test collection and changing what the collection needs to cover are different problems.

The route to this problem starts with prompt engineering. In a banking or creative application, a developer changes instructions, tries more words, and watches what comes out. Koc compares this to developing a medicine for liver disease and discovering that it relieves pain: a useful effect can emerge without the original experiment explaining it well. His rhetorical claim that prompt engineering died in 2023 comes with an acknowledgment that people still work this way. The target of the criticism is unguided wordsmithing.

Context engineering makes the system’s moving parts more explicit. Incoming data, search, retrieval-augmented generation and tool calls create steps that can be inspected separately. A large agent might contain an MCP sales tool; that tool can be tested for its intended behavior independently of the whole application. Decomposition gives evaluation clearer targets and makes failures easier to understand, even though passing component tests does not settle the behavior of the complete agent.

Greater access to tokens accelerates this process. Koc concedes that whether tokens are cheap is debatable, but their availability makes more generation possible. His phrase “tokens become fast food” captures a feedback effect: developers consume more tokens, generate more code, and increase the velocity of application development.

Model capability expands what that generated software can do. Koc describes optimization work in which general-purpose models solve individual ARC-AGI-2 puzzles. He also recalls trying ARC-AGI-3 and finding some puzzles difficult for humans, while emphasizing models’ ability to recognize useful patterns. ARC-AGI-2 uses input-output tasks; ARC-AGI-3 introduces interactive environments with unstated rules and goals. His examples include no model identity, harness, budget or measured score, so they illustrate capability rather than establish completion of either benchmark.

That leads to intent engineering: systems that adapt toward what a user wants. Koc points to OpenClaw and harnesses around Claude and Codex as examples of systems trying to understand users and improve their experience. Personalization creates a new testing difficulty. If the system adapts differently to you and to me, a single expected interaction may no longer describe both experiences.

This increases the need for evaluation and observability. Teams need to understand what happens inside the different layers of an agentic application. A statement such as agents being insecure is too broad to guide a fix; visibility into behavior is what can turn that concern into a specific, actionable problem.

An intent-based outcome requires a different evaluation target. For arithmetic, an exact-answer check can establish that one plus one equals two. For an assistant, the questions may instead concern how it handles ambiguity, expresses personality or meets an organization’s expectations. Rubrics make those criteria explicit, much as a school can evaluate artwork without requiring every student to produce the same picture.

The test suite can also change as usage changes. Koc imagines a workload in which 80% of traced activity is familiar, then asks what happens when the customer base changes. New customers ask different questions and use the agent differently. Those traces provide evidence that the evaluation suite may no longer represent the application’s workload.

His proposed trace-driven process is:

1. Collect traces of the application’s behavior.
2. Feed those traces to an evaluation agent that looks for changes in usage.
3. Notify users or agent owners when something has changed.
4. Revise the evaluation suite to cover the changed behavior.

The agent helps curate the tests rather than leaving that work entirely to people. This is a proposed maintenance loop: the detection of a change supplies a reason to update coverage.

Once agents read traces and perform evaluations, the process can operate as an always-on evaluation and optimization service. Evaluation no longer has to wait for someone to assemble a dataset and launch an offline run.

The next step is telemetry in the loop. Koc mentions a paper he has written on this idea: expose operational information to the harness running the software, MCP tools or agentic system. Two kinds of feedback matter in his description:

• Failures: Give the harness information about what broke so it can attempt a repair and continue.
• Cost: Give it information about resource spending, together with conditions that guide acceptable behavior.

The feedback changes what the harness can respond to. Instead of relying only on failures anticipated during development, it can use evidence from the current run. Koc describes this as enabling self-correction to some degree, not guaranteed recovery from every error.

Koc tentatively calls the alternative eval calcification: evaluation becomes increasingly rigid while the system it describes continues to change. To explore a different pattern, he points to Karpathy’s autoresearch, a Python-based optimization loop that tunes and tweaks toward a goal. The transferable idea is to specify a target, evaluate a candidate change, and use the resulting signal to guide another iteration.

His deliberately everyday example is barbecue. Optimizing for the tastiest mix and optimizing for the cheapest mix are different tasks. The user’s intent determines which reward signal makes sense. A small Python example makes that distinction concrete with illustrative candidate values:

from dataclasses import dataclass


@dataclass(frozen=True)
class Mix:
    name: str
    cost: float
    taste: float


mixes = [
    Mix("smoky", cost=6.0, taste=8.0),
    Mix("pepper", cost=4.0, taste=7.0),
    Mix("sweet", cost=5.0, taste=9.0),
]


def reward(mix: Mix, intent: str) -> float:
    if intent == "tastiest":
        return mix.taste
    if intent == "cheapest":
        return -mix.cost
    raise ValueError(f"Unknown intent: {intent}")


def choose_mix(intent: str) -> Mix:
    return max(mixes, key=lambda mix: reward(mix, intent))


for intent in ("tastiest", "cheapest"):
    print(intent, choose_mix(intent).name)

The candidates stay the same; changing the objective changes the preferred result. The taste values would have to come from some assessment—the function cannot discover what tastes good merely by naming that objective.

An adaptive loop would go further than choosing among those fixed candidates: it would propose changes, assess them against the objective, and use the feedback to correct its next attempt. Evaluation becomes a specification of the desired end state, rather than merely the dataset used at the beginning. That is the broader direction Koc draws from automated optimization: people define what they want the system to reach, and the machine performs the iterative work toward it.

Koc closes with an illustrative split: 80% of behavior is stable and defined through intent, while 20% keeps changing. Those proportions are a thought experiment, not measured workload statistics. The changing portion contains the strange question or unexpected use that can cause serious business problems. His proposal is to have agents monitor and maintain that portion, adapting evaluations as those behaviors emerge.

This makes evaluation something to maintain as code, software or a living agent: a growing, self-optimizing system rather than a point-in-time dataset. The implementation was still in progress at the time of the talk. Koc did not show the planned in-depth Comet demonstration because its end state was unfinished; he expected it to be ready in the following weeks.

The conceptual framework is meant to transfer to other teams’ systems. Agents can shift their own behavior; the surrounding problem and datasets can change too. Evaluation therefore needs its own capacity to observe those changes, revise what it tests and guide correction toward the intended outcome.