Spec-Driven Testing for Agents With A Brain the Size of A Planet — Steven Willmott, Safe Intelligence

How do you know whether an AI system still does the right thing when its input changes? At Safe Intelligence, Steven Willmott’s starting point was formal verification of vision and tabular models. With the model available, the team could examine whole regions of the input space and investigate whether perturbations pushed otherwise correct examples into failure. For language models whose internals they do not have, the approach changes: generate edge cases and test cases intelligently. Willmott describes a language-model testing product released the previous day as analogous work, rather than the same verification guarantee.

The company’s giveaway ducks carry the reminder “Think harder.” The engineering question behind that reminder, connecting this talk to the preceding Braintrust presentation, is how to specify an agent’s intended behavior. This is specification-driven testing, not using a specification to generate application code. Traditional ML practice expresses desired behavior through a dataset, then measures performance with metrics such as F1 and accuracy. Those examples remain useful, but deployment introduces requirements that a collection of input/output pairs does not capture.

A larger model should be better at its job. That is the intuitive starting point, but it does not always hold for a deployed agent. Marvin, the robot in The Hitchhiker’s Guide to the Galaxy, has a brain the size of a planet and gets asked to make tea. His boredom and depression make the mismatch memorable; the book’s five-part “trilogy” supplies the appropriate absurdity. Having more intelligence available does not establish whether it is useful for the assigned role.

Willmott illustrates a possible downside with a harmful instruction wrapped in a poem. A less capable model might fail to interpret the poem; a more capable model might recover the instruction and comply. The proposed mechanism is that better comprehension can also make an attack intelligible. That explanation is a hypothesis, not a demonstrated general relationship between model size and safety.

A broader remit also increases both the surface an attacker can exploit and the behavior a developer must test. Meanwhile, assigning simple arithmetic to a large model incurs token costs and latency that an optimized solution can avoid. For fully automated deployment, the target is therefore enough capability to perform the task, with bounded opportunities for harm.

Those opportunities have two dimensions:

• Instruction flexibility: What instructions can the agent receive, and how freely can users formulate them?
• Action authority: What tools and tasks can it carry out inside the surrounding infrastructure?

An agent that can wire millions of dollars has different consequences from one that only answers questions. Good examples alone leave the behavior outside the dataset unspecified. Even the definition of failure needs care: failing to complete a legitimate task is different from successfully complying with a malicious request.

Begin by designing the task or role benchmark independently of the agent that will perform it. Ground-truth examples are one component. Rules are another: a customer-support agent might never be allowed to offer a discount above 10%, or issue a refund more than 30 days after purchase. These are policy boundaries, and the word never creates a demanding testing problem. A set of passing examples cannot by itself establish that a rule is never violated.

A compact JSON fragment can make those two illustrative boundaries explicit for test generation:

{
  "role": "customer_support",
  "rules": {
    "discount": {
      "maximum_percent": 10
    },
    "refund": {
      "disallow_when_days_since_purchase_exceeds": 30
    }
  }
}

The refund rule excludes purchases older than 30 days; it does not establish that every purchase inside the window qualifies. Encoding the restriction separately from positive examples preserves that distinction.

Next come ontologies and dictionaries. An airline chatbot operates within the set of destinations its airline actually serves. A company may also use internal policy terminology that a general model does not know. Supplying that information to the agent is only half the work: the testing system needs it too, so it can determine which substitutions produce valid test cases.

Domain knowledge further constrains substitution. Gross profit and gross sales are different business quantities, even if a general language model treats the terms as closely related. A variation that swaps one for the other may change the question rather than test equivalent phrasing. Rights and roles add another dependency: expected behavior may differ for a logged-out user, a logged-in user, or someone with particular permissions.

Finally, correctness must hold under specified stress. For runway detection, recognizing the runway in a clear image is only the beginning. The system may need to work at sunrise, sunset, or in fog—and the requirement must address how much fog or camera shake it can tolerate. The agent equivalent is typo tolerance, repeated rephrasing, and stability as input wording changes. How many typos disrupt the task? How often must a frustrated customer rephrase before getting a useful answer? Those questions belong in the benchmark for the particular task and role.

For a product-support agent, an eval should capture more than the test set. An A2A agent card contributes a description of what the agent does. That description sits alongside the examples and the surrounding requirements needed to judge its behavior. From a software-engineering perspective, these contextual tests resemble integration tests: the behavior being checked depends on more than an isolated prompt and answer. Build a growing collection of them, making assumptions explicit instead of leaving them implicit in the implementation.

Safe Intelligence feeds intended behavior into security testing. Willmott’s rationale is that an agent’s assigned domain exposes promising places to probe: the agent is willing to discuss that domain, and its tools often give it the greatest power there. A banking agent, for example, has consequential infrastructure access in the same area where it is expected to help users. The specification identifies both the legitimate activity and the boundaries an attacker might try to cross.

The same context supports robustness testing by defining which input changes should preserve successful behavior.

Security probes target misuse of the agent’s capabilities. Robustness probes explore the range over which legitimate requests still receive correct answers. Both need explicit task context, regardless of which testing infrastructure runs them.

Prompt-management platforms can already record why a test exists. That explanation is useful when generating variants because it preserves the purpose behind the original example. Agent cards supply another piece: descriptions of the agent’s capabilities and skills. But a skill description alone still does not say enough to evaluate the behavior.

Consider a meeting-booking skill. Knowing that the agent can book meetings does not establish which people it may book them for. That missing boundary determines whether changing a person in a test remains a valid variation or changes the expected outcome. Generating variations inside such an envelope is difficult precisely because the generator needs the task’s constraints, not just plausible language. Capture the task and its context alongside the benchmark. Marvin’s specification would at least tell him what he is supposed to do with that enormous brain.

The behavioral specification should outlive the infrastructure that implements it. A team may work with LangSmith or Vertex agents today and move elsewhere later. Its integration tests, unit tests, and penetration tests should remain independently runnable. Otherwise, changing the implementation also risks losing the accumulated definition of correct behavior.

That independence also makes an automated improvement loop possible:

1. Run the agent against the specified tests.
2. Collect results and identify robustness gaps.
3. Iterate on the system to address those gaps, then run the tests again.

Willmott describes this as a kind of backyard reinforcement learning, while explicitly distinguishing it from reinforcement learning on the model itself. The loop is assembled around the agent: evaluation results guide changes to the surrounding system.

The next step is an open way to express and exchange these specifications. Willmott connects that ambition to his earlier API-infrastructure work, recalling that he helped write the OpenAPI specification. The direction he proposes is to keep agent specifications in a GitHub repository, pull their constituent pieces into different tools, and version them thoroughly. This is a proposed approach to interoperability, not an announced completed standard: the durable artifact would be the description of required behavior, maintained independently of any one agent or testing platform.