AI Engineer Europe 2026
Spec-Driven Testing for Agents With A Brain the Size of A Planet — Steven Willmott, Safe Intelligence
Read the talk
Spec-Driven Testing for Agents With a Brain the Size of a Planet
An agent needs more than correct answers on a test set: it needs explicit rules, domain boundaries, permissions, and a defined range of inputs over which its behavior should remain reliable.
From a talk by Steven Willmott
How do you specify what an agent should do?
How do you know whether an AI system still does the right thing when its input changes? At Safe Intelligence, Steven Willmott’s starting point was formal verification of vision and tabular models. With the model available, the team could examine whole regions of the input space and investigate whether perturbations pushed otherwise correct examples into failure. For language models whose internals they do not have, the approach changes: generate edge cases and test cases intelligently. Willmott describes a language-model testing product released the previous day as analogous work, rather than the same verification guarantee.
The company’s giveaway ducks carry the reminder “Think harder.” The engineering question behind that reminder, connecting this talk to the preceding Braintrust presentation, is how to specify an agent’s intended behavior. This is specification-driven testing, not using a specification to generate application code. Traditional ML practice expresses desired behavior through a dataset, then measures performance with metrics such as F1 and accuracy. Those examples remain useful, but deployment introduces requirements that a collection of input/output pairs does not capture.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A smarter agent is a better agent—right?
A larger model should be better at its job. That is the intuitive starting point, but it does not always hold for a deployed agent. Marvin, the robot in The Hitchhiker’s Guide to the Galaxy, has a brain the size of a planet and gets asked to make tea. His boredom and depression make the mismatch memorable; the book’s five-part “trilogy” supplies the appropriate absurdity. Having more intelligence available does not establish whether it is useful for the assigned role.
Willmott illustrates a possible downside with a harmful instruction wrapped in a poem. A less capable model might fail to interpret the poem; a more capable model might recover the instruction and comply. The proposed mechanism is that better comprehension can also make an attack intelligible. That explanation is a hypothesis, not a demonstrated general relationship between model size and safety.
A broader remit also increases both the surface an attacker can exploit and the behavior a developer must test. Meanwhile, assigning simple arithmetic to a large model incurs token costs and latency that an optimized solution can avoid. For fully automated deployment, the target is therefore enough capability to perform the task, with bounded opportunities for harm.
Those opportunities have two dimensions:
- Instruction flexibility: What instructions can the agent receive, and how freely can users formulate them?
- Action authority: What tools and tasks can it carry out inside the surrounding infrastructure?
An agent that can wire millions of dollars has different consequences from one that only answers questions. Good examples alone leave the behavior outside the dataset unspecified. Even the definition of failure needs care: failing to complete a legitimate task is different from successfully complying with a malicious request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Define the role independently of the agent
Begin by designing the task or role benchmark independently of the agent that will perform it. Ground-truth examples are one component. Rules are another: a customer-support agent might never be allowed to offer a discount above 10%, or issue a refund more than 30 days after purchase. These are policy boundaries, and the word never creates a demanding testing problem. A set of passing examples cannot by itself establish that a rule is never violated.
A compact JSON fragment can make those two illustrative boundaries explicit for test generation:
json
{
"role": "customer_support",
"rules": {
"discount": {
"maximum_percent": 10
},
"refund": {
"disallow_when_days_since_purchase_exceeds": 30
}
}
}
The refund rule excludes purchases older than 30 days; it does not establish that every purchase inside the window qualifies. Encoding the restriction separately from positive examples preserves that distinction.
Next come ontologies and dictionaries. An airline chatbot operates within the set of destinations its airline actually serves. A company may also use internal policy terminology that a general model does not know. Supplying that information to the agent is only half the work: the testing system needs it too, so it can determine which substitutions produce valid test cases.
Domain knowledge further constrains substitution. Gross profit and gross sales are different business quantities, even if a general language model treats the terms as closely related. A variation that swaps one for the other may change the question rather than test equivalent phrasing. Rights and roles add another dependency: expected behavior may differ for a logged-out user, a logged-in user, or someone with particular permissions.
Finally, correctness must hold under specified stress. For runway detection, recognizing the runway in a clear image is only the beginning. The system may need to work at sunrise, sunset, or in fog—and the requirement must address how much fog or camera shake it can tolerate. The agent equivalent is typo tolerance, repeated rephrasing, and stability as input wording changes. How many typos disrupt the task? How often must a frustrated customer rephrase before getting a useful answer? Those questions belong in the benchmark for the particular task and role.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the surrounding context part of the test
For a product-support agent, an eval should capture more than the test set. An A2A agent card contributes a description of what the agent does. That description sits alongside the examples and the surrounding requirements needed to judge its behavior. From a software-engineering perspective, these contextual tests resemble integration tests: the behavior being checked depends on more than an isolated prompt and answer. Build a growing collection of them, making assumptions explicit instead of leaving them implicit in the implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use the specification to direct testing
Safe Intelligence feeds intended behavior into security testing. Willmott’s rationale is that an agent’s assigned domain exposes promising places to probe: the agent is willing to discuss that domain, and its tools often give it the greatest power there. A banking agent, for example, has consequential infrastructure access in the same area where it is expected to help users. The specification identifies both the legitimate activity and the boundaries an attacker might try to cross.
The same context supports robustness testing by defining which input changes should preserve successful behavior.
| Testing purpose | Question | How the specification helps |
|---|---|---|
| Security | Can the agent be induced to cross a boundary? | Identifies its domain, authority, and prohibited behavior |
| Robustness | Does it still do its job as inputs vary? | Defines valid variations and expected outcomes |
Security probes target misuse of the agent’s capabilities. Robustness probes explore the range over which legitimate requests still receive correct answers. Both need explicit task context, regardless of which testing infrastructure runs them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A capability description is not an evaluation boundary
Prompt-management platforms can already record why a test exists. That explanation is useful when generating variants because it preserves the purpose behind the original example. Agent cards supply another piece: descriptions of the agent’s capabilities and skills. But a skill description alone still does not say enough to evaluate the behavior.
Consider a meeting-booking skill. Knowing that the agent can book meetings does not establish which people it may book them for. That missing boundary determines whether changing a person in a test remains a valid variation or changes the expected outcome. Generating variations inside such an envelope is difficult precisely because the generator needs the task’s constraints, not just plausible language. Capture the task and its context alongside the benchmark. Marvin’s specification would at least tell him what he is supposed to do with that enormous brain.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Keep the tests when the implementation changes
The behavioral specification should outlive the infrastructure that implements it. A team may work with LangSmith or Vertex agents today and move elsewhere later. Its integration tests, unit tests, and penetration tests should remain independently runnable. Otherwise, changing the implementation also risks losing the accumulated definition of correct behavior.
That independence also makes an automated improvement loop possible:
- Run the agent against the specified tests.
- Collect results and identify robustness gaps.
- Iterate on the system to address those gaps, then run the tests again.
Willmott describes this as a kind of backyard reinforcement learning, while explicitly distinguishing it from reinforcement learning on the model itself. The loop is assembled around the agent: evaluation results guide changes to the surrounding system.
The next step is an open way to express and exchange these specifications. Willmott connects that ambition to his earlier API-infrastructure work, recalling that he helped write the OpenAPI specification. The direction he proposes is to keep agent specifications in a GitHub repository, pull their constituent pieces into different tools, and version them thoroughly. This is a proposed approach to interoperability, not an announced completed standard: the durable artifact would be the description of required behavior, maintained independently of any one agent or testing platform.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Versioned specification for agent discovery, capability descriptions, skill examples, authentication, and interaction.
Further reading
An empirical study of poetic jailbreak prompts, including differences across models and limitations of proposed explanations. Links to the January 2026 revision.
Updates since the talk
Willmott explains Spec27's automated test generation, contextual specifications, and infrastructure-independent testing.
Read the complete timestamped transcript
- 0:00
[upbeat music] So nice to meet you.
- 0:15
I'm, I'm Steve. Um, I'm the CEO of Safe Intelligence. We're a company, we've been around for three years. Um, we really go very, very deep into ML, uh, validation.
- 0:26
Um, and actually, we use formal verification techniques on... Especially, we started out on vision models, tabular data models, a bunch of other types of models, um, where we actually have the model available, and we look at whole regions of the input space and see whether or not the test points that are there actually tip over and do
- 0:41
the wrong thing, um, uh, under perturbations. So that's where the company started out. We have a whole bunch of products in that space. Um, and uh, actually, yesterday, we, we released a new product, which is doing something analogous for, for language models.
- 0:54
Obviously, we don't have the language model, so what we're trying to do instead is, is be very clever about how we generate edge cases and test cases. So I won't talk about the product too much.
- 1:02
We have a booth, so, so come and, come and chat, chat to us, um, at the booth for that. If you've seen these ducks around, these are ours. If you didn't get one, I have a whole box of them here, so feel free.
- 1:13
They say, "Think harder," on the front, so you can put that on your desk just to be reminded, um, about what you should be doing. And today, I'm gonna talk about something which is very similar to what, um, or very related to what Phil talked about just now from Braintrust.
- 1:26
Uh, we like what Braintrust does a lot. Um, and I think one of the inherent problems is, like, how do you actually specify what an agent is supposed to do?
- 1:33
Um, so I think pe-people are familiar with spec-driven development. This is not gonna be about developing code with specs. That's also very important. We do a lot of that in the company for the products that we build.
- 1:43
This is-- But this is about how you specify what an agent or an AI system is supposed to do. And in ML, you typically use a dataset to do that.
- 1:50
You basically have your dataset and you run it, all those examples, and you look at F1 accuracy and things like that. And that's sort of telling you what you want the a- what the, what you want the agent or the system to do.
- 2:02
Um, but as, as we'll see, there's actually a lot more to it, um, uh, when, when you deploy things. So that's what the focus of the talk is, like, how do we actually specify what agents are supposed to do?
- 2:11
And I guess my, my key starting point is this, this seems like an obvious question. Like, a smarter agent is a better agent, right? So if I have a smarter agent, I'm using a bigger model, uh, it's gonna be better at doing the job that it's supposed to do.
- 2:24
Um, in, in general, you'd expect that to be right. But that's-- I think probably most people have the experience that that's not always true, in fact. Um, so there's some problems.
- 2:32
Um, if you're familiar with, uh, the, the book, The Hitchhiker's Guide to the Galaxy, there's a robot called Marvin, uh, who has the brain the size of a planet, and he's normally asked to do things like make the tea.
- 2:42
And he gets extremely bored, and he's extremely depressed. So this depressed robot, uh, kind of is a theme in the book. Um, if you haven't read the book, by the way, you absolutely have to read, read the trilogy, um, which is five, five-part trilogy.
- 2:55
That will tell you something about the, the style of the humor. Um, in any case, Marvin, there are challenges with having, like, massive models. Some of the jailbreaks actually, uh, work better on large models because, uh, they're smarter.
- 3:06
So if you en-encapsulate something in a poem and you give that to a relatively low-end model, the low-end model doesn't even understand the poem. Whereas a larger model will be like, "Oh, I can take this out, and I can execute the bad instruction that's wrapped up in the poem."
- 3:19
So it's not obvious that, um, bigger is safer, and it's not obvious that bigger is better. Another thing is, if you're building agents that have a very broad remit, they can do a lot of things, that creates a lot of surface area for someone to actually exploit, and it creates a lot of surface area to test if
- 3:35
you wanna be sure that the a-agent is actually doing things that you want it to do. And obviously, there is a cost issue, right? So if you're using large models to do something which is relatively simple, like, like just simple math, you're gonna be paying for tokens, and it's gonna be slower, um, rather than something that's very
- 3:50
optimized. So in general, if you're building agents for deployment, for especially automated use, uh, fully automated use, there's this trade-off between smart and safe in some sense, and, um, smart and, and capable in the other direction.
- 4:05
And so what you're really seeking is, like, a model or an agent that's built on a model that's good enough to perform, but it's not capable of doing arbitrary harm.
- 4:12
And that arbitrary harm is, uh, kind of two parts to it. One is, you know, what kind of instructions can it receive? How flexible is it about how those are formulated, what the, what the prompts look like?
- 4:23
Um, that's one part of it. And the other part is, like, what tools and tasks can it carry out in your infrastructure? So if it's able to wire millions of dollars to people, that's obviously a lot more risky than if it can just answer questions, and so on.
- 4:36
Right. So this is the balance that most people-- that you're basically looking for. But how do you actually define what good looks like? I think it's pretty obvious that it's not just a dataset of inputs and outputs that, that are pretty good, and then the rest is, like, guesswork.
- 4:50
Uh, and it's also sometimes hard to define what harm looks like, because maybe an agent doesn't do the right thing, um, but it's, it's kind of just failing at the task, and sometimes it's doing exactly the wrong thing when it's asked to do something bad.
- 5:03
So what is this idea of spec-driven validation? Um, spec-driven testing, um, you could call it that as well. Um, it's basically, what are the things we would want to do if we were just designing the role or the task benchmark by itself, like, independent of the agent?
- 5:19
So we already talked about datasets. So the ground truth, having a bunch of examples of what good looks like is one thing, so that's kinda one component. Often, we see, uh, customers that we work with also have rules.
- 5:31
So if they've got a customer support agent, you wanna say things like, you know, "Don't ever give a discount more than ten percent. Uh, we don't allow refunds if, um, if, you know, it's more than thirty days past the purchase."
- 5:42
And there are sort of these rules, right? So your alarm bells should be going off a little bit already because, like, how do you actually test for sure that a rule is never violated?
- 5:50
It's pretty hard. Um, sometimes you also, um, have, uh, ontologies or dictionaries that are relevant. So an example would be if you're building an airline chat, um, bot- That particular airline might only fly to certain, uh, might only fly to certain destinations, so that's the relevant universe of things you need to think about.
- 6:09
Um, you may have internal terminology in your company that apply to your policies that no one else in the rest of the world actually knows about, so that's also part of the spec, right?
- 6:17
Because if you're gonna actually, um, build an agent, you will be building that into the agent, but if you're gonna test it, you actually need to tell the testing system what these things are and what is a, a valid substitution.
- 6:29
There's domain knowledge, so you may have very specific, you know, scientific, um, finance agents, other things that are-- that need to know what ter-terms are substitutable. So if you, for example, if you do substitutions on something like, um,
- 6:45
uh, you know, gross profit and gross sales, for example, if you're sort of talking to an LLM generally, it might actually confuse those two terms, but in business, they're very different things.
- 6:55
Um, so this specific domain knowledge is relevant to testing as well. And then you might have rights and roles, like the agent may perform differently if you're logged in, if you're logged out, if you have certain rights and permissions and things like that.
- 7:06
And then the last one, which is, um, pretty important, is robustness requirements. So one is I've got my test set. That should work, right? Um, but it m-needs to work under stress.
- 7:18
So in vision, where we started out, it's things like can I detect this runway for the, the plane to land on, but can I detect it at sunset, sunrise, under fog?
- 7:25
Like, and how much fog can there be? How much can the camera shake before the thing doesn't work? And that's actually similar in, in agents. You know, if you're building a customer-facing agent, could typos disrupt it?
- 7:35
How many typos disrupt it? Like, how frustrated will people get rephrasing? How, how stable under change are, are the results? And so really this is... The point here is we need to go beyond the test set to have, like, task and role-specific benchmarks that are for the agent itself.
- 7:53
Um, and what do you then do with that? Or maybe I already talked about some of these examples, but these are just examples of the kind of things if you had a product support agent.
- 8:00
So we've worked with quite a few people doing this. So you can kind of think of it as there's an... In, in LLM land, people have started to call the eval kind of the test set, which sort of makes sense, but I just think that the eval itself, like, we have to think of going beyond the eval
- 8:16
as well. There's this concept of an agent card, which comes from the A2A spec. It's been in other things as round, which describes what the agent does. It's also relevant here.
- 8:26
And then obviously there's all the context around this. And if you're a company deploying agents, you kind of want your, your tests to look, um, like something that has these various elements that are relevant.
- 8:37
That's a fair eval, and you want to build more and more of these tech. These look like integration tests if you're from an engineering perspective. Um, often some of these things are implicit, but you wanna make them explicit.
- 8:48
So what do we... What can you do with this? So what we do with this in our platform, we do two things. We do security checks. So we actually pull the, the specs, um, that an agent is supposed to fulfill into security testing.
- 9:00
Why do we do that? Generally, if you know what an agent is trying to do, you know the edges of where it's vulnerable because it's gonna be willing to talk about those domains that it's supposed to act in, right?
- 9:11
So that's actually where it's most likely to be vulnerable. Second, the tasks it performs, it will have more power to act in the infrastructure on those tasks. Like, if it's a banking agent or something like that, it will have-- be able to work in that area.
- 9:24
So we-- That's a place you can pull things like this spec information in. And then the robustness side is, like, does it do its job properly? Especially the robustness side, can we vary the inputs and see how, how much of a range it has in terms of answering the questions properly?
- 9:39
Um, so we built a product to do this, but my, my point here is not to show the product. Um, I think it's just something if you're testing agents in any context using any infrastructure, trying to, like, be explicit about the various bits of-- that are on this slide and bringing that together is, is a useful thing
- 9:56
to try to do. Um, from an industry perspective, I think there's lots of things going on, but just calling out two. I mean, there are, there are a lot of prompt management platforms that allow you to be fairly elaborate about why this test exists and things like this.
- 10:08
This is all useful when you actually wanna generate, um, variants of the test because you want this context. As I said, from the A2A spec, you've got agent cards.
- 10:18
They're quite long, but here's an example of a skill. Um, you would also realize that even if you have this, that doesn't give you enough to actually evaluate the agent.
- 10:26
You still want to know, well, what, what range of change could be-- is, is valid, you know, uh, for maybe in this case, what kind of people could, could the meeting be booked for and, and so on.
- 10:38
Um, I can talk a lot more about how it, how hard it is to create variations within these sort of envelopes that a spec might create. Um, but I think just in general, my, my point here is, like, as you think about evaluating agents, start thinking about not just the eval data set or benchmark.
- 10:54
Um, uh, also think about the task and the context for the task and how you, how you capture that. So, um, hopefully, we can make Marvin a little bit happier because he has the specs and he kinda knows what he's supposed to do.
- 11:08
Um, um, and then, yeah, specify the behavior of your agents. That's kind of the key thing to do here. Stay independent of the implementation because often you may, you know, may, you may be building in LangSmith or something, uh, or Vertex agents or, or, or, or so on, but then later on you may change to a different
- 11:25
infrastructure. You actually wanna keep those integration tests, the little unit tests and penetration tests and, and run them independently. And this is also a way to close the loop.
- 11:34
So part of our inspiration of thinking about what should go into a spec is, like, what would you need, uh, to actually run the agent automatically, get the results, and then start to iterate and try to fill the robustness gaps that have appeared?
- 11:46
So it's like a backyard type of RL. It's not proper RL because you're not doing it on the model, but you're kinda like, uh, jury rigging something around the outside.
- 11:55
Um, that's the key point. Um, where do we go from here? So we're obviously building product around this, but I've, I've been in computer science for a long time.
- 12:04
My last company, we did API infrastructure, so if you've used OpenAPI spec, I'm a-- I apologize. It's partly my fault. So I helped write that spec way back in the day.
- 12:13
Uh, so we're all about open. So we're thinking about, like, how do you express these things in a way that you could just have in a GitHub repo, pull them into whatever tool you wanna do, and then pull all the different pieces and kind of just version the hell out of that stuff.
- 12:26
So if anyone's interested in stuff, love than that, love to chat. Um, that's my talk. Um, come to our booth. We have a, a game you can play. Uh, if you play by four, uh, you can win some of the Lego prizes, uh, up there.
- 12:39
Uh, you need a bit of knowledge, to be fair, or you need to be insanely lucky. Um, but yeah, that's my talk. Thanks a lot. [audience applauding] [upbeat music]