AI Engineer World's Fair 2026
The Unreasonable Effectiveness of Separating the Task from the Model
Read the talk
Separate the AI Task from Its Implementation
DSPy treats AI workflows as functions with stable contracts, using specifications, executable constraints, and evaluations to make models and harnesses replaceable.
From a talk by Maxime Rivest and Isaac Miller
Before you start: Basic familiarity with Python functions, classes, and language-model prompting will help with the code example.
What should stay fixed when the model changes?
If you repeat an AI task, why not give it the same structure as any other reusable function? Define its name, inputs, and outputs, then put the implementation behind that interface. Callers can reuse, compose, test, optimize, package, and distribute the function without understanding everything inside it. DSPy brings those properties to AI programs through an open-source Python framework.
That boundary becomes valuable when models and techniques change every few weeks. A new prompting strategy or agent architecture is an implementation choice; the task it must accomplish is the contract. Keep the input and output interfaces stable, and you can experiment inside them without rebuilding every integration. The practical question becomes whether a new technique improves your task, rather than whether you have adopted the latest technique.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From farm invoices to pull requests
Maxime Rivest’s first DSPy program extracted tax values from his farm invoices. Another utility used a keyboard command to read clipboard text and correct its grammar. A separate program rewrote text for clarity. Each had a small, repeatable job and a stable interface, so changing the model inside did not require changing how he invoked the utility.
The same structure can describe much larger workflows:
| Task | Inputs | Output |
|---|---|---|
| Draft an email reply | Inbox and incoming email | Draft reply |
| Implement a change | Specification and repository | Pull request |
Rivest proposes Recursive Language Models, or RLMs, as one possible implementation for the inbox task. These are ambitious task boundaries, not just wrappers around short text transformations.
Inside a boundary, the implementation can progress from a simple prompt to an improved prompt, an agent, tool use, and more elaborate loops. The surrounding application still supplies the same inputs and consumes the same outputs. A stable boundary also makes automatic optimization possible—but an input/output signature alone does not tell an optimizer everything it needs to know.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Specify what should happen
The first part of a task definition is what should happen: natural-language instructions and a typed signature. Model configuration sits separately from that specification. In the tax example, the program accepts invoice text and produces string and floating-point outputs. The instructions describe tax extraction and include a condition for returning zero; that business rule belongs in the task definition, independently of the chosen model.
Natural language communicates intent efficiently. When a friend arrives to play a board game, you give them the rules so they can begin. Asking them to infer the game entirely from examples—as in Rivest’s analogy to AlphaGo and AlphaZero—would make for a long evening. Examples matter, but they need not carry information that instructions can express directly.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Enforce what must happen
The second part is what must happen. Requirements that must hold belong in executable code. Rivest’s tax module uses self.extract with dspy.Predict for a direct prediction and self.recheck with dspy.ChainOfThought for a reasoning pass over the same task. Its forward method follows a prescribed sequence:
- Run the direct extractor.
- If the initial tax result fails the truthiness check, rerun extraction with reasoning.
- If the resulting tax value is negative, raise an exception so a human can review it.
The model does not decide whether these safeguards apply.
A Python module expressing that pattern keeps the task signature and the control flow separate:
python
import dspy
class ExtractTaxes(dspy.Signature):
"""Extract the total tax amount from an invoice."""
invoice: str = dspy.InputField()
explanation: str = dspy.OutputField()
taxes: float = dspy.OutputField()
class TaxExtractor(dspy.Module):
def __init__(self):
super().__init__()
self.extract = dspy.Predict(ExtractTaxes)
self.recheck = dspy.ChainOfThought(ExtractTaxes)
def forward(self, invoice: str):
pred = self.extract(invoice=invoice)
if not pred.taxes:
pred = self.recheck(invoice=invoice)
if pred.taxes < 0:
raise ValueError("Negative tax amount requires human review")
return pred
A truthiness check on a floating-point amount also retries a legitimate zero; it does not distinguish zero from an unsuccessful extraction. The essential design is that retry and rejection rules remain outside the predictors. Even a much more capable future model would still have to obey them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Show what good looks like, then optimize
The third part is what good looks like. As a child on the farm, Rivest asked his father how he knew a tree was a maple. His father could recognize it but could not supply a satisfactory instruction, much less code. Rivest learned through examples over time. Mentorship and internships work similarly: they expose people to successful behavior whose long tail is difficult to write down as explicit rules.
Specs, code, and evals therefore express complementary parts of the task:
| Component | Defines | Form |
|---|---|---|
| Specs | Intended behavior | Instructions and signatures |
| Code | Required behavior | Enforced control flow and constraints |
| Evals | Successful behavior | Examples and quality criteria |
Together, they give optimization a goal beyond producing an output of the right type.
With a program and metrics in place, an optimizer such as GEPA can search for better implementations. Rivest describes an expanding scope for that search: in the work preceding ChatGPT, code selected few-shot examples because the models were not capable enough to perform the optimization themselves. Stronger models later made automatic instruction optimization practical. The longer-term ambition is to delegate more implementation decisions—including the adoption of new techniques—while retaining the task definition.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make cheaper implementations compete on the same task
Implementation flexibility has an immediate production consequence: you can search for a cheaper way to meet the same requirements. Isaac Miller connects this to the bitter lesson—search across alternatives rather than committing permanently to one implementation. Lower costs can make larger workloads feasible.
Miller reports that Shopify made its implementation 550 times cheaper by moving from an expensive model to a cheaper one while retaining its evals and iterating on business logic. DSPy’s documentation identifies the workload as metadata extraction across Shopify shops; the talk does not specify the model pair, cost accounting, or quality threshold. The production slide also presents Dropbox and Databricks case studies. The useful mechanism is the preserved evaluation target: it lets a cheaper candidate compete on the business task rather than on model reputation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Try new techniques without changing the signature
DSPy’s ecosystem supplies techniques for subproblems; it cannot guarantee that any particular technique solves your application. Miller returns to Recursive Language Models, introducing Alex Zhang as an MIT PhD student and describing RLMs as a way to address some long-context problems. His integration claim is deliberately small: try the technique with a one-line change while keeping the signature fixed, then measure whether it helps.
The same pattern applies to other research available around DSPy: GEPA, which Miller introduces as a Berkeley prompt optimizer; Better Together; and Multi-Module GRPO. These become alternatives inside an implementation rather than reasons to redefine the application’s task. Miller then previews two directions for DSPy 4: dspy.Flex and qualitative learning. These are presented here in the talk’s preview context; the current documentation’s inclusion of Flex does not establish its release status at the recording.
dspy.Flex extends the optimization target from few-shot examples and prompts to code and harnesses. For a function, the proposal is to learn a custom harness over time: the implementation can change as long as it solves the business problem measured by the specs, code, and evals. The stable contract makes this larger search space useful because it supplies a way to judge the resulting program.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Let feedback improve the evaluation target
Qualitative learning addresses a harder problem: creating the evaluation itself. Miller identifies three difficulties:
- Defining quality: Real tasks rarely have an easy, complete definition of a good result.
- Preserving detail: Labeling an email good or bad conveys less information than explaining what should change in it.
- Representing reality: A dataset and objective are proxies for the environment in which the system must work.
Even an effective optimizer can only climb the hill its evaluation defines.
The research proposal is to use models to interpret textual feedback from the environment and turn it into evals and an optimization objective. Production traces, user actions, product analytics, and questions from the model about how data should be represented could supply evidence. As more feedback arrives, the system would refine the objective while continuing to optimize the program against it. Miller explicitly frames this as an open research question: the goal is to make the hill better reflect the actual business problem, not merely to climb a fixed proxy faster.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Intelligence still needs local context
This is the kind of last-mile problem DSPy’s research ecosystem aims to address. Applied engineering exposes a problem; researchers define it, build a benchmark, and develop techniques; open-source software then distributes the results. That cycle connects a recurring practical difficulty to an implementation others can try.
Would a sufficiently intelligent model make this work unnecessary? Miller distinguishes intelligence from knowing everything about a particular task. His Einstein analogy starts with an obvious knowledge gap: asked to help with email, Einstein might first ask what email is. A future AGI may already understand email, but it still would not automatically know your relationships, how you need to interact with particular people, or the context behind a reply. Those details must be learned over time. Better general capability does not remove the need to communicate the local task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Hold the implementation accountable to the problem
Miller dates DSPy’s focus on programmatic specs, code, and evals to 2022. The techniques have evolved from selecting few-shot examples to optimizing prompts, learning harnesses, and exploring evolving evals. Throughout that progression, the test remains whether a technique lets you solve harder problems or solve your own problem better.
Define the problem, then hold prompts, models, and code accountable to it. That requires data-driven evaluation of each proposed change against the business task. A new technique earns its place through the improvement it produces, while a flexible implementation leaves room to adopt useful inventions from the wider community.
The closing invitation is also a distribution model: use DSPy’s open-source software and open research, join the community on Discord, and contribute new techniques so other developers can apply them behind their own task interfaces. A shared implementation ecosystem becomes more valuable when each application has a clear way to decide what works.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Getting started with task signatures, modules, evaluation and optimization in DSPy.
An inference strategy that lets models inspect external context and recursively process selected portions.
The research paper introducing GEPA's reflective prompt optimization approach.
BetterTogether alternates prompt optimization and model fine-tuning to improve modular language-model programs.
Multi-module GRPO extends policy optimization to modular programs and combines it with prompt optimization.
Further reading
- Moderating Discord with DSPyArticle
Isaac Miller develops a moderation application by separating intent classification, task specification and executable business rules.
Read the complete timestamped transcript
- 0:00
[outro jingle] Please welcome to the stage Maxime Rivest and Isaac Miller. [audience applauding]
- 0:19
Wow! Isaac, myself, all of the DSPy community are so grateful to be here today to get to talk to you about AI programming, DSPy, and the unreasonable effectiveness of separating the task from the model, its harness, and all of the implementation details.
- 0:38
When you think about it, in programming, if we want to repeat a task often, we make it a function. We believe the same should be true for AI programs.
- 0:51
Functions are awesome. Functions are reusable, composable, testable, and optimizable. To make a function, you give it a name, you define some inputs, some outputs, and then you have some implementation logic inside of it.
- 1:08
You get to reuse your functions thousands of times. You can optimize it, but you can also compose it into bigger programs.
- 1:17
One of the really nice things about functions is that you can also package it and distribute it, and someone else can use it, and they just need to know about the contract on top of it to use it, and they can treat it as a black box.
- 1:30
DSPy brings all of these properties to AI programs. And so DSPy is an open-source software in Python that lets you, like I said, bring these properties to your AI workflows and AI programs, and it gives you all of the toolings you need to do that.
- 1:52
Why do you want that? Well, we have been inventing a lot of terms in our fields in the last three years. It's growing fast. We have new models coming every other week.
- 2:04
We have new techniques, new strategies. And if you're like me, you wanna try all of them. But will any of these new specific techniques coming out at a different time really help on your task, on your job?
- 2:19
Well, these are all just implementation tactics, and you wanna put them inside a clear contract. If for your repeated AI task, you define an input interface and an output interface, you get to play in the internals, you get a lot of agility.
- 2:37
Let's make it concrete for AI. So my first AI program I made when I discovered DSPy was that I had some invoices from my farm, and I wanted to extract them to do my taxes.
- 2:49
I wanted to extract the tax values from there.
- 2:53
Then another AI program I did is that on my keyboard in my computer, I have a little command that reads my keyboard shortcuts, read my clipboard, and will correct the grammar for me.
- 3:06
Sometimes I actually want it to also rewrite for clarity, so I have another program that takes text and just rewrites it for clarity, put it back in my keyboard, keyboard, and that's a command, and then I can, like, have a lot of agility and bring it different places.
- 3:19
Inside of that, I can change it how-however I want. A new model comes out, and I can change that. It's super easy 'cause my interface is fixed like that.
- 3:27
I'll skip that one. But they're not, uh, restrained to very easy things and small input output. You can be very ambitious with AI programs. So in these examples, you could have your entire inbox and a new email coming in, and you wanna compose a new drafted reply.
- 3:46
We can do that in DSPy with RLM, recursive language models. This is an idea that came from around our community. Or more like things we probably all do, agentic engineering or vibe coding, you can give it a spec, a repository, and you get a PR.
- 4:02
Those are repeatable tasks. And so, as I have been telling you, when you fix that boundary, you can focus on the how on the top, and then inside of it, you can have a little chat with just a simple prompt.
- 4:17
You can iterate on that prompt. Agents come out, you change it to be an agent. Tools gets invented, you add tools. And then we get into loop engineering, you put that inside of it too.
- 4:28
Anything on the outside of it doesn't change. Your integration and, and anything else doesn't change. And when you have such a hard boundary, you can also start to automatically optimize.
- 4:43
But how can you automatically optimize with just that simple signature? This is not enough. This is not enough to specify your task. And even before ChatGPT came out, the creator of DSPy had started to land on this idea that you need three things to specify your task, and if you have this language and this ability to express
- 5:04
your task in a programming language, you can start to automatically optimize and delegate away the implementation details. So the first one is what should happen. This is instructions. The signatures that I've been showing you are part of that.
- 5:21
Here on the screen, you see the beginning of a real script in DSPy. You set your model at the top, you configure that, and it's fully independent of the signatures here, where you have natural language instruction to extract all taxes and, um,
- 5:36
and if it's eligible to output zero. Then you say that, "I'm gonna give you an input. It's gonna be a string. I want you to give me an output, and it goes-- it's gonna be a string and a float."
- 5:45
This is natural language expressing my needs. This is very powerful and efficient. If you think about it, if you have a friend over coming to play a board game with you and you give them the instructions and they're ready to play.
- 5:58
But if you want to do like AlphaGo or AlphaZero, and you tell them, "You're just gonna learn from example," you're gonna have a long night
- 6:07
And then the second one is what must happen. There are some constraints you have that they have to be listened to, they have to be enforced. The best way to do that is with code.
- 6:20
So I want you to go to the third line, uh, fourth line. You have self.extract and self.recheck. You can see we're doing a predict on the extract taxes, and we're doing a chain of thought on the extract taxes.
- 6:36
The first one is a vanilla program, the second one makes it do some reasoning. Now I'm taking them inside, in the forward, and you can see in the if not pred tax, this is a requirement I have that if my first simple vanilla program doesn't extract my taxes, I want you to rerun with more reasoning.
- 6:56
I mean, I gotta get my taxes right. And then another requirement I have is if the value is below zero, throw. I wanna show that to a human. I don't want to let you go.
- 7:06
This will not change. Like, even if I have AGI, I would hope it doesn't make mistake, but whatever is in the predictor, if they make these mistakes, I still want these things to be true.
- 7:19
So the last one is what good look like.
- 7:23
And when I was young, I was on the farm with my dad, and I asked him, "How do you know that this tree is a maple?" And he couldn't tell me.
- 7:32
He couldn't give me the instruction on how to know this tree is a maple, and he certainly couldn't give me code on how to know this tree is a maple.
- 7:39
And so through time, with example, I learned how to know that a tree is a maple. But this is not limited to things like classifying plants. It's also for all of the long tails in your specifications that are things that are more latent.
- 7:56
These are sometimes the reason why you would do internship, and you would have a mentor and a mentee. You're looking at a lot of g- of examples, and there are long tails of successful behaviors that you have to see and learn.
- 8:07
Now that you have all of these, you have expressed fully, you have all these three languages you can put together, you have the specs, the code, and the evals, and now your goal is fully specified,
- 8:20
and so you can start optimizing. You can use things like JEPA on your metrics and on your program, and you can start optimizing. At the beginning of DSPy, the ChatGPT didn't exist.
- 8:31
The models were not good enough to optimize, and so we were using code to find two shots examples to make the base models, uh, act in the proper way.
- 8:41
Then models got better, and so we could automatically optimize instruction. And in the future, we are starting to be able to be liberated more and more from the implementation details and delegate that away.
- 8:54
And at the end, our hope in DSPy is that you can stick to all of that, and then just the news and the implementation details will be automated for you.
- 9:03
Isaac will talk to you a lot more about what has been released in the last year, what we're releasing now, and all of the future plans we have. Thank you. [audience applauding]
- 9:16
Thanks, Max. So we've given you a pretty big abstract overview of specs, code, and evals. But these aren't things that are just res- restricted to the academic sphere. These are used in production by some of the biggest enterprises for massive gains.
- 9:32
And we see two main benefits when you use DSPy in the enterprise. First is that your implementation becomes cheaper. When you're flexible to what the implementation is, you can use the bitter lesson to search over different solutions, find something that solves your problem cheaply, and you can use this to scale to data sizes that weren't possible with
- 9:55
a more expensive implementation. Shopify, five hundred and fifty times cheaper. They're able to do that because they went from an expensive model to a cheap model, but they could keep the same evals, keep iterating on their business logic inside and try new things.
- 10:13
There's three awesome case studies here, and you should check them out after the talk. They give you a lot of details on how you can do this in your own enterprise.
- 10:22
Now, part of the reason why you want to build in the DSPy ecosystem is that we're constantly adding new techniques for you to try.
- 10:32
And it's important to note none of these techniques we add will definitely solve your problem, because that's your job. What we can do is we can solve subproblems for you that make your implementation easier.
- 10:44
For instance, Alex Zhang, a PhD student at MIT, came out with this paper called Recursive Language Models. Recursive language models are a way to solve some kinds of long context programs.
- 10:58
And guess what? We can bring this into DSPy for you to try. See if it helps your long context tasks. Maybe it will, maybe it won't. But the thing is, it's one line, and your signature stays the same.
- 11:10
That's what's important here. Everything gets to stay constant, and you get to see if this solves your problem or not.
- 11:17
And we've had a number of examples of this just in the last year from people building in and around the DSPy community. We've had RLMs. We've had JEPA, which is an incredible prompt optimizer out of Berkeley.
- 11:29
Better Together, Multi-Module GRPO. All these are incredible research innovations that you get to try in your implementation just by being in the DSPy ecosystem.
- 11:41
And we have more coming in DSPy 4, and I'm excited to talk to you about two of those today: DSPy Flex and qualitative learning.
- 11:51
DSPy.Flex is a new kind of module. In DSPy, when we let you optimize things, it started with few-shot examples, then it became prompts, and now that's becoming code.
- 12:03
For any function that you want to implement, you can actually learn a harness over time to solve that function. And this is completely custom, and you don't care about the implementation as long as it solves your business problem, which you've created ways to measure because you've defined the three core parts of specs, code, and evals.
- 12:26
The second thing I'm excited to talk about is qualitative learning.
- 12:31
One of the hard, hard problems in AI engineering is building evals. And there's a few reasons why this is hard.
- 12:39
One is that defining what good looks like is really challenging for any real-world problem.
- 12:46
The second is that when you define good, oftentimes you have to lose detail. If an email is good or bad, contains a lot less information than if you know what could change in that email in order to improve.
- 12:59
And the third is that whenever you create a hill and a dataset, you're really trying to create a proxy for reality. What if instead we could use reality to inform our evals automatically?
- 13:13
What qualitative learning asks is how do we decrease this question? How do we decrease assistance? And it's a research question right now. But what we believe is that models are now good enough to interpret whatever textual feedback is present in the environment and convert that into evals and a hill that the model can climb.
- 13:33
And so as you get more feedback from production, its traces, its user actions, its product analytics, it's asking you-- it's the model asking you questions about how data should be represented.
- 13:44
As you do this, the model can iteratively refine the hill over time and continue climbing it to solve your actual business problem.
- 13:55
And DSPy focuses on these kinds of last-mile problems. We have a really strong research ecosystem, and we collaborate really closely with them. And that's part of the beauty is that we can see the problems that happen in applied AI engineering, so- define them, build a benchmark, and then solve them with techniques.
- 14:14
And then we get to democratize the results of that to everyone because it's open source, open research.
- 14:21
Now, one common question is what happens when we have AGI?
- 14:27
Well, even when we have an incredibly smart model, the model won't know how to solve your problems. It won't know how to do your tasks or have your context.
- 14:36
And so this genre of last-mile learning is trying to ask how do we efficiently do this learning?
- 14:42
Intelligence is very different from being all-knowing. If you were to ask Albert Einstein to help you with your emails, he'd probably ask what's an email?
- 14:54
But if you a- AGI will know how to do your emails.
- 14:59
Nevertheless, it won't know how to actually solve your problem and interact with the people you need to interact with. It won't understand your relationships without learning this context over time.
- 15:12
Since twenty twenty-two, DSPy has been focused on these three core ideas of specs, code, and evals, all defined as a programmatic interface.
- 15:24
We've certainly evolved over time, and new techniques are incredible. We've gone from evolving few shots to prompts to now harnesses and now evolving your evals over time too.
- 15:35
But what you need to ask for any of these new techniques is how do they help you solve harder problems or solve your own problems better?
- 15:46
And you should ask this question in a data-driven manner.
- 15:50
You should look at this new technique, say how can I apply this to the business problem that I have? You should define your problem, and you should hold your prompts, models, and code accountable to the problem that you need them to solve.
- 16:04
And what's awesome about when you build in this way where you have flexible implementations, what you unlock is you unlock the ecosystem of all the techniques that anyone in this room is constantly inventing.
- 16:17
You unlock access to the collective intelligence of everyone here, all sharing techniques together.
- 16:26
So if you wanna build reliable AI software, I encourage you to come check out DSPy. We're completely open source, open research, and we're here to help you solve your problems by building reliable software.
- 16:41
We have a Discord that you should come join. And when you come up with the next technique, you should come contribute it to DSPy, and we can help you distribute it and make this awesome technique available for everyone.
- 16:52
Thank you. [audience applauding] [upbeat music]