The Unreasonable Effectiveness of Separating the Task from the Model

If you repeat an AI task, why not give it the same structure as any other reusable function? Define its name, inputs, and outputs, then put the implementation behind that interface. Callers can reuse, compose, test, optimize, package, and distribute the function without understanding everything inside it. DSPy brings those properties to AI programs through an open-source Python framework.

That boundary becomes valuable when models and techniques change every few weeks. A new prompting strategy or agent architecture is an implementation choice; the task it must accomplish is the contract. Keep the input and output interfaces stable, and you can experiment inside them without rebuilding every integration. The practical question becomes whether a new technique improves your task, rather than whether you have adopted the latest technique.

Maxime Rivest’s first DSPy program extracted tax values from his farm invoices. Another utility used a keyboard command to read clipboard text and correct its grammar. A separate program rewrote text for clarity. Each had a small, repeatable job and a stable interface, so changing the model inside did not require changing how he invoked the utility.

The same structure can describe much larger workflows:

Rivest proposes Recursive Language Models, or RLMs, as one possible implementation for the inbox task. These are ambitious task boundaries, not just wrappers around short text transformations.

Inside a boundary, the implementation can progress from a simple prompt to an improved prompt, an agent, tool use, and more elaborate loops. The surrounding application still supplies the same inputs and consumes the same outputs. A stable boundary also makes automatic optimization possible—but an input/output signature alone does not tell an optimizer everything it needs to know.

The first part of a task definition is what should happen: natural-language instructions and a typed signature. Model configuration sits separately from that specification. In the tax example, the program accepts invoice text and produces string and floating-point outputs. The instructions describe tax extraction and include a condition for returning zero; that business rule belongs in the task definition, independently of the chosen model.

Natural language communicates intent efficiently. When a friend arrives to play a board game, you give them the rules so they can begin. Asking them to infer the game entirely from examples—as in Rivest’s analogy to AlphaGo and AlphaZero—would make for a long evening. Examples matter, but they need not carry information that instructions can express directly.

The second part is what must happen. Requirements that must hold belong in executable code. Rivest’s tax module uses self.extract with dspy.Predict for a direct prediction and self.recheck with dspy.ChainOfThought for a reasoning pass over the same task. Its forward method follows a prescribed sequence:

1. Run the direct extractor.
2. If the initial tax result fails the truthiness check, rerun extraction with reasoning.
3. If the resulting tax value is negative, raise an exception so a human can review it.

The model does not decide whether these safeguards apply.

A Python module expressing that pattern keeps the task signature and the control flow separate:

import dspy


class ExtractTaxes(dspy.Signature):
    """Extract the total tax amount from an invoice."""

    invoice: str = dspy.InputField()
    explanation: str = dspy.OutputField()
    taxes: float = dspy.OutputField()


class TaxExtractor(dspy.Module):
    def __init__(self):
        super().__init__()
        self.extract = dspy.Predict(ExtractTaxes)
        self.recheck = dspy.ChainOfThought(ExtractTaxes)

    def forward(self, invoice: str):
        pred = self.extract(invoice=invoice)
        if not pred.taxes:
            pred = self.recheck(invoice=invoice)
        if pred.taxes < 0:
            raise ValueError("Negative tax amount requires human review")
        return pred

A truthiness check on a floating-point amount also retries a legitimate zero; it does not distinguish zero from an unsuccessful extraction. The essential design is that retry and rejection rules remain outside the predictors. Even a much more capable future model would still have to obey them.

The third part is what good looks like. As a child on the farm, Rivest asked his father how he knew a tree was a maple. His father could recognize it but could not supply a satisfactory instruction, much less code. Rivest learned through examples over time. Mentorship and internships work similarly: they expose people to successful behavior whose long tail is difficult to write down as explicit rules.

Specs, code, and evals therefore express complementary parts of the task:

Together, they give optimization a goal beyond producing an output of the right type.

With a program and metrics in place, an optimizer such as GEPA can search for better implementations. Rivest describes an expanding scope for that search: in the work preceding ChatGPT, code selected few-shot examples because the models were not capable enough to perform the optimization themselves. Stronger models later made automatic instruction optimization practical. The longer-term ambition is to delegate more implementation decisions—including the adoption of new techniques—while retaining the task definition.

Implementation flexibility has an immediate production consequence: you can search for a cheaper way to meet the same requirements. Isaac Miller connects this to the bitter lesson—search across alternatives rather than committing permanently to one implementation. Lower costs can make larger workloads feasible.

Miller reports that Shopify made its implementation 550 times cheaper by moving from an expensive model to a cheaper one while retaining its evals and iterating on business logic. DSPy’s documentation identifies the workload as metadata extraction across Shopify shops; the talk does not specify the model pair, cost accounting, or quality threshold. The production slide also presents Dropbox and Databricks case studies. The useful mechanism is the preserved evaluation target: it lets a cheaper candidate compete on the business task rather than on model reputation.

DSPy’s ecosystem supplies techniques for subproblems; it cannot guarantee that any particular technique solves your application. Miller returns to Recursive Language Models, introducing Alex Zhang as an MIT PhD student and describing RLMs as a way to address some long-context problems. His integration claim is deliberately small: try the technique with a one-line change while keeping the signature fixed, then measure whether it helps.

The same pattern applies to other research available around DSPy: GEPA, which Miller introduces as a Berkeley prompt optimizer; Better Together; and Multi-Module GRPO. These become alternatives inside an implementation rather than reasons to redefine the application’s task. Miller then previews two directions for DSPy 4: dspy.Flex and qualitative learning. These are presented here in the talk’s preview context; the current documentation’s inclusion of Flex does not establish its release status at the recording.

dspy.Flex extends the optimization target from few-shot examples and prompts to code and harnesses. For a function, the proposal is to learn a custom harness over time: the implementation can change as long as it solves the business problem measured by the specs, code, and evals. The stable contract makes this larger search space useful because it supplies a way to judge the resulting program.

Qualitative learning addresses a harder problem: creating the evaluation itself. Miller identifies three difficulties:

• Defining quality: Real tasks rarely have an easy, complete definition of a good result.
• Preserving detail: Labeling an email good or bad conveys less information than explaining what should change in it.
• Representing reality: A dataset and objective are proxies for the environment in which the system must work.

Even an effective optimizer can only climb the hill its evaluation defines.

The research proposal is to use models to interpret textual feedback from the environment and turn it into evals and an optimization objective. Production traces, user actions, product analytics, and questions from the model about how data should be represented could supply evidence. As more feedback arrives, the system would refine the objective while continuing to optimize the program against it. Miller explicitly frames this as an open research question: the goal is to make the hill better reflect the actual business problem, not merely to climb a fixed proxy faster.

This is the kind of last-mile problem DSPy’s research ecosystem aims to address. Applied engineering exposes a problem; researchers define it, build a benchmark, and develop techniques; open-source software then distributes the results. That cycle connects a recurring practical difficulty to an implementation others can try.

Would a sufficiently intelligent model make this work unnecessary? Miller distinguishes intelligence from knowing everything about a particular task. His Einstein analogy starts with an obvious knowledge gap: asked to help with email, Einstein might first ask what email is. A future AGI may already understand email, but it still would not automatically know your relationships, how you need to interact with particular people, or the context behind a reply. Those details must be learned over time. Better general capability does not remove the need to communicate the local task.

Miller dates DSPy’s focus on programmatic specs, code, and evals to 2022. The techniques have evolved from selecting few-shot examples to optimizing prompts, learning harnesses, and exploring evolving evals. Throughout that progression, the test remains whether a technique lets you solve harder problems or solve your own problem better.

Define the problem, then hold prompts, models, and code accountable to it. That requires data-driven evaluation of each proposed change against the business task. A new technique earns its place through the improvement it produces, while a flexible implementation leaves room to adopt useful inventions from the wider community.

The closing invitation is also a distribution model: use DSPy’s open-source software and open research, join the community on Discord, and contribute new techniques so other developers can apply them behind their own task interfaces. A shared implementation ecosystem becomes more valuable when each application has a clear way to decide what works.