Pydantic is all you need: Jason Liu

Jason Liu frames the central production problem as connecting language models to software that already expects specific schemas, APIs, and data structures. Although many developers encountered these models through chat interfaces, the systems they build often need to process inputs and produce structured outputs compatible with interfaces they cannot change. In that setting, asking a model for JSON and hoping the result can be parsed is not a reliable programming model. 0:14

Even when an answer resembles valid JSON, subtle contract violations can remain invisible until downstream code fails: one response might use a user field while another uses username, or the model might surround its payload with conversational text. Liu argues that these errors should surface through explicit contracts rather than relying on logs, regular expressions, and careful inspection of strings. 2:07

OpenAI function calling improves the situation by allowing developers to specify an output JSON schema and receive structured arguments in a more predictable location. However, Liu emphasizes that parsing JSON into dictionaries still leaves applications exposed to missing keys, incorrect types, spelling differences, and additional handwritten checks. A better interface needs to validate the data and give application code an object it can use directly. 2:07

Liu presents Pydantic as the missing layer between language-model responses and conventional Python systems. Its type hints define data models, its field and model validation enforce expectations, and its JSON schema output provides a format that can be passed into OpenAI function calling. In his delivery example, a timestamp supplied as a string and dimensions supplied as a list of strings can be parsed into the declared datetime and tuple-of-integers representations. 3:04

The benefits extend beyond successful parsing. Once timestamp and dimensions are declared fields, an IDE can expose their types, offer autocomplete, and catch naming mistakes. The prompt also becomes more reviewable: instead of burying the desired structure inside a long text instruction or manually maintained JSON example, developers express the output contract in code they can inspect and evolve. 3:58

Liu describes Instructor as a library he built to connect this contract to OpenAI function calling. In the workflow he presents, developers patch the completion API, declare a Pydantic object as the response model, and receive an instance of that model instead of manually extracting an untyped dictionary. He notes that patching the completion API is a debatable design choice and that this implementation only works with OpenAI function calling; he identifies Marvin as an alternative framework offering access to more language models and additional capabilities. 4:59

A Pydantic model can represent more than a flat response: it can contain nested references, lists of related objects, reusable classes, and methods that define behavior. Liu illustrates this with user details, addresses, best friends, and collections of friends. Because class docstrings and field descriptions become part of the JSON schema sent to the model, documentation influences both how the model interprets its task and how developers understand the resulting objects. 5:57

This produces a single review surface for prompt quality, data quality, and code quality. Good variable names, descriptive fields, and accurate documentation become operational parts of the model interaction rather than separate artifacts that can drift apart. Liu’s argument is not simply that structured generation produces cleaner JSON; it is that the model should capture the prompt, the data, and the behavior together. 5:57

Validation adds an explicit boundary around model output. Standard validators can normalize or reject values and return errors that application code can catch, while an LLM-backed validator can assess a qualitative condition such as whether a statement is objectionable. Instructor can then use a configured retry limit to send validation failures back to the language model and request a corrected response. 6:50

Liu distinguishes this retry loop from broader prompting frameworks: the mechanism is validation, error handling, and re-asking implemented as separate, manageable pieces of software. Rules can include a maximum character count or an external check that a name exists in a database. The tradeoff is that developers must define the relevant constraints and recovery behavior explicitly, rather than assuming the initial prompt will enforce every business rule. 7:51

Structured outputs can express uncertainty without depending on a model to reproduce a specific sentinel phrase. Liu proposes a wrapper that contains an optional extracted user, an error, and an error message, while individual fields such as a role can also be optional. This escape hatch makes an absent result a first-class application state that downstream code can branch on, instead of a fragile string comparison against a phrase meaning that the model does not know. 8:47

Reusable components also make prompt behavior more modular. Liu defines work time and leisure time using the same time-range structure, with start and end fields shared across both. If extraction is unreliable, he suggests adding a chain-of-thought field within that component and comparing configurations with and without it to understand latency and performance tradeoffs between testing and production. 9:49

For open-ended extraction, a list of key-value properties can represent arbitrary attributes without abandoning structure. Consistent property keys can be encouraged through instructions, checked through validators, and repaired through retries; an explicit index can help constrain the list to a desired number of properties. Similarly, users with identifiers and lists of friend identifiers can represent a network that ordinary graph-processing code can traverse. 9:49

Liu applies the same modeling approach to retrieval-augmented generation. Rather than assuming every question maps to a single vector search, he describes structured search requests containing a search type, title, query, and date filter. A method can dispatch video searches and email searches to different backends, while a model-generated list of searches can be processed asynchronously using conventional application code. 11:41

Planning can also be represented as a graph. Liu describes a query plan whose nodes contain an identifier, a question, and dependencies, allowing independent lookups to run in parallel before later nodes combine their results. His example contrasts this single model-generated plan with repeatedly asking an agent what to do next; after the plan is produced, execution becomes an ordinary dependency-management and retrieval problem. 12:37

For knowledge-graph extraction, Liu recommends shaping the generated structure to match the graph visualization API as closely as possible. By aligning model output with the downstream consumer, the code needed to create and render a graph becomes simpler. He presents a description of quantum mechanics as an example input that can be transformed into a visualization through a compact implementation. 13:40

The most stringent example concerns grounded question answering. Liu models an answer as a list of facts, each associated with one or more substring excerpts from the source text. Validators discard facts whose excerpts cannot be found in the original chunk, and a second validation layer retains only facts with at least one verified excerpt. This proves that cited text exists in the supplied material; the mechanism described checks substring presence and does not, by itself, establish that every interpretation or paraphrase is correct. 14:36

Liu’s broader claim is that structured prompting shifts the work from improvising better instructions toward domain modeling. Objects can carry per-object instructions, nested or recursive structures, and behavior that allows generated data to integrate with established software. Once a model produces a graph, workflow, or plan, ordinary code can traverse it or dispatch it to a system such as Airflow instead of relying on an unconstrained loop. 10:50

He also identifies structured evaluation as an area of ongoing experimentation: one interface might assess whether a response is mean while another examines the distribution of numerical attributes and evaluates them against explicit expectations. The presentation characterizes this as open work rather than a completed or universally solved evaluation system. 16:35

Finally, Liu offers multimodal and generative interfaces as prospective applications rather than established results. He imagines extracting image bounding boxes alongside product-search queries, then rendering an interface element for each region, and extends the idea to generated interfaces over images or audio. These examples reinforce the central thesis: the useful unit is not merely a fluent response, but a structured object that existing software can inspect and act on. 16:35