AI Engineer World's Fair 2026
Structuring the Unstructured: Advanced Document Parsing for AI Workflows
Read the talk
Structuring Unstructured Documents with Docling
Turn PDFs into usable AI context by preserving layout, extracting tables and pictures, enriching diagrams, and retrieving through document structure before exposing conversion tools to agents.
From a talk by Cedric Clyburn
Before you start: Basic Python and familiarity with retrieval-augmented generation will help you follow the notebook examples.
What context disappears inside a PDF?
How do you give an AI application useful context when the information lives in PDFs, presentations, contracts, scanned pages, diagrams and tables? A harness can manage an LLM’s context, but it cannot recover information that an ingestion pipeline has already discarded. Cedric Clyburn, introducing himself as an open source engineer at Red Hat, starts with this practical gap between having documents and being able to use them.
Enterprise documents are often scattered across dozens of systems. Processing them may also mean accepting a proprietary service or sending private data to another organization’s servers. The task is therefore more specific than extracting text: preserve the relationships in graphs, tables and images while converting them into representations such as Markdown or JSON. Docling, introduced here as an open source Linux Foundation project, supplies local tools for extraction, parsing and chunking.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An extraction error can become someone else’s evidence
Technical documentation, meeting minutes and invoices can become retrieval context or training data for a specialized model. In either case, document processing affects what the model can answer correctly. Better inference hardware or a different model cannot reliably reconstruct a table relationship that disappeared before the data reached it.
Clyburn cites a report of twenty scientific papers containing a nonsensical term, attributing its origin to words incorrectly joined across columns in an old scanned article. The matching reporting on “vegetative electron microscopy” treated that OCR explanation as a hypothesis; a follow-up investigation also considered Farsi typo or translation explanations. The precise origin remains uncertain, but the example illustrates the propagation problem: questionable text enters papers, and those papers become material that others cite.
A layout-aware view makes the proposed failure easy to understand. In Clyburn’s local Docling view, the two words occupy separated regions and should not be combined. Spatial structure is part of the evidence, not merely a formatting preference. Preserving it gives downstream systems a better basis for extraction and validation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Cheap text extraction loses relationships
The next example puts a PDF containing text sections, a table, an image and captions beside a simple parser’s Markdown output. The parser is fast and inexpensive to run on a CPU, but text is truncated or merged, and the table becomes a linear stream. A question about a particular table entry now requires reconstructing relationships that the output no longer expresses. Unwanted page headers remain, while the image’s content is missing.
A frontier model can produce more convincing Markdown, but its economics and output stability become consequential across a large collection. Clyburn uses an illustrative price of $30 per million output tokens to explain how processing costs accumulate across many PDFs. He also raises a hypothetical model change from version 5.1 to 5.2: even when the output looks accurate, a replacement model can change how it expresses structure. Nondeterminism and hallucinations add another source of variability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recover structure locally before choosing an output
Docling provides a CLI and Python library that combine OCR with specialized vision models to recover document structure. Its output is a Pydantic document object, which can then be exported as Markdown or JSON. In the table example, Clyburn renders the recovered table as HTML; because the table’s structure survives extraction, an application can work with a selected column instead of interpreting flattened text.
Clyburn’s Red Hat team uses Docling with thousands of product-documentation PDFs. Installation begins with pip install docling, and conversion can cover documents and websites while retaining layout information. Running locally also accommodates private data and air-gapped environments without requiring a hosted conversion service. The useful distinction among the approaches is what each preserves and what the application must manage afterward.
| Approach | Useful property | Main concern in the walkthrough |
|---|---|---|
| Simple PDF parser | Fast, cheap CPU extraction | Lost reading order and table relationships |
| Frontier model | Richer interpretation | Token cost and variable output |
| Docling pipeline | Local, structured conversion | Recover layout before downstream use |
The middle ground is to make document structure an explicit intermediate representation, rather than asking every downstream model call to rediscover it.
For scale, Clyburn points to Leandro von Werra’s work preparing Common Crawl PDFs for FinePDFs. Preprocessing and structural extraction help remove unwanted material before training-data export. Clyburn reports roughly 50× lower processing cost in the FinePDFs example, emphasizing that the Docling path runs on CPUs without requiring a GPU. The published cost breakdown compares different routed workloads: customized CPU Docling handles extractable PDFs, while GPU RolmOCR handles scanned or truncated PDFs. That makes the figure a pipeline-specific cost comparison, not a same-document benchmark.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Descriptions and fields serve different needs
Once document content is accessible, two further operations become useful:
- Image enrichment: A vision-language model describes and annotates an image, adding context that a RAG application or human reader can use. This can help explain an organizational diagram even after the subject-matter expert who understood it has left. The model can be local or proprietary.
- Field extraction: An invoice workflow asks for the bill number, total price and sender name, returning those fields in a Pydantic structure. It does not need unrelated headings or every line from the document.
The first operation expands the available description; the second narrows the output to a requested schema. Clyburn then moves into workshop notebooks to demonstrate the underlying conversion steps.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Convert a PDF, then inspect the document object
The notebook starts with Docling’s own research-paper PDF, opened from an online source. It contains titles, subtitles, pictures, captions and tabular content—the elements a plain text extraction can lose. The first operation is straightforward: import DocumentConverter, convert the source, and export the resulting document to Markdown.
For a local copy saved as paper.pdf, the same conversion pattern is:
python
from pathlib import Path
from docling.document_converter import DocumentConverter
source = Path("paper.pdf")
result = DocumentConverter().convert(source)
document = result.document
markdown = document.export_to_markdown()
Path("paper.md").write_text(markdown, encoding="utf-8")
print(markdown)
Clyburn converts an eight-page PDF to Markdown in the notebook. The displayed result includes the paper’s title, authors, abstract and section headings; he does not give a numerical runtime.
Markdown is only one view of the result. The Pydantic object also exposes pages, tables and the relationship between content and its source page. Keeping that object lets the application inspect structure before exporting it as Markdown, HTML or a dictionary:
python
print(f"Pages: {len(document.pages)}")
print(f"Tables: {len(document.tables)}")
html = document.export_to_html()
structured_data = document.export_to_dict()
Path("paper.html").write_text(html, encoding="utf-8")
Keep the structured document as the working representation; export text when a consumer needs text.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Inspect tables, pictures and their positions
Table extraction makes the value of that representation concrete. Each recovered table can become a data frame, which Jupyter can display directly and an application can query without first rebuilding rows and columns from prose. Clyburn exports eight tables from the demonstrated source PDF into data frames.
python
from IPython.display import display
for index, table in enumerate(document.tables, start=1):
frame = table.export_to_dataframe(doc=document)
print(f"Table {index}")
display(frame)
This preserves a useful distinction: a table can be available to retrieval while still remaining tabular data.
Pictures require an additional PDF-pipeline setup. Clyburn configures image scaling before conversion, then inspects the extracted picture content. The notebook brings together the source image, its caption and embedded text elements. That gives a retrieval application more than an isolated image file: it retains the surrounding information needed to ask what the picture depicts.
The layout visualizer then exposes bounding boxes for section headers, text, subtitles and pictures. These coordinates show where extracted elements came from and provide a way to inspect whether the parser has separated the page correctly. Clyburn proposes using this spatial information to support removal of customer personally identifiable information before ingestion. Coordinates support such a workflow, but visualizing them does not itself detect sensitive information or complete redaction.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Add a description beyond the original caption
Extracting a caption and describing an image are different operations. The next pipeline uses a locally running Granite vision model to request a detailed description of the picture. Ollama exposes the local model through an OpenAI-compatible completions interface; a third-party model is another option.
The example returns to the Docling pipeline diagram extracted earlier. Previously, the notebook had the original caption. After enrichment, it also has a generated description of what the diagram shows. This adds retrievable context for questions about visual content while preserving the distinction between text present in the source and interpretation supplied by a model.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Use the document outline as the retrieval index
With the document parsed and enriched, retrieval does not have to begin with fixed chunks. In the demonstrated chunkless RAG pattern, an LLM reads a document outline, chooses a relevant section, and retrieves that section’s full text from the Docling document. It can repeat the operation in an agentic loop. This path uses no chunker, embedding model or vector database.
| Retrieval approach | Index | Selection mechanism |
|---|---|---|
| Vector retrieval | Embedded chunks | Semantic similarity |
| Outline retrieval | Markdown headings and section summaries | LLM chooses sections |
Instead of searching thousands of stored vectors, the agent navigates a compact description of the document’s structure. A getting-started section, for example, provides the route to text explaining that Docling can be installed from PyPI. The outline points to the evidence; the full section supplies it.
The first query is: “What are the main AI models used in Docling?” The retrieval procedure is:
- Read the available section outline.
- Select the section likely to discuss the models.
- Fetch its full text and assess whether it answers the question.
- Answer from that material, or retrieve another section if evidence is missing.
For the Docling-models question, Clyburn allows about five iterations over twenty available sections, and the demonstration returns an answer in one iteration. The significant behavior is that the answer comes from navigating document structure rather than consulting a vector database.
The next example increases the search space. Clyburn’s parsed IBM 2025 annual report contains 418 sections. The question asks about Red Hat’s revenue growth in 2025 and its contribution to the overall software segment. Here the agent must assess whether a selected section is enough and fetch more information when it is not. The walkthrough illustrates repeated relevance checks without supplying a final financial answer. Moving from one report to large collections then raises a separate question: how should conversion itself be deployed?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Expose conversion as a service
Docling Serve exposes document processing through a REST API. The deployment can be a microservice, a container or a Kubernetes workload. In the walkthrough, installation starts with pip install docling-serve, followed by launching the server from its CLI on a selected port.
Clients submit documents with options such as whether to use OCR, which processing backend to select, and whether to annotate images. This separates application code from the conversion process and gives multiple consumers a shared processing endpoint. Current Docling Serve documentation uses a stable v1 API; its request syntax should be taken from that documentation rather than inferred from the recorded setup.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give an agent document-processing tools
The REST service addresses centralized processing; Docling MCP addresses how an agent discovers and invokes document operations. Clyburn presents the service as a route to larger collections, without a throughput measurement. Through the Model Context Protocol, a developer assistant can access document-processing capabilities without the user manually assembling every conversion command.
The local example connects a Qwen model running on a Mac to VS Code. The MCP tools cover conversion, generation and manipulation, including operations on a selected part of a PDF. Clyburn checks the local MLX model server, identifies the running model as Qwen 3.6, and checks the Docling MCP server using uvx. The architecture shown connects Continue in VS Code, the model server and Docling’s document tools.
In the demonstrated client setup, adding the MCP server means editing config.yaml; that filename is specific to the client configuration shown. Current Docling MCP defaults to remote conversion, so reproducing a fully local arrangement requires the local installation extra and explicit local configuration. Starting it with uvx alone does not establish that documents remain local.
Once connected, the agent can turn a user request into document operations: convert a document and summarize it, or create a document with action items, pull a list from another PDF, and export the assembled result as Markdown. Continue, Claude Code, Codex and Cursor are among the client options mentioned. The interface changes from a notebook cell or REST request to an agent tool call, while the underlying parsing capabilities remain available.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One structured document, multiple downstream paths
The architecture ends where the first conversion began: OCR and layout analysis assemble a Pydantic Docling document, and downstream consumers choose how to use it. PDF-to-Markdown or JSON conversion can run entirely locally without a GPU. The same structured result can support format exports, dataset creation, or retrieval through an outline.
Chunkless retrieval is one option, not a requirement imposed by the parser. Docling also offers a hybrid chunker for workflows that need chunks, alongside integrations with RAG frameworks, agentic systems and harnesses. Preserving structure first keeps those choices open: the application can select a retrieval or agent architecture without starting over from a flattened, incomplete copy of the source.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Local document conversion library and CLI with layout analysis, OCR and structured exports.
PDF-derived training corpus with documentation of CPU extraction, GPU OCR, filtering and deduplication.
Run Docling as a REST service using Python or published container images.
Expose document conversion and generation tools to MCP clients.
Further reading
Hands-on lab for retrieving through document structure and comparing the approach with hybrid chunking.
- FinePDFs processing costsArticle
Leandro von Werra's breakdown of document counts, compute hours and extraction spending.
Technical background on the original open-source PDF conversion package.
Reporting on competing explanations for an anomalous phrase appearing in scientific papers.
Read the complete timestamped transcript
- 0:00
Hey, hey. Welcome. My name is Cedric Clyburn. I'm an open source engineer here at Red Hat, and I think we can all agree that context is the most important aspect to building an AI application or an agent, right?
- 0:11
It's the reason that harnesses have become so popular in order to manage the LLM's context. But the thing is, no matter what model or agent that you're using, there is so much data that we're not able to use properly because it's in unstructured formats.
- 0:25
I'm talking everything from PDFs to presentations to contracts and technical docs, even meeting notes, scanned documents, diagrams, tables, images and more. And sorry, I know that's a lot, but you understand what I mean, right?
- 0:38
All this data needs to be transformed into something that an LLM can actually understand. And that's why by the end of this session, you'll understand how to extract structure between raw enterprise documents and use it to power better downstream AI systems like RAG and agents.
- 0:54
So let's get started. Now, I think Jensen from NVIDIA made this point super clear at his keynote, that unstructured data is becoming this new context layer for AI. And the reality, though, for many teams, and I know this personally working at Red Hat, that PDFs and data are spread across dozens of different systems.
- 1:12
So we've got a lot to cover today. As you might know, a large majority of the world's data is unstructured, and so no matter what model you're using, if you're working with data like PDFs and unstructured types of formats, this is a bit tricky to work with because there are solutions out there, but they might be proprietary
- 1:29
or require you to send your private data to someone else's server. And for not just text, how do we take documents and their graphs or tables and images to formats that LLMs can understand, like Markdown or JSON?
- 1:42
And I'm going to show you how in the session today, because we're going to be using an open source tool, part of the Linux Foundation that is called Docling, and learn about extraction, parsing, chunking and much more.
- 1:53
And I've got some live demos for you, so we're going to have some fun. And just in case you'd like it, we have the session slides here on the right and a little overview of the specifics I'll be showing you in today's session.
- 2:04
But without further ado, let's get started. So why is there a need for advanced document processing? As I briefly mentioned before, you might have a lot of technical documentation or meeting minutes or different types of documents and invoices that you need to use in maybe RAG or different type of applications where the, uh, context is provided to
- 2:25
an LLM. So whether it's RAG or retrieval-augmented generation to answer questions based on this data, or you're using this to fine-tune a new specialized model, well, data is this key ingredient behind those applications.
- 2:39
And it doesn't matter if you're using, you know, NVIDIA acceleration or, uh, an open source or proprietary model. That data and the way you process it is the key determining factor in whether your answer is going to be correct or incorrect for the user or customer at the end of the day.
- 2:56
And that's what's most important. And how important is it? Well, I have this viral tweet from earlier where twenty scientific papers now feature a new nonsensical term that doesn't exist because AI misinterpreted a very old article that was scanned and taken to a PDF, merging two different words from two different columns in this PDF.
- 3:17
And because researchers are using these models in order to help them, right, now we have different types of scientific papers that all feature this word and are even being cited by other people.
- 3:28
And so that's how important it is to make sure that the data that we're processing is processed in a way that's accurate and not hallucinated and able to be used confidently in our applications that we're delivering to users and customers.
- 3:42
So it's quite important. Now, if we were to use a tool like Docling that I can run on my own machine, you could see that these two words are quite far away from each other and shouldn't have been combined in the first place.
- 3:54
But that's how we're going to learn about extracting this text here in a second. Now, if we were to try a simple PDF parser for a PDF like this that includes, um, a table here, it has an image, there's captions, and there's regular sections of text.
- 4:08
Well, we might get an answer like this here on the right in Markdown. You know, this might be very fast and cheap to run even on CPU. But the issue is, is that a lot of this text has been truncated, has been merged, and isn't decipherable even by me as a human.
- 4:25
And if I sent this to a model, I don't think I could trust that the model could extract specifics from, say, for example, this table, because the table has been kind of just spit out linearly.
- 4:35
And this information isn't fit for most use cases where I need to ask questions or have an agent do validation and extraction on this s-source data. So this isn't going to cut it, right?
- 4:48
There's undesired page headers. We don't understand the table. And where's the content from the image, right? It's not even there. When we are using frontier models, this is, um, kind of not bad, right?
- 4:59
But quite expensive. If I'm sending this to a model that's maybe thirty dollars per million output tokens, you can see how this can get quite expensive as I scale this up to dozens or hundreds or in a lot of cases, thousands of PDFs that organizations have to work through to use in an AI application.
- 5:16
And the differences between maybe a five point one of a model that was depreciated and the five point two version of a model make it tricky to have structured output that's consistent every single time.
- 5:27
And so while it might be good quality, and I can see that most of this looks accurate in the exported Markdown, we might be susceptible to hallucinations because models are non-deterministic, and this is really tricky at scale.
- 5:40
And so what is the middle ground? Well, that's where Docling comes in. It's a fast and cheap and most importantly, local CLI and library that I can use to take various types of input sources and convert this to Markdown, JSON in a Pydantic data type that I can use in my applications and that I can scale up
- 6:00
if I have thousands of different types of formats that need to be used or translated to something like Markdown. And so we re-rendered this as HTML, but you can see here we've got this export of the specific table that we had here previously, because as a PDF This data type is spread out and it's proprietary, so it's
- 6:22
hard to extract this type of data from the source PDF here. And I'll show you how Docling does it by using a combination of OCR and specific vision models that extract the format and allow me to do things like structured output if I only, say, for example, want a specific column to be outputted from a content source.
- 6:43
So it's really cool. And I've been using it at Red Hat. My team uses it because we have thousands of PDFs that we need to work with, specifically from product documentation.
- 6:52
But also we have content like images that we want to extract using vision models. And what Docling does with a pip install Docling is allow me to convert single documents or websites or anything else to Markdown or whatever file type I want and be able to work with the page layout, so I don't lose the consistency and
- 7:10
the structure of the source document. And there's a lot of other integrations and for situations where maybe you need to run this locally, or you don't want to pay for a service, or you have an air-gapped environment.
- 7:23
So you're able to do this using this open source project that is part of the Linux Foundation. So before I show you the quick demo that I want to highlight the project, I want to talk a little bit about scale and cost.
- 7:36
As I just mentioned it there earlier, but there is a public use case I want to show you from Leandro at Hugging Face, where he compared a source of Common Crawl PDFs and where he did a little bit of pre-work on them and actually extracted the structure using OCR and using Docling in order to remove certain parts
- 7:55
and clean this up for the fine PDFs export, which is thousands of tokens from PDFs around the web that you could use for training a model, et cetera. But the two comparisons he did using GPU and using CPU for Docling allow him to do this at fifty times of a cost savings compared to VLMs and OCR naively.
- 8:19
And so this is what's really cool is that you can really scale this up, and he did this on CPU and not needing GPU, which is really cool. And it's not just document conversion, right?
- 8:29
So his example was getting things ready from PDFs, but let's say we have images, right? This image with a vision language model was able to be described and annotated right from this specific image right here.
- 8:42
Now we have all of this really important context that can be used in a RAG application as an example, but also just for the end user to be able to understand what's happening in this image.
- 8:53
And say, for example, the SME doesn't work there anymore at the organization, now we have a way to understand what this is with the help of an LLM, whether proprietary or local.
- 9:04
And then finally, it's not just document conversion and annotation, but also structured output. Say, for example, I have this invoice and I need to extract specifically the bill number, the total invoice price, uh, the name of the sender.
- 9:17
We're able to get that in a format that is structured, so I don't have to worry about, hey, it's going to pull in like the headings and titles. No, I just want the, you know, total invoice price and the bill number, and I can get that in a format that's Pydantic, but also just, uh, super simple if
- 9:34
I'm just trying to get a couple things out of a huge document. So let's hop over to the demo. Let me show you what I'm talking about. And by the way, we're going to be using this Docling workshop repository for the demos today, so feel free to check it out.
- 9:45
In this first example, we're going to be using Docling to convert a popular file type like a PDF because remember that data is the foundation for all AI systems, and in order to leverage that data, we have to properly ingest different file formats with accuracy.
- 9:58
And without doing that, we can lose information or information can become unreadable from tables and diagrams and images. So with Docling, what we're able to do with a simple pip install down here is start processing some of these different file types.
- 10:12
So I'll show you how we do that. First off, we're going to be importing essential components like the document converter, as well as some other dependencies. And I'll show you the simplest way, which is to just start out with a PDF that we have online, which is Docling's own research paper.
- 10:28
And here you can see we have different titles and subtitles. We have components such as images and captions that we have here. And at the bottom of this PDF, we also have something like a table here that we need to extract, as well as images that might be helpful for our AI application or agent.
- 10:44
So let me come back to the notebook here, and I'll show you how we do this simple conversion by using the Docling's document converter and exporting this to Markdown.
- 10:54
So here you can see is a rough example of how fast this can be for a eight-page PDF to be able to export this in a way that an LLM can start using, right?
- 11:04
But at the same time, this is a Pydantic data type. So you can see we can explore this PDF, the number of pages and tables, and see what is on which page of this PDF and export this into a variety of different formats like Markdown, HTML, dictionary and much more.
- 11:20
But the real value here is not just with basic text and columns, but working with tables. So this PDF here has a variety of different tables that we want to be able to extract, and so by doing a converter for this specific document, we can then extract these tables here and be able to export this to a
- 11:38
data frame so we can render this out in our Jupyter notebook here. So as I run this cell, you can see that we've exported eight different tables from that source PDF, and we can list these but also be able to get these in a format ready to use in a RAG application or just to query with our
- 11:54
LLM. So we've taken a look at how to pull text and tables from a PDF, but what about visualizing and extracting images from that source document as well? Now, for this specific example, what we'll do is set up a PDF pipeline that will allow us to scale up the images and push this into a document converter.
- 12:13
So now when I go to inspect the images and picture content in the cells-
- 12:18
We can see this nicely mapped out where we have the picture, so the source image, the caption of that, and all of the embedded text elements that we could use in some type of retrieval-augmented generation application to ask questions about, hey, what's happening in these different photos in our source PDF?
- 12:35
I think what would help here is also to be able to visualize the document layout by using the bounding boxes provided by the layout visualizer here. So here what we're going to do is visualize all of the different elements and components that can be extracted from, say, for example, that source PDF.
- 12:52
So section headers or text or subtitles or different components such as that photo that we just pulled and extracted from the PDF. So this is one of the models that Docling provides, which can be used for situations where you might have personal identifiable information from a customer that you want to remove from a source document type before
- 13:11
you extract that into your application. We can also use vision language models in order to enrich the source images and diagrams that might be in these document types using something like Ollama or a third-party LLM.
- 13:24
So what we're going to do here is set up a PDF pipeline that's going to use a local running Granite model and say, "Hey, give us a detailed description of what's happening in this image."
- 13:34
And with that document converter, we're going to go ahead and display that enriched document by calling the OpenAI endpoint with Ollama that we have running locally at the completions endpoint.
- 13:45
And so here we have an annotated caption of what's happening with that Docling pipeline image that we were taking a look at earlier, where originally we were just pulling the caption, but now we can use a vision language model to describe what's happening in these photos.
- 13:57
And with all this additional information, this can help us to build a really solid RAG pipeline to where we can do questioning and answering over our source data. And for this example, I want to show you what's known as chunkless RAG or agentic RAG using Docling.
- 14:13
And now by starting off with our document outline, so processing a PDF with Docling like we just did, we can allow an LLM to be able to pick the most relevant part of the document that is related to the user's question and pull that full text from the Docling document itself to try to answer that question for
- 14:33
the user. And this can run in an agentic loop. And what's really important here is that we're doing RAG, but without having to use a chunker or embedding model or vector database, et cetera, et cetera.
- 14:44
So the index ends up being the markdown outline of the document. Now, when the user asks a question, the entire retrieval index typically would be thousands of vectors in a database where we would do semantic similarity to make sure, hey, these sections are similar to the user's question.
- 14:59
But for us, what we're going to do is be able to see a markdown outline of the document with each section summary outline. So if the LLM is looking for something about getting started with Docling, well, it can just pull from this reference of text to see that Docling can be installed for PyPi, and that's the entire
- 15:16
retrieval index. So let's say we have a query, "What are the main AI models used in Docling?" Well, we're setting up a RAG agent here to be able to iterate on that specific question about five times.
- 15:29
So we see that there are twenty sections available when the user's asked the question, and we're going to search for that specific part of text that talks about the AI models and be able to determine, hey, is this relevant to answering the question?
- 15:43
And you can see here the final answer in one iteration was pulled from that source material without having to go in a vector database, but instead search that Docling document structure for the specific text.
- 15:55
So it's a quite interesting way to be able to answer users' questions through this chunkless retrieval-augmented generation pattern. And while this first example might have been simple, what we can do is also pull the IBM twenty twenty-five annual report into the context here, which has four hundred and eighteen sections, so it's much larger, and ask a question
- 16:15
like, "Hey, what was Red Hat's revenue growth in twenty twenty-five, and how did it contribute to an overall software segment?" And so here we're going to be iterating multiple times to figure out, hey, is this section relevant to the user's question?
- 16:29
And if not, we need to pull in more information. And so that is how chunkless RAG can work in a situation where you're using a tool like Docling. But what happens when we have hundreds or hundreds of thousands of PDFs that we want to have processed?
- 16:43
Well, this is where we can deploy Docling as a REST API service using something that's known as Docling Serve. This allows us to scale things up and to run this as a microservice, as a container, or through Kubernetes.
- 16:55
So when we set things up here, we're going to do a pip install docling serve, and we're going to be able to serve this from a CLI with Docling Serve on a specific port.
- 17:05
Or once we have that server started, be able to send things to that specific endpoint with different types of options and arguments like, hey, do we want OCR? Do we want a specific back end, or do we want images annotated?
- 17:18
And so that's how we can scale things up and allow this API endpoint to be able to handle hundreds or thousands of different document types at a time. And let's say, for example, that you're trying to build an AI agent, you can also use the Docling MCP server.
- 17:32
So this allows us to automate things with our AI agent and give it the capabilities that Docling has through that model context protocol and allow us to standardize the communication between, say, for example, Cloud Code or Continue in our developer CLI to the MCP server, which can handle the document processing for us without us having to know
- 17:53
all of those different arguments and commands. So it makes it quite easy. And the example I show here is by using one of the Qwen models on my own Mac itself and connecting this to my VS Code instance.
- 18:05
So let's understand the tools that are available. For the MCP server, we have conversion tools, generation tools, say, for example, if I want to process a specific part of a PDF, and manipulation tools, and this is all provided to the LLM and the agent that we're going to be using with the MCP server.
- 18:23
So here I'm just checking that my local MLX server to run an LLM is running, and it looks like we've got Qwen three point six here. And we're going to verify that the Docling MCP server is also running, which would be done using UVX here.
- 18:37
And so we'll do that in this cell here to make sure that the MCP server is available. Now, in Claude Code or Codex or another type of AI application, we're going to install the extension.
- 18:49
So for us, this means adding an MCP server in the config.yaml. And now, at this point, we can use a model and an MCP server to do things such as, hey, convert this document and give me a summary, or create a document with a section of action items and pull in a list from another PDF and export
- 19:09
that as Markdown. So we can use all of those Docling components through the MCP server in order to agentically process and parse these documents using an AI agent like Cursor or Claude Code or one of the many open source options that are out there.
- 19:26
Now let's head back to the slides and wrap things up. So let's put it all together. With Docling, we've seen that we can take a PDF into a format like Markdown or JSON with a fully local operation from our own machine without even needing a GPU.
- 19:39
It's fast, it's cheap, and most importantly, it's open source. So I encourage you to check it all out. Behind the scenes, there's different pipelines that use a combination of OCR and layout analysis to structure everything together, put it together as a Pydantic Docling document that then you can use to export to different types of formats, create datasets,
- 20:00
chunk that using the hybrid chunker, and do so much more. It integrates with a lot of different RAG frameworks and agentic systems and harnesses, so feel free to try it out.
- 20:09
And I want to give a big thank you to the AI engineering team for having me on. This is something that I'm really passionate about with processing documents, but also doing this in an open source way because, you know, we're here at Red Hat, and we love open source.
- 20:22
So feel free to check out the slides. Connect with me on LinkedIn. I appreciate the opportunity. Enjoy the conference and keep up with the AI engineering ecosystem. Things are looking super bright right now for the open source world and for AI engineers in general.
- 20:37
So see you next time, and thanks for watching.