# EyeLevel Launch: Your RAG is Tripping, Here's the Real Reason Why

Benjamin Fletcher, EyeLevel.ai

AI Engineer World's Fair 2024

[Talk page](https://ai.engineer/talks/8RbTZl7bs5U-eyelevel-launch-your-rag-is-tripping-heres) · [Video](https://www.youtube.com/watch?v=8RbTZl7bs5U)

## EyeLevel Launch: Your RAG is Tripping, Here's the Real Reason Why

Complicated documents can lose meaning before retrieval begins. EyeLevel’s walkthrough follows extraction, context-preserving semantic objects, and query rewriting through an Air France copilot example.

Familiarity with RAG—retrieving source material and passing it to a language model—and with splitting documents into chunks will help.

### Getting documents into RAG is only the beginning

[0:00](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=0s)

How do you turn uploaded documents into content an AI application can reliably retrieve? EyeLevel’s starting promise is a short path: upload documents through no-code tools or APIs, then receive retrievals within minutes. Co-founder Benjamin Fletcher brings experience from IBM Research, IBM Watson, and work with brands including The Weather Channel. He describes EyeLevel as the world’s most accurate and scalable RAG platform, developed over four years beginning with early access to the **GPT-3** beta.

### Three ways the evidence goes missing

[0:53](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=53s)

Getting started with retrieval-augmented generation is easy; mastering it becomes difficult when the knowledge base contains complicated enterprise documents. Fletcher reports that, in EyeLevel’s experience, RAG applications can have error or hallucination rates as high as 35%. His diagnosis points upstream of generation: the retrieved content may already be incomplete, distorted, or irrelevant before the LLM sees it.

The failures fall into three categories:

- **Bad text extraction:** the extracted words do not faithfully represent the source.
- **Missing context:** chunking separates a passage from information elsewhere in its document.
- **Missing visual information:** tables, charts, and other visual elements never reach the retrieval system.

**The quality of the answer depends on the evidence that survives ingestion.** Fletcher argues that fixing these failures with advanced RAG techniques can take hundreds of implementation hours; EyeLevel packages that data-engineering work into its ingestion pipeline.

![Slide with three numbered boxes: Bad text extraction; Retrievals missing context; Visual formats like tables and charts.](https://ai.engineer/api/talk-essay-media/e12a11b67e92d2d168f68355cfe6a5ef0fea6a69534d0d09e1600f686f1f5a27)

Three common causes of RAG failure: bad text extraction, missing context, and visual formats.

Fletcher reports that customers including Air France and Dartmouth say their applications respond correctly more than 95% of the time. He also cites an unnamed study reporting 98% accuracy on complicated real-world documents and up to 120% outperformance over popular competing solutions. The talk does not specify the study’s evaluation protocol, identify the comparators, or define the outperformance calculation, so those figures describe the results Fletcher reports rather than a reproducible comparison supplied in the walkthrough.

### From an Air France document to searchable objects

[2:33](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=153s)

EyeLevel’s architecture does not use a vector database. Instead, it creates **semantic objects** and searches across multiple attributes of each object. Fletcher questions whether vector databases are the best fit for many RAG applications; the alternative he introduces is a richer unit of retrieval than an isolated passage of extracted text.

Air France provides the concrete setting. At the time of the talk, the airline had been using EyeLevel for a year to build a ChatGPT-like copilot for call-center agents. Its knowledge base contains hundreds of thousands of documents, including pages with tables, figures, and text scattered across the layout. Understanding such a page requires identifying its different components before deciding how to extract them.

The ingestion sequence begins with a vision model that locates images, tables, and text. Fletcher says EyeLevel fine-tuned this model with millions of documents. Dedicated multimodal processing pipelines then extract the written and visual information from those regions. The slide labels the detector **GroundX Vision Model** and shows colored region annotations over an Air France document containing a passenger photo, a table, and text. Region detection is the first step; extracting the meaning within those regions is the next.

![Air France document containing a passenger photo, table, and text, with colored region overlays beside the label GroundX Vision Model.](https://ai.engineer/api/talk-essay-media/5397c91da4aef89118b19f34a85f63eb39ee2e60b0ae209e731469a653a970e9)

GroundX Vision Model highlights regions of an Air France document.

### Keep the chunk, restore its surroundings

[3:40](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=220s)

Even a correctly extracted document can lose meaning when split into smaller chunks. A passage may no longer carry the section it belongs to—or even the identity of its document. Fletcher compares this to asking questions about a book and receiving random paragraphs: the paragraphs can be accurate while still being poor evidence for an answer.

A semantic object retains the original chunk and adds automatically generated metadata that preserves surrounding information. It also contains two rewrites, each serving a different purpose:

| Component | Purpose |
| --- | --- |
| Original chunk | Preserve the extracted passage |
| Generated metadata | Retain surrounding document context |
| Search rewrite | Represent the content for retrieval |
| Completion rewrite | Represent the content for answer generation |

The distinction between the last two fields matters: text that helps find a passage need not be the same text that best helps a model answer from it.

For a small book example, suppose a chapter titled “Seedling care” contains the sentence “Water them daily.” A representation of the described structure could be:

```json
{
  "originalText": "Water them daily.",
  "metadata": {
    "document": "A Beginner's Gardening Guide",
    "section": "Seedling care"
  },
  "searchText": "Seedling care: daily watering frequency.",
  "completionText": "The Seedling care section of A Beginner's Gardening Guide recommends watering seedlings daily."
}
```

Here the original sentence remains intact, while the surrounding context resolves what “them” refers to. These illustrative field names express the mechanism, not an SDK contract. Current [GroundX API concepts](https://docs.eyelevel.ai/documentation/fundamentals/api-concepts) document semantic objects, but the walkthrough describes the historical architecture rather than a current API integration.

### What OCR leaves behind

[4:31](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=271s)

The next demonstration returns to a figure from an Air France document. Fletcher first shows the result of a familiar sequence: OCR the figure, extract its text, vectorize that text, and place it in a vector database. He then points to information lost along the way and contrasts that result with EyeLevel’s ingestion output, which includes both the search and completion versions of the text.

The comparison turns on what reaches the index. Recognizing words inside a figure does not necessarily preserve the information conveyed by their arrangement or visual relationships. Once extraction has omitted information, vectorizing the remaining text does not restore it. EyeLevel’s demonstrated response is to process visual and written information during ingestion, then produce representations suited to finding that information and using it in an answer.

### Rewrite the query, search the object, rerank the results

[5:04](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=304s)

At query time, the system applies a corresponding transformation to the user’s request:

1. **Rewrite the query** into a format compatible with the semantic objects.
2. **Search multiple fields:** the original text, automatically generated metadata, and search rewrite.
3. **Rerank the candidates** with a fine-tuned LLM to improve the ordering of the results.

The completed pipeline slide labels the middle stage “Weighted Search of Semantic Objects.” The completion rewrite is part of the object, but Fletcher’s enumeration of searched fields specifically names the original text, metadata, and search version.

![Four labeled icons under The Search Pipeline: Query, Query Rewriter, Weighted Search of Semantic Objects, and Fine-Tuned Rerank Model.](https://ai.engineer/api/talk-essay-media/048439a8e4686337ba727a02dbdcddecfaf85591ac2c9bb24a4e90612278bf1d)

The search pipeline includes query rewriting, semantic-object search, and reranking.

Fletcher says ingestion and search together use more than nine fine-tuned models. The platform’s promise is to make that machinery available without requiring each application team to assemble it: enterprise-quality, production-ready applications in minutes rather than months. He closes by inviting viewers to try EyeLevel’s XRay. The technical substance behind that invitation is the full sequence just demonstrated—extract the document’s information, preserve the context of each chunk, create separate search and completion representations, and retrieve through query rewriting, multifield search, and reranking.

## Resources

- [EyeLevel's RAG accuracy comparison](https://www.eyelevel.ai/post/most-accurate-rag) — Vendor evaluation comparing GroundX, LangChain/Pinecone, and LlamaIndex on questions about complex PDFs, with methodology and answer examples.
- [GroundX API concepts and semantic objects](https://docs.eyelevel.ai/documentation/fundamentals/api-concepts) — Current definitions of semantic objects, document ingestion, X-Ray output, and search.
- [X-Ray document parsing guide](https://docs.eyelevel.ai/documentation/guides/x-ray-document-parsing) — Current instructions for uploading documents and inspecting contextualized parsing results with GroundX SDKs.
- [Air France/KLM customer-support case study](https://www.eyelevel.ai/customers/air-france-klm) — EyeLevel's account of a three-month airline assistant evaluation, including expert scoring and response refinement.

## Slides

### Presenting sponsor

Microsoft is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=5s) [Microsoft is the presenting sponsor.](https://assets.ai.engineer/sites/aiecode2025/sha256/44873888010267d24e850aeb225cead1f3a6fa2a00be588dfd81315228a33028.webp)

PRESENTING SPONSOR

Microsoft

### Innovation and platinum sponsors

[0:10](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=10s) [AWS is the innovation sponsor; MongoDB, Google Cloud, and Neo4j are platinum sponsors.](https://assets.ai.engineer/sites/aiecode2025/sha256/55ca619cc788a7c426c0a272ebdc05351a465f23d65beaabda0ec266a684d8b3.webp)

INNOVATION SPONSOR
aws

PLATINUM SPONSORS
MongoDB®
Google Cloud
neo4j

### Three most common causes of RAG failure

The speaker identifies three content ingestion problems: bad text extraction, document context lost during chunking, and visual elements such as tables and charts that may not be extracted.

[1:40](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=100s) [Three causes are bad text extraction, retrievals missing context, and visual formats such as tables and charts.](https://assets.ai.engineer/sites/aiecode2025/sha256/4f2f9c1cad83fcbddc1ccbe36ff67e0a0487f8b833cc73149744ab1753a488cb.webp)

THREE MOST COMMON CAUSES OF RAG FAILURE

1. Bad text extraction
2. Retrievals missing context
3. Visual formats like tables and charts

AIE
EYELEVEL
Microsoft
smol ai
aws

### 95% accurate RAG or better

EyeLevel says Air France and Dartmouth report correct RAG responses more than 95% of the time. It also reports 98% accuracy on complicated real-world documents in a recent study.

[2:04](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=124s) [Slide claims 95% accurate RAG or better alongside customer logos.](https://assets.ai.engineer/sites/aiecode2025/sha256/0b6b0da5668944d66fe824109f023e05d59bb931ea71eaae04534a470c3a859d.webp)

YOU CAN ACHIEVE
95%
ACCURATE RAG
OR BETTER

AIRFRANCE
DARTMOUTH
The Weather Channel
AMERICA’S TEST KITCHEN
Stride
AskVet
COVALENT METROLOGY

AIE
EYELEVEL
Microsoft
smol ai
aws

### Semantic objects

EyeLevel creates semantic objects and searches across their attributes without using vector databases. The diagram shows an object surrounded by connected nodes.

[2:44](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=164s) [A central object surrounded by connected nodes beside the words “100% vector free.”](https://assets.ai.engineer/sites/aiecode2025/sha256/3272680739428358c109264b263f99d2a9c86d1ee9a28be2bd560446e5f94997.webp)

SEMANTIC OBJECTS

100% VECTOR FREE

AIE
EYELEVEL
Microsoft
smol ai
aws

### GroundX vision model extraction

GroundX routes a document through a fine-tuned vision model to extract text, tables, and figures for pipelines that create LLM-ready data.

[3:49](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=229s) [Diagram shows a document passing through a fine-tuned vision model to identify text, tables, and figures.](https://assets.ai.engineer/sites/aiecode2025/sha256/ff748610baf0af05c256d6d8b1b5fe7fd969b0708f1046cf48f14d130d30f686.webp)

GROUNDX VISION MODEL EXTRACTS COMPLEX DATA FROM DOCUMENTS, SENDS TO PIPELINES TO CREATE LLM READY DATA

DOCUMENT
FINE-TUNED VISION MODEL
TEXT
TABLES
FIGURES

AIE
EYELEVEL
Microsoft
smol ai
aws

### The search pipeline

GroundX rewrites the query to match its objects, searches their text and generated metadata, then uses a fine-tuned model to rerank the results.

[5:20](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=320s) [Four stages: query, query rewriter, weighted search of semantic objects, and fine-tuned rerank model.](https://assets.ai.engineer/sites/aiecode2025/sha256/a439e93e0e18c3bb0a15149566de3b54a01275c7cb19d3a2de0f4ea36273861c.webp)

THE SEARCH PIPELINE

QUERY
QUERY REWRITER
WEIGHTED SEARCH OF SEMANTIC OBJECTS
FINE-TUNED RERANK MODEL

Interface text:
AIE
EYELEVEL
Microsoft
smol ai
aws

### Build Production Ready RAG in Minutes

EyeLevel presents RAG in three stages: ingest, search and complete. The speaker says users can build production-ready applications in minutes.

[5:35](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=335s) [Slide titled “Build Production Ready RAG in Minutes” with three boxes labeled Ingest, Search, and Complete.](https://assets.ai.engineer/sites/aiecode2025/sha256/de6b69addc1e46d19c8323311d327e6c50cb6313b28a7905d0422396fa6d3a27.webp)

BUILD PRODUCTION READY RAG IN MINUTES

INGEST
SEARCH
COMPLETE

AIE
EYELEVEL
Microsoft
smol ai
aws

## Full transcript

[0:00](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=0s) [upbeat music] Hi, my name's Ben. I'm co-founder of EyeLevel.ai, and I've been building applications powered with AI for the last 15 years.

[0:22](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=22s) First at IBM Research, then at IBM Watson, later working with major brands like The Weather Channel, and now at EyeLevel, where we've built the world's most accurate and scalable RAG platform.

[0:36](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=36s) Using our no-code tools and APIs, our users can upload documents and receive the most accurate retrievals in minutes. We've been developing our solution for the last four years, and we're among the first early users that were admitted to the GPT-3 beta program.

[0:53](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=53s) We found it easy to get started with RAG and very difficult to master. In our own experience, RAG applications can have error or hallucination rates as high as thirty-five percent, especially when the knowledge base consists of the kinds of complicated documents that are commonly found in the enter- enterprise.

[1:14](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=74s) The source of these errors is rarely the LLMs or the prompts. Instead, it's typically RAG itself, or more specifically, the quality and relevance of retrieved content. And the problems with content generally fall into one of three categories: bad or improperly extracted text, missing information from the surrounding parts of the document that's lost during

[1:39](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=99s) chunking, or visual elements that are not extracted at all.

[1:44](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=104s) Most commonly, the problems with RAG are content ingestion problems, and advanced RAG techniques that help you solve these problems can take hundreds of hours to implement. We've spent the last four years tackling these difficult data engineering problems and have built the solutions to them into our ingestion pipeline.

[2:05](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=125s) As a result, our users are able to build the most accurate RAG applications in just minutes, and our customers, such as Air France and Dartmouth, tell us that their RAG applications respond correctly more than ninety-five percent of the time.

[2:19](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=139s) In a recent study, our app-- our platform achieved ninety-eight percent accuracy against complicated real-world documents and outperformed some of the most popular solutions in market by as much as a hundred twenty percent.

[2:33](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=153s) I'm gonna quickly, quickly walk you through the unique approach we take to achieve this high level of accuracy, and I'll start by telling you that we don't use vector databases at all, and in fact, we think they may not be the best technology solution for a lot of RAG applications.

[2:49](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=169s) Instead, what we do is we create what we call semantic objects, and we do a multifield search across the attributes of this object. I'll show you what that means with a real example from Air France.

[3:01](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=181s) Air France has been using our platform for the last year to build a ChatGPT-like copilot for their call center agents.

[3:10](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=190s) They wanted to understand their knowledge base, which consists of hundreds of thousands of documents just like this one, filled with tables, figures, and text sca- uh, scattered across the pages.

[3:22](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=202s) In our ingestion pipeline, the first thing we do is run a vision model that we fine-tune with millions of documents to identify where the images, the tables, and the text are.

[3:32](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=212s) Then we run them through dedicated multimodal processing pipelines to extract the visual and written information.

[3:40](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=220s) When you do RAG, you have to break apart this document into smaller chunks. When you do that, you quite often lose information from around the chunks, things like what section of the document it came from or even which document it came from.

[3:55](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=235s) If you were to ask questions about a book and receive random paragraphs from the book, chances aren't great you'd get good answers, and that's kind of what's happening with chunking and the loss of the context.

[4:08](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=248s) That's why we created semantic objects. It consists of the original chunk text as well as auto-generated metadata that preserves the information around the text, and then we re- rewrite the text into two ideal formats, one for search and one for completion.

[4:30](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=270s) Thank you. [laughs]

[4:30](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=270s) Hope you're networking right now.

[4:31](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=271s) Yeah. Let, let me, let me, let me show you what that looks like with an example. So this is a figure from one of Air France's documents. If you were to OCR this and extract the text from it, vectorize it, put it in your vector database, it would look something like this.

[4:47](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=287s) Look at how much information is lost in the process, though. Instead, what comes out of our ingestion pipeline is something like this,

[4:57](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=297s) and this, this includes both the search version as well as the completion version of the text.

[5:04](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=304s) When we receive a search query, we do something similar. We rewrite the query into a format that's compatible with the objects. Then we search the entire object, the original text, the auto-generated metadata, and the search version of the text.

[5:20](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=320s) We use a fine-tuned LLM to re-rank the results and improve the accuracy. And in total, in our ingestion and search, there are more than nine models that are fine-tuned

[5:32](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=332s) to help deliver this kind of accuracy. The end result is the world's most accurate RAG platform, and our users are able to build enter- enterprise-quality, production-ready applications in minutes, not months.

[5:46](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=346s) Uh, I invite you to try it for yourself, though, EyeLevel.ai/XRay. [laughs]

[5:51](https://www.youtube.com/watch?v=8RbTZl7bs5U&t=351s) Thank you very much. Thank you, Ben. All right. [upbeat music]
