HybridRAG: A Fusion of Graph and Vector Retrieval to Enhance Data Interpretation

What connects a speaker, a conference, and an attendee? Naming the people and the event is only the beginning; answering the question requires representing their relationships. That is the starting point for Mitesh Patel’s overview of a HybridRAG project developed with colleagues at NVIDIA and a partner. Patel introduces his developer advocacy team as a publisher of reusable workflows and notebooks: this is an architecture walkthrough, with implementation materials to follow, rather than a live coding session.

A knowledge graph represents entities—people, places, concepts, or events—and the edges connecting them. In Patel’s conference example, Patel is a speaker at the conference, while an attendee attended his session. Each connection can be expressed as a triplet: an entity, a relationship, and another entity. The relationship tells us something that the entity names alone cannot.

Making those connections explicit gives retrieval a structure to explore. Instead of relying only on text that resembles the question, a graph can supply connected facts that help construct an answer. Graphs can also organize information from multiple sources, although Patel acknowledges that this capability is shared with other RAG systems. The distinctive benefit is the representation and use of relationships.

The architecture has four components: source data, data processing, graph or vector index construction, and inference. Processing quality propagates into retrieval quality: better preparation supports a better knowledge graph, which gives the retriever better material to work with. Patel separates the workflow into offline construction—described here as a one-time preparation stage—and online querying. Once the indexes exist, the system retrieves information and turns it into a readable response. Returning a handful of relationships and expecting the user to interpret them is not enough.

The two construction branches retain different representations of the documents:

The vector branch follows the familiar document-to-embedding path. The graph branch requires additional care because its usefulness depends on which entities and relationships survive extraction. This overview establishes both branches, but does not specify a ranking or fusion algorithm for combining their retrieved results.

Consider the ExxonMobil passage used to demonstrate extraction. Patel initially describes it as a results document, but the matching example in NVIDIA’s knowledge-graph article concerns projected exploration-spending cuts. The extraction lesson is precise: ExxonMobil is a company entity, cut is the relationship, and spending on oil and gas exploration is an activity entity. Patel first calls cut a feature, then corrects himself: it belongs on the edge between the entities.

In Python, that distinction can be represented directly as a typed record:

from typing import Literal, TypedDict

class Entity(TypedDict):
    name: str
    type: Literal["company", "activity"]

class Triplet(TypedDict):
    subject: Entity
    relationship: str
    object: Entity

triplet: Triplet = {
    "subject": {"name": "ExxonMobil", "type": "company"},
    "relationship": "cut",
    "object": {
        "name": "spending on oil and gas exploration",
        "type": "activity",
    },
}

print(
    triplet["subject"]["name"],
    triplet["relationship"],
    triplet["object"]["name"],
    sep=" → ",
)

The difficult part is producing the right record from unstructured prose. An LLM can identify the entities and organize their connection into the triplet format needed for storage.

The extraction procedure begins with the application’s ontology: the entity types and relationships the system needs to recognize.

1. Define the ontology for the use case.
2. Include it in the extraction prompt.
3. Ask the LLM to extract ontology-specific information and return structured triplets.
4. Inspect the results and revise the ontology or prompt when the extracted relationships are wrong or noisy.

This is an iterative engineering task. A graph built from noisy triplets will return noisy evidence, so prompt quality cannot be judged merely by whether the output has the right shape. Patel estimates that getting the ontology right can consume 80% of development time.

The semantic branch starts with a more established indexing procedure. Patel uses the first page of Attention Is All You Need to illustrate document chunking. Each chunk passes through an embedding model, and the resulting vector is stored for retrieval. Two parameters matter together: chunk size determines how much text receives one representation, while overlap retains some shared text between neighboring chunks. Without overlap, context connecting the end of one chunk to the beginning of the next can be separated.

Overlap helps with boundaries, but it does not create explicit relationship edges. Patel’s comparison is about how the two representations expose information: vector indexing represents the meaning of a text span, while graph construction identifies entities and stores their connections as relationships that retrieval can follow.

Now consider a question about ExxonMobil’s cuts for the quarter. Graph retrieval must locate the relevant entities and relationships. A single-hop lookup reaches immediate neighbors; a second hop can continue from those neighbors to another set of connected entities. Restricting retrieval to one hop can leave useful connections unexplored.

Deeper traversal can bring in more context, but it also takes more time. The production decision is therefore a balance between the relationships needed to answer the question and the latency the application can tolerate. Tune single-hop, double-hop, and deeper retrieval against that requirement. Patel introduces cuGraph’s NetworkX integration as one way to accelerate supported graph operations and create room for deeper searches. In current usage, GPU acceleration requires installing and enabling a backend such as nx-cugraph; installing ordinary NetworkX alone does not enable it.

A readable answer does not establish that the pipeline retrieved the right evidence or used it faithfully. Patel introduces evaluation at several levels: faithfulness, answer relevancy, precision, and recall for the RAG workflow, alongside helpfulness, correctness, coherence, complexity, and verbosity for generated responses.

Ragas provides a pip-installable framework for evaluating the workflow across its question, retrieved context, and response. The four measures emphasized here divide the inspection into useful questions:

These are the selected evaluation measures, not the library’s entire metric inventory. Examining the components helps distinguish retrieval problems from problems in interpreting the question or constructing the answer.

Ragas uses an LLM for judging. Patel describes a default GPT integration and the option to connect another model through an API. The practical benefit is flexibility: the evaluator can use a model appropriate to the application. He stops short of calling that evaluation comprehensive.

A second approach uses a model trained specifically to judge another LLM’s responses. Patel describes a 340-billion-parameter reward model, calling it “Llama Nemotron.” The description closely matches Nemotron-4-340B-Reward, a Nemotron-family model rather than a Llama model. Its five judging dimensions are helpfulness, correctness, coherence, complexity, and verbosity. The distinction is in the model’s training objective: response assessment is its intended task, rather than an additional role assigned to a general-purpose generator.

Patel frames optimization with an 80/20 analogy: building the initial 80% of a GraphRAG system takes 20% of the time, while the final 20% takes 80%. The remaining work is making the system good enough for its actual application. In the partner project, this meant revisiting graph construction: could fine-tuning improve triplet quality, and could better preprocessing remove characters that interfered with extraction? Candidate changes included regex cleanup and removal of apostrophes, brackets, or other irrelevant characters.

The experiments combined several adjustments:

• Text cleanup: Remove possessive apostrophe-S and other characters the team considered unimportant to triplet generation.
• Output handling: Patel describes making longer outputs smaller to avoid missing output, without giving enough procedural detail to reproduce that adjustment.
• Fine-tuning: Adapt the extraction model rather than relying only on its as-is behavior.

Patel reports that these changes improved the extraction results.

Patel reports extraction accuracy rising from 71% to 87% after introducing LoRA fine-tuning, evaluated on 100 documents. He initially says 100 triplets, then corrects the evaluation unit to documents. In describing the comparison, he names Llama 3.3 for the as-is result and Llama 3.1 for fine-tuning, after earlier uncertainty about versions. The exact model pairing and accuracy scoring procedure are therefore not established; the scores describe the reported extraction experiment, not end-to-end answer accuracy.

Further small adjustments reportedly improved performance, but Patel does not give a final score. He also cautions that accuracy may decline as the document pool grows. The useful result is the improvement observed within this small evaluation: preprocessing, extraction tuning, and incremental corrections all affected the quality of the graph being built.

Extraction accuracy and retrieval latency require different interventions. As a graph grows to millions or billions of nodes, searching it becomes a computational problem in its own right. Patel returns to cuGraph through NetworkX and describes algorithm performance tests with sharply reduced overall execution latency. He gives no numerical timings or hardware conditions in the spoken comparison.

The two experimental tracks address separate parts of the pipeline: better extraction improves the information available in the graph, while accelerated graph operations reduce the time needed to work with it. Together they target the quality and latency requirements that determine whether the system is useful in an application.

Should the application use semantic retrieval, graph retrieval, or both? Start with the data. Retail records, financial-services data, and employee databases often already have well-defined structure, making them promising graph candidates. Unstructured documents can also support a graph-based system if the extraction process can produce a useful knowledge graph from them.

Then examine the questions the application must answer. A graph earns its cost when the answers require understanding relationships. If semantic retrieval supplies the needed evidence, a graph may add unnecessary work. If the questions depend on complex connections, graph retrieval—or a hybrid using both representations—is worth considering. Patel’s recommendation is conditional because these systems are compute-heavy: the data must support the representation, and the application must benefit enough to justify its cost and latency.

The path from this overview to implementation is the accompanying notebook work. Patel says the GitHub materials include Llama 3.1 LoRA fine-tuning, and that a two-hour workshop expands on the talk. He closes by pointing to NVIDIA’s developer programs and interest-based mailing lists for subsequent releases, and invites attendees to continue the discussion at the Neo4j booth across the hall.