Practical GraphRAG: Making LLMs Smarter with Knowledge Graphs

How can an LLM answer an enterprise question when it does not know the enterprise’s data? A fluent response does not establish that the model knows the relevant products, customers or internal processes—or that it can explain where its answer came from. Missing domain knowledge, unverifiable answers, hallucinations and bias are the opening problems for GraphRAG.

Retrieval-augmented generation supplies external information at answer time. That makes data quality central: the model needs accurate, contextual evidence to work with. But a basic vector search retrieves only a selection of fragments, chosen by similarity. Similarity is not the same as relevance, and a group of passages about the right topic can still omit the facts needed to answer the question.

Stephen Chin also questions the maturity, resilience and scalability of some vector database offerings. Those are his assessments of the systems under discussion, rather than inherent limitations of vector indexing. The more general retrieval problem is that similar fragments alone can be incomplete and difficult to explain.

GraphRAG adds explicit knowledge and context to the language model’s ability to generate responses. Chin introduces this through a creative-versus-logical brain analogy; the concrete implementation is a knowledge graph made of nodes, relationships and properties. Nodes represent things, relationships connect them, and properties record their attributes.

Consider two people who live together and share a car. The person who owns the car is not necessarily the person who drives it. A graph preserves that distinction through separate OWNS and DRIVES relationships, alongside connections such as LIVES WITH. Retrieving information about the people and the car is useful, but answering a question about ownership requires preserving which person has which relationship.

The Stack Overflow graph expands this idea from a tiny household example to connected content with rich metadata. Retrieval can start with a relevant item and follow graph relationships to gather related information. The result carries explicit structure and semantics that can be inspected, visualized, analyzed and logged. Security and role-based access can also be applied to the retrieval process. These capabilities support grounding and explanation; they do not automatically make every generated answer correct.

Microsoft Research’s From Local to Global provides an early example of using graph community summaries for questions about an entire collection. Its evaluation found improvements in comprehensiveness and diversity on global sensemaking questions over podcast and news corpora. Its token savings compare community-summary context with map-reduce summarization over source text, not the total cost of every GraphRAG system against vector RAG. Building the graph remains an upfront investment.

The data.world benchmark supplies a more specific interpretation of the talk’s roughly threefold accuracy claim. In an insurance-domain enterprise benchmark, GPT-4 zero-shot question-answering accuracy rose from 16% over SQL to 54% over a knowledge-graph representation. That graph represented the enterprise SQL database using an ontology and mappings, so the result concerns a particular representation and question-answering task, not a general database performance contest.

Chin then turns to Gartner’s 2024 hype-cycle discussion and enterprise adoption as signs of growing interest. These provide market context, rather than controlled evidence of effectiveness. His account of analyst claims about resolving hallucinations is best understood as advocacy for grounding responses in facts, not a guarantee that adding a graph eliminates hallucinations.

The LinkedIn customer-service study measures a practical support outcome. In LinkedIn’s production comparison against traditional manual methods, median per-issue resolution time fell from seven to five hours, a 28.6% reduction. The customer-service team was randomly split between the two approaches across multiple product lines. This production result is distinct from the paper’s offline retrieval comparisons; the talk closes its evidence section by returning to data.world’s accuracy result.

Michael Hunger separates GraphRAG into two activities: constructing the knowledge graph and retrieving from it. Construction proceeds through three layers:

1. Document structure: Turn unstructured information into a lexical graph containing documents, chunks and their relationships.
2. Domain knowledge: Use a graph schema to guide extraction of entities and relationships from the text.
3. Enrichment: Apply graph algorithms and summarization, including PageRank and community summaries.

The resulting graph becomes the basis for local, global and other search approaches. Those search modes are introduced here as options, rather than fully specified algorithms.

This is data engineering work: richer retrieval requires more effort at ingestion. The payoff is reusable structure that can supply context for many later queries. The GraphRAG pattern catalog collects approaches observed in applications and research, with names, descriptions, usage context, example graphs and queries. A pattern can combine a lexical graph with a domain graph, letting a query move between source passages and the entities those passages describe.

A lexical graph represents documents and their constituent elements. A book can contain chapters, chapters can contain sections, and sections can contain paragraphs. A semantically cohesive paragraph may be a good embedding unit, while its graph relationships preserve the surrounding structure that an embedding alone does not express.

Parent and sequence links make document context directly retrievable. A k-nearest-neighbor graph adds similarity relationships between chunks, with a weight recording their similarity. Retrieval can then expand by document membership, sequence or similarity instead of treating every chunk as an isolated result.

The RFP example follows the same procedure: divide the request for proposal into meaningful subsections, connect those elements, attach their text and compute embeddings. Repeating that process across the collection produces a lexical graph. The important decision is not merely where to split text, but which relationships to preserve so a later query can recover its context.

The next layer extracts entities and relationships. LLMs extend established NLP extraction techniques with flexible instructions and multilingual understanding. The input combines source text, an instruction prompt and a graph schema. Hunger mentions extraction inputs of ten thousand or a hundred thousand tokens as possibilities with large context windows, not as measured operating recommendations.

Existing structured data can make this task more precise. If products, genes, partners or clients already have established records, supply those records as ground truth. The model can recognize known entities in the text instead of creating every entity from scratch, then extract relationships and attach additional facts to entities or relationships.

This also provides a route into an existing enterprise graph. A CRM may already contain customers and leads; call transcripts can add information connected to those established records. The lexical graph retains the source structure, while the domain graph connects the extracted information to the business entities it concerns.

Enrichment then looks beyond individual documents. Graph clustering groups related entities into communities, and an LLM can summarize each community. Where a document gives one ordered presentation of information, a community can expose a topic recurring across many documents. This is the basis for gathering context about a collection-wide theme instead of locating only one relevant passage.

A graph retriever begins with an index search. Vector, full-text, hybrid, spatial or another search method identifies entry points in the graph. It then follows relationships from those starting points, stopping at a chosen depth or relevance threshold. Vector search remains useful; it becomes the beginning of retrieval rather than its entire scope.

For a small RFP illustration, suppose index search selects a passage describing support hours. A one-hop expansion can also include its parent section and the next passage containing an exception. This Python example shows that expansion after index search has supplied the entry-point ID:

nodes = {
    "support": {"kind": "section", "text": "Support requirements"},
    "hours": {"kind": "chunk", "text": "Provide support every weekday."},
    "exception": {"kind": "chunk", "text": "Critical incidents need weekend coverage."},
}
edges = [
    ("hours", "PART_OF", "support"),
    ("exception", "PART_OF", "support"),
    ("hours", "NEXT", "exception"),
]


def retrieve_context(entry_ids, max_hops=1):
    selected = set(entry_ids)
    frontier = set(entry_ids)
    for _ in range(max_hops):
        neighbors = set()
        for source, relation, target in edges:
            if source in frontier:
                neighbors.add(target)
            if target in frontier:
                neighbors.add(source)
        frontier = neighbors - selected
        selected.update(frontier)

    return {
        "nodes": [{"id": key, **nodes[key]} for key in sorted(selected)],
        "relationships": [
            {"source": source, "type": relation, "target": target}
            for source, relation, target in edges
            if source in selected and target in selected
        ],
    }


context = retrieve_context(["hours"], max_hops=1)

The selected evidence now includes the weekday requirement, the weekend exception and their structural connections. The example uses a fixed hop limit; a real retrieval policy can also restrict which relationships are relevant to the question.

The question is only one input to that policy. External user context also influences which information to retrieve and how much to include. Someone in finance may need different context from someone in engineering, even when looking at the same underlying data. Context selection should therefore reflect the user’s task as well as the initial search match.

The LLM receives text together with a contextual subgraph: explicit node–relationship–node structures rather than only disconnected fragments. Hunger notes that modern LLMs can process these additional structures, although no comparative benchmark is shown for that claim. Clustering, link prediction and PageRank can supply further enrichment signals for the graph used in retrieval.

The Neo4j LLM Knowledge Graph Builder makes the construction workflow concrete. It accepts PDFs, YouTube transcripts, local documents, web articles and Wikipedia articles. In the demonstration, several sources have already been uploaded, including material about Google DeepMind. The tool supports different LLMs and an optional graph schema. A pattern such as person–works for–company tells the extraction model which kinds of relationships to look for.

The DeepMind Wikipedia extraction exposes both layers of the graph. One part contains the document and its chunks; the other contains connected entities, including companies, locations, people and technologies. The schema guides which entity types are extracted. For querying, the interface offers vector, graph, full-text and entity retrieval options. The builder is an open-source project.

Hunger opens a previously run answer to the question, “What has DeepMind worked on?” The query was run before the presentation because venue internet was unreliable. The useful part of the walkthrough is the evidence inspection:

1. Sources: Inspect the source material used, including AlphaFold, Google DeepMind, Wikipedia and another PDF.
2. Chunks: Examine the text passages retrieved through the search process.
3. Entities: Inspect the graph entities supplied to the LLM alongside their connected text.

This makes the retrieved context visible. It explains what evidence the generator received, rather than proving that every sentence in the answer is correct.

The builder also offers evaluation with Ragas. The demonstration mentions this capability without presenting evaluation scores.

An agentic approach packages individual retrievers as domain-specific tools. Each tool has a query, inputs and a description. An agent can call a tool, inspect its response and make a deeper call when the first result reveals a need for more information. Each call performs a bounded retrieval operation over the relevant domain.

The tooling is presented as available through open-source libraries, including Python. Hunger also shows NeoConverse, which can return charts and network visualizations as well as text. The output format can therefore reflect the structure of the retrieved information rather than always becoming a prose answer.

The orchestration procedure is to break the user’s question into individual tasks, extract the parameters each task needs, and run the corresponding tools in sequence or in a loop. Responses can guide subsequent calls. This extends retrieval from one search into a series of targeted queries, while keeping each query’s purpose and inputs explicit.

The final implementation example brings construction and retrieval together through Neo4j GraphRAG for Python. The package supports building knowledge graphs, implementing retrievers and assembling pipelines. Hunger supplies PDFs and a graph schema, imports the resulting graph into Neo4j, and shows that the data can then be visualized in a Python notebook. The repository’s current examples are an implementation entry point; their API signatures should not be assumed to match the notebook shown in the recording.

The closing destination is the GraphRAG pattern catalog: graph models, retrieval patterns, papers and resources, with an invitation to contribute additional patterns. Chin credits Hunger’s team with building tools including Knowledge Graph Builder. Further questions move to the conference booth, leaving the recording’s practical endpoint at a working construction-and-retrieval workflow and the tools for extending it.