RAG Needs a Map: Using GraphRAG to Retrieve Connected Context — Nyah Macklin, Neo4j

Most hands in Nyah Macklin’s opening poll go up for vector-based RAG; far fewer go up for graph-based retrieval. This Neo4j workshop starts at that gap: how can an agent retrieve relationships between pieces of information, rather than just passages that resemble a question? Macklin connects that engineering problem to a longer-standing interest in tracing and explaining AI decisions.

The build uses GraphAcademy’s hands-on course material, a Neo4j Aura database and an OpenAI API connection. Participants begin by creating a GraphAcademy account and a free Aura instance. The workshop supplies a limited-use API key to reduce setup work. These pieces have different jobs: the course supplies the material, Aura stores the graph, and the model services support extraction and embeddings.

An apple makes the representation problem concrete. Its vector representation is a series of floating-point numbers. Those numbers can support machine retrieval, but a person cannot readily read them and explain what the system knows about the apple.

What becomes easier to inspect when the same apple appears in a knowledge graph? The representation exposes recognizable facts: it comes from a tree, it is round, this particular apple is red, and it is a fruit. That last connection also distinguishes the fruit from the company. The useful change is that the representation names entities and their connections in terms that both software and people can work with.

A graph makes those connections traversable. An agent can move from one node to another through a relationship, leaving a path that can be inspected. Macklin contrasts this with reconstructing connections through table joins, though the workshop provides no performance comparison establishing that graphs are universally faster. The concrete benefit here is the explicit relationship structure.

That structure becomes especially attractive when a decision needs a history. Finance, medicine and legal work are the motivating examples—“Nobody wants Vibe banking.” Agent memory could preserve the context behind earlier decisions, but this workshop does not implement it. Its narrower task is to build the connected data that retrieval can use.

The baseline RAG pipeline starts with documents such as PDFs. Split them into chunks, embed those chunks and index the vectors. At question time, use the same embedding model to represent the query, retrieve the top K relevant chunks, and pass them to an LLM to produce an answer. GraphRAG changes the retrieval path so that it includes a knowledge graph.

The decision depends on what answering the question requires:

• Find a relevant passage. A question that resembles an ordinary search query may be well served by vector retrieval.
• Follow connected facts. Source-code analysis and supply-chain questions may require several relationship hops, rather than one semantically similar passage.
• Combine or count information. Aggregation across documents requires bringing together the relevant records; a small set of nearby chunks may not contain the complete answer.

Macklin offers multiple hops, including a three-hop example, as a practical signal to consider graphs rather than as a measured cutoff.

Vectors remain useful in this design. They find promising material by similarity; the graph supplies relationships that retrieval can follow. The recommendation is to use both, preserving the vector layer while adding connected context.

A retriever is the component that fetches this outside information for the answer. Macklin’s mnemonic is a Golden Retriever: it goes and gets what is needed. Neo4j’s options include full-text search, vector search and text-to-Cypher retrieval. A natural-language question can therefore lead to different ways of searching the stored data.

The news-chatbot example separates three complementary sources:

• News API: fetch current articles and headlines.
• Knowledge graph: recover relationships between topics, including historical connections.
• Document database: store and retrieve the articles, reports and other text.

These are distinct contributions to an answer; the example does not require them to run as a sequential chain.

A knowledge graph itself needs only entities and relationships to get started. Macklin and a colleague can each be nodes, with separate colleague and friend relationships between them. The same pair of entities can participate in more than one relationship—a useful distinction when the question depends on how two things are connected.

Graph construction begins with unstructured text. The teaching example uses the Neo4j Wikipedia page: collect its text, split it into chunks and generate embeddings. An LLM also extracts the entities and relationships expressed in that text. Manual extraction is possible, but the model is introduced to reduce that work.

The extraction prompt asks for entities and relationships in JSON objects. Parsing that structured output supplies the information used to generate the graph. The resulting example distinguishes Neo4j the company from Neo4j the database, connects the database to its developer, and records that it is implemented in Java. Those connections turn statements in prose into facts that can be queried through Cypher, Neo4j’s graph query language.

The hands-on build then moves to Neo4j’s open-source GraphRAG for Python package. It supplies knowledge-graph construction and retrieval capabilities, so the exercise can concentrate on configuring the pipeline and connecting its output to an agent.

The course opens a GitHub Codespace with the workshop project. Its environment template becomes a real .env file containing the database connection details and OpenAI key. Editing only .env.example is a setup trap: the running program needs the configured .env file.

There are two database options. A personal Aura instance is preferred for the later agent exercise; participants who cannot finish that setup can use the small database supplied within GraphAcademy. Either route must provide a working Neo4j connection because that database is where the builder writes the graph.

Before ingestion, the environment test runs and reports five passing tests. This verifies the immediate prerequisites: the Python environment can connect to Neo4j and use the OpenAI key. It does not yet demonstrate that extraction or retrieval produces correct answers.

Running the structured knowledge-graph builder starts populating Aura. Macklin has already loaded data to avoid waiting for the entire process, so the displayed database is not a clean first run. The builder writes the data; querying lets participants inspect what has arrived.

What did ingestion actually create? After correcting the path assignment in a Cypher query, the demonstration returns connected nodes and relationships. Clicking the results exposes their data. Aura’s graph view therefore serves as an interactive query result, giving participants a way to inspect the stored structure rather than merely look at a diagram. Repeating the read query reveals changes as ingestion progresses; it is the builder that generates the graph.

Aura’s agent interface may first need enabling at the organization level. The demonstration turns on Generative AI Assistance and Aura Agent in Settings, then creates an agent from scratch against the database instance. The agent is named GraphRAG Fundamentals Agent and described as providing Neo4j GraphRAG information grounded in this particular GraphAcademy course.

A missing prerequisite interrupts creation: similarity search requires a vector index. Having a populated graph does not by itself make the similarity-search tool available. The workshop returns to the course’s vector-index example and runs its creation command in Aura’s Cypher query interface.

On Macklin’s instance, the index already exists because the command was run earlier. With that dependency satisfied, the build returns to agent configuration and can add similarity search. The ordering matters: construct the data, create the index used to search its embeddings, then configure the tool against that index.

The similarity-search tool is called Search lesson content. Its purpose is to find information in the course so the agent can answer questions about lessons. The configuration selects the chunk-embedding vector index, retrieves the top five matches, and uses OpenAI’s text-embedding-3-small with 1,536 dimensions.

The consequential option is to return connected graph data. Similarity search supplies an entry point; the retrieval query determines what associated graph information comes back with it. Aura supports configuring this through natural language or Cypher, and the demonstration uses a supplied Cypher query.

How does the query connect vector retrieval to graph retrieval? The highlighted requirement is that it work with the retrieved nodes and their similarity scores. Those are the bridge between finding nearby embeddings and returning connected context. The full traversal is not spoken aloud, so the demonstration does not establish which relationship paths or expansion depth it uses.

The finished agent has lesson similarity search and a natural-language-to-Cypher tool. These support different kinds of work: finding relevant lesson content and asking structured questions of the graph. The interface also offers an Update agent action as the underlying knowledge graph changes.

The first question asks how temperature affects an LLM. The agent answers that temperature controls randomness in generated text, discusses higher and lower settings, and displays course sources for further reading. Which material supports that explanation? The accompanying sources make the retrieved basis inspectable. The PDFs came from GraphAcademy’s introductory GenAI course; this build ingests those documents for retrieval rather than demonstrating model-weight training.

The next question changes the task: “How many lessons are there in this course?” Finding a passage about a concept and counting lessons are different operations. A top-five similarity result cannot be assumed to contain every lesson; the count needs the relevant structured records. This is the aggregation use case introduced earlier.

The agent returns 30, while Macklin says the course contains 15 lessons. She attributes the doubled answer to rerunning the graph builder. The demonstration then checks Aura directly, including the option to select the lesson node type, and reports 15. The explanation also mentions not refreshing the agent’s information, so it does not isolate whether duplication, stale agent state or the generated query caused the discrepancy.

The useful result is the ability to investigate the disagreement. A working GraphRAG pipeline can still give a wrong aggregate answer. Direct access to the stored lesson records provides a separate check on the agent’s response, making the graph useful for diagnosis as well as retrieval.

The closing code walkthrough explains what produced the database. The structured builder combines a Neo4j driver for database access, an OpenAI LLM, OpenAI embeddings, a simple knowledge-graph pipeline and a fixed-size text splitter. The splitter uses a chunk size of 500 and an overlap to preserve context across boundaries; the spoken explanation does not specify the size unit or overlap value.

Extraction is guided by a schema with several distinct parts:

• Node types: technology, concept, example, process, challenge and benefit.
• Relationship types: relations such as related to, part of and used in.
• Allowed patterns: combinations of endpoint types and relationships, such as a technology related to another technology, or an example used in a technology.

The patterns give extraction more structure than a vocabulary of labels alone: they describe which kinds of entities a relationship should connect.

Where does that structure enter the implementation? The builder passes the node types, relationship types and patterns into the knowledge-graph pipeline. They guide how the LLM turns document statements into graph records, instead of leaving it to invent a fresh set of types for each piece of text.

For business documents, the schema should fit the business material rather than copy this course’s categories. An LLM can help propose that schema; once chosen, it guides extraction. The demonstration’s inputs include CSV data and lesson PDFs, which the builder processes into the graph queried in Aura. Schema choice therefore shapes what later retrieval can ask about and connect.

The workshop closes by pointing toward hands-on study of Cypher, knowledge graphs and context graphs in GraphAcademy. Those are practical next steps after the agent works: learn to query its evidence directly, and learn to model the relationships that the next set of questions will need.