Graph RAG vs vector RAG: when the graph earns its keep

Graph RAG vs vector RAG gets argued like a matter of taste, and it is much closer to a matter of accounting. Vector retrieval is very good at finding the passage that resembles what you asked. A graph is good at finding what is connected to the thing you asked about. Most people already know which of those their users are actually asking for, and buy the expensive index anyway.

What is the difference between graph RAG and vector RAG?

The difference is the question the retrieval step asks on your behalf. Vector retrieval asks what text resembles this. Graph retrieval asks what is connected to this. The database, the chunk size, the reranker, all of it is downstream of that one choice.

An embedding turns a passage into a point in a space where nearby means similar in meaning. That is the whole trick and it is a very good one, but the space only has distances in it. Two facts that have to be laid end to end sit wherever their wording puts them, which is usually nowhere near each other. I wrote up the shape of that failure in the questions similarity search cannot answer: a fix is recorded in one note, the fork that predates it in a second, and which site runs which fork in a third. No single passage holds the answer, so no amount of similarity retrieves it.

A graph stores the same information as things and relations between things. The three-note answer becomes a walk of two hops. Retrieval matches the entities in the question to nodes, walks outward, turns the small subgraph it collected back into sentences, and the model reads a path instead of a pile. That is the entire mechanical difference, and everything expensive about graph RAG comes from building the nodes and edges in the first place.

What do the benchmarks actually say?

They say the gains are real, concentrated in relational questions, and close to nothing on the questions most systems get asked most of the time.

The cleanest side-by-side I know of is HippoRAG 2 (Gutiérrez et al., February 2025, revised June 2025), which builds a graph over the corpus and runs Personalized PageRank across it, then measures against a strong embedding-only baseline. On 2WikiMultihopQA, passage recall at 5 goes from 76.5 to 90.4. On MuSiQue, 69.7 to 74.7. Then look at the single-hop questions in the same table: on PopQA, recall at 5 moves from 51.0 to 51.7. Almost fourteen points of recall on one benchmark and seven tenths of a point on another, from the same system, on the same metric. The answer scores make the same point a different way: single-hop barely moves, with PopQA F1 landing at 56.2 against the baseline's 55.7, while 2Wiki F1 moves 61.5 to 71.0.

GraphRAG-Bench (Xiang et al., June 2025, revised February 2026) sets out to test exactly this and does not hedge in the abstract:

Despite its conceptual promise, recent studies report that GraphRAG frequently underperforms vanilla RAG on many real-world tasks.

Their measurements split the same way. On the literature corpus, complex reasoning, scored by a GPT-4o-mini judge, goes to HippoRAG 2 at 53.38% against basic RAG with a reranker at 42.93%. Simple fact retrieval on that same corpus goes the other way: basic RAG with a reranker takes 60.92% against Microsoft's GraphRAG at 49.29%. Their summary sentence is the one worth taping to the wall: "basic RAG is comparable to or outperforms GraphRAG in simple fact retrieval tasks that does not require complex reasoning across connected concepts."

One caveat on all of it. Most sensemaking comparisons in this literature are scored by an LLM judge, and judges have preferences. I have not traced every correction for judge bias to its underlying paper myself, so treat win-rate numbers on open-ended sensemaking tasks as softer evidence than the recall numbers above, which are computed against labeled gold passages.

Graph RAG vs vector RAG, side by side

What does the graph actually cost to build?

Whatever you decide to spend, because extraction is a dial rather than a switch. The phrase "add a knowledge graph" covers four different bills.

At the expensive end sits Microsoft's standard GraphRAG, which has a model read every passage and name its entities and relations, then clusters the graph and has a model write a report on every cluster. Their own indexing documentation is refreshingly blunt about where the money goes:

We estimate graph extraction to constitute roughly 75% of indexing cost.

The same page describes FastGraphRAG, where "entities are noun phrases extracted using NLP libraries such as NLTK and spaCy", and says plainly that it is "much cheaper, but the tradeoff is that the extracted graph is less directly relevant for use outside of GraphRAG". Zero model calls, a rougher graph. And LazyGraphRAG (Microsoft Research, 25 November 2024) skips the up-front summaries entirely and defers the model work to query time, which puts its indexing cost at "identical to vector RAG and 0.1% of the costs of full GraphRAG". A thousandfold spread inside one vendor's own product line tells you the cost is a design decision, not a property of graphs.

The underlying asymmetry is simple arithmetic. Embedding a passage is one forward pass through a small encoder, priced per input token. Extraction is a generative call: the model reads the passage and then writes triples, so you pay to read and again to write. On OpenAI's published pricing, checked 3 September 2026, text-embedding-3-small runs $0.02 per million tokens while gpt-4o-mini is $0.15 per million in and $0.60 per million out. Reading a passage with the cheap chat model already costs seven and a half times what embedding it costs, before it emits a single character.

Then there is the update. The LightRAG paper (Guo et al., October 2024) makes the case that community-based designs have to dismantle and rebuild their community structure when new text arrives, while an append-only edge design absorbs the new material with one extraction pass. If your corpus is written once and read forever, that difference is academic. If it changes weekly, it is the whole budget, and it is the reason I care more about update cost than about index cost when I am keeping a knowledge base current.

The hybrid pattern, and what I actually run

Nearly every system that works in production keeps both tools and knows which question is which. The pattern that has held up for me is embeddings as the way in, edges as the way around: similarity picks the entry points, then a short traversal adds what similarity could not reach.

The assistant on this site runs a blend rather than a race. A MySQL full-text match and cosine similarity over embedded chunks are normalized and mixed at 0.45 keyword to 0.55 semantic, anything under a 0.28 noise floor is dropped, and the top five survive. Then the retrieval walks one hop out along a relations table and appends at most two connected chunks. The knowledge base is 283 hand-curated chunks, all embedded at 2,048 dimensions.

SELECT IF(from_id IN ($ph), to_id, from_id) AS nid, MAX(weight) AS w
  FROM chat_knowledge_edges
 WHERE from_id IN ($ph) OR to_id IN ($ph)
 GROUP BY nid ORDER BY w DESC

That is the entire graph step, where $ph is the placeholder list for the chunk ids similarity just returned. Two rules build the edges, and neither one calls a model: a chunk whose text contains another chunk's title (or the part after its colon) gets a mentions edge, and two chunks sharing a tag used by six chunks or fewer get a shared-tag edge, with commoner tags needing a second shared tag to agree before they link. A full rebuild compares every chunk title against every chunk body, 79,806 comparisons, and finishes in under a third of a second on my laptop. It currently yields 227 edges: 33 from mentions, 194 from shared tags.

Three things about that design are load-bearing. The edges rebuild themselves whenever a chunk is saved or deleted, so there is nothing to maintain by hand. The neighbor lookup over-collects candidates before applying the audience filter, so an edge pointing at a staff-only chunk cannot silently consume one of a public visitor's two slots. And if the edges table is missing entirely, which is true of older forks of this engine running on client sites, the whole graph step is a caught exception and retrieval behaves exactly as it did before. A retrieval upgrade that can take a site down is not an upgrade. I described the day this shipped in the harness reads my texts.

What this is not: a knowledge graph in the sense the papers mean. My nodes are documents, not entities, and my relations are two heuristics rather than extracted predicates. It sits at the cheap end of the dial in the diagram above. It buys the one behavior I wanted, which is that a chunk worded nothing like the question can still arrive because something already retrieved points at it, and it buys it for no tokens at all.

How should you decide?

Count your questions before you buy an index. The decision is almost always visible in the query log and almost never visible in the architecture diagram.

• Take fifty questions people actually asked your system, from the logs, not from your imagination. Imagined questions are uniformly more interesting than real ones.
• Label each one: lookup, chain, or global. A chain question names two things and asks how they relate, or asks what happened after something, or which of a set depends on another. A global question asks about the corpus as a whole ("what are the recurring themes here"), which is the case Microsoft's community summaries exist to serve.
• If chains and globals are under a tenth of the traffic, the honest answer is better vector retrieval: a reranker, better chunking, and a hybrid keyword blend will beat a graph you cannot afford to keep fresh.
• If the chain questions are the ones a person currently answers by hand, price the graph against that person's hour rather than against a benchmark point. This is where graphs earn their keep in small businesses, and the calculation has nothing to do with F1.
• Start at the cheap end of the extraction dial and move up only when a real question fails. Derived edges, then parser-extracted nodes, then model extraction, then community summaries.
• Label the route on every retrieved chunk. Mine records whether a chunk arrived by similarity or by edge, and that one field is what makes the next decision evidence rather than intuition.

Where the graph stops earning it

Entity resolution is the real project, and it is nobody's favorite. Notes call the same thing by a first name, a project codename, and a domain; a naive extractor mints three nodes, the graph fragments into islands, and chain queries return nothing while every dashboard reports a healthy index. Every graph write-up I have read puts most of the effort here, and having built the small version I believe them.

Staleness is the second one. A graph states facts with a completely straight face long after they stop being true, so edges need dates and old ones need to lose their vote. And hybrid retrieval means two indexes that can disagree: a chunk edited but not re-embedded, an edge pointing at a deleted node. Both are ordinary engineering, and both are work that a pure vector store does not ask of you. That maintenance surface is part of the price, and it belongs in the comparison alongside the token bill. It is one of the five pieces I keep coming back to in the anatomy of an agent harness.

The thing I want to know next about my own is whether the shared-tag rule is earning its place. It produces 194 of my 227 edges, and it is a heuristic about how I write tags rather than a fact about how the knowledge connects. The way to find out is to log the arrival route on every retrieved chunk for a month of real questions and count how often a tag edge is the one that carried the answer. The label is already in the row. I have not counted it yet.