# What Is RAG? A Practical Guide to RAG, GraphRAG and Hybrid Retrieval
Canonical source: [https://isaiuseful.com/rag](https://isaiuseful.com/rag)
- [Build](https://isaiuseful.com/guides.html.md)
· Retrieval systems · checked 28 July 2026
Start with a small, cited retrieval baseline. Add graph structure only when the questions genuinely depend on entities, connections or whole-corpus themes—and keep both indexes rebuildable from governed source data.
- [Choose a retrieval route](#choose)
- [Build the pipeline](#build)
- [Define the test](#evaluate)
**3**
retrieval routes
VECTOR · GRAPH · HYBRID
**2**
systems to test separately
RETRIEVER + GENERATOR
**1**
governed source of truth
INDEXES ARE REBUILDABLE
01 · Mental model
## How does retrieval-augmented generation work?
Retrieval-Augmented Generation joins a generator to external, non-parametric memory. The documents remain outside the model weights.
### Prepare
Parse governed sources, preserve document and section identity, split them into useful retrieval units and create searchable representations.
### Retrieve
Use the question to select candidate passages, records or graph neighborhoods, then filter and rerank within the user's access boundary.
### Generate
Give the model the selected evidence, require an answer bounded by it and return citations that resolve to the original source—not merely to an index row.
**Evidence boundary:** the [original RAG paper](https://papers.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html) established generation with parametric and retrieved non-parametric memory on specific knowledge-intensive tasks. It did not establish that every modern document-chat system is accurate, secure or citation-faithful.
02 · Architecture decision
## Use the lightest route that can pass the real questions.
GraphRAG is an additional data product and query path, not an automatic upgrade.
**Vector RAG**
Start here when answers usually live in one or a few text passages: policies, manuals, case notes, contracts or knowledge-base articles.
**Search + rerank**
Add keyword retrieval, metadata filters or a reranker when dense similarity misses exact terms, identifiers or authoritative sources.
**GraphRAG**
Test graph retrieval when questions depend on connected entities, multi-document paths, controlled relationship types or themes across a corpus.
**Hybrid**
Combine text and graph context only when the evaluation set shows complementary failures and the improvement justifies two synchronized indexes.
| Route | Best first test | What it preserves | Primary failure | Operating burden |
| --- | --- | --- | --- | --- |
| Vector RAG | “What does this document say?” | Semantic similarity and source chunks | Related wording can outrank the needed fact; evidence may be split across chunks | Lowest of the three; still requires parsing, updates, ACLs and evaluation |
| GraphRAG | “How are these people, events or systems connected?” | Entities, typed relationships, neighborhoods and graph communities | Extraction and entity-resolution errors create missing or false paths | Higher; graph construction, resolution, provenance and query tuning |
| Hybrid RAG | A query needing both an exact source passage and a cross-source relationship | Broad textual evidence plus explicit structure | More context can add noise; ranking and synchronization become harder | Highest; two retrieval paths, merging, budgets and regression tests |
Research context: a [2025 systematic evaluation](https://arxiv.org/abs/2502.11371) reports distinct strengths for RAG and GraphRAG across question answering and query-focused summarization; [HybridRAG](https://arxiv.org/abs/2408.04948) reports gains from combined vector and graph retrieval on a financial-transcript experiment. These results motivate testing routes—they are not universal production guarantees.
03 · Reference architecture
## See exactly what the graph changes.
Both systems retrieve evidence for a generator. Conventional RAG retrieves passages; GraphRAG first builds explicit entities, relationships and corpus-level structure.
Conventional RAG
**Retrieve the best passages.**
Lowest useful complexity for document-grounded answers
**Index time**
Governed source
**Documents + records**
Version · ACL · owner
Prepare
**Parse + chunk**
Page and heading IDs survive
Derived memory
**Text + vector index**
Keywords · embeddings · metadata
**Question time**
Input
**User question**
Identity + allowed scope
Retrieve
**Search + filter + rerank**
Select top-k passages
Context
**Source chunks**
Passages + resolvable citations
Generate
**LLM answer**
Answer · abstention · citations
**Best fit**
Policies, manuals, contracts, case notes and answers that live in a few passages.
GraphRAG
**Retrieve connected evidence.**
Additional structure for relational and whole-corpus questions
**Index time**
Governed source
**Documents + records**
Version · ACL · owner
Extract + resolve
**Entities + relations**
Aliases · claims · timestamps
Derived memory
**Knowledge graph**
Edges · communities · source links
**Question time**
Input
**User question**
Entities + question class
Route + traverse
**Local, global or DRIFT**
Neighborhood · paths · communities
Context
**Subgraph + source chunks**
Nodes · edges · supporting text
Generate
**LLM answer**
Answer · path · citations
**Best fit**
Dependencies, ownership, lineage, investigations, multi-document paths and corpus-wide themes.
**The architectural difference is upstream of the model.** GraphRAG adds extraction, entity resolution, graph maintenance and graph-aware retrieval. A hybrid system keeps the passage path and adds the graph path only where the evaluation set proves it helps.
> Visual: Reference hybrid RAG pipeline
**Visual reading order:**
1. **01** **Govern sources** Rights, classification, owner, version, retention and access policy.
2. **02** **Parse + identify** Document, page, heading, record and stable source identifiers.
3. **03** **Build indexes** Text chunks and embeddings; optional entities, edges and community summaries.
4. **04** **Route + retrieve** Metadata filters, vector or keyword candidates and bounded graph traversal.
5. **05** **Merge + answer** Rerank, enforce a context budget and cite the original evidence.
6. **06** **Measure + refresh** Log versions, misses, path validity, latency, cost and user corrections.
**GraphRAG is more than a graph database.** Microsoft's current implementation extracts entities, relationships and claims, detects communities, produces summaries and embeds text. Its query engine separates entity-focused local search, whole-dataset global search, DRIFT and basic vector search. See the official [indexing overview](https://microsoft.github.io/graphrag/index/overview/) and [query overview](https://microsoft.github.io/graphrag/query/overview/) .
04 · Graph gate
## Earn the graph with questions that require one.
A graph pays for itself only when explicit structure improves an outcome that simpler retrieval cannot reach reliably.
**Strong signal**
### Relationships are the answer.
Ownership, dependency, lineage, supply chains, citations, organizational paths or event sequences must be traversed and explained.
**Strong signal**
### Questions span the corpus.
Readers need themes, clusters or connected evidence across many documents rather than the nearest matching passage.
**Conditional**
### A useful schema exists.
Stable identifiers, entity types and relationship rules already exist—or the workflow value can fund their creation and maintenance.
**Weak signal**
### “Graphs sound smarter.”
A product label, demo or vendor benchmark does not justify graph extraction when ordinary search already passes the acceptance set.
### A GraphRAG index is a governed data pipeline, not an LLM side effect.
Every stage creates an artifact that can be inspected independently. If the final answer is wrong, the trace should reveal whether the source was missing, the entity was split, the relationship was invented, the wrong neighborhood was traversed or the generator ignored valid context.
> Visual: Six stages in a GraphRAG indexing pipeline
**Visual reading order:**
1. **01 · Segment** **Text units** Preserve document, page, heading, version and ACL on every unit.
2. **02 · Extract** **Entities + claims** Find people, systems, events, concepts and candidate relationships.
3. **03 · Resolve** **Canonical identity** Merge aliases; keep homonyms, subsidiaries and versions separate.
4. **04 · Relate** **Typed edges** Direction, predicate, validity time, confidence and supporting source.
5. **05 · Organize** **Communities + summaries** Cluster the graph for broader questions without discarding raw evidence.
6. **06 · Publish** **Queryable index** Version the graph, embeddings, prompts and extraction configuration together.
### Choose the query mode by the shape of the question.
| Query mode | Question shape | Context assembled | What to test | Cost / failure boundary |
| --- | --- | --- | --- | --- |
| Basic text / vector | “What does policy 7.2 say about retention?” | Top matching passages | Exact source appears in top-k; citation resolves | Cheapest baseline; can miss distributed or relational evidence |
| Local graph search | “Who owns service A, and which incidents involved it?” | Seed entities, neighbors, relationships, community context and linked text | Entity mapping, edge direction, path validity and source support | Sensitive to duplicate entities and missing edges |
| Global graph search | “What recurring risks appear across the full incident archive?” | Community reports evaluated and reduced across the corpus | Theme coverage, minority evidence, aggregation bias and token budget | Resource-intensive; summaries can flatten exceptions |
| DRIFT / exploratory | “How might these local failures connect to broader operating patterns?” | Community-informed starting point plus detailed follow-up retrieval | Breadth gained, irrelevant branches, reproducibility and stop conditions | Broader search can add latency and plausible noise |
| Explicit graph query | “List approved suppliers two hops from programme X as of 30 June.” | Schema-bound traversal with filters and validity time | Exact path, filter semantics, authorization and empty-result behavior | Precise only when schema and graph data are precise |
### Graph quality is won or lost at four seams.
**Identity**
“ACME,” “ACME Ltd” and a product called “Acme” cannot be merged because an embedding says they look alike. Use stable keys where available, alias rules where necessary and a review queue for uncertain merges.
**Relationship**
“Uses,” “owns,” “approved by” and “mentioned with” are not interchangeable. Define direction and allowed entity types; reject an edge that cannot name its predicate and source.
**Time**
A graph without valid-from, valid-to and observed-at fields can answer with a relationship that was once true. Preserve event time separately from ingestion time and expose staleness.
**Provenance**
An extracted edge is navigation, not proof. Store the document and text-unit IDs behind it, and make every answer path resolve to original evidence a reviewer can inspect.
### Trace one relational question end to end.
> Visual: Worked GraphRAG query trace
**Visual reading order:**
1. **Question** **Which change caused the outage, and who approved it?** Requires an event, deployment, service, incident and person to connect.
2. **Seed** **Map outage + service** Resolve the incident ID and service alias before expanding.
3. **Traverse** **Incident → deployment → change** Follow only allowed, time-valid edge types within the incident window.
4. **Join** **Change → approval → person** Recover the approval record and responsible identity.
5. **Retrieve** **Open supporting passages** Deployment log, incident timeline and approval record enter context.
6. **Answer** **State path + uncertainty** Cite each hop; abstain if any required edge lacks evidence.
### The graph earns production only if it passes a separate acceptance set.
| Test family | Fixture | Pass rule | Failure it catches | Release action |
| --- | --- | --- | --- | --- |
| Entity resolution | Aliases, homonyms, mergers, renamed systems and versioned products | Known same entities merge; known different entities remain separate | False joins and broken neighborhoods | Block new resolver; review uncertain identity queue |
| Relationship extraction | Positive, negative, hypothetical and historical statements | Predicate, direction, time and source match the reference | Invented, reversed or timeless edges | Quarantine edge type or fall back to text retrieval |
| Path retrieval | Known one-, two- and three-hop questions plus impossible paths | Required path is returned; impossible path produces no fabricated bridge | Traversal gaps and graph completion by guessing | Tune seeding and hop limits; keep abstention |
| Global themes | Dominant, minority and contradictory themes across a frozen corpus | Material themes survive aggregation with traceable evidence | Summary flattening and majority bias | Change community level or return scoped results |
| Authorization | Users with overlapping but different document and entity rights | No node, edge, summary or citation crosses the caller's boundary | Relationship leakage across collections | Stop serving graph results until fixed |
Implementation references: Microsoft's [standard and fast indexing methods](https://microsoft.github.io/graphrag/index/methods/) , [query-mode overview](https://microsoft.github.io/graphrag/query/overview/) and the [systematic RAG versus GraphRAG evaluation](https://arxiv.org/abs/2502.11371) . Microsoft's method notes explicitly trade richer extraction against cost and noisier fast graphs; treat that as an engineering choice to benchmark on your corpus.
**From agent loop to shared graph memory.** The downloadable 11-page study note connects Karpathy's autoresearch loop and AgentHub commit DAG with multi-agent workflows, knowledge-graph provenance and a staged implementation path.
- [Download Graph Engineering (PDF) →](https://isaiuseful.com/downloads/Karpathy-Graph-Engineering-Systems.pdf)
**Graph data can be confidently wrong**
Keep source identifiers and confidence on every extracted claim. Test duplicate entities, aliases, missing edges, contradictory timestamps and unauthorized relationship leakage. If a path cannot resolve back to evidence, do not present it as a citation.
05 · Build sequence
## Build one measured slice before a platform.
The first deliverable is not “chat with everything.” It is a small set of real questions answered within a defined data and authority boundary.
| Stage | Minimum deliverable | Acceptance check | Do not hide | Scale trigger |
| --- | --- | --- | --- | --- |
| 1 · Contract | One audience, corpus, task, owner and answer policy | Known answerable, unanswerable and forbidden questions | Rights, personal data, stale sources and access boundaries | The workflow owner accepts the question set |
| 2 · Baseline | Keyword and vector retrieval over a small representative corpus | Correct evidence appears in top-k for held-out questions | Parser failures, empty pages, tables and duplicate versions | Retrieval misses are understood by category |
| 3 · Grounded answer | Answer, refusal and resolvable source citations | Claims match cited evidence; unsupported questions abstain | Prompt and model version, context used and truncation | A stable regression set passes repeatedly |
| 4 · Optional graph | Only the entity and relationship types needed by failed questions | Valid paths improve the named failures without harming simple QA | Resolution confidence, provenance and graph build cost | Measured gain exceeds added latency and upkeep |
| 5 · Operate | Incremental refresh, deletion, ACL enforcement, monitoring and rollback | Source changes appear on time; revoked data disappears everywhere | Index age, last successful build and partial failures | Restore and re-index drills work |
06 · Evaluation
## Score the retriever before blaming the model.
A fluent wrong answer can begin with a retrieval miss, a ranking mistake, an incomplete graph path or unsupported generation. Preserve the stage boundary.
Retriever
### Did the right evidence arrive?
Measure top-k hit rate, context precision and recall, metadata-filter accuracy and—when graph retrieval is used—entity and path coverage.
Generator
### Did the answer stay inside it?
Measure answer correctness, citation entailment, refusal behavior and faithfulness to the retrieved context. Review consequential failures manually.
System
### Was it useful repeatedly?
Track end-to-end latency, cost, freshness, access-control failures, user corrections and pass rate across repeated runs—not only a best attempt.
**Metric caution:** Ragas documents context precision, context recall, response relevance and faithfulness as separable RAG measures. Some of these use model-based scoring. Calibrate them against references and human review rather than treating one automated score as independent proof. See the [current metric catalogue](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/) and this site's [evidence policy](https://isaiuseful.com/evidence.html.md) .
**Ten real trials before more authority**
Freeze the corpus snapshot, questions, expected evidence and pass rules. Run vector, graph and hybrid variants on the same set. Save every retrieved context and answer. Expand only the route that improves the named workflow without unacceptable security, latency or maintenance regressions.
07 · Production boundaries
## Retrieval creates a new route to governed data.
Embeddings, chunks, graph edges, traces and caches can reveal sensitive material even when the original repository is protected.
**Authorize first**
Filter candidates by the user's entitlement before generation. Do not retrieve broadly and ask the model to redact afterward.
**Preserve provenance**
Carry source ID, section or page, observed time, version and classification through every index and response.
**Delete everywhere**
Define how revocation reaches chunks, vectors, graph facts, summaries, caches, traces, backups and derived evaluations.
**Expose freshness**
Show the source date and last successful index build. Stop or warn when an update fails instead of silently serving stale evidence.
### Map policy to every derived surface.
A source repository's controls do not automatically follow copied text, embeddings, community summaries or traces. Each derived surface needs an explicit owner, authorization check, retention rule and deletion path.
| Surface | What it can reveal | Minimum control | Deletion / correction path | Production signal |
| --- | --- | --- | --- | --- |
| Parsed text + chunks | Full passages, hidden fields, OCR mistakes and prior versions | Encrypted storage, document ACL, parser allowlist and quarantine for unreadable files | Remove by stable source/version ID, then rebuild affected indexes | Parse success, skipped content and last valid version |
| Embeddings + search index | Similarity, membership and retrievable sensitive concepts | Tenant/collection isolation, pre-retrieval filters, private endpoints and tested backups | Delete vector and metadata by source ID; verify it cannot be retrieved | Index age, document count, filter result and restore test |
| Graph nodes + edges | Sensitive relationships that no single document states plainly | Node and edge authorization, provenance, time validity and restricted traversal | Retract affected claims, summaries and neighborhoods; re-run resolution | Orphan rate, unresolved identities, invalid edges and ACL denials |
| Community summaries + caches | Cross-document conclusions and stale aggregate facts | Scope summaries to compatible ACL domains; attach input version set and expiry | Invalidate every summary whose dependency set changed | Dependency version, expiry and cache-hit freshness |
| Prompts, traces + eval sets | Questions, retrieved evidence, model answers, corrections and user intent | Redaction, access-limited observability, retention limits and separation from product analytics | Purge by run, user and source ID without destroying required audit records | Sampling rate, redaction failures, retention age and reviewer access |
### Treat refresh as a versioned release.
> Visual: Safe retrieval index refresh lifecycle
**Visual reading order:**
1. **01 · Detect** **Source changed** Create, update, revoke or classification change enters a durable queue.
2. **02 · Rebuild** **Derived artifacts** Reparse affected units; re-embed; re-extract graph claims and summaries.
3. **03 · Validate** **Quality + policy** Parser, retrieval, graph, ACL, deletion and freshness checks run before publish.
4. **04 · Publish** **Atomic version** Promote compatible index, graph, prompt and configuration versions together.
5. **05 · Observe** **Canary questions** Compare hit rate, groundedness, latency, empty results and authorization denials.
6. **06 · Roll back** **Last valid release** Keep a known-good manifest; never serve a half-updated vector/graph pair.
### Define the failure response before the first incident.
| Failure | How it appears | Automatic response | Owner decision | Evidence to retain |
| --- | --- | --- | --- | --- |
| Source or parser failure | Missing pages, zero-length chunks, broken tables or unexpected document-count drop | Quarantine source; keep last valid version with a visible stale flag | Repair parser, accept exclusion or stop the collection | File hash, parser version, errors and skipped ranges |
| Retrieval regression | Known evidence falls out of top-k or irrelevant context dominates | Fail canary; hold index promotion; fall back to last valid release | Change chunking, embeddings, filters or reranker | Question, expected source, candidates, scores and versions |
| Graph corruption | Entity explosion, suspicious new hubs, reversed edges or impossible paths | Disable affected edge types or graph route; retain text RAG | Re-run resolution, correct schema or rebuild graph | Extraction prompt, claim source, merge history and graph diff |
| Grounding failure | Answer claim is not supported by the supplied passage or path | Abstain or return evidence without synthesized claim | Adjust prompt, context assembly, model or answer policy | Full context, answer, citations, grader and human adjudication |
| Authorization leak | Restricted chunk, entity, relationship or summary reaches an unauthorized run | Stop affected route, revoke caches and preserve incident evidence | Breach process, user notification and safe re-enable criteria | Caller identity, policy decision, retrieved IDs and access logs |
| Refresh drift | Vector and graph versions disagree or revoked facts remain in summaries | Mark release unhealthy and route to a coherent known-good version | Complete rebuild or targeted dependency invalidation | Release manifest, dependency graph and deletion verification |
### Make ownership as explicit as the architecture.
Data owner
### Controls the source boundary.
Approves corpus, rights, classification, retention, authoritative versions and who may see which documents and relationships.
Retrieval owner
### Controls the derived memory.
Owns parsers, chunking, indexes, entity resolution, refresh, deletion, restore, latency and retrieval regression tests.
Workflow owner
### Controls the useful answer.
Defines questions, acceptance rules, abstention, human escalation, outcome measurement and the authority the answer may trigger.
**Cost lever, not a RAG property:** vector quantization can reduce index memory and storage, but it is lossy and system-specific. Azure AI Search currently documents up to 28× index-size reduction for binary quantization and recommends oversampling and rescoring to offset information loss. Treat compression as an experiment with the same recall set. [Read the official guide →](https://learn.microsoft.com/en-us/azure/search/vector-search-how-to-quantization)
08 · Replaceable components
## Choose interfaces before brands.
Keep the raw corpus, parsing output, retrieval evaluation and source identifiers portable. The index is derived infrastructure.
### Prepare + retrieve
Document parsers, chunkers, keyword search, vector stores, metadata filters and rerankers. Start with the fewest services that meet the data boundary.
- [Compare retrieval tools →](https://isaiuseful.com/tools.html.md#tools-build-knowledge-and-workflows)
- [See a bounded local knowledge recipe →](https://isaiuseful.com/guides.html.md#second-brain)
### Relate + traverse
Entity extraction, resolution, graph storage and query generation. A graph store is optional; source provenance and repeatable graph builds are not.
- [Compare graph options →](https://isaiuseful.com/tools.html.md#tools-build-knowledge-and-workflows)
- [See an operational graph stack →](https://isaiuseful.com/diy-palantir.html.md#stack)
### Trace + evaluate
Capture the query, route, retrieved evidence, graph path, prompt, answer, model and versioned grader result so failures can be reproduced.
- [Compare evaluation tools →](https://isaiuseful.com/tools.html.md#tools-fine-tune-evaluate-and-reproduce)
- [Choose an evaluation pattern →](https://isaiuseful.com/benchmarks.html.md#database)
The practical default
Start with cited vector retrieval. Let failed questions earn more structure.
- [Choose the route](#choose)
- [Build one slice](#build)
- [Compare retrieval with training](https://isaiuseful.com/training-models.html.md#chooser)