Enhancing RAG with RAPTOR

Previous Next

Core idea

AI-generated

RAPTOR—Recursive Abstractive Processing for Tree-Organized Retrieval—builds a hierarchy over document chunks. Nearby chunks are clustered, summarized, and then clustered again. Retrieval can search both detailed leaves and higher-level summaries, helping with questions that span several passages.

Indexing flow

AI-generated

  1. Split documents into leaf chunks with source metadata.
  2. Embed and cluster related chunks.
  3. Summarize each cluster into a parent node; repeat as needed.
  4. Store the tree with source lineage.

RAPTOR indexing and retrieval treeChunks are recursively clustered and summarized; retrieval searches leaves and summaries while retaining source lineage.RAPTOR indexing and retrieval tree Document chunks are embedded, clustered, and summarized recursively; retrieval may search detailed leaves and higher-level summaries while citations resolve to source chunks. {"provenance": "AI-generated", "sourceAssets": []} RAPTOR builds and searches a hierarchy Chunk 1 Chunk 2 Chunk 3 Chunk 4 Chunk 5 Chunk 6 Cluster Asummary Cluster Bsummary Cluster Csummary Higher-level summary query search summaries search leaves Answer evidence retains lineage back to the original chunks.

At query time, either search all levels together or traverse from broad nodes to their most relevant children. The second approach can reduce search space but is more sensitive to an early routing mistake.

Trade-offs and evaluation

AI-generated

  • Summaries cost more to build and may omit details.
  • Updates may require regenerating ancestor nodes.
  • Keep citations to original chunks and compare with a flat or hybrid baseline.

Evaluate retrieval recall separately from answer quality. Compare against a flat chunk baseline using the same corpus, questions, model, and token budget.

Source: RAPTOR paper.

Building Systems with ChatGPT LangChain