Back to AI Glossary
Retrieval & DataRAG

Retrieval-Augmented Generation

A technique that grounds a language model's output in external data by retrieving relevant documents before generating a response.

Why it matters

RAG is the most practical way to give an LLM access to your private data without fine-tuning. It is the foundation of most enterprise AI deployments today.

How it works

RAG pipelines typically follow a three-step process: index documents into a vector store, retrieve the most relevant chunks at query time, and augment the LLM prompt with those chunks so the model can cite real sources instead of hallucinating.

Key variations

The basic retrieve-then-read pattern has evolved into several specialized variants:

  • Naive RAG — single retrieval pass, straightforward but limited for complex queries.
  • Agentic RAG — an LLM decides when and what to retrieve, routing queries and iterating on results.
  • GraphRAG — combines vector retrieval with knowledge graph traversal for relationship-aware answers.
  • Contextual Retrieval — prepends document-level context to each chunk before embedding, dramatically improving recall.

Common pitfalls

Poor chunking strategy is the number one source of bad RAG quality. Splitting on arbitrary token counts loses semantic coherence. Use semantic chunking or document-aware splitting instead. Also monitor retrieval recall — if the right documents aren't in the top-k results, no amount of prompt engineering will fix the output.

Related Terms

Embeddings— Dense numerical representations of text (or other data) in a high-dimensional vector space, where similar meanings are placed closer together.Vector Database— A database optimized for storing, indexing, and querying high-dimensional vector embeddings, enabling fast similarity search at scale.Knowledge Graph(KG)— A structured representation of information as a network of entities and their relationships, enabling machines to reason about connections between concepts.Hallucination— When an AI model generates information that sounds plausible but is factually incorrect, fabricated, or unsupported by its training data or provided context.

On the AI Radar

Agentic RAG— Embeds autonomous AI agents into the RAG pipeline, replacing static retrieve-then-generate workflows with systems that plan retrieval strategies, use multiple tools, reflect on evidence, and iterate across multiple search steps for complex multi-hop queries.Contextual Retrieval— A technique from Anthropic that prepends chunk-specific explanatory context to each document chunk before embedding and BM25 indexing. Uses an LLM to generate a short context snippet situating each chunk within its source document, improving retrieval accuracy significantly.Knowledge Graphs— Graph-based knowledge representation that enables AI systems to reason over structured relationships rather than relying solely on unstructured text retrieval. The convergence with LLMs through GraphRAG has made knowledge graphs practical for reducing hallucinations and enabling multi-hop reasoning.GraphRAG— Combining knowledge graphs with retrieval-augmented generation for structured, relationship-aware AI reasoning. Moving from research pattern to production standard as naive RAG reaches its limits.
Back to AI Glossary