> ## Documentation Index > Fetch the complete documentation index at: https://builditwithai.xyz/llms.txt > Use this file to discover all available pages before exploring further. # RAG (Retrieval Augmented Generation) > How to give AI access to your documents and data # RAG (Retrieval Augmented Generation) ## The Problem: AI Doesn't Know Your Data You've probably experienced this: * You ask ChatGPT about your company's policies → It doesn't know * You want Claude to analyze your research papers → It can't access them * You need Gemini to answer questions from your documentation → It has no idea **Why?** LLMs are trained on public data. They don't have access to: * Your company documents * Your personal files * Private databases * Recent information (after their training cutoff) * Proprietary knowledge **RAG solves this problem.** *** ## What is RAG? RAG stands for **Retrieval Augmented Generation**. It's a technique that gives AI access to external information by: 1. **Retrieving** relevant information from your documents 2. **Augmenting** the AI's prompt with that information 3. **Generating** a response based on both its training and your data Think of it as giving the AI a "cheat sheet" with exactly the information it needs to answer your question. *** ## How RAG Works (Simplified) Your documents are split into chunks and converted into numerical representations (embeddings) that capture their meaning. ``` Document: "Our return policy allows 30-day returns..." → Chunk 1: "Return policy: 30 days" → Embedding: [0.23, -0.45, 0.67, ...] (vector) ``` These embeddings are stored in a searchable database. Common tools: Pinecone, Weaviate, ChromaDB, FAISS Your question is also converted to an embedding. ``` Question: "What's your return policy?" → Embedding: [0.25, -0.43, 0.65, ...] ``` The system finds the most similar chunks from your documents. ``` Top matches: 1. "Return policy: 30 days for unused items" 2. "Refunds processed within 5-7 business days" 3. "Original receipt required for returns" ``` The retrieved information is added to your question. ``` Context: [Retrieved chunks] Question: "What's your return policy?" Instructions: Answer based on the context provided. ``` The LLM generates an answer using both its training and your documents. ``` "Our return policy allows returns within 30 days for unused items with the original receipt. Refunds are processed within 5-7 business days." ``` *** ## RAG vs Other Approaches ### RAG vs Fine-Tuning | Aspect | RAG | Fine-Tuning | | ------------ | ----------------------- | ----------------------- | | **Cost** | Low (just storage) | High (retraining model) | | **Speed** | Fast to set up | Slow (days/weeks) | | **Updates** | Easy (add new docs) | Requires retraining | | **Use Case** | Factual Q\&A, documents | Changing behavior/style | | **Accuracy** | Can cite sources | No source attribution | **Use RAG when:** You need to give AI access to documents or data **Use Fine-Tuning when:** You need to change how the AI behaves or writes ### RAG vs Long Context Windows Some models (like Claude with 200K tokens or Gemini with 1M tokens) can handle very long inputs. Why use RAG? **Advantages of RAG:** * ✅ Cost-effective (only send relevant chunks) * ✅ Works with any model * ✅ Can search across millions of documents * ✅ Faster responses * ✅ Can cite specific sources **When to use long context instead:** * You need to analyze a specific document in full * The entire context is relevant * You're willing to pay for large context windows *** ## Common Use Cases **Problem:** Support agents need to answer questions from hundreds of help articles **RAG Solution:** * Index all help articles * Agent asks question * RAG retrieves relevant articles * AI generates answer with citations **Tools:** Intercom AI, Zendesk AI, custom solutions **Problem:** Employees need to find information across company docs, wikis, Slack **RAG Solution:** * Index all company documents * Employee asks question * RAG finds relevant information * AI provides answer with sources **Tools:** Glean, Guru, Notion AI, custom solutions **Problem:** Researchers need to query across hundreds of papers **RAG Solution:** * Index research papers * Ask questions about findings * RAG retrieves relevant sections * AI synthesizes information **Tools:** Elicit, Consensus, Perplexity, custom solutions **Problem:** Need to answer questions from contracts, regulations, policies **RAG Solution:** * Index legal documents * Query specific clauses or requirements * RAG finds exact references * AI explains in plain language **Tools:** Harvey AI, custom solutions *** ## Tools That Use RAG ### No-Code Solutions Upload documents directly to ChatGPT (Plus/Team/Enterprise) Upload PDFs and documents to Claude Searches the web and cites sources (RAG over the internet) Queries your Notion workspace ### Low-Code Platforms * **Stack AI** - Build RAG apps without code * **Voiceflow** - Create chatbots with knowledge bases * **Chatbase** - Train chatbots on your documents * **CustomGPT** - Create custom GPTs with your data ### Developer Tools * **LangChain** - Python/JS framework for RAG * **LlamaIndex** - Data framework for LLM applications * **Pinecone** - Vector database * **Weaviate** - Open-source vector database *** ## Best Practices for RAG ### 1. Document Preparation ✅ **Do:** * Clean and format documents consistently * Remove irrelevant content * Add metadata (date, author, category) * Use clear headings and structure ❌ **Don't:** * Include duplicate content * Mix unrelated topics in one document * Use poor formatting (all caps, no structure) ### 2. Chunking Strategy **Chunk size matters:** * Too small → Loses context * Too large → Retrieves irrelevant information **Typical approach:** * 500-1000 tokens per chunk * Overlap chunks by 10-20% * Respect document structure (don't split mid-sentence) ### 3. Retrieval Quality **Improve retrieval with:** * Better embeddings (OpenAI, Cohere, open-source) * Hybrid search (keyword + semantic) * Metadata filtering (date, category, author) * Re-ranking retrieved results ### 4. Prompt Engineering **Good RAG prompt:** ``` Context: {retrieved_chunks} Question: {user_question} Instructions: - Answer based only on the context provided - If the answer isn't in the context, say so - Cite specific sources when possible - Be concise and accurate ``` *** ## Limitations and Challenges **Retrieval Accuracy** * May miss relevant information * May retrieve irrelevant chunks * Depends on query phrasing **Context Window Limits** * Can only include limited chunks * May need to prioritize what to include **Cost** * Embedding generation costs * Vector database storage * LLM API calls **Maintenance** * Need to update documents * Re-index when content changes * Monitor quality over time *** ## RAG vs Agents RAG and Agents often work together: **RAG alone:** * You ask a question * System retrieves and generates answer **RAG + Agents:** * Agent decides when to use RAG * Agent can query multiple knowledge bases * Agent can combine RAG with other tools (web search, calculations) Example: An agent might: 1. Search your documents (RAG) 2. Search the web for recent info 3. Combine both sources 4. Generate comprehensive answer *** ## Getting Started with RAG ### For Non-Technical Users Use ChatGPT Plus or Claude Pro with file uploads Experiment with Chatbase or CustomGPT Test with real questions from your use case ### For Technical Users LangChain + OpenAI + Pinecone (popular combo) Clean, chunk, and embed your data Implement search and ranking Connect to GPT-4, Claude, or open-source model Test, measure, and improve retrieval quality *** ## Curated Resources DataCamp's introduction to RAG Build your first RAG application Anthropic's guide to effective RAG Understanding vector databases *** ## Next Steps Learn how to build AI systems that use RAG and other tools autonomously