chore: init monorepo snapshot

2025-11-23 10:55:04 +08:00
commit c70ff52869
941 changed files with 246586 additions and 0 deletions
@@ -0,0 +1,487 @@
+# 📊 RAGAS-based Evaluation Framework
+
+## What is RAGAS?
+
+**RAGAS** (Retrieval Augmented Generation Assessment) is a framework for reference-free evaluation of RAG systems using LLMs. RAGAS uses state-of-the-art evaluation metrics:
+
+### Core Metrics
+
+| Metric | What It Measures | Good Score |
+|--------|-----------------|-----------|
+| **Faithfulness** | Is the answer factually accurate based on retrieved context? | > 0.80 |
+| **Answer Relevance** | Is the answer relevant to the user's question? | > 0.80 |
+| **Context Recall** | Was all relevant information retrieved from documents? | > 0.80 |
+| **Context Precision** | Is retrieved context clean without irrelevant noise? | > 0.80 |
+| **RAGAS Score** | Overall quality metric (average of above) | > 0.80 |
+
+### 📁 LightRAG Evalua'tion Framework Directory Structure
+
+```
+lightrag/evaluation/
+├── eval_rag_quality.py      # Main evaluation script
+├── sample_dataset.json        # 3 test questions about LightRAG
+├── sample_documents/          # Matching markdown files for testing
+│   ├── 01_lightrag_overview.md
+│   ├── 02_rag_architecture.md
+│   ├── 03_lightrag_improvements.md
+│   ├── 04_supported_databases.md
+│   ├── 05_evaluation_and_deployment.md
+│   └── README.md
+├── __init__.py              # Package init
+├── results/                 # Output directory
+│   ├── results_YYYYMMDD_HHMMSS.json    # Raw metrics in JSON
+│   └── results_YYYYMMDD_HHMMSS.csv     # Metrics in CSV format
+└── README.md                # This file
+```
+
+**Quick Test:** Index files from `sample_documents/` into LightRAG, then run the evaluator to reproduce results (~89-100% RAGAS score per question).
+
+
+
+## 🚀 Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pip install ragas datasets langfuse
+```
+
+Or use your project dependencies (already included in pyproject.toml):
+
+```bash
+pip install -e ".[evaluation]"
+```
+
+### 2. Run Evaluation
+
+**Basic usage (uses defaults):**
+```bash
+cd /path/to/LightRAG
+python lightrag/evaluation/eval_rag_quality.py
+```
+
+**Specify custom dataset:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --dataset my_test.json
+```
+
+**Specify custom RAG endpoint:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --ragendpoint http://my-server.com:9621
+```
+
+**Specify both (short form):**
+```bash
+python lightrag/evaluation/eval_rag_quality.py -d my_test.json -r http://localhost:9621
+```
+
+**Get help:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --help
+```
+
+### 3. View Results
+
+Results are saved automatically in `lightrag/evaluation/results/`:
+
+```
+results/
+├── results_20241023_143022.json     ← Raw metrics in JSON format
+└── results_20241023_143022.csv      ← Metrics in CSV format (for spreadsheets)
+```
+
+**Results include:**
+- ✅ Overall RAGAS score
+- 📊 Per-metric averages (Faithfulness, Answer Relevance, Context Recall, Context Precision)
+- 📋 Individual test case results
+- 📈 Performance breakdown by question
+
+
+
+## 📋 Command-Line Arguments
+
+The evaluation script supports command-line arguments for easy configuration:
+
+| Argument | Short | Default | Description |
+|----------|-------|---------|-------------|
+| `--dataset` | `-d` | `sample_dataset.json` | Path to test dataset JSON file |
+| `--ragendpoint` | `-r` | `http://localhost:9621` or `$LIGHTRAG_API_URL` | LightRAG API endpoint URL |
+
+### Usage Examples
+
+**Use default dataset and endpoint:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py
+```
+
+**Custom dataset with default endpoint:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --dataset path/to/my_dataset.json
+```
+
+**Default dataset with custom endpoint:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --ragendpoint http://my-server.com:9621
+```
+
+**Custom dataset and endpoint:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py -d my_dataset.json -r http://localhost:9621
+```
+
+**Absolute path to dataset:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py -d /path/to/custom_dataset.json
+```
+
+**Show help message:**
+```bash
+python lightrag/evaluation/eval_rag_quality.py --help
+```
+
+
+
+## ⚙️ Configuration
+
+### Environment Variables
+
+The evaluation framework supports customization through environment variables:
+
+**⚠️ IMPORTANT: Both LLM and Embedding endpoints MUST be OpenAI-compatible**
+- The RAGAS framework requires OpenAI-compatible API interfaces
+- Custom endpoints must implement the OpenAI API format (e.g., vLLM, SGLang, LocalAI)
+- Non-compatible endpoints will cause evaluation failures
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| **LLM Configuration** | | |
+| `EVAL_LLM_MODEL` | `gpt-4o-mini` | LLM model used for RAGAS evaluation |
+| `EVAL_LLM_BINDING_API_KEY` | falls back to `OPENAI_API_KEY` | API key for LLM evaluation |
+| `EVAL_LLM_BINDING_HOST` | (optional) | Custom OpenAI-compatible endpoint URL for LLM |
+| **Embedding Configuration** | | |
+| `EVAL_EMBEDDING_MODEL` | `text-embedding-3-large` | Embedding model for evaluation |
+| `EVAL_EMBEDDING_BINDING_API_KEY` | falls back to `EVAL_LLM_BINDING_API_KEY` → `OPENAI_API_KEY` | API key for embeddings |
+| `EVAL_EMBEDDING_BINDING_HOST` | falls back to `EVAL_LLM_BINDING_HOST` | Custom OpenAI-compatible endpoint URL for embeddings |
+| **Performance Tuning** | | |
+| `EVAL_MAX_CONCURRENT` | 2 | Number of concurrent test case evaluations (1=serial) |
+| `EVAL_QUERY_TOP_K` | 10 | Number of documents to retrieve per query |
+| `EVAL_LLM_MAX_RETRIES` | 5 | Maximum LLM request retries |
+| `EVAL_LLM_TIMEOUT` | 180 | LLM request timeout in seconds |
+
+### Usage Examples
+
+**Example 1: Default Configuration (OpenAI Official API)**
+```bash
+export OPENAI_API_KEY=sk-xxx
+python lightrag/evaluation/eval_rag_quality.py
+```
+Both LLM and embeddings use OpenAI's official API with default models.
+
+**Example 2: Custom Models on OpenAI**
+```bash
+export OPENAI_API_KEY=sk-xxx
+export EVAL_LLM_MODEL=gpt-4o-mini
+export EVAL_EMBEDDING_MODEL=text-embedding-3-large
+python lightrag/evaluation/eval_rag_quality.py
+```
+
+**Example 3: Same Custom OpenAI-Compatible Endpoint for Both**
+```bash
+# Both LLM and embeddings use the same custom endpoint
+export EVAL_LLM_BINDING_API_KEY=your-custom-key
+export EVAL_LLM_BINDING_HOST=http://localhost:8000/v1
+export EVAL_LLM_MODEL=qwen-plus
+export EVAL_EMBEDDING_MODEL=BAAI/bge-m3
+python lightrag/evaluation/eval_rag_quality.py
+```
+Embeddings automatically inherit LLM endpoint configuration.
+
+**Example 4: Separate Endpoints (Cost Optimization)**
+```bash
+# Use OpenAI for LLM (high quality)
+export EVAL_LLM_BINDING_API_KEY=sk-openai-key
+export EVAL_LLM_MODEL=gpt-4o-mini
+# No EVAL_LLM_BINDING_HOST means use OpenAI official API
+
+# Use local vLLM for embeddings (cost-effective)
+export EVAL_EMBEDDING_BINDING_API_KEY=local-key
+export EVAL_EMBEDDING_BINDING_HOST=http://localhost:8001/v1
+export EVAL_EMBEDDING_MODEL=BAAI/bge-m3
+
+python lightrag/evaluation/eval_rag_quality.py
+```
+LLM uses OpenAI official API, embeddings use local custom endpoint.
+
+**Example 5: Different Custom Endpoints for LLM and Embeddings**
+```bash
+# LLM on one OpenAI-compatible server
+export EVAL_LLM_BINDING_API_KEY=key1
+export EVAL_LLM_BINDING_HOST=http://llm-server:8000/v1
+export EVAL_LLM_MODEL=custom-llm
+
+# Embeddings on another OpenAI-compatible server
+export EVAL_EMBEDDING_BINDING_API_KEY=key2
+export EVAL_EMBEDDING_BINDING_HOST=http://embedding-server:8001/v1
+export EVAL_EMBEDDING_MODEL=custom-embedding
+
+python lightrag/evaluation/eval_rag_quality.py
+```
+Both use different custom OpenAI-compatible endpoints.
+
+**Example 6: Using Environment Variables from .env File**
+```bash
+# Create .env file in project root
+cat > .env << EOF
+EVAL_LLM_BINDING_API_KEY=your-key
+EVAL_LLM_BINDING_HOST=http://localhost:8000/v1
+EVAL_LLM_MODEL=qwen-plus
+EVAL_EMBEDDING_MODEL=BAAI/bge-m3
+EOF
+
+# Run evaluation (automatically loads .env)
+python lightrag/evaluation/eval_rag_quality.py
+```
+
+### Concurrency Control & Rate Limiting
+
+The evaluation framework includes built-in concurrency control to prevent API rate limiting issues:
+
+**Why Concurrency Control Matters:**
+- RAGAS internally makes many concurrent LLM calls for each test case
+- Context Precision metric calls LLM once per retrieved document
+- Without control, this can easily exceed API rate limits
+
+**Default Configuration (Conservative):**
+```bash
+EVAL_MAX_CONCURRENT=2    # Serial evaluation (one test at a time)
+EVAL_QUERY_TOP_K=10      # OP_K query parameter of LightRAG
+EVAL_LLM_MAX_RETRIES=5   # Retry failed requests 5 times
+EVAL_LLM_TIMEOUT=180     # 3-minute timeout per request
+```
+
+**Common Issues and Solutions:**
+
+| Issue | Solution |
+|-------|----------|
+| **Warning: "LM returned 1 generations instead of 3"** | Reduce `EVAL_MAX_CONCURRENT` to 1 or decrease `EVAL_QUERY_TOP_K` |
+| **Context Precision returns NaN** | Lower `EVAL_QUERY_TOP_K` to reduce LLM calls per test case |
+| **Rate limit errors (429)** | Increase `EVAL_LLM_MAX_RETRIES` and decrease `EVAL_MAX_CONCURRENT` |
+| **Request timeouts** | Increase `EVAL_LLM_TIMEOUT` to 180 or higher |
+
+
+
+## 📝 Test Dataset
+
+`sample_dataset.json` contains 3 generic questions about LightRAG. Replace with questions matching YOUR indexed documents.
+
+**Custom Test Cases:**
+
+```json
+{
+  "test_cases": [
+    {
+      "question": "Your question here",
+      "ground_truth": "Expected answer from your data",
+      "project": "evaluation_project_name"
+    }
+  ]
+}
+```
+
+---
+
+## 📊 Interpreting Results
+
+### Score Ranges
+
+- **0.80-1.00**: ✅ Excellent (Production-ready)
+- **0.60-0.80**: ⚠️ Good (Room for improvement)
+- **0.40-0.60**: ❌ Poor (Needs optimization)
+- **0.00-0.40**: 🔴 Critical (Major issues)
+
+### What Low Scores Mean
+
+| Metric | Low Score Indicates |
+|--------|-------------------|
+| **Faithfulness** | Responses contain hallucinations or incorrect information |
+| **Answer Relevance** | Answers don't match what users asked |
+| **Context Recall** | Missing important information in retrieval |
+| **Context Precision** | Retrieved documents contain irrelevant noise |
+
+### Optimization Tips
+
+1. **Low Faithfulness**:
+   - Improve entity extraction quality
+   - Better document chunking
+   - Tune retrieval temperature
+
+2. **Low Answer Relevance**:
+   - Improve prompt engineering
+   - Better query understanding
+   - Check semantic similarity threshold
+
+3. **Low Context Recall**:
+   - Increase retrieval `top_k` results
+   - Improve embedding model
+   - Better document preprocessing
+
+4. **Low Context Precision**:
+   - Smaller, focused chunks
+   - Better filtering
+   - Improve chunking strategy
+
+---
+
+## 📚 Resources
+
+- [RAGAS Documentation](https://docs.ragas.io/)
+- [RAGAS GitHub](https://github.com/explodinggradients/ragas)
+
+---
+
+## 🐛 Troubleshooting
+
+### "ModuleNotFoundError: No module named 'ragas'"
+
+```bash
+pip install ragas datasets
+```
+
+### "Warning: LM returned 1 generations instead of requested 3" or Context Precision NaN
+
+**Cause**: This warning indicates API rate limiting or concurrent request overload:
+- RAGAS makes multiple LLM calls per test case (faithfulness, relevancy, recall, precision)
+- Context Precision calls LLM once per retrieved document (with `EVAL_QUERY_TOP_K=10`, that's 10 calls)
+- Concurrent evaluation multiplies these calls: `EVAL_MAX_CONCURRENT × LLM calls per test`
+
+**Solutions** (in order of effectiveness):
+
+1. **Serial Evaluation** (Default):
+   ```bash
+   export EVAL_MAX_CONCURRENT=1
+   python lightrag/evaluation/eval_rag_quality.py
+   ```
+
+2. **Reduce Retrieved Documents**:
+   ```bash
+   export EVAL_QUERY_TOP_K=5  # Halves Context Precision LLM calls
+   python lightrag/evaluation/eval_rag_quality.py
+   ```
+
+3. **Increase Retry & Timeout**:
+   ```bash
+   export EVAL_LLM_MAX_RETRIES=10
+   export EVAL_LLM_TIMEOUT=180
+   python lightrag/evaluation/eval_rag_quality.py
+   ```
+
+4. **Use Higher Quota API** (if available):
+   - Upgrade to OpenAI Tier 2+ for higher RPM limits
+   - Use self-hosted OpenAI-compatible service with no rate limits
+
+### "AttributeError: 'InstructorLLM' object has no attribute 'agenerate_prompt'" or NaN results
+
+This error occurs with RAGAS 0.3.x when LLM and Embeddings are not explicitly configured. The evaluation framework now handles this automatically by:
+- Using environment variables to configure evaluation models
+- Creating proper LLM and Embeddings instances for RAGAS
+
+**Solution**: Ensure you have set one of the following:
+- `OPENAI_API_KEY` environment variable (default)
+- `EVAL_LLM_BINDING_API_KEY` for custom API key
+
+The framework will automatically configure the evaluation models.
+
+### "No sample_dataset.json found"
+
+Make sure you're running from the project root:
+
+```bash
+cd /path/to/LightRAG
+python lightrag/evaluation/eval_rag_quality.py
+```
+
+### "LightRAG query API errors during evaluation"
+
+The evaluation uses your configured LLM (OpenAI by default). Ensure:
+- API keys are set in `.env`
+- Network connection is stable
+
+### Evaluation requires running LightRAG API
+
+The evaluator queries a running LightRAG API server at `http://localhost:9621`. Make sure:
+1. LightRAG API server is running (`python lightrag/api/lightrag_server.py`)
+2. Documents are indexed in your LightRAG instance
+3. API is accessible at the configured URL
+
+
+
+## 📝 Next Steps
+
+1. Start LightRAG API server
+2. Upload sample documents into LightRAG  throught  WebUI
+3. Run `python lightrag/evaluation/eval_rag_quality.py`
+4. Review results (JSON/CSV) in `results/` folder
+
+Evaluation Result Sample:
+
+```
+INFO: ======================================================================
+INFO: 🔍 RAGAS Evaluation - Using Real LightRAG API
+INFO: ======================================================================
+INFO: Evaluation Models:
+INFO:   • LLM Model:            gpt-4.1
+INFO:   • Embedding Model:      text-embedding-3-large
+INFO:   • Endpoint:             OpenAI Official API
+INFO: Concurrency & Rate Limiting:
+INFO:   • Query Top-K:          10 Entities/Relations
+INFO:   • LLM Max Retries:      5
+INFO:   • LLM Timeout:          180 seconds
+INFO: Test Configuration:
+INFO:   • Total Test Cases:     6
+INFO:   • Test Dataset:         sample_dataset.json
+INFO:   • LightRAG API:         http://localhost:9621
+INFO:   • Results Directory:    results
+INFO: ======================================================================
+INFO: 🚀 Starting RAGAS Evaluation of LightRAG System
+INFO: 🔧 RAGAS Evaluation (Stage 2): 2 concurrent
+INFO: ======================================================================
+INFO:
+INFO: ===================================================================================================================
+INFO: 📊 EVALUATION RESULTS SUMMARY
+INFO: ===================================================================================================================
+INFO: #    | Question                                           |  Faith | AnswRel | CtxRec | CtxPrec |  RAGAS | Status
+INFO: -------------------------------------------------------------------------------------------------------------------
+INFO: 1    | How does LightRAG solve the hallucination probl... | 1.0000 |  1.0000 | 1.0000 |  1.0000 | 1.0000 |      ✓
+INFO: 2    | What are the three main components required in ... | 0.8500 |  0.5790 | 1.0000 |  1.0000 | 0.8573 |      ✓
+INFO: 3    | How does LightRAG's retrieval performance compa... | 0.8056 |  1.0000 | 1.0000 |  1.0000 | 0.9514 |      ✓
+INFO: 4    | What vector databases does LightRAG support and... | 0.8182 |  0.9807 | 1.0000 |  1.0000 | 0.9497 |      ✓
+INFO: 5    | What are the four key metrics for evaluating RA... | 1.0000 |  0.7452 | 1.0000 |  1.0000 | 0.9363 |      ✓
+INFO: 6    | What are the core benefits of LightRAG and how ... | 0.9583 |  0.8829 | 1.0000 |  1.0000 | 0.9603 |      ✓
+INFO: ===================================================================================================================
+INFO:
+INFO: ======================================================================
+INFO: 📊 EVALUATION COMPLETE
+INFO: ======================================================================
+INFO: Total Tests:    6
+INFO: Successful:     6
+INFO: Failed:         0
+INFO: Success Rate:   100.00%
+INFO: Elapsed Time:   161.10 seconds
+INFO: Avg Time/Test:  26.85 seconds
+INFO:
+INFO: ======================================================================
+INFO: 📈 BENCHMARK RESULTS (Average)
+INFO: ======================================================================
+INFO: Average Faithfulness:      0.9053
+INFO: Average Answer Relevance:  0.8646
+INFO: Average Context Recall:    1.0000
+INFO: Average Context Precision: 1.0000
+INFO: Average RAGAS Score:       0.9425
+INFO: ----------------------------------------------------------------------
+INFO: Min RAGAS Score:           0.8573
+INFO: Max RAGAS Score:           1.0000
+```
+
+---
+
+**Happy Evaluating! 🚀**
@@ -0,0 +1,25 @@
+"""
+LightRAG Evaluation Module
+
+RAGAS-based evaluation framework for assessing RAG system quality.
+
+Usage:
+    from lightrag.evaluation import RAGEvaluator
+
+    evaluator = RAGEvaluator()
+    results = await evaluator.run()
+
+Note: RAGEvaluator is imported lazily to avoid import errors
+when ragas/datasets are not installed.
+"""
+
+__all__ = ["RAGEvaluator"]
+
+
+def __getattr__(name):
+    """Lazy import to avoid dependency errors when ragas is not installed."""
+    if name == "RAGEvaluator":
+        from .eval_rag_quality import RAGEvaluator
+
+        return RAGEvaluator
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -0,0 +1,34 @@
+{
+  "test_cases": [
+    {
+      "question": "How does LightRAG solve the hallucination problem in large language models?",
+      "ground_truth": "LightRAG solves the hallucination problem by combining large language models with external knowledge retrieval. The framework ensures accurate responses by grounding LLM outputs in actual documents. LightRAG provides contextual responses that reduce hallucinations significantly.",
+      "project": "lightrag_evaluation_sample"
+    },
+    {
+      "question": "What are the three main components required in a RAG system?",
+      "ground_truth": "A RAG system requires three main components: a retrieval system (vector database or search engine) to find relevant documents, an embedding model to convert text into vector representations for similarity search, and a large language model (LLM) to generate responses based on retrieved context.",
+      "project": "lightrag_evaluation_sample"
+    },
+    {
+      "question": "How does LightRAG's retrieval performance compare to traditional RAG approaches?",
+      "ground_truth": "LightRAG delivers faster retrieval performance than traditional RAG approaches. The framework optimizes document retrieval operations for speed. Traditional RAG systems often suffer from slow query response times. LightRAG achieves high quality results with improved performance. The framework combines speed with accuracy in retrieval operations, prioritizing ease of use without sacrificing quality.",
+      "project": "lightrag_evaluation_sample"
+    },
+    {
+      "question": "What vector databases does LightRAG support and what are their key characteristics?",
+      "ground_truth": "LightRAG supports multiple vector databases including ChromaDB for simple deployment and efficient similarity search, Neo4j for graph-based knowledge representation with vector capabilities, Milvus for high-performance vector search at scale, Qdrant for fast similarity search with filtering and production-ready infrastructure, MongoDB Atlas for combined document storage and vector search, Redis for in-memory low-latency vector search, and a built-in nano-vectordb that eliminates external dependencies for small projects. This multi-database support enables developers to choose appropriate backends based on scale, performance, and infrastructure requirements.",
+      "project": "lightrag_evaluation_sample"
+    },
+    {
+      "question": "What are the four key metrics for evaluating RAG system quality and what does each metric measure?",
+      "ground_truth": "RAG system quality is measured through four key metrics: Faithfulness measures whether answers are factually grounded in retrieved context and detects hallucinations. Answer Relevance measures how well answers address the user question and evaluates response appropriateness. Context Recall measures completeness of retrieval and whether all relevant information was retrieved from documents. Context Precision measures quality and relevance of retrieved documents without noise or irrelevant content.",
+      "project": "lightrag_evaluation_sample"
+    },
+    {
+      "question": "What are the core benefits of LightRAG and how does it improve upon traditional RAG systems?",
+      "ground_truth": "LightRAG offers five core benefits: accuracy through document-grounded responses, up-to-date information without model retraining, domain expertise through specialized document collections, cost-effectiveness by avoiding expensive fine-tuning, and transparency by showing source documents. Compared to traditional RAG systems, LightRAG provides a simpler API with intuitive interfaces, faster retrieval performance with optimized operations, better integration with multiple vector database backends for flexible selection, and optimized prompting strategies with refined templates. LightRAG prioritizes ease of use while maintaining quality and combines speed with accuracy.",
+      "project": "lightrag_evaluation_sample"
+    }
+  ]
+}
@@ -0,0 +1,17 @@
+# LightRAG Framework Overview
+
+## What is LightRAG?
+
+**LightRAG** is a Simple and Fast Retrieval-Augmented Generation framework. LightRAG was developed by HKUDS (Hong Kong University Data Science Lab). The framework provides developers with tools to build RAG applications efficiently.
+
+## Problem Statement
+
+Large language models face several limitations. LLMs have a knowledge cutoff date that prevents them from accessing recent information. Large language models generate hallucinations when providing responses without factual grounding. LLMs lack domain-specific expertise in specialized fields.
+
+## How LightRAG Solves These Problems
+
+LightRAG solves the hallucination problem by combining large language models with external knowledge retrieval. The framework ensures accurate responses by grounding LLM outputs in actual documents. LightRAG provides contextual responses that reduce hallucinations significantly. The system enables efficient retrieval from external knowledge bases to supplement LLM capabilities.
+
+## Core Benefits
+
+LightRAG offers accuracy through document-grounded responses. The framework provides up-to-date information without model retraining. LightRAG enables domain expertise through specialized document collections. The system delivers cost-effectiveness by avoiding expensive model fine-tuning. LightRAG ensures transparency by showing source documents for each response.
@@ -0,0 +1,21 @@
+# RAG System Architecture
+
+## Main Components of RAG Systems
+
+A RAG system consists of three main components that work together to provide intelligent responses.
+
+### Component 1: Retrieval System
+
+The retrieval system is the first component of a RAG system. A retrieval system finds relevant documents from large document collections. Vector databases serve as the primary storage for the retrieval system. Search engines can also function as retrieval systems in RAG architectures.
+
+### Component 2: Embedding Model
+
+The embedding model is the second component of a RAG system. An embedding model converts text into vector representations for similarity search. The embedding model transforms documents and queries into numerical vectors. These vector representations enable semantic similarity matching between queries and documents.
+
+### Component 3: Large Language Model
+
+The large language model is the third component of a RAG system. An LLM generates responses based on retrieved context from documents. The large language model synthesizes information from multiple sources into coherent answers. LLMs provide natural language generation capabilities for the RAG system.
+
+## How Components Work Together
+
+The retrieval system fetches relevant documents for a user query. The embedding model enables similarity matching between query and documents. The LLM generates the final response using retrieved context. These three components collaborate to provide accurate, contextual responses.
@@ -0,0 +1,25 @@
+# LightRAG Improvements Over Traditional RAG
+
+## Key Improvements
+
+LightRAG improves upon traditional RAG approaches in several significant ways.
+
+### Simpler API Design
+
+LightRAG offers a simpler API compared to traditional RAG frameworks. The framework provides intuitive interfaces for developers. Traditional RAG systems often require complex configuration and setup. LightRAG focuses on ease of use while maintaining functionality.
+
+### Faster Retrieval Performance
+
+LightRAG delivers faster retrieval performance than traditional RAG approaches. The framework optimizes document retrieval operations for speed. Traditional RAG systems often suffer from slow query response times. LightRAG achieves high quality results with improved performance.
+
+### Better Vector Database Integration
+
+LightRAG provides better integration with various vector databases. The framework supports multiple vector database backends seamlessly. Traditional RAG approaches typically lock developers into specific database choices. LightRAG enables flexible storage backend selection.
+
+### Optimized Prompting Strategies
+
+LightRAG implements optimized prompting strategies for better results. The framework uses refined prompt templates for accurate responses. Traditional RAG systems often use generic prompting approaches. LightRAG balances simplicity with high quality output.
+
+## Design Philosophy
+
+LightRAG prioritizes ease of use without sacrificing quality. The framework combines speed with accuracy in retrieval operations. LightRAG maintains flexibility in database and model selection.
@@ -0,0 +1,37 @@
+# LightRAG Vector Database Support
+
+## Supported Vector Databases
+
+LightRAG supports multiple vector databases for flexible deployment options.
+
+### ChromaDB
+
+ChromaDB is a vector database supported by LightRAG. ChromaDB provides simple deployment for development environments. The database offers efficient vector similarity search capabilities.
+
+### Neo4j
+
+Neo4j is a graph database supported by LightRAG. Neo4j enables graph-based knowledge representation alongside vector search. The database combines relationship modeling with vector capabilities.
+
+### Milvus
+
+Milvus is a vector database supported by LightRAG. Milvus provides high-performance vector search at scale. The database handles large-scale vector collections efficiently.
+
+### Qdrant
+
+Qdrant is a vector database supported by LightRAG. Qdrant offers fast similarity search with filtering capabilities. The database provides production-ready vector search infrastructure.
+
+### MongoDB Atlas Vector Search
+
+MongoDB Atlas Vector Search is supported by LightRAG. MongoDB Atlas combines document storage with vector search capabilities. The database enables unified data management for RAG applications.
+
+### Redis
+
+Redis is supported by LightRAG for vector search operations. Redis provides in-memory vector search with low latency. The database offers fast retrieval for real-time applications.
+
+### Built-in Nano-VectorDB
+
+LightRAG includes a built-in nano-vectordb for simple deployments. Nano-vectordb eliminates external database dependencies for small projects. The built-in database provides basic vector search functionality without additional setup.
+
+## Database Selection Benefits
+
+The multiple database support enables developers to choose appropriate storage backends. LightRAG adapts to different deployment scenarios from development to production. Users can select databases based on scale, performance, and infrastructure requirements.
@@ -0,0 +1,41 @@
+# RAG Evaluation Metrics and Deployment
+
+## Key RAG Evaluation Metrics
+
+RAG system quality is measured through four key metrics.
+
+### Faithfulness Metric
+
+Faithfulness measures whether answers are factually grounded in retrieved context. The faithfulness metric detects hallucinations in LLM responses. High faithfulness scores indicate answers based on actual document content. The metric evaluates factual accuracy of generated responses.
+
+### Answer Relevance Metric
+
+Answer Relevance measures how well answers address the user question. The answer relevance metric evaluates response quality and appropriateness. High answer relevance scores show responses that directly answer user queries. The metric assesses the connection between questions and generated answers.
+
+### Context Recall Metric
+
+Context Recall measures completeness of retrieval from documents. The context recall metric evaluates whether all relevant information was retrieved. High context recall scores indicate comprehensive document retrieval. The metric assesses retrieval system effectiveness.
+
+### Context Precision Metric
+
+Context Precision measures quality and relevance of retrieved documents. The context precision metric evaluates retrieval accuracy without noise. High context precision scores show clean retrieval without irrelevant content. The metric measures retrieval system selectivity.
+
+## LightRAG Deployment Options
+
+LightRAG can be deployed in production through multiple approaches.
+
+### Docker Container Deployment
+
+Docker containers enable consistent LightRAG deployment across environments. Docker provides isolated runtime environments for the framework. Container deployment simplifies dependency management and scaling.
+
+### REST API Server with FastAPI
+
+FastAPI serves as the REST API framework for LightRAG deployment. The FastAPI server exposes LightRAG functionality through HTTP endpoints. REST API deployment enables client-server architecture for RAG applications.
+
+### Direct Python Integration
+
+Direct Python integration embeds LightRAG into Python applications. Python integration provides programmatic access to RAG capabilities. Direct integration supports custom application workflows and pipelines.
+
+### Deployment Features
+
+LightRAG supports environment-based configuration for different deployment scenarios. The framework integrates with multiple LLM providers for flexibility. LightRAG enables horizontal scaling for production workloads.
@@ -0,0 +1,21 @@
+# Sample Documents for Evaluation
+
+These markdown files correspond to test questions in `../sample_dataset.json`.
+
+## Usage
+
+1. **Index documents** into LightRAG (via WebUI, API, or Python)
+2. **Run evaluation**: `python lightrag/evaluation/eval_rag_quality.py`
+3. **Expected results**: ~91-100% RAGAS score per question
+
+## Files
+
+- `01_lightrag_overview.md` - LightRAG framework and hallucination problem
+- `02_rag_architecture.md` - RAG system components
+- `03_lightrag_improvements.md` - LightRAG vs traditional RAG
+- `04_supported_databases.md` - Vector database support
+- `05_evaluation_and_deployment.md` - Metrics and deployment
+
+## Note
+
+Documents use clear entity-relationship patterns for LightRAG's default entity extraction prompts. For better results with your data, customize `lightrag/prompt.py`.