chore: init monorepo snapshot
This commit is contained in:
liaibo
@@ -0,0 +1,4 @@
|
||||
|
||||
*Figure 1: LightRAG Indexing Flowchart - Img Caption : [Source](https://learnopencv.com/lightrag/)*
|
||||
|
||||
*Figure 2: LightRAG Retrieval and Querying Flowchart - Img Caption : [Source](https://learnopencv.com/lightrag/)*
|
||||
@@ -0,0 +1,135 @@
|
||||
# LightRAG Docker Deployment
|
||||
|
||||
A lightweight Knowledge Graph Retrieval-Augmented Generation system with multiple LLM backend support.
|
||||
|
||||
## 🚀 Preparation
|
||||
|
||||
### Clone the repository:
|
||||
|
||||
```bash
|
||||
# Linux/MacOS
|
||||
git clone https://github.com/HKUDS/LightRAG.git
|
||||
cd LightRAG
|
||||
```
|
||||
```powershell
|
||||
# Windows PowerShell
|
||||
git clone https://github.com/HKUDS/LightRAG.git
|
||||
cd LightRAG
|
||||
```
|
||||
|
||||
### Configure your environment:
|
||||
|
||||
```bash
|
||||
# Linux/MacOS
|
||||
cp .env.example .env
|
||||
# Edit .env with your preferred configuration
|
||||
```
|
||||
```powershell
|
||||
# Windows PowerShell
|
||||
Copy-Item .env.example .env
|
||||
# Edit .env with your preferred configuration
|
||||
```
|
||||
|
||||
LightRAG can be configured using environment variables in the `.env` file:
|
||||
|
||||
**Server Configuration**
|
||||
|
||||
- `HOST`: Server host (default: 0.0.0.0)
|
||||
- `PORT`: Server port (default: 9621)
|
||||
|
||||
**LLM Configuration**
|
||||
|
||||
- `LLM_BINDING`: LLM backend to use (lollms/ollama/openai)
|
||||
- `LLM_BINDING_HOST`: LLM server host URL
|
||||
- `LLM_MODEL`: Model name to use
|
||||
|
||||
**Embedding Configuration**
|
||||
|
||||
- `EMBEDDING_BINDING`: Embedding backend (lollms/ollama/openai)
|
||||
- `EMBEDDING_BINDING_HOST`: Embedding server host URL
|
||||
- `EMBEDDING_MODEL`: Embedding model name
|
||||
|
||||
**RAG Configuration**
|
||||
|
||||
- `MAX_ASYNC`: Maximum async operations
|
||||
- `MAX_TOKENS`: Maximum token size
|
||||
- `EMBEDDING_DIM`: Embedding dimensions
|
||||
|
||||
## 🐳 Docker Deployment
|
||||
|
||||
Docker instructions work the same on all platforms with Docker Desktop installed.
|
||||
|
||||
### Build Optimization
|
||||
|
||||
The Dockerfile uses BuildKit cache mounts to significantly improve build performance:
|
||||
|
||||
- **Automatic cache management**: BuildKit is automatically enabled via `# syntax=docker/dockerfile:1` directive
|
||||
- **Faster rebuilds**: Only downloads changed dependencies when `uv.lock` or `bun.lock` files are modified
|
||||
- **Efficient package caching**: UV and Bun package downloads are cached across builds
|
||||
- **No manual configuration needed**: Works out of the box in Docker Compose and GitHub Actions
|
||||
|
||||
### Start LightRAG server:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
LightRAG Server uses the following paths for data storage:
|
||||
|
||||
```
|
||||
data/
|
||||
├── rag_storage/ # RAG data persistence
|
||||
└── inputs/ # Input documents
|
||||
```
|
||||
|
||||
### Updates
|
||||
|
||||
To update the Docker container:
|
||||
```bash
|
||||
docker compose pull
|
||||
docker compose down
|
||||
docker compose up
|
||||
```
|
||||
|
||||
### Offline deployment
|
||||
|
||||
Software packages requiring `transformers`, `torch`, or `cuda` will is not preinstalled in the dokcer images. Consequently, document extraction tools such as Docling, as well as local LLM models like Hugging Face and LMDeploy, can not be used in an off line enviroment. These high-compute-resource-demanding services should not be integrated into LightRAG. Docling will be decoupled and deployed as a standalone service.
|
||||
|
||||
## 📦 Build Docker Images
|
||||
|
||||
### For local development and testing
|
||||
|
||||
```bash
|
||||
# Build and run with Docker Compose (BuildKit automatically enabled)
|
||||
docker compose up --build
|
||||
|
||||
# Or explicitly enable BuildKit if needed
|
||||
DOCKER_BUILDKIT=1 docker compose up --build
|
||||
```
|
||||
|
||||
**Note**: BuildKit is automatically enabled by the `# syntax=docker/dockerfile:1` directive in the Dockerfile, ensuring optimal caching performance.
|
||||
|
||||
### For production release
|
||||
|
||||
**multi-architecture build and push**:
|
||||
|
||||
```bash
|
||||
# Use the provided build script
|
||||
./docker-build-push.sh
|
||||
```
|
||||
|
||||
**The build script will**:
|
||||
|
||||
- Check Docker registry login status
|
||||
- Create/use buildx builder automatically
|
||||
- Build for both AMD64 and ARM64 architectures
|
||||
- Push to GitHub Container Registry (ghcr.io)
|
||||
- Verify the multi-architecture manifest
|
||||
|
||||
**Prerequisites**:
|
||||
|
||||
Before building multi-architecture images, ensure you have:
|
||||
|
||||
- Docker 20.10+ with Buildx support
|
||||
- Sufficient disk space (20GB+ recommended for offline image)
|
||||
- Registry access credentials (if pushing images)
|
||||
@@ -0,0 +1,207 @@
|
||||
# Frontend Build Guide
|
||||
|
||||
## Overview
|
||||
|
||||
The LightRAG project includes a React-based WebUI frontend. This guide explains how frontend building works in different scenarios.
|
||||
|
||||
## Key Principle
|
||||
|
||||
- **Git Repository**: Frontend build results are **NOT** included (kept clean)
|
||||
- **PyPI Package**: Frontend build results **ARE** included (ready to use)
|
||||
- **Build Tool**: Uses **Bun** (not npm/yarn)
|
||||
|
||||
## Installation Scenarios
|
||||
|
||||
### 1. End Users (From PyPI) ✨
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
pip install lightrag-hku[api]
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
- Frontend is already built and included in the package
|
||||
- No additional steps needed
|
||||
- Web interface works immediately
|
||||
|
||||
---
|
||||
|
||||
### 2. Development Mode (Recommended for Contributors) 🔧
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/HKUDS/LightRAG.git
|
||||
cd LightRAG
|
||||
|
||||
# Install in editable mode (no frontend build required yet)
|
||||
pip install -e ".[api]"
|
||||
|
||||
# Build frontend when needed (can be done anytime)
|
||||
cd lightrag_webui
|
||||
bun install --frozen-lockfile
|
||||
bun run build
|
||||
cd ..
|
||||
```
|
||||
|
||||
**Advantages:**
|
||||
- Install first, build later (flexible workflow)
|
||||
- Changes take effect immediately (symlink mode)
|
||||
- Frontend can be rebuilt anytime without reinstalling
|
||||
|
||||
**How it works:**
|
||||
- Creates symlinks to source directory
|
||||
- Frontend build output goes to `lightrag/api/webui/`
|
||||
- Changes are immediately visible in installed package
|
||||
|
||||
---
|
||||
|
||||
### 3. Normal Installation (Testing Package Build) 📦
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/HKUDS/LightRAG.git
|
||||
cd LightRAG
|
||||
|
||||
# ⚠️ MUST build frontend FIRST
|
||||
cd lightrag_webui
|
||||
bun install --frozen-lockfile
|
||||
bun run build
|
||||
cd ..
|
||||
|
||||
# Now install
|
||||
pip install ".[api]"
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
- Frontend files are **copied** to site-packages
|
||||
- Post-build modifications won't affect installed package
|
||||
- Requires rebuild + reinstall to update
|
||||
|
||||
**When to use:**
|
||||
- Testing complete installation process
|
||||
- Verifying package configuration
|
||||
- Simulating PyPI user experience
|
||||
|
||||
---
|
||||
|
||||
### 4. Creating Distribution Package 🚀
|
||||
|
||||
**Command:**
|
||||
```bash
|
||||
# Build frontend first
|
||||
cd lightrag_webui
|
||||
bun install --frozen-lockfile --production
|
||||
bun run build
|
||||
cd ..
|
||||
|
||||
# Create distribution packages
|
||||
python -m build
|
||||
|
||||
# Output: dist/lightrag_hku-*.whl and dist/lightrag_hku-*.tar.gz
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
- `setup.py` checks if frontend is built
|
||||
- If missing, installation fails with helpful error message
|
||||
- Generated package includes all frontend files
|
||||
|
||||
---
|
||||
|
||||
## GitHub Actions (Automated Release)
|
||||
|
||||
When creating a release on GitHub:
|
||||
|
||||
1. **Automatically builds frontend** using Bun
|
||||
2. **Verifies** build completed successfully
|
||||
3. **Creates Python package** with frontend included
|
||||
4. **Publishes to PyPI** using existing trusted publisher setup
|
||||
|
||||
**No manual intervention required!**
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Scenario | Command | Frontend Required | Can Build After |
|
||||
|----------|---------|-------------------|-----------------|
|
||||
| From PyPI | `pip install lightrag-hku[api]` | Included | No (already installed) |
|
||||
| Development | `pip install -e ".[api]"` | No | ✅ Yes (anytime) |
|
||||
| Normal Install | `pip install ".[api]"` | ✅ Yes (before) | No (must reinstall) |
|
||||
| Create Package | `python -m build` | ✅ Yes (before) | N/A |
|
||||
|
||||
---
|
||||
|
||||
## Bun Installation
|
||||
|
||||
If you don't have Bun installed:
|
||||
|
||||
```bash
|
||||
# macOS/Linux
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
# Windows
|
||||
powershell -c "irm bun.sh/install.ps1 | iex"
|
||||
```
|
||||
|
||||
Official documentation: https://bun.sh
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
LightRAG/
|
||||
├── lightrag_webui/ # Frontend source code
|
||||
│ ├── src/ # React components
|
||||
│ ├── package.json # Dependencies
|
||||
│ └── vite.config.ts # Build configuration
|
||||
│ └── outDir: ../lightrag/api/webui # Build output
|
||||
│
|
||||
├── lightrag/
|
||||
│ └── api/
|
||||
│ └── webui/ # Frontend build output (gitignored)
|
||||
│ ├── index.html # Built files (after running bun run build)
|
||||
│ └── assets/ # Built assets
|
||||
│
|
||||
├── setup.py # Build checks
|
||||
├── pyproject.toml # Package configuration
|
||||
└── .gitignore # Excludes lightrag/api/webui/* (except .gitkeep)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Q: I installed in development mode but the web interface doesn't work
|
||||
|
||||
**A:** Build the frontend:
|
||||
```bash
|
||||
cd lightrag_webui && bun run build
|
||||
```
|
||||
|
||||
### Q: I built the frontend but it's not in my installed package
|
||||
|
||||
**A:** You probably used `pip install .` after building. Either:
|
||||
- Use `pip install -e ".[api]"` for development
|
||||
- Or reinstall: `pip uninstall lightrag-hku && pip install ".[api]"`
|
||||
|
||||
### Q: Where are the built frontend files?
|
||||
|
||||
**A:** In `lightrag/api/webui/` after running `bun run build`
|
||||
|
||||
### Q: Can I use npm or yarn instead of Bun?
|
||||
|
||||
**A:** The project is configured for Bun. While npm/yarn might work, Bun is recommended per project standards.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
✅ **PyPI users**: No action needed, frontend included
|
||||
✅ **Developers**: Use `pip install -e ".[api]"`, build frontend when needed
|
||||
✅ **CI/CD**: Automatic build in GitHub Actions
|
||||
✅ **Git**: Frontend build output never committed
|
||||
|
||||
For questions or issues, please open a GitHub issue.
|
||||
@@ -0,0 +1,114 @@
|
||||
## LightRAG Multi-Document Processing: Concurrent Control Strategy
|
||||
|
||||
LightRAG employs a multi-layered concurrent control strategy when processing multiple documents. This article provides an in-depth analysis of the concurrent control mechanisms at document level, chunk level, and LLM request level, helping you understand why specific concurrent behaviors occur.
|
||||
|
||||
### 1. Document-Level Concurrent Control
|
||||
|
||||
**Control Parameter**: `max_parallel_insert`
|
||||
|
||||
This parameter controls the number of documents processed simultaneously. The purpose is to prevent excessive parallelism from overwhelming system resources, which could lead to extended processing times for individual files. Document-level concurrency is governed by the `max_parallel_insert` attribute within LightRAG, which defaults to 2 and is configurable via the `MAX_PARALLEL_INSERT` environment variable. `max_parallel_insert` is recommended to be set between 2 and 10, typically `llm_model_max_async/3`. Setting this value too high can increase the likelihood of naming conflicts among entities and relationships across different documents during the merge phase, thereby reducing its overall efficiency.
|
||||
|
||||
### 2. Chunk-Level Concurrent Control
|
||||
|
||||
**Control Parameter**: `llm_model_max_async`
|
||||
|
||||
This parameter controls the number of chunks processed simultaneously in the extraction stage within a document. The purpose is to prevent a high volume of concurrent requests from monopolizing LLM processing resources, which would impede the efficient parallel processing of multiple files. Chunk-Level Concurrent Control is governed by the `llm_model_max_async` attribute within LightRAG, which defaults to 4 and is configurable via the `MAX_ASYNC` environment variable. The purpose of this parameter is to fully leverage the LLM's concurrency capabilities when processing individual documents.
|
||||
|
||||
In the `extract_entities` function, **each document independently creates** its own chunk semaphore. Since each document independently creates chunk semaphores, the theoretical chunk concurrency of the system is:
|
||||
$$
|
||||
ChunkConcurrency = Max Parallel Insert × LLM Model Max Async
|
||||
$$
|
||||
For example:
|
||||
- `max_parallel_insert = 2` (process 2 documents simultaneously)
|
||||
- `llm_model_max_async = 4` (maximum 4 chunk concurrency per document)
|
||||
- Theoretical chunk-level concurrent: 2 × 4 = 8
|
||||
|
||||
### 3. Graph-Level Concurrent Control
|
||||
|
||||
**Control Parameter**: `llm_model_max_async * 2`
|
||||
|
||||
This parameter controls the number of entities and relations processed simultaneously in the merging stage within a document. The purpose is to prevent a high volume of concurrent requests from monopolizing LLM processing resources, which would impede the efficient parallel processing of multiple files. Graph-level concurrency is governed by the `llm_model_max_async` attribute within LightRAG, which defaults to 4 and is configurable via the `MAX_ASYNC` environment variable. Graph-level parallelism control parameters are equally applicable to managing parallelism during the entity relationship reconstruction phase after document deletion.
|
||||
|
||||
Given that the entity relationship merging phase doesn't necessitate LLM interaction for every operation, its parallelism is set at double the LLM's parallelism. This optimizes machine utilization while concurrently preventing excessive queuing resource contention for the LLM.
|
||||
|
||||
### 4. LLM-Level Concurrent Control
|
||||
|
||||
**Control Parameter**: `llm_model_max_async`
|
||||
|
||||
This parameter governs the **concurrent volume** of LLM requests dispatched by the entire LightRAG system, encompassing the document extraction stage, merging stage, and user query handling.
|
||||
|
||||
LLM request prioritization is managed via a global priority queue, which **systematically prioritizes user queries** over merging-related requests, and merging-related requests over extraction-related requests. This strategic prioritization **minimizes user query latency**.
|
||||
|
||||
LLM-level concurrency is governed by the `llm_model_max_async` attribute within LightRAG, which defaults to 4 and is configurable via the `MAX_ASYNC` environment variable.
|
||||
|
||||
### 5. Complete Concurrent Hierarchy Diagram
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
classDef doc fill:#e6f3ff,stroke:#5b9bd5,stroke-width:2px;
|
||||
classDef chunk fill:#fbe5d6,stroke:#ed7d31,stroke-width:1px;
|
||||
classDef merge fill:#e2f0d9,stroke:#70ad47,stroke-width:2px;
|
||||
|
||||
A["Multiple Documents<br>max_parallel_insert = 2"] --> A1
|
||||
A --> B1
|
||||
|
||||
A1[DocA: split to n chunks] --> A_chunk;
|
||||
B1[DocB: split to m chunks] --> B_chunk;
|
||||
|
||||
subgraph A_chunk[Extraction Stage]
|
||||
A_chunk_title[Entity Relation Extraction<br>llm_model_max_async = 4];
|
||||
A_chunk_title --> A_chunk1[Chunk A1]:::chunk;
|
||||
A_chunk_title --> A_chunk2[Chunk A2]:::chunk;
|
||||
A_chunk_title --> A_chunk3[Chunk A3]:::chunk;
|
||||
A_chunk_title --> A_chunk4[Chunk A4]:::chunk;
|
||||
A_chunk1 & A_chunk2 & A_chunk3 & A_chunk4 --> A_chunk_done([Extraction Complete]);
|
||||
end
|
||||
|
||||
subgraph B_chunk[Extraction Stage]
|
||||
B_chunk_title[Entity Relation Extraction<br>llm_model_max_async = 4];
|
||||
B_chunk_title --> B_chunk1[Chunk B1]:::chunk;
|
||||
B_chunk_title --> B_chunk2[Chunk B2]:::chunk;
|
||||
B_chunk_title --> B_chunk3[Chunk B3]:::chunk;
|
||||
B_chunk_title --> B_chunk4[Chunk B4]:::chunk;
|
||||
B_chunk1 & B_chunk2 & B_chunk3 & B_chunk4 --> B_chunk_done([Extraction Complete]);
|
||||
end
|
||||
A_chunk -.->|LLM Request| LLM_Queue;
|
||||
|
||||
A_chunk --> A_merge;
|
||||
B_chunk --> B_merge;
|
||||
|
||||
subgraph A_merge[Merge Stage]
|
||||
A_merge_title[Entity Relation Merging<br>llm_model_max_async * 2 = 8];
|
||||
A_merge_title --> A1_entity[Ent a1]:::merge;
|
||||
A_merge_title --> A2_entity[Ent a2]:::merge;
|
||||
A_merge_title --> A3_entity[Rel a3]:::merge;
|
||||
A_merge_title --> A4_entity[Rel a4]:::merge;
|
||||
A1_entity & A2_entity & A3_entity & A4_entity --> A_done([Merge Complete])
|
||||
end
|
||||
|
||||
subgraph B_merge[Merge Stage]
|
||||
B_merge_title[Entity Relation Merging<br>llm_model_max_async * 2 = 8];
|
||||
B_merge_title --> B1_entity[Ent b1]:::merge;
|
||||
B_merge_title --> B2_entity[Ent b2]:::merge;
|
||||
B_merge_title --> B3_entity[Rel b3]:::merge;
|
||||
B_merge_title --> B4_entity[Rel b4]:::merge;
|
||||
B1_entity & B2_entity & B3_entity & B4_entity --> B_done([Merge Complete])
|
||||
end
|
||||
|
||||
A_merge -.->|LLM Request| LLM_Queue["LLM Request Prioritized Queue<br>llm_model_max_async = 4"];
|
||||
B_merge -.->|LLM Request| LLM_Queue;
|
||||
B_chunk -.->|LLM Request| LLM_Queue;
|
||||
|
||||
```
|
||||
|
||||
> The extraction and merge stages share a global prioritized LLM queue, regulated by `llm_model_max_async`. While numerous entity and relation extraction and merging operations may be "actively processing", **only a limited number will concurrently execute LLM requests** the remainder will be queued and awaiting their turn.
|
||||
|
||||
### 6. Performance Optimization Recommendations
|
||||
|
||||
* **Increase LLM Concurrent Setting based on the capabilities of your LLM server or API provider**
|
||||
|
||||
During the file processing phase, the performance and concurrency capabilities of the LLM are critical bottlenecks. When deploying LLMs locally, the service's concurrency capacity must adequately account for the context length requirements of LightRAG. LightRAG recommends that LLMs support a minimum context length of 32KB; therefore, server concurrency should be calculated based on this benchmark. For API providers, LightRAG will retry requests up to three times if the client's request is rejected due to concurrent request limits. Backend logs can be used to determine if LLM retries are occurring, thereby indicating whether `MAX_ASYNC` has exceeded the API provider's limits.
|
||||
|
||||
* **Align Parallel Document Insertion Settings with LLM Concurrency Configurations**
|
||||
|
||||
The recommended number of parallel document processing tasks is 1/4 of the LLM's concurrency, with a minimum of 2 and a maximum of 10. Setting a higher number of parallel document processing tasks typically does not accelerate overall document processing speed, as even a small number of concurrently processed documents can fully utilize the LLM's parallel processing capabilities. Excessive parallel document processing can significantly increase the processing time for each individual document. Since LightRAG commits processing results on a file-by-file basis, a large number of concurrent files would necessitate caching a substantial amount of data. In the event of a system error, all documents in the middle stage would require reprocessing, thereby increasing error handling costs. For instance, setting `MAX_PARALLEL_INSERT` to 3 is appropriate when `MAX_ASYNC` is configured to 12.
|
||||
@@ -0,0 +1,317 @@
|
||||
# LightRAG Offline Deployment Guide
|
||||
|
||||
This guide provides comprehensive instructions for deploying LightRAG in offline environments where internet access is limited or unavailable.
|
||||
|
||||
If you deploy LightRAG using Docker, there is no need to refer to this document, as the LightRAG Docker image is pre-configured for offline operation.
|
||||
|
||||
> Software packages requiring `transformers`, `torch`, or `cuda` will not be included in the offline dependency group. Consequently, document extraction tools such as Docling, as well as local LLM models like Hugging Face and LMDeploy, are outside the scope of offline installation support. These high-compute-resource-demanding services should not be integrated into LightRAG. Docling will be decoupled and deployed as a standalone service.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Layered Dependencies](#layered-dependencies)
|
||||
- [Tiktoken Cache Management](#tiktoken-cache-management)
|
||||
- [Complete Offline Deployment Workflow](#complete-offline-deployment-workflow)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
## Overview
|
||||
|
||||
LightRAG uses dynamic package installation (`pipmaster`) for optional features based on file types and configurations. In offline environments, these dynamic installations will fail. This guide shows you how to pre-install all necessary dependencies and cache files.
|
||||
|
||||
### What Gets Dynamically Installed?
|
||||
|
||||
LightRAG dynamically installs packages for:
|
||||
|
||||
- **Storage Backends**: `redis`, `neo4j`, `pymilvus`, `pymongo`, `asyncpg`, `qdrant-client`
|
||||
- **LLM Providers**: `openai`, `anthropic`, `ollama`, `zhipuai`, `aioboto3`, `voyageai`, `llama-index`, `lmdeploy`, `transformers`, `torch`
|
||||
- **Tiktoken Models**: BPE encoding models downloaded from OpenAI CDN
|
||||
|
||||
**Note**: Document processing dependencies (`pypdf`, `python-docx`, `python-pptx`, `openpyxl`) are now pre-installed with the `api` extras group and no longer require dynamic installation.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Option 1: Using pip with Offline Extras
|
||||
|
||||
```bash
|
||||
# Online environment: Install all offline dependencies
|
||||
pip install lightrag-hku[offline]
|
||||
|
||||
# Download tiktoken cache
|
||||
lightrag-download-cache
|
||||
|
||||
# Create offline package
|
||||
pip download lightrag-hku[offline] -d ./offline-packages
|
||||
tar -czf lightrag-offline.tar.gz ./offline-packages ~/.tiktoken_cache
|
||||
|
||||
# Transfer to offline server
|
||||
scp lightrag-offline.tar.gz user@offline-server:/path/to/
|
||||
|
||||
# Offline environment: Install
|
||||
tar -xzf lightrag-offline.tar.gz
|
||||
pip install --no-index --find-links=./offline-packages lightrag-hku[offline]
|
||||
export TIKTOKEN_CACHE_DIR=~/.tiktoken_cache
|
||||
```
|
||||
|
||||
### Option 2: Using Requirements Files
|
||||
|
||||
```bash
|
||||
# Online environment: Download packages
|
||||
pip download -r requirements-offline.txt -d ./packages
|
||||
|
||||
# Transfer to offline server
|
||||
tar -czf packages.tar.gz ./packages
|
||||
scp packages.tar.gz user@offline-server:/path/to/
|
||||
|
||||
# Offline environment: Install
|
||||
tar -xzf packages.tar.gz
|
||||
pip install --no-index --find-links=./packages -r requirements-offline.txt
|
||||
```
|
||||
|
||||
## Layered Dependencies
|
||||
|
||||
LightRAG provides flexible dependency groups for different use cases:
|
||||
|
||||
### Available Dependency Groups
|
||||
|
||||
| Group | Description | Use Case |
|
||||
|-------|-------------|----------|
|
||||
| `api` | API server + document processing | FastAPI server with PDF, DOCX, PPTX, XLSX support |
|
||||
| `offline-storage` | Storage backends | Redis, Neo4j, MongoDB, PostgreSQL, etc. |
|
||||
| `offline-llm` | LLM providers | OpenAI, Anthropic, Ollama, etc. |
|
||||
| `offline` | Complete offline package | API + Storage + LLM (all features) |
|
||||
|
||||
**Note**: Document processing (PDF, DOCX, PPTX, XLSX) is included in the `api` extras group. The previous `offline-docs` group has been merged into `api` for better integration.
|
||||
|
||||
> Software packages requiring `transformers`, `torch`, or `cuda` will not be included in the offline dependency group.
|
||||
|
||||
### Installation Examples
|
||||
|
||||
```bash
|
||||
# Install API with document processing
|
||||
pip install lightrag-hku[api]
|
||||
|
||||
# Install API and storage backends
|
||||
pip install lightrag-hku[api,offline-storage]
|
||||
|
||||
# Install all offline dependencies (recommended for offline deployment)
|
||||
pip install lightrag-hku[offline]
|
||||
```
|
||||
|
||||
### Using Individual Requirements Files
|
||||
|
||||
```bash
|
||||
# Storage backends only
|
||||
pip install -r requirements-offline-storage.txt
|
||||
|
||||
# LLM providers only
|
||||
pip install -r requirements-offline-llm.txt
|
||||
|
||||
# All offline dependencies
|
||||
pip install -r requirements-offline.txt
|
||||
```
|
||||
|
||||
## Tiktoken Cache Management
|
||||
|
||||
Tiktoken downloads BPE encoding models on first use. In offline environments, you must pre-download these models.
|
||||
|
||||
### Using the CLI Command
|
||||
|
||||
After installing LightRAG, use the built-in command:
|
||||
|
||||
```bash
|
||||
# Download to default location (~/.tiktoken_cache)
|
||||
lightrag-download-cache
|
||||
|
||||
# Download to specific directory
|
||||
lightrag-download-cache --cache-dir ./tiktoken_cache
|
||||
|
||||
# Download specific models only
|
||||
lightrag-download-cache --models gpt-4o-mini gpt-4
|
||||
```
|
||||
|
||||
### Default Models Downloaded
|
||||
|
||||
- `gpt-4o-mini` (LightRAG default)
|
||||
- `gpt-4o`
|
||||
- `gpt-4`
|
||||
- `gpt-3.5-turbo`
|
||||
- `text-embedding-ada-002`
|
||||
- `text-embedding-3-small`
|
||||
- `text-embedding-3-large`
|
||||
|
||||
### Setting Cache Location in Offline Environment
|
||||
|
||||
```bash
|
||||
# Option 1: Environment variable (temporary)
|
||||
export TIKTOKEN_CACHE_DIR=/path/to/tiktoken_cache
|
||||
|
||||
# Option 2: Add to ~/.bashrc or ~/.zshrc (persistent)
|
||||
echo 'export TIKTOKEN_CACHE_DIR=~/.tiktoken_cache' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
|
||||
# Option 3: Copy to default location
|
||||
cp -r /path/to/tiktoken_cache ~/.tiktoken_cache/
|
||||
```
|
||||
|
||||
## Complete Offline Deployment Workflow
|
||||
|
||||
### Step 1: Prepare in Online Environment
|
||||
|
||||
```bash
|
||||
# 1. Install LightRAG with offline dependencies
|
||||
pip install lightrag-hku[offline]
|
||||
|
||||
# 2. Download tiktoken cache
|
||||
lightrag-download-cache --cache-dir ./offline_cache/tiktoken
|
||||
|
||||
# 3. Download all Python packages
|
||||
pip download lightrag-hku[offline] -d ./offline_cache/packages
|
||||
|
||||
# 4. Create archive for transfer
|
||||
tar -czf lightrag-offline-complete.tar.gz ./offline_cache
|
||||
|
||||
# 5. Verify contents
|
||||
tar -tzf lightrag-offline-complete.tar.gz | head -20
|
||||
```
|
||||
|
||||
### Step 2: Transfer to Offline Environment
|
||||
|
||||
```bash
|
||||
# Using scp
|
||||
scp lightrag-offline-complete.tar.gz user@offline-server:/tmp/
|
||||
|
||||
# Or using USB/physical media
|
||||
# Copy lightrag-offline-complete.tar.gz to USB drive
|
||||
```
|
||||
|
||||
### Step 3: Install in Offline Environment
|
||||
|
||||
```bash
|
||||
# 1. Extract archive
|
||||
cd /tmp
|
||||
tar -xzf lightrag-offline-complete.tar.gz
|
||||
|
||||
# 2. Install Python packages
|
||||
pip install --no-index \
|
||||
--find-links=/tmp/offline_cache/packages \
|
||||
lightrag-hku[offline]
|
||||
|
||||
# 3. Set up tiktoken cache
|
||||
mkdir -p ~/.tiktoken_cache
|
||||
cp -r /tmp/offline_cache/tiktoken/* ~/.tiktoken_cache/
|
||||
export TIKTOKEN_CACHE_DIR=~/.tiktoken_cache
|
||||
|
||||
# 4. Add to shell profile for persistence
|
||||
echo 'export TIKTOKEN_CACHE_DIR=~/.tiktoken_cache' >> ~/.bashrc
|
||||
```
|
||||
|
||||
### Step 4: Verify Installation
|
||||
|
||||
```bash
|
||||
# Test Python import
|
||||
python -c "from lightrag import LightRAG; print('✓ LightRAG imported')"
|
||||
|
||||
# Test tiktoken
|
||||
python -c "from lightrag.utils import TiktokenTokenizer; t = TiktokenTokenizer(); print('✓ Tiktoken working')"
|
||||
|
||||
# Test optional dependencies (if installed)
|
||||
python -c "import docling; print('✓ Docling available')"
|
||||
python -c "import redis; print('✓ Redis available')"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Tiktoken fails with network error
|
||||
|
||||
**Problem**: `Unable to load tokenizer for model gpt-4o-mini`
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Ensure TIKTOKEN_CACHE_DIR is set
|
||||
echo $TIKTOKEN_CACHE_DIR
|
||||
|
||||
# Verify cache files exist
|
||||
ls -la ~/.tiktoken_cache/
|
||||
|
||||
# If empty, you need to download cache in online environment first
|
||||
```
|
||||
|
||||
### Issue: Dynamic package installation fails
|
||||
|
||||
**Problem**: `Error installing package xxx`
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Pre-install the specific package you need
|
||||
# For API with document processing:
|
||||
pip install lightrag-hku[api]
|
||||
|
||||
# For storage backends:
|
||||
pip install lightrag-hku[offline-storage]
|
||||
|
||||
# For LLM providers:
|
||||
pip install lightrag-hku[offline-llm]
|
||||
```
|
||||
|
||||
### Issue: Missing dependencies at runtime
|
||||
|
||||
**Problem**: `ModuleNotFoundError: No module named 'xxx'`
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check what you have installed
|
||||
pip list | grep -i xxx
|
||||
|
||||
# Install missing component
|
||||
pip install lightrag-hku[offline] # Install all offline deps
|
||||
```
|
||||
|
||||
### Issue: Permission denied on tiktoken cache
|
||||
|
||||
**Problem**: `PermissionError: [Errno 13] Permission denied`
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Ensure cache directory has correct permissions
|
||||
chmod 755 ~/.tiktoken_cache
|
||||
chmod 644 ~/.tiktoken_cache/*
|
||||
|
||||
# Or use a user-writable directory
|
||||
export TIKTOKEN_CACHE_DIR=~/my_tiktoken_cache
|
||||
mkdir -p ~/my_tiktoken_cache
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Test in Online Environment First**: Always test your complete setup in an online environment before going offline.
|
||||
|
||||
2. **Keep Cache Updated**: Periodically update your offline cache when new models are released.
|
||||
|
||||
3. **Document Your Setup**: Keep notes on which optional dependencies you actually need.
|
||||
|
||||
4. **Version Pinning**: Consider pinning specific versions in production:
|
||||
```bash
|
||||
pip freeze > requirements-production.txt
|
||||
```
|
||||
|
||||
5. **Minimal Installation**: Only install what you need:
|
||||
```bash
|
||||
# If you only need API with document processing
|
||||
pip install lightrag-hku[api]
|
||||
# Then manually add specific LLM: pip install openai
|
||||
```
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [LightRAG GitHub Repository](https://github.com/HKUDS/LightRAG)
|
||||
- [Docker Deployment Guide](./DockerDeployment.md)
|
||||
- [API Documentation](../lightrag/api/README.md)
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues not covered in this guide:
|
||||
|
||||
1. Check the [GitHub Issues](https://github.com/HKUDS/LightRAG/issues)
|
||||
2. Review the [project documentation](../README.md)
|
||||
3. Create a new issue with your offline deployment details
|
||||
@@ -0,0 +1,170 @@
|
||||
# uv.lock Update Guide
|
||||
|
||||
## What is uv.lock?
|
||||
|
||||
`uv.lock` is uv's lock file. It captures the exact version of every dependency, including transitive ones, much like:
|
||||
- Node.js `package-lock.json`
|
||||
- Rust `Cargo.lock`
|
||||
- Python Poetry `poetry.lock`
|
||||
|
||||
Keeping `uv.lock` in version control guarantees that everyone installs the same dependency set.
|
||||
|
||||
## When does uv.lock change?
|
||||
|
||||
### Situations where it does *not* change automatically
|
||||
|
||||
- Running `uv sync --frozen`
|
||||
- Building Docker images that call `uv sync --frozen`
|
||||
- Editing source code without touching dependency metadata
|
||||
|
||||
### Situations where it will change
|
||||
|
||||
1. **`uv lock` or `uv lock --upgrade`**
|
||||
|
||||
```bash
|
||||
uv lock # Resolve according to current constraints
|
||||
uv lock --upgrade # Re-resolve and upgrade to the newest compatible releases
|
||||
```
|
||||
|
||||
Use these commands after modifying `pyproject.toml`, when you want fresh dependency versions, or if the lock file was deleted or corrupted.
|
||||
|
||||
2. **`uv add`**
|
||||
|
||||
```bash
|
||||
uv add requests # Adds the dependency and updates both files
|
||||
uv add --dev pytest # Adds a dev dependency
|
||||
```
|
||||
|
||||
`uv add` edits `pyproject.toml` and refreshes `uv.lock` in one step.
|
||||
|
||||
3. **`uv remove`**
|
||||
|
||||
```bash
|
||||
uv remove requests
|
||||
```
|
||||
|
||||
This removes the dependency from `pyproject.toml` and rewrites `uv.lock`.
|
||||
|
||||
4. **`uv sync` without `--frozen`**
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
```
|
||||
|
||||
Normally this only installs what is already locked. However, if `pyproject.toml` and `uv.lock` disagree or the lock file is missing, uv will regenerate and update `uv.lock`. In CI and production builds you should prefer `uv sync --frozen` to prevent unintended updates.
|
||||
|
||||
## Example workflows
|
||||
|
||||
### Scenario 1: Add a new dependency
|
||||
|
||||
```bash
|
||||
# Recommended: let uv handle both files
|
||||
uv add fastapi
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Add fastapi dependency"
|
||||
|
||||
# Manual alternative
|
||||
# 1. Edit pyproject.toml
|
||||
# 2. Regenerate the lock file
|
||||
uv lock
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Add fastapi dependency"
|
||||
```
|
||||
|
||||
### Scenario 2: Relax or tighten a version constraint
|
||||
|
||||
```bash
|
||||
# 1. Edit the requirement in pyproject.toml,
|
||||
# e.g. openai>=1.0.0,<2.0.0 -> openai>=1.5.0,<2.0.0
|
||||
|
||||
# 2. Re-resolve the lock file
|
||||
uv lock
|
||||
|
||||
# 3. Commit both files
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Update openai to >=1.5.0"
|
||||
```
|
||||
|
||||
### Scenario 3: Upgrade everything to the newest compatible versions
|
||||
|
||||
```bash
|
||||
uv lock --upgrade
|
||||
git diff uv.lock
|
||||
git add uv.lock
|
||||
git commit -m "Upgrade dependencies to latest compatible versions"
|
||||
```
|
||||
|
||||
### Scenario 4: Teammate syncing the project
|
||||
|
||||
```bash
|
||||
git pull # Fetch latest code and lock file
|
||||
uv sync --frozen # Install exactly what uv.lock specifies
|
||||
```
|
||||
|
||||
## Using uv.lock in Docker
|
||||
|
||||
```dockerfile
|
||||
RUN uv sync --frozen --no-dev --extra api
|
||||
```
|
||||
|
||||
`--frozen` guarantees reproducible builds because uv will refuse to deviate from the locked versions.
|
||||
`--extra api` install API server
|
||||
|
||||
## Generating a lock file that includes offline dependencies
|
||||
|
||||
If you need `uv.lock` to capture the optional offline stacks, regenerate it with the relevant extras enabled:
|
||||
|
||||
```bash
|
||||
uv lock --extra api --extra offline
|
||||
```
|
||||
|
||||
This command resolves the base project requirements plus both the `api` and `offline` optional dependency sets, ensuring downstream `uv sync --frozen --extra api --extra offline` installs work without further resolution.
|
||||
|
||||
## Frequently asked questions
|
||||
|
||||
- **`uv.lock` is almost 1 MB. Does that matter?**
|
||||
No. The file is read only during dependency resolution.
|
||||
|
||||
- **Should we commit `uv.lock`?**
|
||||
Yes. Commit it so collaborators and CI jobs share the same dependency graph.
|
||||
|
||||
- **Deleted the lock file by accident?**
|
||||
Run `uv lock` to regenerate it from `pyproject.toml`.
|
||||
|
||||
- **Can `uv.lock` and `requirements.txt` coexist?**
|
||||
They can, but maintaining both is redundant. Prefer relying on `uv.lock` alone whenever possible.
|
||||
|
||||
- **How do I inspect locked versions?**
|
||||
```bash
|
||||
uv tree
|
||||
grep -A5 'name = "openai"' uv.lock
|
||||
```
|
||||
|
||||
## Best practices
|
||||
|
||||
### Recommended
|
||||
|
||||
1. Commit `uv.lock` alongside `pyproject.toml`.
|
||||
2. Use `uv sync --frozen` in CI, Docker, and other reproducible environments.
|
||||
3. Use plain `uv sync` during local development if you want uv to reconcile the lock for you.
|
||||
4. Run `uv lock --upgrade` periodically to pick up the latest compatible releases.
|
||||
5. Regenerate the lock file immediately after changing dependency constraints.
|
||||
|
||||
### Avoid
|
||||
|
||||
1. Running `uv sync` without `--frozen` in CI or production pipelines.
|
||||
2. Editing `uv.lock` by hand—uv will overwrite manual edits.
|
||||
3. Ignoring lock file diffs in code reviews—unexpected dependency changes can break builds.
|
||||
|
||||
## Summary
|
||||
|
||||
| Command | Updates `uv.lock` | Typical use |
|
||||
|-----------------------|-------------------|-------------------------------------------|
|
||||
| `uv lock` | ✅ Yes | After editing constraints |
|
||||
| `uv lock --upgrade` | ✅ Yes | Upgrade to the newest compatible versions |
|
||||
| `uv add <pkg>` | ✅ Yes | Add a dependency |
|
||||
| `uv remove <pkg>` | ✅ Yes | Remove a dependency |
|
||||
| `uv sync` | ⚠️ Maybe | Local development; can regenerate the lock |
|
||||
| `uv sync --frozen` | ❌ No | CI/CD, Docker, reproducible builds |
|
||||
|
||||
Remember: `uv.lock` only changes when you run a command that tells it to. Keep it in sync with your project and commit it whenever it changes.
|
||||
Reference in New Issue
Block a user