chore: init monorepo snapshot
This commit is contained in:
liaibo
@@ -0,0 +1,2 @@
|
||||
inputs
|
||||
rag_storage
|
||||
@@ -0,0 +1,626 @@
|
||||
# LightRAG 服务器和 WebUI
|
||||
|
||||
LightRAG 服务器旨在提供 Web 界面和 API 支持。Web 界面便于文档索引、知识图谱探索和简单的 RAG 查询界面。LightRAG 服务器还提供了与 Ollama 兼容的接口，旨在将 LightRAG 模拟为 Ollama 聊天模型。这使得 AI 聊天机器人（如 Open WebUI）可以轻松访问 LightRAG。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 入门指南
|
||||
|
||||
### 安装
|
||||
|
||||
* 从 PyPI 安装
|
||||
|
||||
```bash
|
||||
# 使用 uv (推荐)
|
||||
uv pip install "lightrag-hku[api]"
|
||||
|
||||
# 或使用 pip
|
||||
# pip install "lightrag-hku[api]"
|
||||
```
|
||||
|
||||
* 从源代码安装
|
||||
|
||||
```bash
|
||||
# 克隆仓库
|
||||
git clone https://github.com/HKUDS/lightrag.git
|
||||
|
||||
# 进入仓库目录
|
||||
cd lightrag
|
||||
|
||||
# 使用 uv (推荐)
|
||||
# 注意: uv sync 会自动在 .venv/ 目录创建虚拟环境
|
||||
uv sync --extra api
|
||||
source .venv/bin/activate # 激活虚拟环境 (Linux/macOS)
|
||||
# Windows 系统: .venv\Scripts\activate
|
||||
|
||||
# 或使用 pip 与虚拟环境
|
||||
# python -m venv .venv
|
||||
# source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
# pip install -e ".[api]"
|
||||
|
||||
# 构建前端代码
|
||||
cd lightrag_webui
|
||||
bun install --frozen-lockfile
|
||||
bun run build
|
||||
cd ..
|
||||
```
|
||||
|
||||
### 启动 LightRAG 服务器前的准备
|
||||
|
||||
LightRAG 需要同时集成 LLM（大型语言模型）和嵌入模型以有效执行文档索引和查询操作。在首次部署 LightRAG 服务器之前，必须配置 LLM 和嵌入模型的设置。LightRAG 支持绑定到各种 LLM/嵌入后端：
|
||||
|
||||
* ollama
|
||||
* lollms
|
||||
* openai 或 openai 兼容
|
||||
* azure_openai
|
||||
* aws_bedrock
|
||||
|
||||
建议使用环境变量来配置 LightRAG 服务器。项目根目录中有一个名为 `env.example` 的示例环境变量文件。请将此文件复制到启动目录并重命名为 `.env`。之后，您可以在 `.env` 文件中修改与 LLM 和嵌入模型相关的参数。需要注意的是，LightRAG 服务器每次启动时都会将 `.env` 中的环境变量加载到系统环境变量中。**LightRAG 服务器会优先使用系统环境变量中的设置**。
|
||||
|
||||
> 由于安装了 Python 扩展的 VS Code 可能会在集成终端中自动加载 .env 文件，请在每次修改 .env 文件后打开新的终端会话。
|
||||
|
||||
以下是 LLM 和嵌入模型的一些常见设置示例：
|
||||
|
||||
* OpenAI LLM + Ollama 嵌入
|
||||
|
||||
```
|
||||
LLM_BINDING=openai
|
||||
LLM_MODEL=gpt-4o
|
||||
LLM_BINDING_HOST=https://api.openai.com/v1
|
||||
LLM_BINDING_API_KEY=your_api_key
|
||||
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
# EMBEDDING_BINDING_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
* Ollama LLM + Ollama 嵌入
|
||||
|
||||
```
|
||||
LLM_BINDING=ollama
|
||||
LLM_MODEL=mistral-nemo:latest
|
||||
LLM_BINDING_HOST=http://localhost:11434
|
||||
# LLM_BINDING_API_KEY=your_api_key
|
||||
### Ollama 服务器上下文 token 数（必须大于 MAX_TOTAL_TOKENS+2000）
|
||||
OLLAMA_LLM_NUM_CTX=8192
|
||||
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
# EMBEDDING_BINDING_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
> **重要提示**：在文档索引前必须确定使用的Embedding模型，且在文档查询阶段必须沿用与索引阶段相同的模型。有些存储（例如PostgreSQL）在首次建立数表的时候需要确定向量维度，因此更换Embedding模型后需要删除向量相关库表，以便让LightRAG重建新的库表。
|
||||
|
||||
### 启动 LightRAG 服务器
|
||||
|
||||
LightRAG 服务器支持两种运行模式：
|
||||
* 简单高效的 Uvicorn 模式
|
||||
|
||||
```
|
||||
lightrag-server
|
||||
```
|
||||
* 多进程 Gunicorn + Uvicorn 模式（生产模式，不支持 Windows 环境）
|
||||
|
||||
```
|
||||
lightrag-gunicorn --workers 4
|
||||
```
|
||||
启动LightRAG的时候，当前工作目录必须含有`.env`配置文件。**要求将.env文件置于启动目录中是经过特意设计的**。 这样做的目的是支持用户同时启动多个LightRAG实例，并为不同实例配置不同的.env文件。**修改.env文件后，您需要重新打开终端以使新设置生效**。 这是因为每次启动时，LightRAG Server会将.env文件中的环境变量加载至系统环境变量，且系统环境变量的设置具有更高优先级。
|
||||
|
||||
启动时可以通过命令行参数覆盖`.env`文件中的配置。常用的命令行参数包括：
|
||||
|
||||
- `--host`：服务器监听地址（默认：0.0.0.0）
|
||||
- `--port`：服务器监听端口（默认：9621）
|
||||
- `--timeout`：LLM 请求超时时间（默认：150 秒）
|
||||
- `--log-level`：日志级别（默认：INFO）
|
||||
- `--working-dir`：数据库持久化目录（默认：./rag_storage）
|
||||
- `--input-dir`：上传文件存放目录（默认：./inputs）
|
||||
- `--workspace`: 工作空间名称，用于逻辑上隔离多个LightRAG实例之间的数据（默认：空）
|
||||
|
||||
### 使用 Docker 启动 LightRAG 服务器
|
||||
|
||||
使用 Docker Compose 是部署和运行 LightRAG Server 最便捷的方式。
|
||||
- 创建一个项目目录。
|
||||
- 将 LightRAG 仓库中的 `docker-compose.yml` 文件复制到您的项目目录中。
|
||||
- 准备 `.env` 文件：复制示例文件 [`env.example`](https://ai.znipower.com:5013/c/env.example) 创建自定义的 `.env` 文件，并根据您的具体需求配置 LLM 和嵌入参数。
|
||||
|
||||
* 通过以下命令启动 LightRAG 服务器：
|
||||
|
||||
```shell
|
||||
docker compose up
|
||||
# 如果希望启动后让程序退到后台运行，需要在命令的最后添加 -d 参数
|
||||
```
|
||||
> 可以通过以下链接获取官方的docker compose文件：[docker-compose.yml]( https://raw.githubusercontent.com/HKUDS/LightRAG/refs/heads/main/docker-compose.yml) 。如需获取LightRAG的历史版本镜像，可以访问以下链接: [LightRAG Docker Images]( https://github.com/HKUDS/LightRAG/pkgs/container/lightrag). 如需获取更多关于docker部署的信息，请参阅 [DockerDeployment.md](./../../docs/DockerDeployment.md).
|
||||
|
||||
### 离线部署
|
||||
|
||||
官方的 LightRAG Docker 镜像完全兼容离线或隔离网络环境。如需搭建自己的离线部署环境，请参考 [离线部署指南](./../../docs/OfflineDeployment.md)。
|
||||
|
||||
### 启动多个LightRAG实例
|
||||
|
||||
有两种方式可以启动多个LightRAG实例。第一种方式是为每个实例配置一个完全独立的工作环境。此时需要为每个实例创建一个独立的工作目录，然后在这个工作目录上放置一个当前实例专用的`.env`配置文件。不同实例的配置文件中的服务器监听端口不能重复，然后在工作目录上执行 lightrag-server 启动服务即可。
|
||||
|
||||
第二种方式是所有实例共享一套相同的`.env`配置文件，然后通过命令行参数来为每个实例指定不同的服务器监听端口和工作空间。你可以在同一个工作目录中通过不同的命令行参数启动多个LightRAG实例。例如：
|
||||
|
||||
```
|
||||
# 启动实例1
|
||||
lightrag-server --port 9621 --workspace space1
|
||||
|
||||
# 启动实例2
|
||||
lightrag-server --port 9622 --workspace space2
|
||||
```
|
||||
|
||||
工作空间的作用是实现不同实例之间的数据隔离。因此不同实例之间的`workspace`参数必须不同，否则会导致数据混乱，数据将会被破坏。
|
||||
|
||||
通过 Docker Compose 启动多个 LightRAG 实例时，只需在 `docker-compose.yml` 中为每个容器指定不同的 `WORKSPACE` 和 `PORT` 环境变量即可。即使所有实例共享同一个 `.env` 文件，Compose 中定义的容器环境变量也会优先覆盖 `.env` 文件中的同名设置，从而确保每个实例拥有独立的配置。
|
||||
|
||||
### LightRAG实例间的数据隔离
|
||||
|
||||
每个实例配置一个独立的工作目录和专用`.env`配置文件通常能够保证内存数据库中的本地持久化文件保存在各自的工作目录，实现数据的相互隔离。LightRAG默认存储全部都是内存数据库，通过这种方式进行数据隔离是没有问题的。但是如果使用的是外部数据库，如果不同实例访问的是同一个数据库实例，就需要通过配置工作空间来实现数据隔离，否则不同实例的数据将会出现冲突并被破坏。
|
||||
|
||||
命令行的 workspace 参数和`.env`文件中的环境变量`WORKSPACE` 都可以用于指定当前实例的工作空间名字，命令行参数的优先级别更高。下面是不同类型的存储实现工作空间的方式：
|
||||
|
||||
- **对于本地基于文件的数据库，数据隔离通过工作空间子目录实现：** JsonKVStorage, JsonDocStatusStorage, NetworkXStorage, NanoVectorDBStorage, FaissVectorDBStorage。
|
||||
- **对于将数据存储在集合（collection）中的数据库，通过在集合名称前添加工作空间前缀来实现：** RedisKVStorage, RedisDocStatusStorage, MilvusVectorDBStorage, QdrantVectorDBStorage, MongoKVStorage, MongoDocStatusStorage, MongoVectorDBStorage, MongoGraphStorage, PGGraphStorage。
|
||||
- **对于关系型数据库，数据隔离通过向表中添加 `workspace` 字段进行数据的逻辑隔离：** PGKVStorage, PGVectorStorage, PGDocStatusStorage。
|
||||
|
||||
* **对于Neo4j图数据库，通过label来实现数据的逻辑隔离**：Neo4JStorage
|
||||
|
||||
为了保持对遗留数据的兼容，在未配置工作空间时PostgreSQL的默认工作空间为`default`，Neo4j的默认工作空间为`base`。对于所有的外部存储，系统都提供了专用的工作空间环境变量，用于覆盖公共的 `WORKSPACE`环境变量配置。这些适用于指定存储类型的工作空间环境变量为：`REDIS_WORKSPACE`, `MILVUS_WORKSPACE`, `QDRANT_WORKSPACE`, `MONGODB_WORKSPACE`, `POSTGRES_WORKSPACE`, `NEO4J_WORKSPACE`。
|
||||
|
||||
### Gunicorn + Uvicorn 的多工作进程
|
||||
|
||||
LightRAG 服务器可以在 `Gunicorn + Uvicorn` 预加载模式下运行。Gunicorn 的多工作进程（多进程）功能可以防止文档索引任务阻塞 RAG 查询。使用 CPU 密集型文档提取工具（如 docling）在纯 Uvicorn 模式下可能会导致整个系统被阻塞。
|
||||
|
||||
虽然 LightRAG 服务器使用一个工作进程来处理文档索引流程，但通过 Uvicorn 的异步任务支持，可以并行处理多个文件。文档索引速度的瓶颈主要在于 LLM。如果您的 LLM 支持高并发，您可以通过增加 LLM 的并发级别来加速文档索引。以下是几个与并发处理相关的环境变量及其默认值：
|
||||
|
||||
```
|
||||
### 工作进程数，数字不大于 (2 x 核心数) + 1
|
||||
WORKERS=2
|
||||
### 一批中并行处理的文件数
|
||||
MAX_PARALLEL_INSERT=2
|
||||
# LLM 的最大并发请求数
|
||||
MAX_ASYNC=4
|
||||
```
|
||||
|
||||
### 将 Lightrag 安装为 Linux 服务
|
||||
|
||||
从示例文件 `lightrag.service.example` 创建您的服务文件 `lightrag.service`。修改服务文件中的服务启动定义：
|
||||
|
||||
```text
|
||||
# Set Enviroment to your Python virtual enviroment
|
||||
Environment="PATH=/home/netman/lightrag-xyj/venv/bin"
|
||||
WorkingDirectory=/home/netman/lightrag-xyj
|
||||
# ExecStart=/home/netman/lightrag-xyj/venv/bin/lightrag-server
|
||||
ExecStart=/home/netman/lightrag-xyj/venv/bin/lightrag-gunicorn
|
||||
```
|
||||
> ExecStart命令必须是 lightrag-gunicorn 或 lightrag-server 中的一个，不能使用其它脚本包裹它们。因为停止服务必须要求主进程必须是这两个进程。
|
||||
|
||||
安装 LightRAG 服务。如果您的系统是 Ubuntu，以下命令将生效：
|
||||
|
||||
```shell
|
||||
sudo cp lightrag.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl start lightrag.service
|
||||
sudo systemctl status lightrag.service
|
||||
sudo systemctl enable lightrag.service
|
||||
```
|
||||
|
||||
## Ollama 模拟
|
||||
|
||||
我们为 LightRAG 提供了 Ollama 兼容接口，旨在将 LightRAG 模拟为 Ollama 聊天模型。这使得支持 Ollama 的 AI 聊天前端（如 Open WebUI）可以轻松访问 LightRAG。
|
||||
|
||||
### 将 Open WebUI 连接到 LightRAG
|
||||
|
||||
启动 lightrag-server 后，您可以在 Open WebUI 管理面板中添加 Ollama 类型的连接。然后，一个名为 `lightrag:latest` 的模型将出现在 Open WebUI 的模型管理界面中。用户随后可以通过聊天界面向 LightRAG 发送查询。对于这种用例，最好将 LightRAG 安装为服务。
|
||||
|
||||
Open WebUI 使用 LLM 来执行会话标题和会话关键词生成任务。因此，Ollama 聊天补全 API 会检测并将 OpenWebUI 会话相关请求直接转发给底层 LLM。Open WebUI 的截图：
|
||||
|
||||
|
||||
|
||||
### 在聊天中选择查询模式
|
||||
|
||||
如果您从 LightRAG 的 Ollama 接口发送消息（查询），默认查询模式是 `hybrid`。您可以通过发送带有查询前缀的消息来选择查询模式。
|
||||
|
||||
查询字符串中的查询前缀可以决定使用哪种 LightRAG 查询模式来生成响应。支持的前缀包括：
|
||||
|
||||
```
|
||||
/local
|
||||
/global
|
||||
/hybrid
|
||||
/naive
|
||||
/mix
|
||||
|
||||
/bypass
|
||||
/context
|
||||
/localcontext
|
||||
/globalcontext
|
||||
/hybridcontext
|
||||
/naivecontext
|
||||
/mixcontext
|
||||
```
|
||||
|
||||
例如，聊天消息 "/mix 唐僧有几个徒弟" 将触发 LightRAG 的混合模式查询。没有查询前缀的聊天消息默认会触发混合模式查询。
|
||||
|
||||
"/bypass" 不是 LightRAG 查询模式，它会告诉 API 服务器将查询连同聊天历史直接传递给底层 LLM。因此用户可以使用 LLM 基于聊天历史回答问题。如果您使用 Open WebUI 作为前端，您可以直接切换到普通 LLM 模型，而不是使用 /bypass 前缀。
|
||||
|
||||
"/context" 也不是 LightRAG 查询模式，它会告诉 LightRAG 只返回为 LLM 准备的上下文信息。您可以检查上下文是否符合您的需求，或者自行处理上下文。
|
||||
|
||||
### 在聊天中添加用户提示词
|
||||
|
||||
使用LightRAG进行内容查询时，应避免将搜索过程与无关的输出处理相结合，这会显著影响查询效果。用户提示（user prompt）正是为解决这一问题而设计 -- 它不参与RAG检索阶段，而是在查询完成后指导大语言模型（LLM）如何处理检索结果。我们可以在查询前缀末尾添加方括号，从而向LLM传递用户提示词：
|
||||
|
||||
```
|
||||
/[使用mermaid格式画图] 请画出 Scrooge 的人物关系图谱
|
||||
/mix[使用mermaid格式画图] 请画出 Scrooge 的人物关系图谱
|
||||
```
|
||||
|
||||
## API 密钥和认证
|
||||
|
||||
默认情况下，LightRAG 服务器可以在没有任何认证的情况下访问。我们可以使用 API 密钥或账户凭证配置服务器以确保其安全。
|
||||
|
||||
* API 密钥
|
||||
|
||||
```
|
||||
LIGHTRAG_API_KEY=your-secure-api-key-here
|
||||
WHITELIST_PATHS=/health,/api/*
|
||||
```
|
||||
|
||||
> 健康检查和 Ollama 模拟端点默认不进行 API 密钥检查。为了安全原因，如果不需要提供Ollama服务，应该把`/api/*`从WHITELIST_PATHS中移除。
|
||||
|
||||
API Key使用的请求头是 `X-API-Key` 。以下是使用API访问LightRAG Server的一个例子：
|
||||
|
||||
```
|
||||
curl -X 'POST' \
|
||||
'http://localhost:9621/documents/scan' \
|
||||
-H 'accept: application/json' \
|
||||
-H 'X-API-Key: your-secure-api-key-here-123' \
|
||||
-d ''
|
||||
```
|
||||
|
||||
* 账户凭证（Web 界面需要登录后才能访问）
|
||||
|
||||
LightRAG API 服务器使用基于 HS256 算法的 JWT 认证。要启用安全访问控制，需要以下环境变量：
|
||||
|
||||
```bash
|
||||
# JWT 认证
|
||||
AUTH_ACCOUNTS='admin:admin123,user1:pass456'
|
||||
TOKEN_SECRET='your-key'
|
||||
TOKEN_EXPIRE_HOURS=4
|
||||
```
|
||||
|
||||
> 目前仅支持配置一个管理员账户和密码。尚未开发和实现完整的账户系统。
|
||||
|
||||
如果未配置账户凭证，Web 界面将以访客身份访问系统。因此，即使仅配置了 API 密钥，所有 API 仍然可以通过访客账户访问，这仍然不安全。因此，要保护 API，需要同时配置这两种认证方法。
|
||||
|
||||
## Azure OpenAI 后端配置
|
||||
|
||||
可以使用以下 Azure CLI 命令创建 Azure OpenAI API（您需要先从 [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) 安装 Azure CLI）：
|
||||
|
||||
```bash
|
||||
# 根据需要更改资源组名称、位置和 OpenAI 资源名称
|
||||
RESOURCE_GROUP_NAME=LightRAG
|
||||
LOCATION=swedencentral
|
||||
RESOURCE_NAME=LightRAG-OpenAI
|
||||
|
||||
az login
|
||||
az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
|
||||
az cognitiveservices account create --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --kind OpenAI --sku S0 --location swedencentral
|
||||
az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name gpt-4o --model-name gpt-4o --model-version "2024-08-06" --sku-capacity 100 --sku-name "Standard"
|
||||
az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name text-embedding-3-large --model-name text-embedding-3-large --model-version "1" --sku-capacity 80 --sku-name "Standard"
|
||||
az cognitiveservices account show --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --query "properties.endpoint"
|
||||
az cognitiveservices account keys list --name $RESOURCE_NAME -g $RESOURCE_GROUP_NAME
|
||||
```
|
||||
|
||||
最后一个命令的输出将提供 OpenAI API 的端点和密钥。您可以使用这些值在 `.env` 文件中设置环境变量。
|
||||
|
||||
```
|
||||
# .env 中的 Azure OpenAI 配置
|
||||
LLM_BINDING=azure_openai
|
||||
LLM_BINDING_HOST=your-azure-endpoint
|
||||
LLM_MODEL=your-model-deployment-name
|
||||
LLM_BINDING_API_KEY=your-azure-api-key
|
||||
### API Version可选，默认为最新版本
|
||||
AZURE_OPENAI_API_VERSION=2024-08-01-preview
|
||||
|
||||
### 如果使用 Azure OpenAI 进行嵌入
|
||||
EMBEDDING_BINDING=azure_openai
|
||||
EMBEDDING_MODEL=your-embedding-deployment-name
|
||||
```
|
||||
|
||||
## LightRAG 服务器详细配置
|
||||
|
||||
API 服务器可以通过三种方式配置（优先级从高到低）：
|
||||
|
||||
* 命令行参数
|
||||
* 环境变量或 .env 文件
|
||||
* Config.ini（仅用于存储配置）
|
||||
|
||||
大多数配置都有默认设置，详细信息请查看示例文件：`.env.example`。数据存储配置也可以通过 config.ini 设置。为方便起见，提供了示例文件 `config.ini.example`。
|
||||
|
||||
### 支持的 LLM 和嵌入后端
|
||||
|
||||
LightRAG 支持绑定到各种 LLM/嵌入后端：
|
||||
|
||||
* ollama
|
||||
* openai (含openai 兼容)
|
||||
* azure_openai
|
||||
* lollms
|
||||
* aws_bedrock
|
||||
|
||||
使用环境变量 `LLM_BINDING` 或 CLI 参数 `--llm-binding` 选择 LLM 后端类型。使用环境变量 `EMBEDDING_BINDING` 或 CLI 参数 `--embedding-binding` 选择嵌入后端类型。
|
||||
|
||||
LLM和Embedding配置例子请查看项目根目录的 env.example 文件。OpenAI和Ollama兼容LLM接口的支持的完整配置选型可以通过一下命令查看：
|
||||
|
||||
```
|
||||
lightrag-server --llm-binding openai --help
|
||||
lightrag-server --llm-binding ollama --help
|
||||
lightrag-server --embedding-binding ollama --help
|
||||
```
|
||||
|
||||
> 请使用openai兼容方式访问OpenRouter、vLLM或SLang部署的LLM。可以通过 `OPENAI_LLM_EXTRA_BODY` 环境变量给OpenRouter、vLLM或SGLang推理框架传递额外的参数，实现推理模式的关闭或者其它个性化控制。
|
||||
|
||||
设置 `max_tokens` 参数旨在**防止在实体关系提取阶段出现LLM 响应输出过长或无休止的循环输出的问题**。设置 `max_tokens` 参数的目的是在超时发生之前截断 LLM 输出，从而防止文档提取失败。这解决了某些包含大量实体和关系的文本块（例如表格或引文）可能导致 LLM 产生过长甚至无限循环输出的问题。此设置对于本地部署的小参数模型尤为重要。`max_tokens` 值可以通过以下公式计算：
|
||||
|
||||
```
|
||||
# For vLLM/SGLang doployed models, or most of OpenAI compatible API provider
|
||||
OPENAI_LLM_MAX_TOKENS=9000
|
||||
|
||||
# For Ollama Deployed Modeles
|
||||
OLLAMA_LLM_NUM_PREDICT=9000
|
||||
|
||||
# For OpenAI o1-mini or newer modles
|
||||
OPENAI_LLM_MAX_COMPLETION_TOKENS=9000
|
||||
```
|
||||
|
||||
### 实体提取配置
|
||||
|
||||
* ENABLE_LLM_CACHE_FOR_EXTRACT：为实体提取启用 LLM 缓存（默认：true）
|
||||
|
||||
在测试环境中将 `ENABLE_LLM_CACHE_FOR_EXTRACT` 设置为 true 以减少 LLM 调用成本是很常见的做法。
|
||||
|
||||
### 支持的存储类型
|
||||
|
||||
LightRAG 使用 4 种类型的存储用于不同目的：
|
||||
|
||||
* KV_STORAGE：llm 响应缓存、文本块、文档信息
|
||||
* VECTOR_STORAGE：实体向量、关系向量、块向量
|
||||
* GRAPH_STORAGE：实体关系图
|
||||
* DOC_STATUS_STORAGE：文档索引状态
|
||||
|
||||
每种存储类型都有多种存储实现方式。LightRAG Server默认的存储实现为内存数据库，数据通过文件持久化保存到WORKING_DIR目录。LightRAG还支持PostgreSQL、MongoDB、FAISS、Milvus、Qdrant、Neo4j、Memgraph和Redis等存储实现方式。详细的存储支持方式请参考根目录下的`README.md`文件中关于存储的相关内容。
|
||||
|
||||
您可以通过环境变量选择存储实现。例如，在首次启动 API 服务器之前，您可以将以下环境变量设置为特定的存储实现名称：
|
||||
|
||||
```
|
||||
LIGHTRAG_KV_STORAGE=PGKVStorage
|
||||
LIGHTRAG_VECTOR_STORAGE=PGVectorStorage
|
||||
LIGHTRAG_GRAPH_STORAGE=PGGraphStorage
|
||||
LIGHTRAG_DOC_STATUS_STORAGE=PGDocStatusStorage
|
||||
```
|
||||
|
||||
在向 LightRAG 添加文档后，您不能更改存储实现选择。目前尚不支持从一个存储实现迁移到另一个存储实现。更多配置信息请阅读示例 `env.exampl`e文件。
|
||||
|
||||
### 在不同存储类型之间迁移LLM缓存
|
||||
|
||||
当LightRAG更换存储实现方式的时候，可以LLM缓存从就的存储迁移到新的存储。先以后在新的存储上重新上传文件时，将利用利用原有存储的LLM缓存大幅度加快文件处理的速度。LLM缓存迁移工具的使用方法请参考[README_MIGRATE_LLM_CACHE.md](../tools/README_MIGRATE_LLM_CACHE.md)
|
||||
|
||||
### LightRag API 服务器命令行选项
|
||||
|
||||
| 参数 | 默认值 | 描述 |
|
||||
|-----------|---------|-------------|
|
||||
| --host | 0.0.0.0 | 服务器主机 |
|
||||
| --port | 9621 | 服务器端口 |
|
||||
| --working-dir | ./rag_storage | RAG 存储的工作目录 |
|
||||
| --input-dir | ./inputs | 包含输入文档的目录 |
|
||||
| --max-async | 4 | 最大异步操作数 |
|
||||
| --log-level | INFO | 日志级别（DEBUG、INFO、WARNING、ERROR、CRITICAL） |
|
||||
| --verbose | - | 详细调试输出（True、False） |
|
||||
| --key | None | 用于认证的 API 密钥。保护 lightrag 服务器免受未授权访问 |
|
||||
| --ssl | False | 启用 HTTPS |
|
||||
| --ssl-certfile | None | SSL 证书文件路径（如果启用 --ssl 则必需） |
|
||||
| --ssl-keyfile | None | SSL 私钥文件路径（如果启用 --ssl 则必需） |
|
||||
| --llm-binding | ollama | LLM 绑定类型（lollms、ollama、openai、openai-ollama、azure_openai、aws_bedrock） |
|
||||
| --embedding-binding | ollama | 嵌入绑定类型（lollms、ollama、openai、azure_openai、aws_bedrock） |
|
||||
|
||||
### Reranking 配置
|
||||
|
||||
Reranking 查询召回的块可以显著提高检索质量，它通过基于优化的相关性评分模型对文档重新排序。LightRAG 目前支持以下 rerank 提供商：
|
||||
|
||||
- **Cohere / vLLM**：提供与 Cohere AI 的 `v2/rerank` 端点的完整 API 集成。由于 vLLM 提供了与 Cohere 兼容的 reranker API，因此也支持所有通过 vLLM 部署的 reranker 模型。
|
||||
- **Jina AI**：提供与所有 Jina rerank 模型的完全实现兼容性。
|
||||
- **阿里云**：具有旨在支持阿里云 rerank API 格式的自定义实现。
|
||||
|
||||
Rerank 提供商通过 `.env` 文件进行配置。以下是使用 vLLM 本地部署的 rerank 模型的示例配置：
|
||||
|
||||
```
|
||||
RERANK_BINDING=cohere
|
||||
RERANK_MODEL=BAAI/bge-reranker-v2-m3
|
||||
RERANK_BINDING_HOST=http://localhost:8000/v1/rerank
|
||||
RERANK_BINDING_API_KEY=your_rerank_api_key_here
|
||||
```
|
||||
|
||||
以下是使用阿里云提供的 Reranker 服务的示例配置：
|
||||
|
||||
```
|
||||
RERANK_BINDING=aliyun
|
||||
RERANK_MODEL=gte-rerank-v2
|
||||
RERANK_BINDING_HOST=https://dashscope.aliyuncs.com/api/v1/services/rerank/text-rerank/text-rerank
|
||||
RERANK_BINDING_API_KEY=your_rerank_api_key_here
|
||||
```
|
||||
|
||||
有关完整的 reranker 配置示例，请参阅 `env.example` 文件。
|
||||
|
||||
### 启用 Reranking
|
||||
|
||||
可以按查询启用或禁用 Reranking。
|
||||
|
||||
`/query` 和 `/query/stream` API 端点包含一个 `enable_rerank` 参数，默认设置为 `true`，用于控制当前查询是否激活 reranking。要将 `enable_rerank` 参数的默认值更改为 `false`，请设置以下环境变量：
|
||||
|
||||
```
|
||||
RERANK_BY_DEFAULT=False
|
||||
```
|
||||
|
||||
### .env 文件示例
|
||||
|
||||
```bash
|
||||
### Server Configuration
|
||||
# HOST=0.0.0.0
|
||||
PORT=9621
|
||||
WORKERS=2
|
||||
|
||||
### Settings for document indexing
|
||||
ENABLE_LLM_CACHE_FOR_EXTRACT=true
|
||||
SUMMARY_LANGUAGE=Chinese
|
||||
MAX_PARALLEL_INSERT=2
|
||||
|
||||
### LLM Configuration (Use valid host. For local services installed with docker, you can use host.docker.internal)
|
||||
TIMEOUT=150
|
||||
MAX_ASYNC=4
|
||||
|
||||
LLM_BINDING=openai
|
||||
LLM_MODEL=gpt-4o-mini
|
||||
LLM_BINDING_HOST=https://api.openai.com/v1
|
||||
LLM_BINDING_API_KEY=your-api-key
|
||||
|
||||
### Embedding Configuration (Use valid host. For local services installed with docker, you can use host.docker.internal)
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
|
||||
### For JWT Auth
|
||||
# AUTH_ACCOUNTS='admin:admin123,user1:pass456'
|
||||
# TOKEN_SECRET=your-key-for-LightRAG-API-Server-xxx
|
||||
# TOKEN_EXPIRE_HOURS=48
|
||||
|
||||
# LIGHTRAG_API_KEY=your-secure-api-key-here-123
|
||||
# WHITELIST_PATHS=/api/*
|
||||
# WHITELIST_PATHS=/health,/api/*
|
||||
```
|
||||
|
||||
#### 使用 ollama 默认本地服务器作为 llm 和嵌入后端运行 Lightrag 服务器
|
||||
|
||||
Ollama 是 llm 和嵌入的默认后端，因此默认情况下您可以不带参数运行 lightrag-server，将使用默认值。确保已安装 ollama 并且正在运行，且默认模型已安装在 ollama 上。
|
||||
|
||||
```bash
|
||||
# 使用 ollama 运行 lightrag，llm 使用 mistral-nemo:latest，嵌入使用 bge-m3:latest
|
||||
lightrag-server
|
||||
|
||||
# 使用认证密钥
|
||||
lightrag-server --key my-key
|
||||
```
|
||||
|
||||
#### 使用 lollms 默认本地服务器作为 llm 和嵌入后端运行 Lightrag 服务器
|
||||
|
||||
```bash
|
||||
# 使用 lollms 运行 lightrag，llm 使用 mistral-nemo:latest，嵌入使用 bge-m3:latest
|
||||
# 在 .env 或 config.ini 中配置 LLM_BINDING=lollms 和 EMBEDDING_BINDING=lollms
|
||||
lightrag-server
|
||||
|
||||
# 使用认证密钥
|
||||
lightrag-server --key my-key
|
||||
```
|
||||
|
||||
#### 使用 openai 服务器作为 llm 和嵌入后端运行 Lightrag 服务器
|
||||
|
||||
```bash
|
||||
# 使用 openai 运行 lightrag，llm 使用 GPT-4o-mini，嵌入使用 text-embedding-3-small
|
||||
# 在 .env 或 config.ini 中配置：
|
||||
# LLM_BINDING=openai
|
||||
# LLM_MODEL=GPT-4o-mini
|
||||
# EMBEDDING_BINDING=openai
|
||||
# EMBEDDING_MODEL=text-embedding-3-small
|
||||
lightrag-server
|
||||
|
||||
# 使用认证密钥
|
||||
lightrag-server --key my-key
|
||||
```
|
||||
|
||||
#### 使用 azure openai 服务器作为 llm 和嵌入后端运行 Lightrag 服务器
|
||||
|
||||
```bash
|
||||
# 使用 azure_openai 运行 lightrag
|
||||
# 在 .env 或 config.ini 中配置：
|
||||
# LLM_BINDING=azure_openai
|
||||
# LLM_MODEL=your-model
|
||||
# EMBEDDING_BINDING=azure_openai
|
||||
# EMBEDDING_MODEL=your-embedding-model
|
||||
lightrag-server
|
||||
|
||||
# 使用认证密钥
|
||||
lightrag-server --key my-key
|
||||
```
|
||||
|
||||
**重要说明：**
|
||||
- 对于 LoLLMs：确保指定的模型已安装在您的 LoLLMs 实例中
|
||||
- 对于 Ollama：确保指定的模型已安装在您的 Ollama 实例中
|
||||
- 对于 OpenAI：确保您已设置 OPENAI_API_KEY 环境变量
|
||||
- 对于 Azure OpenAI：按照先决条件部分所述构建和配置您的服务器
|
||||
|
||||
要获取任何服务器的帮助，使用 --help 标志：
|
||||
```bash
|
||||
lightrag-server --help
|
||||
```
|
||||
|
||||
注意：如果您不需要 API 功能，可以使用以下命令安装不带 API 支持的基本包：
|
||||
```bash
|
||||
pip install lightrag-hku
|
||||
```
|
||||
|
||||
## 文档和块处理逻辑说明
|
||||
|
||||
LightRAG 中的文档处理流程有些复杂，分为两个主要阶段：提取阶段（实体和关系提取）和合并阶段（实体和关系合并）。有两个关键参数控制流程并发性：并行处理的最大文件数（`MAX_PARALLEL_INSERT`）和最大并发 LLM 请求数（`MAX_ASYNC`）。工作流程描述如下：
|
||||
|
||||
1. `MAX_ASYNC` 限制系统中并发 LLM 请求的总数，包括查询、提取和合并的请求。LLM 请求具有不同的优先级：查询操作优先级最高，其次是合并，然后是提取。
|
||||
2. `MAX_PARALLEL_INSERT` 控制提取阶段并行处理的文件数量。`MAX_PARALLEL_INSERT`建议设置为2～10之间，通常设置为 `MAX_ASYNC/3`，设置太大会导致合并阶段不同文档之间实体和关系重名的机会增大，降低合并阶段的效率。
|
||||
3. 在单个文件中，来自不同文本块的实体和关系提取是并发处理的，并发度由 `MAX_ASYNC` 设置。只有在处理完 `MAX_ASYNC` 个文本块后，系统才会继续处理同一文件中的下一批文本块。
|
||||
4. 当一个文件完成实体和关系提后，将进入实体和关系合并阶段。这一阶段也会并发处理多个实体和关系，其并发度同样是由 `MAX_ASYNC` 控制。
|
||||
5. 合并阶段的 LLM 请求的优先级别高于提取阶段，目的是让进入合并阶段的文件尽快完成处理，并让处理结果尽快更新到向量数据库中。
|
||||
6. 为防止竞争条件，合并阶段会避免并发处理同一个实体或关系，当多个文件中都涉及同一个实体或关系需要合并的时候他们会串行执行。
|
||||
7. 每个文件在流程中被视为一个原子处理单元。只有当其所有文本块都完成提取和合并后，文件才会被标记为成功处理。如果在处理过程中发生任何错误，整个文件将被标记为失败，并且必须重新处理。
|
||||
8. 当由于错误而重新处理文件时，由于 LLM 缓存，先前处理的文本块可以快速跳过。尽管 LLM 缓存在合并阶段也会被利用，但合并顺序的不一致可能会限制其在此阶段的有效性。
|
||||
9. 如果在提取过程中发生错误，系统不会保留任何中间结果。如果在合并过程中发生错误，已合并的实体和关系可能会被保留；当重新处理同一文件时，重新提取的实体和关系将与现有实体和关系合并，而不会影响查询结果。
|
||||
10. 在合并阶段结束时，所有实体和关系数据都会在向量数据库中更新。如果此时发生错误，某些更新可能会被保留。但是，下一次处理尝试将覆盖先前结果，确保成功重新处理的文件不会影响未来查询结果的完整性。
|
||||
|
||||
大型文件应分割成较小的片段以启用增量处理。可以通过在 Web UI 上按“扫描”按钮来启动失败文件的重新处理。
|
||||
|
||||
## API 端点
|
||||
|
||||
所有服务器（LoLLMs、Ollama、OpenAI 和 Azure OpenAI）都为 RAG 功能提供相同的 REST API 端点。当 API 服务器运行时，访问：
|
||||
|
||||
- Swagger UI：http://localhost:9621/docs
|
||||
- ReDoc：http://localhost:9621/redoc
|
||||
|
||||
您可以使用提供的 curl 命令或通过 Swagger UI 界面测试 API 端点。确保：
|
||||
|
||||
1. 启动适当的后端服务（LoLLMs、Ollama 或 OpenAI）
|
||||
2. 启动 RAG 服务器
|
||||
3. 使用文档管理端点上传一些文档
|
||||
4. 使用查询端点查询系统
|
||||
5. 如果在输入目录中放入新文件，触发文档扫描
|
||||
|
||||
## 异步文档索引与进度跟踪
|
||||
|
||||
LightRAG采用异步文档索引机制，便于前端监控和查询文档处理进度。用户通过指定端点上传文件或插入文本时，系统将返回唯一的跟踪ID，以便实时监控处理进度。
|
||||
|
||||
**支持生成跟踪ID的API端点：**
|
||||
* `/documents/upload`
|
||||
* `/documents/text`
|
||||
* `/documents/texts`
|
||||
|
||||
**文档处理状态查询端点：**
|
||||
* `/track_status/{track_id}`
|
||||
|
||||
该端点提供全面的状态信息，包括：
|
||||
* 文档处理状态（待处理/处理中/已处理/失败）
|
||||
* 内容摘要和元数据
|
||||
* 处理失败时的错误信息
|
||||
* 创建和更新时间戳
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 374 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 357 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 530 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 381 KiB |
@@ -0,0 +1,623 @@
|
||||
# LightRAG Server and WebUI
|
||||
|
||||
The LightRAG Server is designed to provide a Web UI and API support. The Web UI facilitates document indexing, knowledge graph exploration, and a simple RAG query interface. LightRAG Server also provides an Ollama-compatible interface, aiming to emulate LightRAG as an Ollama chat model. This allows AI chat bots, such as Open WebUI, to access LightRAG easily.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Installation
|
||||
|
||||
* Install from PyPI
|
||||
|
||||
```bash
|
||||
# Using uv (recommended)
|
||||
uv pip install "lightrag-hku[api]"
|
||||
|
||||
# Or using pip
|
||||
# pip install "lightrag-hku[api]"
|
||||
```
|
||||
|
||||
* Installation from Source
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/HKUDS/lightrag.git
|
||||
|
||||
# Change to the repository directory
|
||||
cd lightrag
|
||||
|
||||
# Using uv (recommended)
|
||||
# Note: uv sync automatically creates a virtual environment in .venv/
|
||||
uv sync --extra api
|
||||
source .venv/bin/activate # Activate the virtual environment (Linux/macOS)
|
||||
# Or on Windows: .venv\Scripts\activate
|
||||
|
||||
# Or using pip with virtual environment
|
||||
# python -m venv .venv
|
||||
# source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
# pip install -e ".[api]"
|
||||
|
||||
# Build front-end artifacts
|
||||
cd lightrag_webui
|
||||
bun install --frozen-lockfile
|
||||
bun run build
|
||||
cd ..
|
||||
```
|
||||
|
||||
### Before Starting LightRAG Server
|
||||
|
||||
LightRAG necessitates the integration of both an LLM (Large Language Model) and an Embedding Model to effectively execute document indexing and querying operations. Prior to the initial deployment of the LightRAG server, it is essential to configure the settings for both the LLM and the Embedding Model. LightRAG supports binding to various LLM/Embedding backends:
|
||||
|
||||
* ollama
|
||||
* lollms
|
||||
* openai or openai compatible
|
||||
* azure_openai
|
||||
* aws_bedrock
|
||||
* gemini
|
||||
|
||||
It is recommended to use environment variables to configure the LightRAG Server. There is an example environment variable file named `env.example` in the root directory of the project. Please copy this file to the startup directory and rename it to `.env`. After that, you can modify the parameters related to the LLM and Embedding models in the `.env` file. It is important to note that the LightRAG Server will load the environment variables from `.env` into the system environment variables each time it starts. **LightRAG Server will prioritize the settings in the system environment variables to .env file**.
|
||||
|
||||
> Since VS Code with the Python extension may automatically load the .env file in the integrated terminal, please open a new terminal session after each modification to the .env file.
|
||||
|
||||
Here are some examples of common settings for LLM and Embedding models:
|
||||
|
||||
* OpenAI LLM + Ollama Embedding:
|
||||
|
||||
```
|
||||
LLM_BINDING=openai
|
||||
LLM_MODEL=gpt-4o
|
||||
LLM_BINDING_HOST=https://api.openai.com/v1
|
||||
LLM_BINDING_API_KEY=your_api_key
|
||||
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
# EMBEDDING_BINDING_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
> When targeting Google Gemini, set `LLM_BINDING=gemini`, choose a model such as `LLM_MODEL=gemini-flash-latest`, and provide your Gemini key via `LLM_BINDING_API_KEY` (or `GEMINI_API_KEY`).
|
||||
|
||||
* Ollama LLM + Ollama Embedding:
|
||||
|
||||
```
|
||||
LLM_BINDING=ollama
|
||||
LLM_MODEL=mistral-nemo:latest
|
||||
LLM_BINDING_HOST=http://localhost:11434
|
||||
# LLM_BINDING_API_KEY=your_api_key
|
||||
### Ollama Server context length (Must be larger than MAX_TOTAL_TOKENS+2000)
|
||||
OLLAMA_LLM_NUM_CTX=16384
|
||||
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
# EMBEDDING_BINDING_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
> **Important Note**: The Embedding model must be determined before document indexing, and the same model must be used during the document query phase. For certain storage solutions (e.g., PostgreSQL), the vector dimension must be defined upon initial table creation. Therefore, when changing embedding models, it is necessary to delete the existing vector-related tables and allow LightRAG to recreate them with the new dimensions.
|
||||
|
||||
### Starting LightRAG Server
|
||||
|
||||
The LightRAG Server supports two operational modes:
|
||||
* The simple and efficient Uvicorn mode:
|
||||
|
||||
```
|
||||
lightrag-server
|
||||
```
|
||||
* The multiprocess Gunicorn + Uvicorn mode (production mode, not supported on Windows environments):
|
||||
|
||||
```
|
||||
lightrag-gunicorn --workers 4
|
||||
```
|
||||
|
||||
When starting LightRAG, the current working directory must contain the `.env` configuration file. **It is intentionally designed that the `.env` file must be placed in the startup directory**. The purpose of this is to allow users to launch multiple LightRAG instances simultaneously and configure different `.env` files for different instances. **After modifying the `.env` file, you need to reopen the terminal for the new settings to take effect.** This is because each time LightRAG Server starts, it loads the environment variables from the `.env` file into the system environment variables, and system environment variables have higher precedence.
|
||||
|
||||
During startup, configurations in the `.env` file can be overridden by command-line parameters. Common command-line parameters include:
|
||||
|
||||
- `--host`: Server listening address (default: 0.0.0.0)
|
||||
- `--port`: Server listening port (default: 9621)
|
||||
- `--timeout`: LLM request timeout (default: 150 seconds)
|
||||
- `--log-level`: Log level (default: INFO)
|
||||
- `--working-dir`: Database persistence directory (default: ./rag_storage)
|
||||
- `--input-dir`: Directory for uploaded files (default: ./inputs)
|
||||
- `--workspace`: Workspace name, used to logically isolate data between multiple LightRAG instances (default: empty)
|
||||
|
||||
### Launching LightRAG Server with Docker
|
||||
|
||||
Using Docker Compose is the most convenient way to deploy and run the LightRAG Server.
|
||||
|
||||
* Create a project directory.
|
||||
|
||||
* Copy the `docker-compose.yml` file from the LightRAG repository into your project directory.
|
||||
|
||||
* Prepare the `.env` file: Duplicate the sample file [`env.example`](https://ai.znipower.com:5013/c/env.example)to create a customized `.env` file, and configure the LLM and embedding parameters according to your specific requirements.
|
||||
|
||||
* Start the LightRAG Server with the following command:
|
||||
|
||||
```shell
|
||||
docker compose up
|
||||
# If you want the program to run in the background after startup, add the -d parameter at the end of the command.
|
||||
```
|
||||
|
||||
You can get the official docker compose file from here: [docker-compose.yml](https://raw.githubusercontent.com/HKUDS/LightRAG/refs/heads/main/docker-compose.yml). For historical versions of LightRAG docker images, visit this link: [LightRAG Docker Images](https://github.com/HKUDS/LightRAG/pkgs/container/lightrag). For more details about docker deployment, please refer to [DockerDeployment.md](./../../docs/DockerDeployment.md).
|
||||
|
||||
### Offline Deployment
|
||||
|
||||
Official LightRAG Docker images are fully compatible with offline or air-gapped environments. If you want to build up you own offline enviroment, please refer to [Offline Deployment Guide](./../../docs/OfflineDeployment.md).
|
||||
|
||||
### Starting Multiple LightRAG Instances
|
||||
|
||||
There are two ways to start multiple LightRAG instances. The first way is to configure a completely independent working environment for each instance. This requires creating a separate working directory for each instance and placing a dedicated `.env` configuration file in that directory. The server listening ports in the configuration files of different instances cannot be the same. Then, you can start the service by running `lightrag-server` in the working directory.
|
||||
|
||||
The second way is for all instances to share the same set of `.env` configuration files, and then use command-line arguments to specify different server listening ports and workspaces for each instance. You can start multiple LightRAG instances in the same working directory with different command-line arguments. For example:
|
||||
|
||||
```
|
||||
# Start instance 1
|
||||
lightrag-server --port 9621 --workspace space1
|
||||
|
||||
# Start instance 2
|
||||
lightrag-server --port 9622 --workspace space2
|
||||
```
|
||||
|
||||
The purpose of a workspace is to achieve data isolation between different instances. Therefore, the `workspace` parameter must be different for different instances; otherwise, it will lead to data confusion and corruption.
|
||||
|
||||
When launching multiple LightRAG instances via Docker Compose, simply specify unique `WORKSPACE` and `PORT` environment variables for each container within your `docker-compose.yml`. Even if all instances share a common `.env` file, the container-specific environment variables defined in Compose will take precedence, ensuring independent configurations for each instance.
|
||||
|
||||
### Data Isolation Between LightRAG Instances
|
||||
|
||||
Configuring an independent working directory and a dedicated `.env` configuration file for each instance can generally ensure that locally persisted files in the in-memory database are saved in their respective working directories, achieving data isolation. By default, LightRAG uses all in-memory databases, and this method of data isolation is sufficient. However, if you are using an external database, and different instances access the same database instance, you need to use workspaces to achieve data isolation; otherwise, the data of different instances will conflict and be destroyed.
|
||||
|
||||
The command-line `workspace` argument and the `WORKSPACE` environment variable in the `.env` file can both be used to specify the workspace name for the current instance, with the command-line argument having higher priority. Here is how workspaces are implemented for different types of storage:
|
||||
|
||||
- **For local file-based databases, data isolation is achieved through workspace subdirectories:** `JsonKVStorage`, `JsonDocStatusStorage`, `NetworkXStorage`, `NanoVectorDBStorage`, `FaissVectorDBStorage`.
|
||||
- **For databases that store data in collections, it's done by adding a workspace prefix to the collection name:** `RedisKVStorage`, `RedisDocStatusStorage`, `MilvusVectorDBStorage`, `MongoKVStorage`, `MongoDocStatusStorage`, `MongoVectorDBStorage`, `MongoGraphStorage`, `PGGraphStorage`.
|
||||
- **For Qdrant vector database, data isolation is achieved through payload-based partitioning (Qdrant's recommended multitenancy approach):** `QdrantVectorDBStorage` uses shared collections with payload filtering for unlimited workspace scalability.
|
||||
- **For relational databases, data isolation is achieved by adding a `workspace` field to the tables for logical data separation:** `PGKVStorage`, `PGVectorStorage`, `PGDocStatusStorage`.
|
||||
- **For graph databases, logical data isolation is achieved through labels:** `Neo4JStorage`, `MemgraphStorage`
|
||||
|
||||
To maintain compatibility with legacy data, the default workspace for PostgreSQL is `default` and for Neo4j is `base` when no workspace is configured. For all external storages, the system provides dedicated workspace environment variables to override the common `WORKSPACE` environment variable configuration. These storage-specific workspace environment variables are: `REDIS_WORKSPACE`, `MILVUS_WORKSPACE`, `QDRANT_WORKSPACE`, `MONGODB_WORKSPACE`, `POSTGRES_WORKSPACE`, `NEO4J_WORKSPACE`, `MEMGRAPH_WORKSPACE`.
|
||||
|
||||
### Multiple workers for Gunicorn + Uvicorn
|
||||
|
||||
The LightRAG Server can operate in the `Gunicorn + Uvicorn` preload mode. Gunicorn's multiple worker (multiprocess) capability prevents document indexing tasks from blocking RAG queries. Using CPU-exhaustive document extraction tools, such as docling, can lead to the entire system being blocked in pure Uvicorn mode.
|
||||
|
||||
Though LightRAG Server uses one worker to process the document indexing pipeline, with the async task support of Uvicorn, multiple files can be processed in parallel. The bottleneck of document indexing speed mainly lies with the LLM. If your LLM supports high concurrency, you can accelerate document indexing by increasing the concurrency level of the LLM. Below are several environment variables related to concurrent processing, along with their default values:
|
||||
|
||||
```
|
||||
### Number of worker processes, not greater than (2 x number_of_cores) + 1
|
||||
WORKERS=2
|
||||
### Number of parallel files to process in one batch
|
||||
MAX_PARALLEL_INSERT=2
|
||||
### Max concurrent requests to the LLM
|
||||
MAX_ASYNC=4
|
||||
```
|
||||
|
||||
### Install LightRAG as a Linux Service
|
||||
|
||||
Create your service file `lightrag.service` from the sample file: `lightrag.service.example`. Modify the start options the service file:
|
||||
|
||||
```text
|
||||
# Set Enviroment to your Python virtual enviroment
|
||||
Environment="PATH=/home/netman/lightrag-xyj/venv/bin"
|
||||
WorkingDirectory=/home/netman/lightrag-xyj
|
||||
# ExecStart=/home/netman/lightrag-xyj/venv/bin/lightrag-server
|
||||
ExecStart=/home/netman/lightrag-xyj/venv/bin/lightrag-gunicorn
|
||||
|
||||
```
|
||||
|
||||
> The ExecStart command must be either `lightrag-gunicorn` or `lightrag-server`; no wrapper scripts are allowed. This is because service termination requires the main process to be one of these two executables.
|
||||
|
||||
Install LightRAG service. If your system is Ubuntu, the following commands will work:
|
||||
|
||||
```shell
|
||||
sudo cp lightrag.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl start lightrag.service
|
||||
sudo systemctl status lightrag.service
|
||||
sudo systemctl enable lightrag.service
|
||||
```
|
||||
|
||||
## Ollama Emulation
|
||||
|
||||
We provide Ollama-compatible interfaces for LightRAG, aiming to emulate LightRAG as an Ollama chat model. This allows AI chat frontends supporting Ollama, such as Open WebUI, to access LightRAG easily.
|
||||
|
||||
### Connect Open WebUI to LightRAG
|
||||
|
||||
After starting the lightrag-server, you can add an Ollama-type connection in the Open WebUI admin panel. And then a model named `lightrag:latest` will appear in Open WebUI's model management interface. Users can then send queries to LightRAG through the chat interface. You should install LightRAG as a service for this use case.
|
||||
|
||||
Open WebUI uses an LLM to do the session title and session keyword generation task. So the Ollama chat completion API detects and forwards OpenWebUI session-related requests directly to the underlying LLM. Screenshot from Open WebUI:
|
||||
|
||||
|
||||
|
||||
### Choose Query mode in chat
|
||||
|
||||
The default query mode is `hybrid` if you send a message (query) from the Ollama interface of LightRAG. You can select query mode by sending a message with a query prefix.
|
||||
|
||||
A query prefix in the query string can determine which LightRAG query mode is used to generate the response for the query. The supported prefixes include:
|
||||
|
||||
```
|
||||
/local
|
||||
/global
|
||||
/hybrid
|
||||
/naive
|
||||
/mix
|
||||
|
||||
/bypass
|
||||
/context
|
||||
/localcontext
|
||||
/globalcontext
|
||||
/hybridcontext
|
||||
/naivecontext
|
||||
/mixcontext
|
||||
```
|
||||
|
||||
For example, the chat message `/mix What's LightRAG?` will trigger a mix mode query for LightRAG. A chat message without a query prefix will trigger a hybrid mode query by default.
|
||||
|
||||
`/bypass` is not a LightRAG query mode; it will tell the API Server to pass the query directly to the underlying LLM, including the chat history. So the user can use the LLM to answer questions based on the chat history. If you are using Open WebUI as a front end, you can just switch the model to a normal LLM instead of using the `/bypass` prefix.
|
||||
|
||||
`/context` is also not a LightRAG query mode; it will tell LightRAG to return only the context information prepared for the LLM. You can check the context if it's what you want, or process the context by yourself.
|
||||
|
||||
### Add user prompt in chat
|
||||
|
||||
When using LightRAG for content queries, avoid combining the search process with unrelated output processing, as this significantly impacts query effectiveness. User prompt is specifically designed to address this issue — it does not participate in the RAG retrieval phase, but rather guides the LLM on how to process the retrieved results after the query is completed. We can append square brackets to the query prefix to provide the LLM with the user prompt:
|
||||
|
||||
```
|
||||
/[Use mermaid format for diagrams] Please draw a character relationship diagram for Scrooge
|
||||
/mix[Use mermaid format for diagrams] Please draw a character relationship diagram for Scrooge
|
||||
```
|
||||
|
||||
## API Key and Authentication
|
||||
|
||||
By default, the LightRAG Server can be accessed without any authentication. We can configure the server with an API Key or account credentials to secure it.
|
||||
|
||||
* API Key:
|
||||
|
||||
```
|
||||
LIGHTRAG_API_KEY=your-secure-api-key-here
|
||||
WHITELIST_PATHS=/health,/api/*
|
||||
```
|
||||
|
||||
> Health check and Ollama emulation endpoints are excluded from API Key check by default. For security reasons, remove `/api/*` from `WHITELIST_PATHS` if the Ollama service is not required.
|
||||
|
||||
The API key is passed using the request header `X-API-Key`. Below is an example of accessing the LightRAG Server via API:
|
||||
|
||||
```
|
||||
curl -X 'POST' \
|
||||
'http://localhost:9621/documents/scan' \
|
||||
-H 'accept: application/json' \
|
||||
-H 'X-API-Key: your-secure-api-key-here-123' \
|
||||
-d ''
|
||||
```
|
||||
|
||||
* Account credentials (the Web UI requires login before access can be granted):
|
||||
|
||||
LightRAG API Server implements JWT-based authentication using the HS256 algorithm. To enable secure access control, the following environment variables are required:
|
||||
|
||||
```bash
|
||||
# For jwt auth
|
||||
AUTH_ACCOUNTS='admin:admin123,user1:pass456'
|
||||
TOKEN_SECRET='your-key'
|
||||
TOKEN_EXPIRE_HOURS=4
|
||||
```
|
||||
|
||||
> Currently, only the configuration of an administrator account and password is supported. A comprehensive account system is yet to be developed and implemented.
|
||||
|
||||
If Account credentials are not configured, the Web UI will access the system as a Guest. Therefore, even if only an API Key is configured, all APIs can still be accessed through the Guest account, which remains insecure. Hence, to safeguard the API, it is necessary to configure both authentication methods simultaneously.
|
||||
|
||||
## For Azure OpenAI Backend
|
||||
|
||||
Azure OpenAI API can be created using the following commands in Azure CLI (you need to install Azure CLI first from [https://docs.microsoft.com/en-us/cli/azure/install-azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)):
|
||||
|
||||
```bash
|
||||
# Change the resource group name, location, and OpenAI resource name as needed
|
||||
RESOURCE_GROUP_NAME=LightRAG
|
||||
LOCATION=swedencentral
|
||||
RESOURCE_NAME=LightRAG-OpenAI
|
||||
|
||||
az login
|
||||
az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
|
||||
az cognitiveservices account create --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --kind OpenAI --sku S0 --location swedencentral
|
||||
az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name gpt-4o --model-name gpt-4o --model-version "2024-08-06" --sku-capacity 100 --sku-name "Standard"
|
||||
az cognitiveservices account deployment create --resource-group $RESOURCE_GROUP_NAME --model-format OpenAI --name $RESOURCE_NAME --deployment-name text-embedding-3-large --model-name text-embedding-3-large --model-version "1" --sku-capacity 80 --sku-name "Standard"
|
||||
az cognitiveservices account show --name $RESOURCE_NAME --resource-group $RESOURCE_GROUP_NAME --query "properties.endpoint"
|
||||
az cognitiveservices account keys list --name $RESOURCE_NAME -g $RESOURCE_GROUP_NAME
|
||||
|
||||
```
|
||||
|
||||
The output of the last command will give you the endpoint and the key for the OpenAI API. You can use these values to set the environment variables in the `.env` file.
|
||||
|
||||
```
|
||||
# Azure OpenAI Configuration in .env:
|
||||
LLM_BINDING=azure_openai
|
||||
LLM_BINDING_HOST=your-azure-endpoint
|
||||
LLM_MODEL=your-model-deployment-name
|
||||
LLM_BINDING_API_KEY=your-azure-api-key
|
||||
### API version is optional, defaults to latest version
|
||||
AZURE_OPENAI_API_VERSION=2024-08-01-preview
|
||||
|
||||
### If using Azure OpenAI for embeddings
|
||||
EMBEDDING_BINDING=azure_openai
|
||||
EMBEDDING_MODEL=your-embedding-deployment-name
|
||||
```
|
||||
|
||||
## LightRAG Server Configuration in Detail
|
||||
|
||||
The API Server can be configured in three ways (highest priority first):
|
||||
|
||||
* Command line arguments
|
||||
* Environment variables or .env file
|
||||
* Config.ini (Only for storage configuration)
|
||||
|
||||
Most of the configurations come with default settings; check out the details in the sample file: `.env.example`. Data storage configuration can also be set by config.ini. A sample file `config.ini.example` is provided for your convenience.
|
||||
|
||||
### LLM and Embedding Backend Supported
|
||||
|
||||
LightRAG supports binding to various LLM/Embedding backends:
|
||||
|
||||
* ollama
|
||||
* openai (including openai compatible)
|
||||
* azure_openai
|
||||
* lollms
|
||||
* aws_bedrock
|
||||
|
||||
Use environment variables `LLM_BINDING` or CLI argument `--llm-binding` to select the LLM backend type. Use environment variables `EMBEDDING_BINDING` or CLI argument `--embedding-binding` to select the Embedding backend type.
|
||||
|
||||
For LLM and embedding configuration examples, please refer to the `env.example` file in the project's root directory. To view the complete list of configurable options for OpenAI and Ollama-compatible LLM interfaces, use the following commands:
|
||||
```
|
||||
lightrag-server --llm-binding openai --help
|
||||
lightrag-server --llm-binding ollama --help
|
||||
lightrag-server --embedding-binding ollama --help
|
||||
```
|
||||
|
||||
> Please use OpenAI-compatible method to access LLMs deployed by OpenRouter or vLLM/SGLang. You can pass additional parameters to OpenRouter or vLLM/SGLang through the `OPENAI_LLM_EXTRA_BODY` environment variable to disable reasoning mode or achieve other personalized controls.
|
||||
|
||||
Set the max_tokens to **prevent excessively long or endless output loop** during the entity relationship extraction phase for Large Language Model (LLM) responses. The purpose of setting max_tokens parameter is to truncate LLM output before timeouts occur, thereby preventing document extraction failures. This addresses issues where certain text blocks (e.g., tables or citations) containing numerous entities and relationships can lead to overly long or even endless loop outputs from LLMs. This setting is particularly crucial for locally deployed, smaller-parameter models. Max tokens value can be calculated by this formula: `LLM_TIMEOUT * llm_output_tokens/second` (i.e. `180s * 50 tokens/s = 9000`)
|
||||
|
||||
```
|
||||
# For vLLM/SGLang doployed models, or most of OpenAI compatible API provider
|
||||
OPENAI_LLM_MAX_TOKENS=9000
|
||||
|
||||
# For Ollama Deployed Modeles
|
||||
OLLAMA_LLM_NUM_PREDICT=9000
|
||||
|
||||
# For OpenAI o1-mini or newer modles
|
||||
OPENAI_LLM_MAX_COMPLETION_TOKENS=9000
|
||||
```
|
||||
|
||||
### Entity Extraction Configuration
|
||||
|
||||
* ENABLE_LLM_CACHE_FOR_EXTRACT: Enable LLM cache for entity extraction (default: true)
|
||||
|
||||
It's very common to set `ENABLE_LLM_CACHE_FOR_EXTRACT` to true for a test environment to reduce the cost of LLM calls.
|
||||
|
||||
### Storage Types Supported
|
||||
|
||||
LightRAG uses 4 types of storage for different purposes:
|
||||
|
||||
* KV_STORAGE: llm response cache, text chunks, document information
|
||||
* VECTOR_STORAGE: entities vectors, relation vectors, chunks vectors
|
||||
* GRAPH_STORAGE: entity relation graph
|
||||
* DOC_STATUS_STORAGE: document indexing status
|
||||
|
||||
LightRAG Server offers various storage implementations, with the default being an in-memory database that persists data to the WORKING_DIR directory. Additionally, LightRAG supports a wide range of storage solutions including PostgreSQL, MongoDB, FAISS, Milvus, Qdrant, Neo4j, Memgraph, and Redis. For detailed information on supported storage options, please refer to the storage section in the README.md file located in the root directory.
|
||||
|
||||
You can select the storage implementation by configuring environment variables. For instance, prior to the initial launch of the API server, you can set the following environment variable to specify your desired storage implementation:
|
||||
|
||||
```
|
||||
LIGHTRAG_KV_STORAGE=PGKVStorage
|
||||
LIGHTRAG_VECTOR_STORAGE=PGVectorStorage
|
||||
LIGHTRAG_GRAPH_STORAGE=PGGraphStorage
|
||||
LIGHTRAG_DOC_STATUS_STORAGE=PGDocStatusStorage
|
||||
```
|
||||
|
||||
You cannot change storage implementation selection after adding documents to LightRAG. Data migration from one storage implementation to another is not supported yet. For further information, please read the sample env file or config.ini file.
|
||||
|
||||
### LLM Cache Migration Between Storage Types
|
||||
|
||||
When switching the storage implementation in LightRAG, the LLM cache can be migrated from the existing storage to the new one. Subsequently, when re-uploading files to the new storage, the pre-existing LLM cache will significantly accelerate file processing. For detailed instructions on using the LLM cache migration tool, please refer to[README_MIGRATE_LLM_CACHE.md](../tools/README_MIGRATE_LLM_CACHE.md)
|
||||
|
||||
### LightRAG API Server Command Line Options
|
||||
|
||||
| Parameter | Default | Description |
|
||||
| --------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| --host | 0.0.0.0 | Server host |
|
||||
| --port | 9621 | Server port |
|
||||
| --working-dir | ./rag_storage | Working directory for RAG storage |
|
||||
| --input-dir | ./inputs | Directory containing input documents |
|
||||
| --max-async | 4 | Maximum number of async operations |
|
||||
| --log-level | INFO | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
|
||||
| --verbose | - | Verbose debug output (True, False) |
|
||||
| --key | None | API key for authentication. Protects the LightRAG server against unauthorized access |
|
||||
| --ssl | False | Enable HTTPS |
|
||||
| --ssl-certfile | None | Path to SSL certificate file (required if --ssl is enabled) |
|
||||
| --ssl-keyfile | None | Path to SSL private key file (required if --ssl is enabled) |
|
||||
| --llm-binding | ollama | LLM binding type (lollms, ollama, openai, openai-ollama, azure_openai, aws_bedrock) |
|
||||
| --embedding-binding | ollama | Embedding binding type (lollms, ollama, openai, azure_openai, aws_bedrock) |
|
||||
|
||||
### Reranking Configuration
|
||||
|
||||
Reranking query-recalled chunks can significantly enhance retrieval quality by re-ordering documents based on an optimized relevance scoring model. LightRAG currently supports the following rerank providers:
|
||||
|
||||
- **Cohere / vLLM**: Offers full API integration with Cohere AI's `v2/rerank` endpoint. As vLLM provides a Cohere-compatible reranker API, all reranker models deployed via vLLM are also supported.
|
||||
- **Jina AI**: Provides complete implementation compatibility with all Jina rerank models.
|
||||
- **Aliyun**: Features a custom implementation designed to support Aliyun's rerank API format.
|
||||
|
||||
The rerank provider is configured via the `.env` file. Below is an example configuration for a rerank model deployed locally using vLLM:
|
||||
|
||||
```
|
||||
RERANK_BINDING=cohere
|
||||
RERANK_MODEL=BAAI/bge-reranker-v2-m3
|
||||
RERANK_BINDING_HOST=http://localhost:8000/v1/rerank
|
||||
RERANK_BINDING_API_KEY=your_rerank_api_key_here
|
||||
```
|
||||
|
||||
Here is an example configuration for utilizing the Reranker service provided by Aliyun:
|
||||
|
||||
```
|
||||
RERANK_BINDING=aliyun
|
||||
RERANK_MODEL=gte-rerank-v2
|
||||
RERANK_BINDING_HOST=https://dashscope.aliyuncs.com/api/v1/services/rerank/text-rerank/text-rerank
|
||||
RERANK_BINDING_API_KEY=your_rerank_api_key_here
|
||||
```
|
||||
|
||||
For comprehensive reranker configuration examples, please refer to the `env.example` file.
|
||||
|
||||
### Enable Reranking
|
||||
|
||||
Reranking can be enabled or disabled on a per-query basis.
|
||||
|
||||
The `/query` and `/query/stream` API endpoints include an `enable_rerank` parameter, which is set to `true` by default, controlling whether reranking is active for the current query. To change the default value of the `enable_rerank` parameter to `false`, set the following environment variable:
|
||||
|
||||
```
|
||||
RERANK_BY_DEFAULT=False
|
||||
```
|
||||
|
||||
### Include Chunk Content in References
|
||||
|
||||
By default, the `/query` and `/query/stream` endpoints return references with only `reference_id` and `file_path`. For evaluation, debugging, or citation purposes, you can request the actual retrieved chunk content to be included in references.
|
||||
|
||||
The `include_chunk_content` parameter (default: `false`) controls whether the actual text content of retrieved chunks is included in the response references. This is particularly useful for:
|
||||
|
||||
- **RAG Evaluation**: Testing systems like RAGAS that need access to retrieved contexts
|
||||
- **Debugging**: Verifying what content was actually used to generate the answer
|
||||
- **Citation Display**: Showing users the exact text passages that support the response
|
||||
- **Transparency**: Providing full visibility into the RAG retrieval process
|
||||
|
||||
**Important**: The `content` field is an **array of strings**, where each string represents a chunk from the same file. A single file may correspond to multiple chunks, so the content is returned as a list to preserve chunk boundaries.
|
||||
|
||||
**Example API Request:**
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "What is LightRAG?",
|
||||
"mode": "mix",
|
||||
"include_references": true,
|
||||
"include_chunk_content": true
|
||||
}
|
||||
```
|
||||
|
||||
**Example Response (with chunk content):**
|
||||
|
||||
```json
|
||||
{
|
||||
"response": "LightRAG is a graph-based RAG system...",
|
||||
"references": [
|
||||
{
|
||||
"reference_id": "1",
|
||||
"file_path": "/documents/intro.md",
|
||||
"content": [
|
||||
"LightRAG is a retrieval-augmented generation system that combines knowledge graphs with vector similarity search...",
|
||||
"The system uses a dual-indexing approach with both vector embeddings and graph structures for enhanced retrieval..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"reference_id": "2",
|
||||
"file_path": "/documents/features.md",
|
||||
"content": [
|
||||
"The system provides multiple query modes including local, global, hybrid, and mix modes..."
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**:
|
||||
- This parameter only works when `include_references=true`. Setting `include_chunk_content=true` without including references has no effect.
|
||||
- **Breaking Change**: Prior versions returned `content` as a single concatenated string. Now it returns an array of strings to preserve individual chunk boundaries. If you need a single string, join the array elements with your preferred separator (e.g., `"\n\n".join(content)`).
|
||||
|
||||
### .env Examples
|
||||
|
||||
```bash
|
||||
### Server Configuration
|
||||
# HOST=0.0.0.0
|
||||
PORT=9621
|
||||
WORKERS=2
|
||||
|
||||
### Settings for document indexing
|
||||
ENABLE_LLM_CACHE_FOR_EXTRACT=true
|
||||
SUMMARY_LANGUAGE=Chinese
|
||||
MAX_PARALLEL_INSERT=2
|
||||
|
||||
### LLM Configuration (Use valid host. For local services installed with docker, you can use host.docker.internal)
|
||||
TIMEOUT=150
|
||||
MAX_ASYNC=4
|
||||
|
||||
LLM_BINDING=openai
|
||||
LLM_MODEL=gpt-4o-mini
|
||||
LLM_BINDING_HOST=https://api.openai.com/v1
|
||||
LLM_BINDING_API_KEY=your-api-key
|
||||
|
||||
### Embedding Configuration (Use valid host. For local services installed with docker, you can use host.docker.internal)
|
||||
# see also env.ollama-binding-options.example for fine tuning ollama
|
||||
EMBEDDING_MODEL=bge-m3:latest
|
||||
EMBEDDING_DIM=1024
|
||||
EMBEDDING_BINDING=ollama
|
||||
EMBEDDING_BINDING_HOST=http://localhost:11434
|
||||
|
||||
### For JWT Auth
|
||||
# AUTH_ACCOUNTS='admin:admin123,user1:pass456'
|
||||
# TOKEN_SECRET=your-key-for-LightRAG-API-Server-xxx
|
||||
# TOKEN_EXPIRE_HOURS=48
|
||||
|
||||
# LIGHTRAG_API_KEY=your-secure-api-key-here-123
|
||||
# WHITELIST_PATHS=/api/*
|
||||
# WHITELIST_PATHS=/health,/api/*
|
||||
|
||||
```
|
||||
|
||||
## Document and Chunk Processing Login Clarification
|
||||
|
||||
The document processing pipeline in LightRAG is somewhat complex and is divided into two primary stages: the Extraction stage (entity and relationship extraction) and the Merging stage (entity and relationship merging). There are two key parameters that control pipeline concurrency: the maximum number of files processed in parallel (MAX_PARALLEL_INSERT) and the maximum number of concurrent LLM requests (MAX_ASYNC). The workflow is described as follows:
|
||||
|
||||
1. MAX_ASYNC limits the total number of concurrent LLM requests in the system, including those for querying, extraction, and merging. LLM requests have different priorities: query operations have the highest priority, followed by merging, and then extraction.
|
||||
2. MAX_PARALLEL_INSERT controls the number of files processed in parallel during the extraction stage. For optimal performance, MAX_PARALLEL_INSERT is recommended to be set between 2 and 10, typically MAX_ASYNC/3. Setting this value too high can increase the likelihood of naming conflicts among entities and relationships across different documents during the merge phase, thereby reducing its overall efficiency.
|
||||
3. Within a single file, entity and relationship extractions from different text blocks are processed concurrently, with the degree of concurrency set by MAX_ASYNC. Only after MAX_ASYNC text blocks are processed will the system proceed to the next batch within the same file.
|
||||
4. When a file completes entity and relationship extraction, it enters the entity and relationship merging stage. This stage also processes multiple entities and relationships concurrently, with the concurrency level also controlled by `MAX_ASYNC`.
|
||||
5. LLM requests for the merging stage are prioritized over the extraction stage to ensure that files in the merging phase are processed quickly and their results are promptly updated in the vector database.
|
||||
6. To prevent race conditions, the merging stage avoids concurrent processing of the same entity or relationship. When multiple files involve the same entity or relationship that needs to be merged, they are processed serially.
|
||||
7. Each file is treated as an atomic processing unit in the pipeline. A file is marked as successfully processed only after all its text blocks have completed extraction and merging. If any error occurs during processing, the entire file is marked as failed and must be reprocessed.
|
||||
8. When a file is reprocessed due to errors, previously processed text blocks can be quickly skipped thanks to LLM caching. Although LLM cache is also utilized during the merging stage, inconsistencies in merging order may limit its effectiveness in this stage.
|
||||
9. If an error occurs during extraction, the system does not retain any intermediate results. If an error occurs during merging, already merged entities and relationships might be preserved; when the same file is reprocessed, re-extracted entities and relationships will be merged with the existing ones, without impacting the query results.
|
||||
10. At the end of the merging stage, all entity and relationship data are updated in the vector database. Should an error occur at this point, some updates may be retained. However, the next processing attempt will overwrite previous results, ensuring that successfully reprocessed files do not affect the integrity of future query results.
|
||||
|
||||
Large files should be divided into smaller segments to enable incremental processing. Reprocessing of failed files can be initiated by pressing the "Scan" button on the web UI.
|
||||
|
||||
## API Endpoints
|
||||
|
||||
All servers (LoLLMs, Ollama, OpenAI and Azure OpenAI) provide the same REST API endpoints for RAG functionality. When the API Server is running, visit:
|
||||
|
||||
- Swagger UI: http://localhost:9621/docs
|
||||
- ReDoc: http://localhost:9621/redoc
|
||||
|
||||
You can test the API endpoints using the provided curl commands or through the Swagger UI interface. Make sure to:
|
||||
|
||||
1. Start the appropriate backend service (LoLLMs, Ollama, or OpenAI)
|
||||
2. Start the RAG server
|
||||
3. Upload some documents using the document management endpoints
|
||||
4. Query the system using the query endpoints
|
||||
5. Trigger document scan if new files are put into the inputs directory
|
||||
|
||||
## Asynchronous Document Indexing with Progress Tracking
|
||||
|
||||
LightRAG implements asynchronous document indexing to enable frontend monitoring and querying of document processing progress. Upon uploading files or inserting text through designated endpoints, a unique Track ID is returned to facilitate real-time progress monitoring.
|
||||
|
||||
**API Endpoints Supporting Track ID Generation:**
|
||||
|
||||
* `/documents/upload`
|
||||
* `/documents/text`
|
||||
* `/documents/texts`
|
||||
|
||||
**Document Processing Status Query Endpoint:**
|
||||
* `/track_status/{track_id}`
|
||||
|
||||
This endpoint provides comprehensive status information including:
|
||||
* Document processing status (pending/processing/processed/failed)
|
||||
* Content summary and metadata
|
||||
* Error messages if processing failed
|
||||
* Timestamps for creation and updates
|
||||
@@ -0,0 +1 @@
|
||||
__api_version__ = "0254"
|
||||
@@ -0,0 +1,109 @@
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import jwt
|
||||
from dotenv import load_dotenv
|
||||
from fastapi import HTTPException, status
|
||||
from pydantic import BaseModel
|
||||
|
||||
from .config import global_args
|
||||
|
||||
# use the .env that is inside the current folder
|
||||
# allows to use different .env file for each lightrag instance
|
||||
# the OS environment variables take precedence over the .env file
|
||||
load_dotenv(dotenv_path=".env", override=False)
|
||||
|
||||
|
||||
class TokenPayload(BaseModel):
|
||||
sub: str # Username
|
||||
exp: datetime # Expiration time
|
||||
role: str = "user" # User role, default is regular user
|
||||
metadata: dict = {} # Additional metadata
|
||||
|
||||
|
||||
class AuthHandler:
|
||||
def __init__(self):
|
||||
self.secret = global_args.token_secret
|
||||
self.algorithm = global_args.jwt_algorithm
|
||||
self.expire_hours = global_args.token_expire_hours
|
||||
self.guest_expire_hours = global_args.guest_token_expire_hours
|
||||
self.accounts = {}
|
||||
auth_accounts = global_args.auth_accounts
|
||||
if auth_accounts:
|
||||
for account in auth_accounts.split(","):
|
||||
username, password = account.split(":", 1)
|
||||
self.accounts[username] = password
|
||||
|
||||
def create_token(
|
||||
self,
|
||||
username: str,
|
||||
role: str = "user",
|
||||
custom_expire_hours: int = None,
|
||||
metadata: dict = None,
|
||||
) -> str:
|
||||
"""
|
||||
Create JWT token
|
||||
|
||||
Args:
|
||||
username: Username
|
||||
role: User role, default is "user", guest is "guest"
|
||||
custom_expire_hours: Custom expiration time (hours), if None use default value
|
||||
metadata: Additional metadata
|
||||
|
||||
Returns:
|
||||
str: Encoded JWT token
|
||||
"""
|
||||
# Choose default expiration time based on role
|
||||
if custom_expire_hours is None:
|
||||
if role == "guest":
|
||||
expire_hours = self.guest_expire_hours
|
||||
else:
|
||||
expire_hours = self.expire_hours
|
||||
else:
|
||||
expire_hours = custom_expire_hours
|
||||
|
||||
expire = datetime.utcnow() + timedelta(hours=expire_hours)
|
||||
|
||||
# Create payload
|
||||
payload = TokenPayload(
|
||||
sub=username, exp=expire, role=role, metadata=metadata or {}
|
||||
)
|
||||
|
||||
return jwt.encode(payload.dict(), self.secret, algorithm=self.algorithm)
|
||||
|
||||
def validate_token(self, token: str) -> dict:
|
||||
"""
|
||||
Validate JWT token
|
||||
|
||||
Args:
|
||||
token: JWT token
|
||||
|
||||
Returns:
|
||||
dict: Dictionary containing user information
|
||||
|
||||
Raises:
|
||||
HTTPException: If token is invalid or expired
|
||||
"""
|
||||
try:
|
||||
payload = jwt.decode(token, self.secret, algorithms=[self.algorithm])
|
||||
expire_timestamp = payload["exp"]
|
||||
expire_time = datetime.utcfromtimestamp(expire_timestamp)
|
||||
|
||||
if datetime.utcnow() > expire_time:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED, detail="Token expired"
|
||||
)
|
||||
|
||||
# Return complete payload instead of just username
|
||||
return {
|
||||
"username": payload["sub"],
|
||||
"role": payload.get("role", "user"),
|
||||
"metadata": payload.get("metadata", {}),
|
||||
"exp": expire_time,
|
||||
}
|
||||
except jwt.PyJWTError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token"
|
||||
)
|
||||
|
||||
|
||||
auth_handler = AuthHandler()
|
||||
@@ -0,0 +1,549 @@
|
||||
"""
|
||||
Configs for the LightRAG API.
|
||||
"""
|
||||
|
||||
import os
|
||||
import argparse
|
||||
import logging
|
||||
from dotenv import load_dotenv
|
||||
from lightrag.utils import get_env_value
|
||||
from lightrag.llm.binding_options import (
|
||||
GeminiEmbeddingOptions,
|
||||
GeminiLLMOptions,
|
||||
OllamaEmbeddingOptions,
|
||||
OllamaLLMOptions,
|
||||
OpenAILLMOptions,
|
||||
)
|
||||
from lightrag.base import OllamaServerInfos
|
||||
import sys
|
||||
|
||||
from lightrag.constants import (
|
||||
DEFAULT_WOKERS,
|
||||
DEFAULT_TIMEOUT,
|
||||
DEFAULT_TOP_K,
|
||||
DEFAULT_CHUNK_TOP_K,
|
||||
DEFAULT_HISTORY_TURNS,
|
||||
DEFAULT_MAX_ENTITY_TOKENS,
|
||||
DEFAULT_MAX_RELATION_TOKENS,
|
||||
DEFAULT_MAX_TOTAL_TOKENS,
|
||||
DEFAULT_COSINE_THRESHOLD,
|
||||
DEFAULT_RELATED_CHUNK_NUMBER,
|
||||
DEFAULT_MIN_RERANK_SCORE,
|
||||
DEFAULT_FORCE_LLM_SUMMARY_ON_MERGE,
|
||||
DEFAULT_MAX_ASYNC,
|
||||
DEFAULT_SUMMARY_MAX_TOKENS,
|
||||
DEFAULT_SUMMARY_LENGTH_RECOMMENDED,
|
||||
DEFAULT_SUMMARY_CONTEXT_SIZE,
|
||||
DEFAULT_SUMMARY_LANGUAGE,
|
||||
DEFAULT_EMBEDDING_FUNC_MAX_ASYNC,
|
||||
DEFAULT_EMBEDDING_BATCH_NUM,
|
||||
DEFAULT_OLLAMA_MODEL_NAME,
|
||||
DEFAULT_OLLAMA_MODEL_TAG,
|
||||
DEFAULT_RERANK_BINDING,
|
||||
DEFAULT_ENTITY_TYPES,
|
||||
)
|
||||
|
||||
# use the .env that is inside the current folder
|
||||
# allows to use different .env file for each lightrag instance
|
||||
# the OS environment variables take precedence over the .env file
|
||||
load_dotenv(dotenv_path=".env", override=False)
|
||||
|
||||
|
||||
ollama_server_infos = OllamaServerInfos()
|
||||
|
||||
|
||||
class DefaultRAGStorageConfig:
|
||||
KV_STORAGE = "JsonKVStorage"
|
||||
VECTOR_STORAGE = "NanoVectorDBStorage"
|
||||
GRAPH_STORAGE = "NetworkXStorage"
|
||||
DOC_STATUS_STORAGE = "JsonDocStatusStorage"
|
||||
|
||||
|
||||
def get_default_host(binding_type: str) -> str:
|
||||
default_hosts = {
|
||||
"ollama": os.getenv("LLM_BINDING_HOST", "http://localhost:11434"),
|
||||
"lollms": os.getenv("LLM_BINDING_HOST", "http://localhost:9600"),
|
||||
"azure_openai": os.getenv("AZURE_OPENAI_ENDPOINT", "https://api.openai.com/v1"),
|
||||
"openai": os.getenv("LLM_BINDING_HOST", "https://api.openai.com/v1"),
|
||||
"gemini": os.getenv(
|
||||
"LLM_BINDING_HOST", "https://generativelanguage.googleapis.com"
|
||||
),
|
||||
}
|
||||
return default_hosts.get(
|
||||
binding_type, os.getenv("LLM_BINDING_HOST", "http://localhost:11434")
|
||||
) # fallback to ollama if unknown
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""
|
||||
Parse command line arguments with environment variable fallback
|
||||
|
||||
Args:
|
||||
is_uvicorn_mode: Whether running under uvicorn mode
|
||||
|
||||
Returns:
|
||||
argparse.Namespace: Parsed arguments
|
||||
"""
|
||||
|
||||
parser = argparse.ArgumentParser(description="LightRAG API Server")
|
||||
|
||||
# Server configuration
|
||||
parser.add_argument(
|
||||
"--host",
|
||||
default=get_env_value("HOST", "0.0.0.0"),
|
||||
help="Server host (default: from env or 0.0.0.0)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--port",
|
||||
type=int,
|
||||
default=get_env_value("PORT", 9621, int),
|
||||
help="Server port (default: from env or 9621)",
|
||||
)
|
||||
|
||||
# Directory configuration
|
||||
parser.add_argument(
|
||||
"--working-dir",
|
||||
default=get_env_value("WORKING_DIR", "./rag_storage"),
|
||||
help="Working directory for RAG storage (default: from env or ./rag_storage)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--input-dir",
|
||||
default=get_env_value("INPUT_DIR", "./inputs"),
|
||||
help="Directory containing input documents (default: from env or ./inputs)",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
default=get_env_value("TIMEOUT", DEFAULT_TIMEOUT, int, special_none=True),
|
||||
type=int,
|
||||
help="Timeout in seconds (useful when using slow AI). Use None for infinite timeout",
|
||||
)
|
||||
|
||||
# RAG configuration
|
||||
parser.add_argument(
|
||||
"--max-async",
|
||||
type=int,
|
||||
default=get_env_value("MAX_ASYNC", DEFAULT_MAX_ASYNC, int),
|
||||
help=f"Maximum async operations (default: from env or {DEFAULT_MAX_ASYNC})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--summary-max-tokens",
|
||||
type=int,
|
||||
default=get_env_value("SUMMARY_MAX_TOKENS", DEFAULT_SUMMARY_MAX_TOKENS, int),
|
||||
help=f"Maximum token size for entity/relation summary(default: from env or {DEFAULT_SUMMARY_MAX_TOKENS})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--summary-context-size",
|
||||
type=int,
|
||||
default=get_env_value(
|
||||
"SUMMARY_CONTEXT_SIZE", DEFAULT_SUMMARY_CONTEXT_SIZE, int
|
||||
),
|
||||
help=f"LLM Summary Context size (default: from env or {DEFAULT_SUMMARY_CONTEXT_SIZE})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--summary-length-recommended",
|
||||
type=int,
|
||||
default=get_env_value(
|
||||
"SUMMARY_LENGTH_RECOMMENDED", DEFAULT_SUMMARY_LENGTH_RECOMMENDED, int
|
||||
),
|
||||
help=f"LLM Summary Context size (default: from env or {DEFAULT_SUMMARY_LENGTH_RECOMMENDED})",
|
||||
)
|
||||
|
||||
# Logging configuration
|
||||
parser.add_argument(
|
||||
"--log-level",
|
||||
default=get_env_value("LOG_LEVEL", "INFO"),
|
||||
choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
|
||||
help="Logging level (default: from env or INFO)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
default=get_env_value("VERBOSE", False, bool),
|
||||
help="Enable verbose debug output(only valid for DEBUG log-level)",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--key",
|
||||
type=str,
|
||||
default=get_env_value("LIGHTRAG_API_KEY", None),
|
||||
help="API key for authentication. This protects lightrag server against unauthorized access",
|
||||
)
|
||||
|
||||
# Optional https parameters
|
||||
parser.add_argument(
|
||||
"--ssl",
|
||||
action="store_true",
|
||||
default=get_env_value("SSL", False, bool),
|
||||
help="Enable HTTPS (default: from env or False)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ssl-certfile",
|
||||
default=get_env_value("SSL_CERTFILE", None),
|
||||
help="Path to SSL certificate file (required if --ssl is enabled)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ssl-keyfile",
|
||||
default=get_env_value("SSL_KEYFILE", None),
|
||||
help="Path to SSL private key file (required if --ssl is enabled)",
|
||||
)
|
||||
|
||||
# Ollama model configuration
|
||||
parser.add_argument(
|
||||
"--simulated-model-name",
|
||||
type=str,
|
||||
default=get_env_value("OLLAMA_EMULATING_MODEL_NAME", DEFAULT_OLLAMA_MODEL_NAME),
|
||||
help="Name for the simulated Ollama model (default: from env or lightrag)",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--simulated-model-tag",
|
||||
type=str,
|
||||
default=get_env_value("OLLAMA_EMULATING_MODEL_TAG", DEFAULT_OLLAMA_MODEL_TAG),
|
||||
help="Tag for the simulated Ollama model (default: from env or latest)",
|
||||
)
|
||||
|
||||
# Namespace
|
||||
parser.add_argument(
|
||||
"--workspace",
|
||||
type=str,
|
||||
default=get_env_value("WORKSPACE", ""),
|
||||
help="Default workspace for all storage",
|
||||
)
|
||||
|
||||
# Server workers configuration
|
||||
parser.add_argument(
|
||||
"--workers",
|
||||
type=int,
|
||||
default=get_env_value("WORKERS", DEFAULT_WOKERS, int),
|
||||
help="Number of worker processes (default: from env or 1)",
|
||||
)
|
||||
|
||||
# LLM and embedding bindings
|
||||
parser.add_argument(
|
||||
"--llm-binding",
|
||||
type=str,
|
||||
default=get_env_value("LLM_BINDING", "ollama"),
|
||||
choices=[
|
||||
"lollms",
|
||||
"ollama",
|
||||
"openai",
|
||||
"openai-ollama",
|
||||
"azure_openai",
|
||||
"aws_bedrock",
|
||||
"gemini",
|
||||
],
|
||||
help="LLM binding type (default: from env or ollama)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--embedding-binding",
|
||||
type=str,
|
||||
default=get_env_value("EMBEDDING_BINDING", "ollama"),
|
||||
choices=[
|
||||
"lollms",
|
||||
"ollama",
|
||||
"openai",
|
||||
"azure_openai",
|
||||
"aws_bedrock",
|
||||
"jina",
|
||||
"gemini",
|
||||
],
|
||||
help="Embedding binding type (default: from env or ollama)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--rerank-binding",
|
||||
type=str,
|
||||
default=get_env_value("RERANK_BINDING", DEFAULT_RERANK_BINDING),
|
||||
choices=["null", "cohere", "jina", "aliyun"],
|
||||
help=f"Rerank binding type (default: from env or {DEFAULT_RERANK_BINDING})",
|
||||
)
|
||||
|
||||
# Document loading engine configuration
|
||||
parser.add_argument(
|
||||
"--docling",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Enable DOCLING document loading engine (default: from env or DEFAULT)",
|
||||
)
|
||||
|
||||
# Conditionally add binding options defined in binding_options module
|
||||
# This will add command line arguments for all binding options (e.g., --ollama-embedding-num_ctx)
|
||||
# and corresponding environment variables (e.g., OLLAMA_EMBEDDING_NUM_CTX)
|
||||
if "--llm-binding" in sys.argv:
|
||||
try:
|
||||
idx = sys.argv.index("--llm-binding")
|
||||
if idx + 1 < len(sys.argv) and sys.argv[idx + 1] == "ollama":
|
||||
OllamaLLMOptions.add_args(parser)
|
||||
except IndexError:
|
||||
pass
|
||||
elif os.environ.get("LLM_BINDING") == "ollama":
|
||||
OllamaLLMOptions.add_args(parser)
|
||||
|
||||
if "--embedding-binding" in sys.argv:
|
||||
try:
|
||||
idx = sys.argv.index("--embedding-binding")
|
||||
if idx + 1 < len(sys.argv):
|
||||
if sys.argv[idx + 1] == "ollama":
|
||||
OllamaEmbeddingOptions.add_args(parser)
|
||||
elif sys.argv[idx + 1] == "gemini":
|
||||
GeminiEmbeddingOptions.add_args(parser)
|
||||
except IndexError:
|
||||
pass
|
||||
else:
|
||||
env_embedding_binding = os.environ.get("EMBEDDING_BINDING")
|
||||
if env_embedding_binding == "ollama":
|
||||
OllamaEmbeddingOptions.add_args(parser)
|
||||
elif env_embedding_binding == "gemini":
|
||||
GeminiEmbeddingOptions.add_args(parser)
|
||||
|
||||
# Add OpenAI LLM options when llm-binding is openai or azure_openai
|
||||
if "--llm-binding" in sys.argv:
|
||||
try:
|
||||
idx = sys.argv.index("--llm-binding")
|
||||
if idx + 1 < len(sys.argv) and sys.argv[idx + 1] in [
|
||||
"openai",
|
||||
"azure_openai",
|
||||
]:
|
||||
OpenAILLMOptions.add_args(parser)
|
||||
except IndexError:
|
||||
pass
|
||||
elif os.environ.get("LLM_BINDING") in ["openai", "azure_openai"]:
|
||||
OpenAILLMOptions.add_args(parser)
|
||||
|
||||
if "--llm-binding" in sys.argv:
|
||||
try:
|
||||
idx = sys.argv.index("--llm-binding")
|
||||
if idx + 1 < len(sys.argv) and sys.argv[idx + 1] == "gemini":
|
||||
GeminiLLMOptions.add_args(parser)
|
||||
except IndexError:
|
||||
pass
|
||||
elif os.environ.get("LLM_BINDING") == "gemini":
|
||||
GeminiLLMOptions.add_args(parser)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# convert relative path to absolute path
|
||||
args.working_dir = os.path.abspath(args.working_dir)
|
||||
args.input_dir = os.path.abspath(args.input_dir)
|
||||
|
||||
# Inject storage configuration from environment variables
|
||||
args.kv_storage = get_env_value(
|
||||
"LIGHTRAG_KV_STORAGE", DefaultRAGStorageConfig.KV_STORAGE
|
||||
)
|
||||
args.doc_status_storage = get_env_value(
|
||||
"LIGHTRAG_DOC_STATUS_STORAGE", DefaultRAGStorageConfig.DOC_STATUS_STORAGE
|
||||
)
|
||||
args.graph_storage = get_env_value(
|
||||
"LIGHTRAG_GRAPH_STORAGE", DefaultRAGStorageConfig.GRAPH_STORAGE
|
||||
)
|
||||
args.vector_storage = get_env_value(
|
||||
"LIGHTRAG_VECTOR_STORAGE", DefaultRAGStorageConfig.VECTOR_STORAGE
|
||||
)
|
||||
|
||||
# Get MAX_PARALLEL_INSERT from environment
|
||||
args.max_parallel_insert = get_env_value("MAX_PARALLEL_INSERT", 2, int)
|
||||
|
||||
# Get MAX_GRAPH_NODES from environment
|
||||
args.max_graph_nodes = get_env_value("MAX_GRAPH_NODES", 1000, int)
|
||||
|
||||
# Handle openai-ollama special case
|
||||
if args.llm_binding == "openai-ollama":
|
||||
args.llm_binding = "openai"
|
||||
args.embedding_binding = "ollama"
|
||||
|
||||
# Ollama ctx_num
|
||||
args.ollama_num_ctx = get_env_value("OLLAMA_NUM_CTX", 32768, int)
|
||||
|
||||
args.llm_binding_host = get_env_value(
|
||||
"LLM_BINDING_HOST", get_default_host(args.llm_binding)
|
||||
)
|
||||
args.embedding_binding_host = get_env_value(
|
||||
"EMBEDDING_BINDING_HOST", get_default_host(args.embedding_binding)
|
||||
)
|
||||
args.llm_binding_api_key = get_env_value("LLM_BINDING_API_KEY", None)
|
||||
args.embedding_binding_api_key = get_env_value("EMBEDDING_BINDING_API_KEY", "")
|
||||
|
||||
# Inject model configuration
|
||||
args.llm_model = get_env_value("LLM_MODEL", "mistral-nemo:latest")
|
||||
args.embedding_model = get_env_value("EMBEDDING_MODEL", "bge-m3:latest")
|
||||
args.embedding_dim = get_env_value("EMBEDDING_DIM", 1024, int)
|
||||
args.embedding_send_dim = get_env_value("EMBEDDING_SEND_DIM", False, bool)
|
||||
|
||||
# Inject chunk configuration
|
||||
args.chunk_size = get_env_value("CHUNK_SIZE", 1200, int)
|
||||
args.chunk_overlap_size = get_env_value("CHUNK_OVERLAP_SIZE", 100, int)
|
||||
|
||||
# Inject LLM cache configuration
|
||||
args.enable_llm_cache_for_extract = get_env_value(
|
||||
"ENABLE_LLM_CACHE_FOR_EXTRACT", True, bool
|
||||
)
|
||||
args.enable_llm_cache = get_env_value("ENABLE_LLM_CACHE", True, bool)
|
||||
|
||||
# Set document_loading_engine from --docling flag
|
||||
if args.docling:
|
||||
args.document_loading_engine = "DOCLING"
|
||||
else:
|
||||
args.document_loading_engine = get_env_value(
|
||||
"DOCUMENT_LOADING_ENGINE", "DEFAULT"
|
||||
)
|
||||
|
||||
# PDF decryption password
|
||||
args.pdf_decrypt_password = get_env_value("PDF_DECRYPT_PASSWORD", None)
|
||||
|
||||
# Add environment variables that were previously read directly
|
||||
args.cors_origins = get_env_value("CORS_ORIGINS", "*")
|
||||
args.summary_language = get_env_value("SUMMARY_LANGUAGE", DEFAULT_SUMMARY_LANGUAGE)
|
||||
args.entity_types = get_env_value("ENTITY_TYPES", DEFAULT_ENTITY_TYPES, list)
|
||||
args.whitelist_paths = get_env_value("WHITELIST_PATHS", "/health,/api/*")
|
||||
|
||||
# For JWT Auth
|
||||
args.auth_accounts = get_env_value("AUTH_ACCOUNTS", "")
|
||||
args.token_secret = get_env_value("TOKEN_SECRET", "lightrag-jwt-default-secret")
|
||||
args.token_expire_hours = get_env_value("TOKEN_EXPIRE_HOURS", 48, int)
|
||||
args.guest_token_expire_hours = get_env_value("GUEST_TOKEN_EXPIRE_HOURS", 24, int)
|
||||
args.jwt_algorithm = get_env_value("JWT_ALGORITHM", "HS256")
|
||||
|
||||
# Rerank model configuration
|
||||
args.rerank_model = get_env_value("RERANK_MODEL", None)
|
||||
args.rerank_binding_host = get_env_value("RERANK_BINDING_HOST", None)
|
||||
args.rerank_binding_api_key = get_env_value("RERANK_BINDING_API_KEY", None)
|
||||
# Note: rerank_binding is already set by argparse, no need to override from env
|
||||
|
||||
# Min rerank score configuration
|
||||
args.min_rerank_score = get_env_value(
|
||||
"MIN_RERANK_SCORE", DEFAULT_MIN_RERANK_SCORE, float
|
||||
)
|
||||
|
||||
# Query configuration
|
||||
args.history_turns = get_env_value("HISTORY_TURNS", DEFAULT_HISTORY_TURNS, int)
|
||||
args.top_k = get_env_value("TOP_K", DEFAULT_TOP_K, int)
|
||||
args.chunk_top_k = get_env_value("CHUNK_TOP_K", DEFAULT_CHUNK_TOP_K, int)
|
||||
args.max_entity_tokens = get_env_value(
|
||||
"MAX_ENTITY_TOKENS", DEFAULT_MAX_ENTITY_TOKENS, int
|
||||
)
|
||||
args.max_relation_tokens = get_env_value(
|
||||
"MAX_RELATION_TOKENS", DEFAULT_MAX_RELATION_TOKENS, int
|
||||
)
|
||||
args.max_total_tokens = get_env_value(
|
||||
"MAX_TOTAL_TOKENS", DEFAULT_MAX_TOTAL_TOKENS, int
|
||||
)
|
||||
args.cosine_threshold = get_env_value(
|
||||
"COSINE_THRESHOLD", DEFAULT_COSINE_THRESHOLD, float
|
||||
)
|
||||
args.related_chunk_number = get_env_value(
|
||||
"RELATED_CHUNK_NUMBER", DEFAULT_RELATED_CHUNK_NUMBER, int
|
||||
)
|
||||
|
||||
# Add missing environment variables for health endpoint
|
||||
args.force_llm_summary_on_merge = get_env_value(
|
||||
"FORCE_LLM_SUMMARY_ON_MERGE", DEFAULT_FORCE_LLM_SUMMARY_ON_MERGE, int
|
||||
)
|
||||
args.embedding_func_max_async = get_env_value(
|
||||
"EMBEDDING_FUNC_MAX_ASYNC", DEFAULT_EMBEDDING_FUNC_MAX_ASYNC, int
|
||||
)
|
||||
args.embedding_batch_num = get_env_value(
|
||||
"EMBEDDING_BATCH_NUM", DEFAULT_EMBEDDING_BATCH_NUM, int
|
||||
)
|
||||
|
||||
# Embedding token limit configuration
|
||||
args.embedding_token_limit = get_env_value(
|
||||
"EMBEDDING_TOKEN_LIMIT", None, int, special_none=True
|
||||
)
|
||||
|
||||
ollama_server_infos.LIGHTRAG_NAME = args.simulated_model_name
|
||||
ollama_server_infos.LIGHTRAG_TAG = args.simulated_model_tag
|
||||
|
||||
return args
|
||||
|
||||
|
||||
def update_uvicorn_mode_config():
|
||||
# If in uvicorn mode and workers > 1, force it to 1 and log warning
|
||||
if global_args.workers > 1:
|
||||
original_workers = global_args.workers
|
||||
global_args.workers = 1
|
||||
# Log warning directly here
|
||||
logging.warning(
|
||||
f">> Forcing workers=1 in uvicorn mode(Ignoring workers={original_workers})"
|
||||
)
|
||||
|
||||
|
||||
# Global configuration with lazy initialization
|
||||
_global_args = None
|
||||
_initialized = False
|
||||
|
||||
|
||||
def initialize_config(args=None, force=False):
|
||||
"""Initialize global configuration
|
||||
|
||||
This function allows explicit initialization of the configuration,
|
||||
which is useful for programmatic usage, testing, or embedding LightRAG
|
||||
in other applications.
|
||||
|
||||
Args:
|
||||
args: Pre-parsed argparse.Namespace or None to parse from sys.argv
|
||||
force: Force re-initialization even if already initialized
|
||||
|
||||
Returns:
|
||||
argparse.Namespace: The configured arguments
|
||||
|
||||
Example:
|
||||
# Use parsed command line arguments (default)
|
||||
initialize_config()
|
||||
|
||||
# Use custom configuration programmatically
|
||||
custom_args = argparse.Namespace(
|
||||
host='localhost',
|
||||
port=8080,
|
||||
working_dir='./custom_rag',
|
||||
# ... other config
|
||||
)
|
||||
initialize_config(custom_args)
|
||||
"""
|
||||
global _global_args, _initialized
|
||||
|
||||
if _initialized and not force:
|
||||
return _global_args
|
||||
|
||||
_global_args = args if args is not None else parse_args()
|
||||
_initialized = True
|
||||
return _global_args
|
||||
|
||||
|
||||
def get_config():
|
||||
"""Get global configuration, auto-initializing if needed
|
||||
|
||||
Returns:
|
||||
argparse.Namespace: The configured arguments
|
||||
"""
|
||||
if not _initialized:
|
||||
initialize_config()
|
||||
return _global_args
|
||||
|
||||
|
||||
class _GlobalArgsProxy:
|
||||
"""Proxy object that auto-initializes configuration on first access
|
||||
|
||||
This maintains backward compatibility with existing code while
|
||||
allowing programmatic control over initialization timing.
|
||||
"""
|
||||
|
||||
def __getattr__(self, name):
|
||||
if not _initialized:
|
||||
initialize_config()
|
||||
return getattr(_global_args, name)
|
||||
|
||||
def __setattr__(self, name, value):
|
||||
if not _initialized:
|
||||
initialize_config()
|
||||
setattr(_global_args, name, value)
|
||||
|
||||
def __repr__(self):
|
||||
if not _initialized:
|
||||
return "<GlobalArgsProxy: Not initialized>"
|
||||
return repr(_global_args)
|
||||
|
||||
|
||||
# Create proxy instance for backward compatibility
|
||||
# Existing code like `from config import global_args` continues to work
|
||||
# The proxy will auto-initialize on first attribute access
|
||||
global_args = _GlobalArgsProxy()
|
||||
@@ -0,0 +1,162 @@
|
||||
# gunicorn_config.py
|
||||
import os
|
||||
import logging
|
||||
from lightrag.kg.shared_storage import finalize_share_data
|
||||
from lightrag.utils import setup_logger, get_env_value
|
||||
from lightrag.constants import (
|
||||
DEFAULT_LOG_MAX_BYTES,
|
||||
DEFAULT_LOG_BACKUP_COUNT,
|
||||
DEFAULT_LOG_FILENAME,
|
||||
)
|
||||
|
||||
|
||||
# Get log directory path from environment variable
|
||||
log_dir = os.getenv("LOG_DIR", os.getcwd())
|
||||
log_file_path = os.path.abspath(os.path.join(log_dir, DEFAULT_LOG_FILENAME))
|
||||
|
||||
# Ensure log directory exists
|
||||
os.makedirs(os.path.dirname(log_file_path), exist_ok=True)
|
||||
|
||||
# Get log file max size and backup count from environment variables
|
||||
log_max_bytes = get_env_value("LOG_MAX_BYTES", DEFAULT_LOG_MAX_BYTES, int)
|
||||
log_backup_count = get_env_value("LOG_BACKUP_COUNT", DEFAULT_LOG_BACKUP_COUNT, int)
|
||||
|
||||
# These variables will be set by run_with_gunicorn.py
|
||||
workers = None
|
||||
bind = None
|
||||
loglevel = None
|
||||
certfile = None
|
||||
keyfile = None
|
||||
|
||||
# Enable preload_app option
|
||||
preload_app = True
|
||||
|
||||
# Use Uvicorn worker
|
||||
worker_class = "uvicorn.workers.UvicornWorker"
|
||||
|
||||
# Other Gunicorn configurations
|
||||
|
||||
# Logging configuration
|
||||
errorlog = os.getenv("ERROR_LOG", log_file_path) # Default write to lightrag.log
|
||||
accesslog = os.getenv("ACCESS_LOG", log_file_path) # Default write to lightrag.log
|
||||
|
||||
logconfig_dict = {
|
||||
"version": 1,
|
||||
"disable_existing_loggers": False,
|
||||
"formatters": {
|
||||
"standard": {"format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s"},
|
||||
},
|
||||
"handlers": {
|
||||
"console": {
|
||||
"class": "logging.StreamHandler",
|
||||
"formatter": "standard",
|
||||
"stream": "ext://sys.stdout",
|
||||
},
|
||||
"file": {
|
||||
"class": "logging.handlers.RotatingFileHandler",
|
||||
"formatter": "standard",
|
||||
"filename": log_file_path,
|
||||
"maxBytes": log_max_bytes,
|
||||
"backupCount": log_backup_count,
|
||||
"encoding": "utf8",
|
||||
},
|
||||
},
|
||||
"filters": {
|
||||
"path_filter": {
|
||||
"()": "lightrag.utils.LightragPathFilter",
|
||||
},
|
||||
},
|
||||
"loggers": {
|
||||
"lightrag": {
|
||||
"handlers": ["console", "file"],
|
||||
"level": loglevel.upper() if loglevel else "INFO",
|
||||
"propagate": False,
|
||||
},
|
||||
"gunicorn": {
|
||||
"handlers": ["console", "file"],
|
||||
"level": loglevel.upper() if loglevel else "INFO",
|
||||
"propagate": False,
|
||||
},
|
||||
"gunicorn.error": {
|
||||
"handlers": ["console", "file"],
|
||||
"level": loglevel.upper() if loglevel else "INFO",
|
||||
"propagate": False,
|
||||
},
|
||||
"gunicorn.access": {
|
||||
"handlers": ["console", "file"],
|
||||
"level": loglevel.upper() if loglevel else "INFO",
|
||||
"propagate": False,
|
||||
"filters": ["path_filter"],
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def on_starting(server):
|
||||
"""
|
||||
Executed when Gunicorn starts, before forking the first worker processes
|
||||
You can use this function to do more initialization tasks for all processes
|
||||
"""
|
||||
print("=" * 80)
|
||||
print(f"GUNICORN MASTER PROCESS: on_starting jobs for {workers} worker(s)")
|
||||
print(f"Process ID: {os.getpid()}")
|
||||
print("=" * 80)
|
||||
|
||||
# Memory usage monitoring
|
||||
try:
|
||||
import psutil
|
||||
|
||||
process = psutil.Process(os.getpid())
|
||||
memory_info = process.memory_info()
|
||||
msg = (
|
||||
f"Memory usage after initialization: {memory_info.rss / 1024 / 1024:.2f} MB"
|
||||
)
|
||||
print(msg)
|
||||
except ImportError:
|
||||
print("psutil not installed, skipping memory usage reporting")
|
||||
|
||||
# Log the location of the LightRAG log file
|
||||
print(f"LightRAG log file: {log_file_path}\n")
|
||||
|
||||
print("Gunicorn initialization complete, forking workers...\n")
|
||||
|
||||
|
||||
def on_exit(server):
|
||||
"""
|
||||
Executed when Gunicorn is shutting down.
|
||||
This is a good place to release shared resources.
|
||||
"""
|
||||
print("=" * 80)
|
||||
print("GUNICORN MASTER PROCESS: Shutting down")
|
||||
print(f"Process ID: {os.getpid()}")
|
||||
|
||||
print("Finalizing shared storage...")
|
||||
finalize_share_data()
|
||||
|
||||
print("Gunicorn shutdown complete")
|
||||
print("=" * 80)
|
||||
|
||||
|
||||
def post_fork(server, worker):
|
||||
"""
|
||||
Executed after a worker has been forked.
|
||||
This is a good place to set up worker-specific configurations.
|
||||
"""
|
||||
# Set up main loggers
|
||||
log_level = loglevel.upper() if loglevel else "INFO"
|
||||
setup_logger("uvicorn", log_level, add_filter=False, log_file_path=log_file_path)
|
||||
setup_logger(
|
||||
"uvicorn.access", log_level, add_filter=True, log_file_path=log_file_path
|
||||
)
|
||||
setup_logger("lightrag", log_level, add_filter=True, log_file_path=log_file_path)
|
||||
|
||||
# Set up lightrag submodule loggers
|
||||
for name in logging.root.manager.loggerDict:
|
||||
if name.startswith("lightrag."):
|
||||
setup_logger(name, log_level, add_filter=True, log_file_path=log_file_path)
|
||||
|
||||
# Disable uvicorn.error logger
|
||||
uvicorn_error_logger = logging.getLogger("uvicorn.error")
|
||||
uvicorn_error_logger.handlers = []
|
||||
uvicorn_error_logger.setLevel(logging.CRITICAL)
|
||||
uvicorn_error_logger.propagate = False
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
This module contains all the routers for the LightRAG API.
|
||||
"""
|
||||
|
||||
from .document_routes import router as document_router
|
||||
from .query_routes import router as query_router
|
||||
from .graph_routes import router as graph_router
|
||||
from .ollama_api import OllamaAPI
|
||||
|
||||
__all__ = ["document_router", "query_router", "graph_router", "OllamaAPI"]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,688 @@
|
||||
"""
|
||||
This module contains all graph-related routes for the LightRAG API.
|
||||
"""
|
||||
|
||||
from typing import Optional, Dict, Any
|
||||
import traceback
|
||||
from fastapi import APIRouter, Depends, Query, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from lightrag.utils import logger
|
||||
from ..utils_api import get_combined_auth_dependency
|
||||
|
||||
router = APIRouter(tags=["graph"])
|
||||
|
||||
|
||||
class EntityUpdateRequest(BaseModel):
|
||||
entity_name: str
|
||||
updated_data: Dict[str, Any]
|
||||
allow_rename: bool = False
|
||||
allow_merge: bool = False
|
||||
|
||||
|
||||
class RelationUpdateRequest(BaseModel):
|
||||
source_id: str
|
||||
target_id: str
|
||||
updated_data: Dict[str, Any]
|
||||
|
||||
|
||||
class EntityMergeRequest(BaseModel):
|
||||
entities_to_change: list[str] = Field(
|
||||
...,
|
||||
description="List of entity names to be merged and deleted. These are typically duplicate or misspelled entities.",
|
||||
min_length=1,
|
||||
examples=[["Elon Msk", "Ellon Musk"]],
|
||||
)
|
||||
entity_to_change_into: str = Field(
|
||||
...,
|
||||
description="Target entity name that will receive all relationships from the source entities. This entity will be preserved.",
|
||||
min_length=1,
|
||||
examples=["Elon Musk"],
|
||||
)
|
||||
|
||||
|
||||
class EntityCreateRequest(BaseModel):
|
||||
entity_name: str = Field(
|
||||
...,
|
||||
description="Unique name for the new entity",
|
||||
min_length=1,
|
||||
examples=["Tesla"],
|
||||
)
|
||||
entity_data: Dict[str, Any] = Field(
|
||||
...,
|
||||
description="Dictionary containing entity properties. Common fields include 'description' and 'entity_type'.",
|
||||
examples=[
|
||||
{
|
||||
"description": "Electric vehicle manufacturer",
|
||||
"entity_type": "ORGANIZATION",
|
||||
}
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
class RelationCreateRequest(BaseModel):
|
||||
source_entity: str = Field(
|
||||
...,
|
||||
description="Name of the source entity. This entity must already exist in the knowledge graph.",
|
||||
min_length=1,
|
||||
examples=["Elon Musk"],
|
||||
)
|
||||
target_entity: str = Field(
|
||||
...,
|
||||
description="Name of the target entity. This entity must already exist in the knowledge graph.",
|
||||
min_length=1,
|
||||
examples=["Tesla"],
|
||||
)
|
||||
relation_data: Dict[str, Any] = Field(
|
||||
...,
|
||||
description="Dictionary containing relationship properties. Common fields include 'description', 'keywords', and 'weight'.",
|
||||
examples=[
|
||||
{
|
||||
"description": "Elon Musk is the CEO of Tesla",
|
||||
"keywords": "CEO, founder",
|
||||
"weight": 1.0,
|
||||
}
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def create_graph_routes(rag, api_key: Optional[str] = None):
|
||||
combined_auth = get_combined_auth_dependency(api_key)
|
||||
|
||||
@router.get("/graph/label/list", dependencies=[Depends(combined_auth)])
|
||||
async def get_graph_labels():
|
||||
"""
|
||||
Get all graph labels
|
||||
|
||||
Returns:
|
||||
List[str]: List of graph labels
|
||||
"""
|
||||
try:
|
||||
return await rag.get_graph_labels()
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting graph labels: {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error getting graph labels: {str(e)}"
|
||||
)
|
||||
|
||||
@router.get("/graph/label/popular", dependencies=[Depends(combined_auth)])
|
||||
async def get_popular_labels(
|
||||
limit: int = Query(
|
||||
300, description="Maximum number of popular labels to return", ge=1, le=1000
|
||||
),
|
||||
):
|
||||
"""
|
||||
Get popular labels by node degree (most connected entities)
|
||||
|
||||
Args:
|
||||
limit (int): Maximum number of labels to return (default: 300, max: 1000)
|
||||
|
||||
Returns:
|
||||
List[str]: List of popular labels sorted by degree (highest first)
|
||||
"""
|
||||
try:
|
||||
return await rag.chunk_entity_relation_graph.get_popular_labels(limit)
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting popular labels: {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error getting popular labels: {str(e)}"
|
||||
)
|
||||
|
||||
@router.get("/graph/label/search", dependencies=[Depends(combined_auth)])
|
||||
async def search_labels(
|
||||
q: str = Query(..., description="Search query string"),
|
||||
limit: int = Query(
|
||||
50, description="Maximum number of search results to return", ge=1, le=100
|
||||
),
|
||||
):
|
||||
"""
|
||||
Search labels with fuzzy matching
|
||||
|
||||
Args:
|
||||
q (str): Search query string
|
||||
limit (int): Maximum number of results to return (default: 50, max: 100)
|
||||
|
||||
Returns:
|
||||
List[str]: List of matching labels sorted by relevance
|
||||
"""
|
||||
try:
|
||||
return await rag.chunk_entity_relation_graph.search_labels(q, limit)
|
||||
except Exception as e:
|
||||
logger.error(f"Error searching labels with query '{q}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error searching labels: {str(e)}"
|
||||
)
|
||||
|
||||
@router.get("/graphs", dependencies=[Depends(combined_auth)])
|
||||
async def get_knowledge_graph(
|
||||
label: str = Query(..., description="Label to get knowledge graph for"),
|
||||
max_depth: int = Query(3, description="Maximum depth of graph", ge=1),
|
||||
max_nodes: int = Query(1000, description="Maximum nodes to return", ge=1),
|
||||
):
|
||||
"""
|
||||
Retrieve a connected subgraph of nodes where the label includes the specified label.
|
||||
When reducing the number of nodes, the prioritization criteria are as follows:
|
||||
1. Hops(path) to the staring node take precedence
|
||||
2. Followed by the degree of the nodes
|
||||
|
||||
Args:
|
||||
label (str): Label of the starting node
|
||||
max_depth (int, optional): Maximum depth of the subgraph,Defaults to 3
|
||||
max_nodes: Maxiumu nodes to return
|
||||
|
||||
Returns:
|
||||
Dict[str, List[str]]: Knowledge graph for label
|
||||
"""
|
||||
try:
|
||||
# Log the label parameter to check for leading spaces
|
||||
logger.debug(
|
||||
f"get_knowledge_graph called with label: '{label}' (length: {len(label)}, repr: {repr(label)})"
|
||||
)
|
||||
|
||||
return await rag.get_knowledge_graph(
|
||||
node_label=label,
|
||||
max_depth=max_depth,
|
||||
max_nodes=max_nodes,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting knowledge graph for label '{label}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error getting knowledge graph: {str(e)}"
|
||||
)
|
||||
|
||||
@router.get("/graph/entity/exists", dependencies=[Depends(combined_auth)])
|
||||
async def check_entity_exists(
|
||||
name: str = Query(..., description="Entity name to check"),
|
||||
):
|
||||
"""
|
||||
Check if an entity with the given name exists in the knowledge graph
|
||||
|
||||
Args:
|
||||
name (str): Name of the entity to check
|
||||
|
||||
Returns:
|
||||
Dict[str, bool]: Dictionary with 'exists' key indicating if entity exists
|
||||
"""
|
||||
try:
|
||||
exists = await rag.chunk_entity_relation_graph.has_node(name)
|
||||
return {"exists": exists}
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking entity existence for '{name}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error checking entity existence: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post("/graph/entity/edit", dependencies=[Depends(combined_auth)])
|
||||
async def update_entity(request: EntityUpdateRequest):
|
||||
"""
|
||||
Update an entity's properties in the knowledge graph
|
||||
|
||||
This endpoint allows updating entity properties, including renaming entities.
|
||||
When renaming to an existing entity name, the behavior depends on allow_merge:
|
||||
|
||||
Args:
|
||||
request (EntityUpdateRequest): Request containing:
|
||||
- entity_name (str): Name of the entity to update
|
||||
- updated_data (Dict[str, Any]): Dictionary of properties to update
|
||||
- allow_rename (bool): Whether to allow entity renaming (default: False)
|
||||
- allow_merge (bool): Whether to merge into existing entity when renaming
|
||||
causes name conflict (default: False)
|
||||
|
||||
Returns:
|
||||
Dict with the following structure:
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Entity updated successfully" | "Entity merged successfully into 'target_name'",
|
||||
"data": {
|
||||
"entity_name": str, # Final entity name
|
||||
"description": str, # Entity description
|
||||
"entity_type": str, # Entity type
|
||||
"source_id": str, # Source chunk IDs
|
||||
... # Other entity properties
|
||||
},
|
||||
"operation_summary": {
|
||||
"merged": bool, # Whether entity was merged into another
|
||||
"merge_status": str, # "success" | "failed" | "not_attempted"
|
||||
"merge_error": str | None, # Error message if merge failed
|
||||
"operation_status": str, # "success" | "partial_success" | "failure"
|
||||
"target_entity": str | None, # Target entity name if renaming/merging
|
||||
"final_entity": str, # Final entity name after operation
|
||||
"renamed": bool # Whether entity was renamed
|
||||
}
|
||||
}
|
||||
|
||||
operation_status values explained:
|
||||
- "success": All operations completed successfully
|
||||
* For simple updates: entity properties updated
|
||||
* For renames: entity renamed successfully
|
||||
* For merges: non-name updates applied AND merge completed
|
||||
|
||||
- "partial_success": Update succeeded but merge failed
|
||||
* Non-name property updates were applied successfully
|
||||
* Merge operation failed (entity not merged)
|
||||
* Original entity still exists with updated properties
|
||||
* Use merge_error for failure details
|
||||
|
||||
- "failure": Operation failed completely
|
||||
* If merge_status == "failed": Merge attempted but both update and merge failed
|
||||
* If merge_status == "not_attempted": Regular update failed
|
||||
* No changes were applied to the entity
|
||||
|
||||
merge_status values explained:
|
||||
- "success": Entity successfully merged into target entity
|
||||
- "failed": Merge operation was attempted but failed
|
||||
- "not_attempted": No merge was attempted (normal update/rename)
|
||||
|
||||
Behavior when renaming to an existing entity:
|
||||
- If allow_merge=False: Raises ValueError with 400 status (default behavior)
|
||||
- If allow_merge=True: Automatically merges the source entity into the existing target entity,
|
||||
preserving all relationships and applying non-name updates first
|
||||
|
||||
Example Request (simple update):
|
||||
POST /graph/entity/edit
|
||||
{
|
||||
"entity_name": "Tesla",
|
||||
"updated_data": {"description": "Updated description"},
|
||||
"allow_rename": false,
|
||||
"allow_merge": false
|
||||
}
|
||||
|
||||
Example Response (simple update success):
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Entity updated successfully",
|
||||
"data": { ... },
|
||||
"operation_summary": {
|
||||
"merged": false,
|
||||
"merge_status": "not_attempted",
|
||||
"merge_error": null,
|
||||
"operation_status": "success",
|
||||
"target_entity": null,
|
||||
"final_entity": "Tesla",
|
||||
"renamed": false
|
||||
}
|
||||
}
|
||||
|
||||
Example Request (rename with auto-merge):
|
||||
POST /graph/entity/edit
|
||||
{
|
||||
"entity_name": "Elon Msk",
|
||||
"updated_data": {
|
||||
"entity_name": "Elon Musk",
|
||||
"description": "Corrected description"
|
||||
},
|
||||
"allow_rename": true,
|
||||
"allow_merge": true
|
||||
}
|
||||
|
||||
Example Response (merge success):
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Entity merged successfully into 'Elon Musk'",
|
||||
"data": { ... },
|
||||
"operation_summary": {
|
||||
"merged": true,
|
||||
"merge_status": "success",
|
||||
"merge_error": null,
|
||||
"operation_status": "success",
|
||||
"target_entity": "Elon Musk",
|
||||
"final_entity": "Elon Musk",
|
||||
"renamed": true
|
||||
}
|
||||
}
|
||||
|
||||
Example Response (partial success - update succeeded but merge failed):
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Entity updated successfully",
|
||||
"data": { ... }, # Data reflects updated "Elon Msk" entity
|
||||
"operation_summary": {
|
||||
"merged": false,
|
||||
"merge_status": "failed",
|
||||
"merge_error": "Target entity locked by another operation",
|
||||
"operation_status": "partial_success",
|
||||
"target_entity": "Elon Musk",
|
||||
"final_entity": "Elon Msk", # Original entity still exists
|
||||
"renamed": true
|
||||
}
|
||||
}
|
||||
"""
|
||||
try:
|
||||
result = await rag.aedit_entity(
|
||||
entity_name=request.entity_name,
|
||||
updated_data=request.updated_data,
|
||||
allow_rename=request.allow_rename,
|
||||
allow_merge=request.allow_merge,
|
||||
)
|
||||
|
||||
# Extract operation_summary from result, with fallback for backward compatibility
|
||||
operation_summary = result.get(
|
||||
"operation_summary",
|
||||
{
|
||||
"merged": False,
|
||||
"merge_status": "not_attempted",
|
||||
"merge_error": None,
|
||||
"operation_status": "success",
|
||||
"target_entity": None,
|
||||
"final_entity": request.updated_data.get(
|
||||
"entity_name", request.entity_name
|
||||
),
|
||||
"renamed": request.updated_data.get(
|
||||
"entity_name", request.entity_name
|
||||
)
|
||||
!= request.entity_name,
|
||||
},
|
||||
)
|
||||
|
||||
# Separate entity data from operation_summary for clean response
|
||||
entity_data = dict(result)
|
||||
entity_data.pop("operation_summary", None)
|
||||
|
||||
# Generate appropriate response message based on merge status
|
||||
response_message = (
|
||||
f"Entity merged successfully into '{operation_summary['final_entity']}'"
|
||||
if operation_summary.get("merged")
|
||||
else "Entity updated successfully"
|
||||
)
|
||||
return {
|
||||
"status": "success",
|
||||
"message": response_message,
|
||||
"data": entity_data,
|
||||
"operation_summary": operation_summary,
|
||||
}
|
||||
except ValueError as ve:
|
||||
logger.error(
|
||||
f"Validation error updating entity '{request.entity_name}': {str(ve)}"
|
||||
)
|
||||
raise HTTPException(status_code=400, detail=str(ve))
|
||||
except Exception as e:
|
||||
logger.error(f"Error updating entity '{request.entity_name}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error updating entity: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post("/graph/relation/edit", dependencies=[Depends(combined_auth)])
|
||||
async def update_relation(request: RelationUpdateRequest):
|
||||
"""Update a relation's properties in the knowledge graph
|
||||
|
||||
Args:
|
||||
request (RelationUpdateRequest): Request containing source ID, target ID and updated data
|
||||
|
||||
Returns:
|
||||
Dict: Updated relation information
|
||||
"""
|
||||
try:
|
||||
result = await rag.aedit_relation(
|
||||
source_entity=request.source_id,
|
||||
target_entity=request.target_id,
|
||||
updated_data=request.updated_data,
|
||||
)
|
||||
return {
|
||||
"status": "success",
|
||||
"message": "Relation updated successfully",
|
||||
"data": result,
|
||||
}
|
||||
except ValueError as ve:
|
||||
logger.error(
|
||||
f"Validation error updating relation between '{request.source_id}' and '{request.target_id}': {str(ve)}"
|
||||
)
|
||||
raise HTTPException(status_code=400, detail=str(ve))
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Error updating relation between '{request.source_id}' and '{request.target_id}': {str(e)}"
|
||||
)
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error updating relation: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post("/graph/entity/create", dependencies=[Depends(combined_auth)])
|
||||
async def create_entity(request: EntityCreateRequest):
|
||||
"""
|
||||
Create a new entity in the knowledge graph
|
||||
|
||||
This endpoint creates a new entity node in the knowledge graph with the specified
|
||||
properties. The system automatically generates vector embeddings for the entity
|
||||
to enable semantic search and retrieval.
|
||||
|
||||
Request Body:
|
||||
entity_name (str): Unique name identifier for the entity
|
||||
entity_data (dict): Entity properties including:
|
||||
- description (str): Textual description of the entity
|
||||
- entity_type (str): Category/type of the entity (e.g., PERSON, ORGANIZATION, LOCATION)
|
||||
- source_id (str): Related chunk_id from which the description originates
|
||||
- Additional custom properties as needed
|
||||
|
||||
Response Schema:
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Entity 'Tesla' created successfully",
|
||||
"data": {
|
||||
"entity_name": "Tesla",
|
||||
"description": "Electric vehicle manufacturer",
|
||||
"entity_type": "ORGANIZATION",
|
||||
"source_id": "chunk-123<SEP>chunk-456"
|
||||
... (other entity properties)
|
||||
}
|
||||
}
|
||||
|
||||
HTTP Status Codes:
|
||||
200: Entity created successfully
|
||||
400: Invalid request (e.g., missing required fields, duplicate entity)
|
||||
500: Internal server error
|
||||
|
||||
Example Request:
|
||||
POST /graph/entity/create
|
||||
{
|
||||
"entity_name": "Tesla",
|
||||
"entity_data": {
|
||||
"description": "Electric vehicle manufacturer",
|
||||
"entity_type": "ORGANIZATION"
|
||||
}
|
||||
}
|
||||
"""
|
||||
try:
|
||||
# Use the proper acreate_entity method which handles:
|
||||
# - Graph lock for concurrency
|
||||
# - Vector embedding creation in entities_vdb
|
||||
# - Metadata population and defaults
|
||||
# - Index consistency via _edit_entity_done
|
||||
result = await rag.acreate_entity(
|
||||
entity_name=request.entity_name,
|
||||
entity_data=request.entity_data,
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"message": f"Entity '{request.entity_name}' created successfully",
|
||||
"data": result,
|
||||
}
|
||||
except ValueError as ve:
|
||||
logger.error(
|
||||
f"Validation error creating entity '{request.entity_name}': {str(ve)}"
|
||||
)
|
||||
raise HTTPException(status_code=400, detail=str(ve))
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating entity '{request.entity_name}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error creating entity: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post("/graph/relation/create", dependencies=[Depends(combined_auth)])
|
||||
async def create_relation(request: RelationCreateRequest):
|
||||
"""
|
||||
Create a new relationship between two entities in the knowledge graph
|
||||
|
||||
This endpoint establishes an undirected relationship between two existing entities.
|
||||
The provided source/target order is accepted for convenience, but the backend
|
||||
stored edge is undirected and may be returned with the entities swapped.
|
||||
Both entities must already exist in the knowledge graph. The system automatically
|
||||
generates vector embeddings for the relationship to enable semantic search and graph traversal.
|
||||
|
||||
Prerequisites:
|
||||
- Both source_entity and target_entity must exist in the knowledge graph
|
||||
- Use /graph/entity/create to create entities first if they don't exist
|
||||
|
||||
Request Body:
|
||||
source_entity (str): Name of the source entity (relationship origin)
|
||||
target_entity (str): Name of the target entity (relationship destination)
|
||||
relation_data (dict): Relationship properties including:
|
||||
- description (str): Textual description of the relationship
|
||||
- keywords (str): Comma-separated keywords describing the relationship type
|
||||
- source_id (str): Related chunk_id from which the description originates
|
||||
- weight (float): Relationship strength/importance (default: 1.0)
|
||||
- Additional custom properties as needed
|
||||
|
||||
Response Schema:
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Relation created successfully between 'Elon Musk' and 'Tesla'",
|
||||
"data": {
|
||||
"src_id": "Elon Musk",
|
||||
"tgt_id": "Tesla",
|
||||
"description": "Elon Musk is the CEO of Tesla",
|
||||
"keywords": "CEO, founder",
|
||||
"source_id": "chunk-123<SEP>chunk-456"
|
||||
"weight": 1.0,
|
||||
... (other relationship properties)
|
||||
}
|
||||
}
|
||||
|
||||
HTTP Status Codes:
|
||||
200: Relationship created successfully
|
||||
400: Invalid request (e.g., missing entities, invalid data, duplicate relationship)
|
||||
500: Internal server error
|
||||
|
||||
Example Request:
|
||||
POST /graph/relation/create
|
||||
{
|
||||
"source_entity": "Elon Musk",
|
||||
"target_entity": "Tesla",
|
||||
"relation_data": {
|
||||
"description": "Elon Musk is the CEO of Tesla",
|
||||
"keywords": "CEO, founder",
|
||||
"weight": 1.0
|
||||
}
|
||||
}
|
||||
"""
|
||||
try:
|
||||
# Use the proper acreate_relation method which handles:
|
||||
# - Graph lock for concurrency
|
||||
# - Entity existence validation
|
||||
# - Duplicate relation checks
|
||||
# - Vector embedding creation in relationships_vdb
|
||||
# - Index consistency via _edit_relation_done
|
||||
result = await rag.acreate_relation(
|
||||
source_entity=request.source_entity,
|
||||
target_entity=request.target_entity,
|
||||
relation_data=request.relation_data,
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"message": f"Relation created successfully between '{request.source_entity}' and '{request.target_entity}'",
|
||||
"data": result,
|
||||
}
|
||||
except ValueError as ve:
|
||||
logger.error(
|
||||
f"Validation error creating relation between '{request.source_entity}' and '{request.target_entity}': {str(ve)}"
|
||||
)
|
||||
raise HTTPException(status_code=400, detail=str(ve))
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Error creating relation between '{request.source_entity}' and '{request.target_entity}': {str(e)}"
|
||||
)
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error creating relation: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post("/graph/entities/merge", dependencies=[Depends(combined_auth)])
|
||||
async def merge_entities(request: EntityMergeRequest):
|
||||
"""
|
||||
Merge multiple entities into a single entity, preserving all relationships
|
||||
|
||||
This endpoint consolidates duplicate or misspelled entities while preserving the entire
|
||||
graph structure. It's particularly useful for cleaning up knowledge graphs after document
|
||||
processing or correcting entity name variations.
|
||||
|
||||
What the Merge Operation Does:
|
||||
1. Deletes the specified source entities from the knowledge graph
|
||||
2. Transfers all relationships from source entities to the target entity
|
||||
3. Intelligently merges duplicate relationships (if multiple sources have the same relationship)
|
||||
4. Updates vector embeddings for accurate retrieval and search
|
||||
5. Preserves the complete graph structure and connectivity
|
||||
6. Maintains relationship properties and metadata
|
||||
|
||||
Use Cases:
|
||||
- Fixing spelling errors in entity names (e.g., "Elon Msk" -> "Elon Musk")
|
||||
- Consolidating duplicate entities discovered after document processing
|
||||
- Merging name variations (e.g., "NY", "New York", "New York City")
|
||||
- Cleaning up the knowledge graph for better query performance
|
||||
- Standardizing entity names across the knowledge base
|
||||
|
||||
Request Body:
|
||||
entities_to_change (list[str]): List of entity names to be merged and deleted
|
||||
entity_to_change_into (str): Target entity that will receive all relationships
|
||||
|
||||
Response Schema:
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Successfully merged 2 entities into 'Elon Musk'",
|
||||
"data": {
|
||||
"merged_entity": "Elon Musk",
|
||||
"deleted_entities": ["Elon Msk", "Ellon Musk"],
|
||||
"relationships_transferred": 15,
|
||||
... (merge operation details)
|
||||
}
|
||||
}
|
||||
|
||||
HTTP Status Codes:
|
||||
200: Entities merged successfully
|
||||
400: Invalid request (e.g., empty entity list, target entity doesn't exist)
|
||||
500: Internal server error
|
||||
|
||||
Example Request:
|
||||
POST /graph/entities/merge
|
||||
{
|
||||
"entities_to_change": ["Elon Msk", "Ellon Musk"],
|
||||
"entity_to_change_into": "Elon Musk"
|
||||
}
|
||||
|
||||
Note:
|
||||
- The target entity (entity_to_change_into) must exist in the knowledge graph
|
||||
- Source entities will be permanently deleted after the merge
|
||||
- This operation cannot be undone, so verify entity names before merging
|
||||
"""
|
||||
try:
|
||||
result = await rag.amerge_entities(
|
||||
source_entities=request.entities_to_change,
|
||||
target_entity=request.entity_to_change_into,
|
||||
)
|
||||
return {
|
||||
"status": "success",
|
||||
"message": f"Successfully merged {len(request.entities_to_change)} entities into '{request.entity_to_change_into}'",
|
||||
"data": result,
|
||||
}
|
||||
except ValueError as ve:
|
||||
logger.error(
|
||||
f"Validation error merging entities {request.entities_to_change} into '{request.entity_to_change_into}': {str(ve)}"
|
||||
)
|
||||
raise HTTPException(status_code=400, detail=str(ve))
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Error merging entities {request.entities_to_change} into '{request.entity_to_change_into}': {str(e)}"
|
||||
)
|
||||
logger.error(traceback.format_exc())
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"Error merging entities: {str(e)}"
|
||||
)
|
||||
|
||||
return router
|
||||
@@ -0,0 +1,734 @@
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
from typing import List, Dict, Any, Optional, Type
|
||||
from lightrag.utils import logger
|
||||
import time
|
||||
import json
|
||||
import re
|
||||
from enum import Enum
|
||||
from fastapi.responses import StreamingResponse
|
||||
import asyncio
|
||||
from ascii_colors import trace_exception
|
||||
from lightrag import LightRAG, QueryParam
|
||||
from lightrag.utils import TiktokenTokenizer
|
||||
from lightrag.api.utils_api import get_combined_auth_dependency
|
||||
from fastapi import Depends
|
||||
|
||||
|
||||
# query mode according to query prefix (bypass is not LightRAG quer mode)
|
||||
class SearchMode(str, Enum):
|
||||
naive = "naive"
|
||||
local = "local"
|
||||
global_ = "global"
|
||||
hybrid = "hybrid"
|
||||
mix = "mix"
|
||||
bypass = "bypass"
|
||||
context = "context"
|
||||
|
||||
|
||||
class OllamaMessage(BaseModel):
|
||||
role: str
|
||||
content: str
|
||||
images: Optional[List[str]] = None
|
||||
|
||||
|
||||
class OllamaChatRequest(BaseModel):
|
||||
model: str
|
||||
messages: List[OllamaMessage]
|
||||
stream: bool = True
|
||||
options: Optional[Dict[str, Any]] = None
|
||||
system: Optional[str] = None
|
||||
|
||||
|
||||
class OllamaChatResponse(BaseModel):
|
||||
model: str
|
||||
created_at: str
|
||||
message: OllamaMessage
|
||||
done: bool
|
||||
|
||||
|
||||
class OllamaGenerateRequest(BaseModel):
|
||||
model: str
|
||||
prompt: str
|
||||
system: Optional[str] = None
|
||||
stream: bool = False
|
||||
options: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class OllamaGenerateResponse(BaseModel):
|
||||
model: str
|
||||
created_at: str
|
||||
response: str
|
||||
done: bool
|
||||
context: Optional[List[int]]
|
||||
total_duration: Optional[int]
|
||||
load_duration: Optional[int]
|
||||
prompt_eval_count: Optional[int]
|
||||
prompt_eval_duration: Optional[int]
|
||||
eval_count: Optional[int]
|
||||
eval_duration: Optional[int]
|
||||
|
||||
|
||||
class OllamaVersionResponse(BaseModel):
|
||||
version: str
|
||||
|
||||
|
||||
class OllamaModelDetails(BaseModel):
|
||||
parent_model: str
|
||||
format: str
|
||||
family: str
|
||||
families: List[str]
|
||||
parameter_size: str
|
||||
quantization_level: str
|
||||
|
||||
|
||||
class OllamaModel(BaseModel):
|
||||
name: str
|
||||
model: str
|
||||
size: int
|
||||
digest: str
|
||||
modified_at: str
|
||||
details: OllamaModelDetails
|
||||
|
||||
|
||||
class OllamaTagResponse(BaseModel):
|
||||
models: List[OllamaModel]
|
||||
|
||||
|
||||
class OllamaRunningModelDetails(BaseModel):
|
||||
parent_model: str
|
||||
format: str
|
||||
family: str
|
||||
families: List[str]
|
||||
parameter_size: str
|
||||
quantization_level: str
|
||||
|
||||
|
||||
class OllamaRunningModel(BaseModel):
|
||||
name: str
|
||||
model: str
|
||||
size: int
|
||||
digest: str
|
||||
details: OllamaRunningModelDetails
|
||||
expires_at: str
|
||||
size_vram: int
|
||||
|
||||
|
||||
class OllamaPsResponse(BaseModel):
|
||||
models: List[OllamaRunningModel]
|
||||
|
||||
|
||||
async def parse_request_body(
|
||||
request: Request, model_class: Type[BaseModel]
|
||||
) -> BaseModel:
|
||||
"""
|
||||
Parse request body based on Content-Type header.
|
||||
Supports both application/json and application/octet-stream.
|
||||
|
||||
Args:
|
||||
request: The FastAPI Request object
|
||||
model_class: The Pydantic model class to parse the request into
|
||||
|
||||
Returns:
|
||||
An instance of the provided model_class
|
||||
"""
|
||||
content_type = request.headers.get("content-type", "").lower()
|
||||
|
||||
try:
|
||||
if content_type.startswith("application/json"):
|
||||
# FastAPI already handles JSON parsing for us
|
||||
body = await request.json()
|
||||
elif content_type.startswith("application/octet-stream"):
|
||||
# Manually parse octet-stream as JSON
|
||||
body_bytes = await request.body()
|
||||
body = json.loads(body_bytes.decode("utf-8"))
|
||||
else:
|
||||
# Try to parse as JSON for any other content type
|
||||
body_bytes = await request.body()
|
||||
body = json.loads(body_bytes.decode("utf-8"))
|
||||
|
||||
# Create an instance of the model
|
||||
return model_class(**body)
|
||||
except json.JSONDecodeError:
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON in request body")
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"Error parsing request body: {str(e)}"
|
||||
)
|
||||
|
||||
|
||||
def estimate_tokens(text: str) -> int:
|
||||
"""Estimate the number of tokens in text using tiktoken"""
|
||||
tokens = TiktokenTokenizer().encode(text)
|
||||
return len(tokens)
|
||||
|
||||
|
||||
def parse_query_mode(query: str) -> tuple[str, SearchMode, bool, Optional[str]]:
|
||||
"""Parse query prefix to determine search mode
|
||||
Returns tuple of (cleaned_query, search_mode, only_need_context, user_prompt)
|
||||
|
||||
Examples:
|
||||
- "/local[use mermaid format for diagrams] query string" -> (cleaned_query, SearchMode.local, False, "use mermaid format for diagrams")
|
||||
- "/[use mermaid format for diagrams] query string" -> (cleaned_query, SearchMode.hybrid, False, "use mermaid format for diagrams")
|
||||
- "/local query string" -> (cleaned_query, SearchMode.local, False, None)
|
||||
"""
|
||||
# Initialize user_prompt as None
|
||||
user_prompt = None
|
||||
|
||||
# First check if there's a bracket format for user prompt
|
||||
bracket_pattern = r"^/([a-z]*)\[(.*?)\](.*)"
|
||||
bracket_match = re.match(bracket_pattern, query)
|
||||
|
||||
if bracket_match:
|
||||
mode_prefix = bracket_match.group(1)
|
||||
user_prompt = bracket_match.group(2)
|
||||
remaining_query = bracket_match.group(3).lstrip()
|
||||
|
||||
# Reconstruct query, removing the bracket part
|
||||
query = f"/{mode_prefix} {remaining_query}".strip()
|
||||
|
||||
# Unified handling of mode and only_need_context determination
|
||||
mode_map = {
|
||||
"/local ": (SearchMode.local, False),
|
||||
"/global ": (
|
||||
SearchMode.global_,
|
||||
False,
|
||||
), # global_ is used because 'global' is a Python keyword
|
||||
"/naive ": (SearchMode.naive, False),
|
||||
"/hybrid ": (SearchMode.hybrid, False),
|
||||
"/mix ": (SearchMode.mix, False),
|
||||
"/bypass ": (SearchMode.bypass, False),
|
||||
"/context": (
|
||||
SearchMode.mix,
|
||||
True,
|
||||
),
|
||||
"/localcontext": (SearchMode.local, True),
|
||||
"/globalcontext": (SearchMode.global_, True),
|
||||
"/hybridcontext": (SearchMode.hybrid, True),
|
||||
"/naivecontext": (SearchMode.naive, True),
|
||||
"/mixcontext": (SearchMode.mix, True),
|
||||
}
|
||||
|
||||
for prefix, (mode, only_need_context) in mode_map.items():
|
||||
if query.startswith(prefix):
|
||||
# After removing prefix and leading spaces
|
||||
cleaned_query = query[len(prefix) :].lstrip()
|
||||
return cleaned_query, mode, only_need_context, user_prompt
|
||||
|
||||
return query, SearchMode.mix, False, user_prompt
|
||||
|
||||
|
||||
class OllamaAPI:
|
||||
def __init__(self, rag: LightRAG, top_k: int = 60, api_key: Optional[str] = None):
|
||||
self.rag = rag
|
||||
self.ollama_server_infos = rag.ollama_server_infos
|
||||
self.top_k = top_k
|
||||
self.api_key = api_key
|
||||
self.router = APIRouter(tags=["ollama"])
|
||||
self.setup_routes()
|
||||
|
||||
def setup_routes(self):
|
||||
# Create combined auth dependency for Ollama API routes
|
||||
combined_auth = get_combined_auth_dependency(self.api_key)
|
||||
|
||||
@self.router.get("/version", dependencies=[Depends(combined_auth)])
|
||||
async def get_version():
|
||||
"""Get Ollama version information"""
|
||||
return OllamaVersionResponse(version="0.9.3")
|
||||
|
||||
@self.router.get("/tags", dependencies=[Depends(combined_auth)])
|
||||
async def get_tags():
|
||||
"""Return available models acting as an Ollama server"""
|
||||
return OllamaTagResponse(
|
||||
models=[
|
||||
{
|
||||
"name": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"modified_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"size": self.ollama_server_infos.LIGHTRAG_SIZE,
|
||||
"digest": self.ollama_server_infos.LIGHTRAG_DIGEST,
|
||||
"details": {
|
||||
"parent_model": "",
|
||||
"format": "gguf",
|
||||
"family": self.ollama_server_infos.LIGHTRAG_NAME,
|
||||
"families": [self.ollama_server_infos.LIGHTRAG_NAME],
|
||||
"parameter_size": "13B",
|
||||
"quantization_level": "Q4_0",
|
||||
},
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
@self.router.get("/ps", dependencies=[Depends(combined_auth)])
|
||||
async def get_running_models():
|
||||
"""List Running Models - returns currently running models"""
|
||||
return OllamaPsResponse(
|
||||
models=[
|
||||
{
|
||||
"name": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"size": self.ollama_server_infos.LIGHTRAG_SIZE,
|
||||
"digest": self.ollama_server_infos.LIGHTRAG_DIGEST,
|
||||
"details": {
|
||||
"parent_model": "",
|
||||
"format": "gguf",
|
||||
"family": "llama",
|
||||
"families": ["llama"],
|
||||
"parameter_size": "7.2B",
|
||||
"quantization_level": "Q4_0",
|
||||
},
|
||||
"expires_at": "2050-12-31T14:38:31.83753-07:00",
|
||||
"size_vram": self.ollama_server_infos.LIGHTRAG_SIZE,
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
@self.router.post(
|
||||
"/generate", dependencies=[Depends(combined_auth)], include_in_schema=True
|
||||
)
|
||||
async def generate(raw_request: Request):
|
||||
"""Handle generate completion requests acting as an Ollama model
|
||||
For compatibility purpose, the request is not processed by LightRAG,
|
||||
and will be handled by underlying LLM model.
|
||||
Supports both application/json and application/octet-stream Content-Types.
|
||||
"""
|
||||
try:
|
||||
# Parse the request body manually
|
||||
request = await parse_request_body(raw_request, OllamaGenerateRequest)
|
||||
|
||||
query = request.prompt
|
||||
start_time = time.time_ns()
|
||||
prompt_tokens = estimate_tokens(query)
|
||||
|
||||
if request.system:
|
||||
self.rag.llm_model_kwargs["system_prompt"] = request.system
|
||||
|
||||
if request.stream:
|
||||
response = await self.rag.llm_model_func(
|
||||
query, stream=True, **self.rag.llm_model_kwargs
|
||||
)
|
||||
|
||||
async def stream_generator():
|
||||
try:
|
||||
first_chunk_time = None
|
||||
last_chunk_time = time.time_ns()
|
||||
total_response = ""
|
||||
|
||||
# Ensure response is an async generator
|
||||
if isinstance(response, str):
|
||||
# If it's a string, send in two parts
|
||||
first_chunk_time = start_time
|
||||
last_chunk_time = time.time_ns()
|
||||
total_response = response
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": response,
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
|
||||
completion_tokens = estimate_tokens(total_response)
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": "",
|
||||
"done": True,
|
||||
"done_reason": "stop",
|
||||
"context": [],
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
else:
|
||||
try:
|
||||
async for chunk in response:
|
||||
if chunk:
|
||||
if first_chunk_time is None:
|
||||
first_chunk_time = time.time_ns()
|
||||
|
||||
last_chunk_time = time.time_ns()
|
||||
|
||||
total_response += chunk
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": chunk,
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
except (asyncio.CancelledError, Exception) as e:
|
||||
error_msg = str(e)
|
||||
if isinstance(e, asyncio.CancelledError):
|
||||
error_msg = "Stream was cancelled by server"
|
||||
else:
|
||||
error_msg = f"Provider error: {error_msg}"
|
||||
|
||||
logger.error(f"Stream error: {error_msg}")
|
||||
|
||||
# Send error message to client
|
||||
error_data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": f"\n\nError: {error_msg}",
|
||||
"error": f"\n\nError: {error_msg}",
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(error_data, ensure_ascii=False)}\n"
|
||||
|
||||
# Send final message to close the stream
|
||||
final_data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": "",
|
||||
"done": True,
|
||||
}
|
||||
yield f"{json.dumps(final_data, ensure_ascii=False)}\n"
|
||||
return
|
||||
if first_chunk_time is None:
|
||||
first_chunk_time = start_time
|
||||
completion_tokens = estimate_tokens(total_response)
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": "",
|
||||
"done": True,
|
||||
"done_reason": "stop",
|
||||
"context": [],
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
return
|
||||
|
||||
except Exception as e:
|
||||
trace_exception(e)
|
||||
raise
|
||||
|
||||
return StreamingResponse(
|
||||
stream_generator(),
|
||||
media_type="application/x-ndjson",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"Content-Type": "application/x-ndjson",
|
||||
"X-Accel-Buffering": "no", # Ensure proper handling of streaming responses in Nginx proxy
|
||||
},
|
||||
)
|
||||
else:
|
||||
first_chunk_time = time.time_ns()
|
||||
response_text = await self.rag.llm_model_func(
|
||||
query, stream=False, **self.rag.llm_model_kwargs
|
||||
)
|
||||
last_chunk_time = time.time_ns()
|
||||
|
||||
if not response_text:
|
||||
response_text = "No response generated"
|
||||
|
||||
completion_tokens = estimate_tokens(str(response_text))
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
return {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"response": str(response_text),
|
||||
"done": True,
|
||||
"done_reason": "stop",
|
||||
"context": [],
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
except Exception as e:
|
||||
trace_exception(e)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@self.router.post(
|
||||
"/chat", dependencies=[Depends(combined_auth)], include_in_schema=True
|
||||
)
|
||||
async def chat(raw_request: Request):
|
||||
"""Process chat completion requests by acting as an Ollama model.
|
||||
Routes user queries through LightRAG by selecting query mode based on query prefix.
|
||||
Detects and forwards OpenWebUI session-related requests (for meta data generation task) directly to LLM.
|
||||
Supports both application/json and application/octet-stream Content-Types.
|
||||
"""
|
||||
try:
|
||||
# Parse the request body manually
|
||||
request = await parse_request_body(raw_request, OllamaChatRequest)
|
||||
|
||||
# Get all messages
|
||||
messages = request.messages
|
||||
if not messages:
|
||||
raise HTTPException(status_code=400, detail="No messages provided")
|
||||
|
||||
# Validate that the last message is from a user
|
||||
if messages[-1].role != "user":
|
||||
raise HTTPException(
|
||||
status_code=400, detail="Last message must be from user role"
|
||||
)
|
||||
|
||||
# Get the last message as query and previous messages as history
|
||||
query = messages[-1].content
|
||||
# Convert OllamaMessage objects to dictionaries
|
||||
conversation_history = [
|
||||
{"role": msg.role, "content": msg.content} for msg in messages[:-1]
|
||||
]
|
||||
|
||||
# Check for query prefix
|
||||
cleaned_query, mode, only_need_context, user_prompt = parse_query_mode(
|
||||
query
|
||||
)
|
||||
|
||||
start_time = time.time_ns()
|
||||
prompt_tokens = estimate_tokens(cleaned_query)
|
||||
|
||||
param_dict = {
|
||||
"mode": mode.value,
|
||||
"stream": request.stream,
|
||||
"only_need_context": only_need_context,
|
||||
"conversation_history": conversation_history,
|
||||
"top_k": self.top_k,
|
||||
}
|
||||
|
||||
# Add user_prompt to param_dict
|
||||
if user_prompt is not None:
|
||||
param_dict["user_prompt"] = user_prompt
|
||||
|
||||
query_param = QueryParam(**param_dict)
|
||||
|
||||
if request.stream:
|
||||
# Determine if the request is prefix with "/bypass"
|
||||
if mode == SearchMode.bypass:
|
||||
if request.system:
|
||||
self.rag.llm_model_kwargs["system_prompt"] = request.system
|
||||
response = await self.rag.llm_model_func(
|
||||
cleaned_query,
|
||||
stream=True,
|
||||
history_messages=conversation_history,
|
||||
**self.rag.llm_model_kwargs,
|
||||
)
|
||||
else:
|
||||
response = await self.rag.aquery(
|
||||
cleaned_query, param=query_param
|
||||
)
|
||||
|
||||
async def stream_generator():
|
||||
try:
|
||||
first_chunk_time = None
|
||||
last_chunk_time = time.time_ns()
|
||||
total_response = ""
|
||||
|
||||
# Ensure response is an async generator
|
||||
if isinstance(response, str):
|
||||
# If it's a string, send in two parts
|
||||
first_chunk_time = start_time
|
||||
last_chunk_time = time.time_ns()
|
||||
total_response = response
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": response,
|
||||
"images": None,
|
||||
},
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
|
||||
completion_tokens = estimate_tokens(total_response)
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"images": None,
|
||||
},
|
||||
"done_reason": "stop",
|
||||
"done": True,
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
else:
|
||||
try:
|
||||
async for chunk in response:
|
||||
if chunk:
|
||||
if first_chunk_time is None:
|
||||
first_chunk_time = time.time_ns()
|
||||
|
||||
last_chunk_time = time.time_ns()
|
||||
|
||||
total_response += chunk
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": chunk,
|
||||
"images": None,
|
||||
},
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
except (asyncio.CancelledError, Exception) as e:
|
||||
error_msg = str(e)
|
||||
if isinstance(e, asyncio.CancelledError):
|
||||
error_msg = "Stream was cancelled by server"
|
||||
else:
|
||||
error_msg = f"Provider error: {error_msg}"
|
||||
|
||||
logger.error(f"Stream error: {error_msg}")
|
||||
|
||||
# Send error message to client
|
||||
error_data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": f"\n\nError: {error_msg}",
|
||||
"images": None,
|
||||
},
|
||||
"error": f"\n\nError: {error_msg}",
|
||||
"done": False,
|
||||
}
|
||||
yield f"{json.dumps(error_data, ensure_ascii=False)}\n"
|
||||
|
||||
# Send final message to close the stream
|
||||
final_data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"images": None,
|
||||
},
|
||||
"done": True,
|
||||
}
|
||||
yield f"{json.dumps(final_data, ensure_ascii=False)}\n"
|
||||
return
|
||||
|
||||
if first_chunk_time is None:
|
||||
first_chunk_time = start_time
|
||||
completion_tokens = estimate_tokens(total_response)
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
data = {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "",
|
||||
"images": None,
|
||||
},
|
||||
"done_reason": "stop",
|
||||
"done": True,
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
yield f"{json.dumps(data, ensure_ascii=False)}\n"
|
||||
|
||||
except Exception as e:
|
||||
trace_exception(e)
|
||||
raise
|
||||
|
||||
return StreamingResponse(
|
||||
stream_generator(),
|
||||
media_type="application/x-ndjson",
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"Content-Type": "application/x-ndjson",
|
||||
"X-Accel-Buffering": "no", # Ensure proper handling of streaming responses in Nginx proxy
|
||||
},
|
||||
)
|
||||
else:
|
||||
first_chunk_time = time.time_ns()
|
||||
|
||||
# Determine if the request is prefix with "/bypass" or from Open WebUI's session title and session keyword generation task
|
||||
match_result = re.search(
|
||||
r"\n<chat_history>\nUSER:", cleaned_query, re.MULTILINE
|
||||
)
|
||||
if match_result or mode == SearchMode.bypass:
|
||||
if request.system:
|
||||
self.rag.llm_model_kwargs["system_prompt"] = request.system
|
||||
|
||||
response_text = await self.rag.llm_model_func(
|
||||
cleaned_query,
|
||||
stream=False,
|
||||
history_messages=conversation_history,
|
||||
**self.rag.llm_model_kwargs,
|
||||
)
|
||||
else:
|
||||
response_text = await self.rag.aquery(
|
||||
cleaned_query, param=query_param
|
||||
)
|
||||
|
||||
last_chunk_time = time.time_ns()
|
||||
|
||||
if not response_text:
|
||||
response_text = "No response generated"
|
||||
|
||||
completion_tokens = estimate_tokens(str(response_text))
|
||||
total_time = last_chunk_time - start_time
|
||||
prompt_eval_time = first_chunk_time - start_time
|
||||
eval_time = last_chunk_time - first_chunk_time
|
||||
|
||||
return {
|
||||
"model": self.ollama_server_infos.LIGHTRAG_MODEL,
|
||||
"created_at": self.ollama_server_infos.LIGHTRAG_CREATED_AT,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": str(response_text),
|
||||
"images": None,
|
||||
},
|
||||
"done_reason": "stop",
|
||||
"done": True,
|
||||
"total_duration": total_time,
|
||||
"load_duration": 0,
|
||||
"prompt_eval_count": prompt_tokens,
|
||||
"prompt_eval_duration": prompt_eval_time,
|
||||
"eval_count": completion_tokens,
|
||||
"eval_duration": eval_time,
|
||||
}
|
||||
except Exception as e:
|
||||
trace_exception(e)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,282 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Start LightRAG server with Gunicorn
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import platform
|
||||
import pipmaster as pm
|
||||
from lightrag.api.utils_api import display_splash_screen, check_env_file
|
||||
from lightrag.api.config import global_args
|
||||
from lightrag.utils import get_env_value
|
||||
from lightrag.kg.shared_storage import initialize_share_data
|
||||
|
||||
from lightrag.constants import (
|
||||
DEFAULT_WOKERS,
|
||||
DEFAULT_TIMEOUT,
|
||||
)
|
||||
|
||||
|
||||
def check_and_install_dependencies():
|
||||
"""Check and install required dependencies"""
|
||||
required_packages = [
|
||||
"gunicorn",
|
||||
"tiktoken",
|
||||
"psutil",
|
||||
# Add other required packages here
|
||||
]
|
||||
|
||||
for package in required_packages:
|
||||
if not pm.is_installed(package):
|
||||
print(f"Installing {package}...")
|
||||
pm.install(package)
|
||||
print(f"{package} installed successfully")
|
||||
|
||||
|
||||
def main():
|
||||
# Explicitly initialize configuration for Gunicorn mode
|
||||
from lightrag.api.config import initialize_config
|
||||
|
||||
initialize_config()
|
||||
|
||||
# Set Gunicorn mode flag for lifespan cleanup detection
|
||||
os.environ["LIGHTRAG_GUNICORN_MODE"] = "1"
|
||||
|
||||
# Check .env file
|
||||
if not check_env_file():
|
||||
sys.exit(1)
|
||||
|
||||
# Check DOCLING compatibility with Gunicorn multi-worker mode on macOS
|
||||
if (
|
||||
platform.system() == "Darwin"
|
||||
and global_args.document_loading_engine == "DOCLING"
|
||||
and global_args.workers > 1
|
||||
):
|
||||
print("\n" + "=" * 80)
|
||||
print("❌ ERROR: Incompatible configuration detected!")
|
||||
print("=" * 80)
|
||||
print(
|
||||
"\nDOCLING engine with Gunicorn multi-worker mode is not supported on macOS"
|
||||
)
|
||||
print("\nReason:")
|
||||
print(" PyTorch (required by DOCLING) has known compatibility issues with")
|
||||
print(" fork-based multiprocessing on macOS, which can cause crashes or")
|
||||
print(" unexpected behavior when using Gunicorn with multiple workers.")
|
||||
print("\nCurrent configuration:")
|
||||
print(" - Operating System: macOS (Darwin)")
|
||||
print(f" - Document Engine: {global_args.document_loading_engine}")
|
||||
print(f" - Workers: {global_args.workers}")
|
||||
print("\nPossible solutions:")
|
||||
print(" 1. Use single worker mode:")
|
||||
print(" --workers 1")
|
||||
print("\n 2. Change document loading engine in .env:")
|
||||
print(" DOCUMENT_LOADING_ENGINE=DEFAULT")
|
||||
print("\n 3. Deploy on Linux where multi-worker mode is fully supported")
|
||||
print("=" * 80 + "\n")
|
||||
sys.exit(1)
|
||||
|
||||
# Check macOS fork safety environment variable for multi-worker mode
|
||||
if (
|
||||
platform.system() == "Darwin"
|
||||
and global_args.workers > 1
|
||||
and os.environ.get("OBJC_DISABLE_INITIALIZE_FORK_SAFETY") != "YES"
|
||||
):
|
||||
print("\n" + "=" * 80)
|
||||
print("❌ ERROR: Missing required environment variable on macOS!")
|
||||
print("=" * 80)
|
||||
print("\nmacOS with Gunicorn multi-worker mode requires:")
|
||||
print(" OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES")
|
||||
print("\nReason:")
|
||||
print(" NumPy uses macOS's Accelerate framework (Objective-C based) for")
|
||||
print(" vector computations. The Objective-C runtime has fork safety checks")
|
||||
print(" that will crash worker processes when embedding functions are called.")
|
||||
print("\nCurrent configuration:")
|
||||
print(" - Operating System: macOS (Darwin)")
|
||||
print(f" - Workers: {global_args.workers}")
|
||||
print(
|
||||
f" - Environment Variable: {os.environ.get('OBJC_DISABLE_INITIALIZE_FORK_SAFETY', 'NOT SET')}"
|
||||
)
|
||||
print("\nHow to fix:")
|
||||
print(" Option 1 - Set environment variable before starting (recommended):")
|
||||
print(" export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES")
|
||||
print(" lightrag-server")
|
||||
print("\n Option 2 - Add to your shell profile (~/.zshrc or ~/.bash_profile):")
|
||||
print(" echo 'export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES' >> ~/.zshrc")
|
||||
print(" source ~/.zshrc")
|
||||
print("\n Option 3 - Use single worker mode (no multiprocessing):")
|
||||
print(" lightrag-server --workers 1")
|
||||
print("=" * 80 + "\n")
|
||||
sys.exit(1)
|
||||
|
||||
# Check and install dependencies
|
||||
check_and_install_dependencies()
|
||||
|
||||
# Note: Signal handlers are NOT registered here because:
|
||||
# - Master cleanup already handled by gunicorn_config.on_exit()
|
||||
|
||||
# Display startup information
|
||||
display_splash_screen(global_args)
|
||||
|
||||
print("🚀 Starting LightRAG with Gunicorn")
|
||||
print(f"🔄 Worker management: Gunicorn (workers={global_args.workers})")
|
||||
print("🔍 Preloading app: Enabled")
|
||||
print("📝 Note: Using Gunicorn's preload feature for shared data initialization")
|
||||
print("\n\n" + "=" * 80)
|
||||
print("MAIN PROCESS INITIALIZATION")
|
||||
print(f"Process ID: {os.getpid()}")
|
||||
print(f"Workers setting: {global_args.workers}")
|
||||
print("=" * 80 + "\n")
|
||||
|
||||
# Import Gunicorn's StandaloneApplication
|
||||
from gunicorn.app.base import BaseApplication
|
||||
|
||||
# Define a custom application class that loads our config
|
||||
class GunicornApp(BaseApplication):
|
||||
def __init__(self, app, options=None):
|
||||
self.options = options or {}
|
||||
self.application = app
|
||||
super().__init__()
|
||||
|
||||
def load_config(self):
|
||||
# Define valid Gunicorn configuration options
|
||||
valid_options = {
|
||||
"bind",
|
||||
"workers",
|
||||
"worker_class",
|
||||
"timeout",
|
||||
"keepalive",
|
||||
"preload_app",
|
||||
"errorlog",
|
||||
"accesslog",
|
||||
"loglevel",
|
||||
"certfile",
|
||||
"keyfile",
|
||||
"limit_request_line",
|
||||
"limit_request_fields",
|
||||
"limit_request_field_size",
|
||||
"graceful_timeout",
|
||||
"max_requests",
|
||||
"max_requests_jitter",
|
||||
}
|
||||
|
||||
# Special hooks that need to be set separately
|
||||
special_hooks = {
|
||||
"on_starting",
|
||||
"on_reload",
|
||||
"on_exit",
|
||||
"pre_fork",
|
||||
"post_fork",
|
||||
"pre_exec",
|
||||
"pre_request",
|
||||
"post_request",
|
||||
"worker_init",
|
||||
"worker_exit",
|
||||
"nworkers_changed",
|
||||
"child_exit",
|
||||
}
|
||||
|
||||
# Import and configure the gunicorn_config module
|
||||
from lightrag.api import gunicorn_config
|
||||
|
||||
# Set configuration variables in gunicorn_config, prioritizing command line arguments
|
||||
gunicorn_config.workers = (
|
||||
global_args.workers
|
||||
if global_args.workers
|
||||
else get_env_value("WORKERS", DEFAULT_WOKERS, int)
|
||||
)
|
||||
|
||||
# Bind configuration prioritizes command line arguments
|
||||
host = (
|
||||
global_args.host
|
||||
if global_args.host != "0.0.0.0"
|
||||
else os.getenv("HOST", "0.0.0.0")
|
||||
)
|
||||
port = (
|
||||
global_args.port
|
||||
if global_args.port != 9621
|
||||
else get_env_value("PORT", 9621, int)
|
||||
)
|
||||
gunicorn_config.bind = f"{host}:{port}"
|
||||
|
||||
# Log level configuration prioritizes command line arguments
|
||||
gunicorn_config.loglevel = (
|
||||
global_args.log_level.lower()
|
||||
if global_args.log_level
|
||||
else os.getenv("LOG_LEVEL", "info")
|
||||
)
|
||||
|
||||
# Timeout configuration prioritizes command line arguments
|
||||
gunicorn_config.timeout = (
|
||||
global_args.timeout + 30
|
||||
if global_args.timeout is not None
|
||||
else get_env_value(
|
||||
"TIMEOUT", DEFAULT_TIMEOUT + 30, int, special_none=True
|
||||
)
|
||||
)
|
||||
|
||||
# Keepalive configuration
|
||||
gunicorn_config.keepalive = get_env_value("KEEPALIVE", 5, int)
|
||||
|
||||
# SSL configuration prioritizes command line arguments
|
||||
if global_args.ssl or os.getenv("SSL", "").lower() in (
|
||||
"true",
|
||||
"1",
|
||||
"yes",
|
||||
"t",
|
||||
"on",
|
||||
):
|
||||
gunicorn_config.certfile = (
|
||||
global_args.ssl_certfile
|
||||
if global_args.ssl_certfile
|
||||
else os.getenv("SSL_CERTFILE")
|
||||
)
|
||||
gunicorn_config.keyfile = (
|
||||
global_args.ssl_keyfile
|
||||
if global_args.ssl_keyfile
|
||||
else os.getenv("SSL_KEYFILE")
|
||||
)
|
||||
|
||||
# Set configuration options from the module
|
||||
for key in dir(gunicorn_config):
|
||||
if key in valid_options:
|
||||
value = getattr(gunicorn_config, key)
|
||||
# Skip functions like on_starting and None values
|
||||
if not callable(value) and value is not None:
|
||||
self.cfg.set(key, value)
|
||||
# Set special hooks
|
||||
elif key in special_hooks:
|
||||
value = getattr(gunicorn_config, key)
|
||||
if callable(value):
|
||||
self.cfg.set(key, value)
|
||||
|
||||
if hasattr(gunicorn_config, "logconfig_dict"):
|
||||
self.cfg.set(
|
||||
"logconfig_dict", getattr(gunicorn_config, "logconfig_dict")
|
||||
)
|
||||
|
||||
def load(self):
|
||||
# Import the application
|
||||
from lightrag.api.lightrag_server import get_application
|
||||
|
||||
return get_application(global_args)
|
||||
|
||||
# Create the application
|
||||
app = GunicornApp("")
|
||||
|
||||
# Force workers to be an integer and greater than 1 for multi-process mode
|
||||
workers_count = global_args.workers
|
||||
if workers_count > 1:
|
||||
# Set a flag to indicate we're in the main process
|
||||
os.environ["LIGHTRAG_MAIN_PROCESS"] = "1"
|
||||
initialize_share_data(workers_count)
|
||||
else:
|
||||
initialize_share_data(1)
|
||||
|
||||
# Run the application
|
||||
print("\nStarting Gunicorn with direct Python API...")
|
||||
app.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 628 B |
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,342 @@
|
||||
"""
|
||||
Utility functions for the LightRAG API.
|
||||
"""
|
||||
|
||||
import os
|
||||
import argparse
|
||||
from typing import Optional, List, Tuple
|
||||
import sys
|
||||
from ascii_colors import ASCIIColors
|
||||
from lightrag.api import __api_version__ as api_version
|
||||
from lightrag import __version__ as core_version
|
||||
from lightrag.constants import (
|
||||
DEFAULT_FORCE_LLM_SUMMARY_ON_MERGE,
|
||||
)
|
||||
from fastapi import HTTPException, Security, Request, status
|
||||
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
|
||||
from starlette.status import HTTP_403_FORBIDDEN
|
||||
from .auth import auth_handler
|
||||
from .config import ollama_server_infos, global_args, get_env_value
|
||||
|
||||
|
||||
def check_env_file():
|
||||
"""
|
||||
Check if .env file exists and handle user confirmation if needed.
|
||||
Returns True if should continue, False if should exit.
|
||||
"""
|
||||
if not os.path.exists(".env"):
|
||||
warning_msg = "Warning: Startup directory must contain .env file for multi-instance support."
|
||||
ASCIIColors.yellow(warning_msg)
|
||||
|
||||
# Check if running in interactive terminal
|
||||
if sys.stdin.isatty():
|
||||
response = input("Do you want to continue? (yes/no): ")
|
||||
if response.lower() != "yes":
|
||||
ASCIIColors.red("Server startup cancelled")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
# Get whitelist paths from global_args, only once during initialization
|
||||
whitelist_paths = global_args.whitelist_paths.split(",")
|
||||
|
||||
# Pre-compile path matching patterns
|
||||
whitelist_patterns: List[Tuple[str, bool]] = []
|
||||
for path in whitelist_paths:
|
||||
path = path.strip()
|
||||
if path:
|
||||
# If path ends with /*, match all paths with that prefix
|
||||
if path.endswith("/*"):
|
||||
prefix = path[:-2]
|
||||
whitelist_patterns.append((prefix, True)) # (prefix, is_prefix_match)
|
||||
else:
|
||||
whitelist_patterns.append((path, False)) # (exact_path, is_prefix_match)
|
||||
|
||||
# Global authentication configuration
|
||||
auth_configured = bool(auth_handler.accounts)
|
||||
|
||||
|
||||
def get_combined_auth_dependency(api_key: Optional[str] = None):
|
||||
"""
|
||||
Create a combined authentication dependency that implements authentication logic
|
||||
based on API key, OAuth2 token, and whitelist paths.
|
||||
|
||||
Args:
|
||||
api_key (Optional[str]): API key for validation
|
||||
|
||||
Returns:
|
||||
Callable: A dependency function that implements the authentication logic
|
||||
"""
|
||||
# Use global whitelist_patterns and auth_configured variables
|
||||
# whitelist_patterns and auth_configured are already initialized at module level
|
||||
|
||||
# Only calculate api_key_configured as it depends on the function parameter
|
||||
api_key_configured = bool(api_key)
|
||||
|
||||
# Create security dependencies with proper descriptions for Swagger UI
|
||||
oauth2_scheme = OAuth2PasswordBearer(
|
||||
tokenUrl="login", auto_error=False, description="OAuth2 Password Authentication"
|
||||
)
|
||||
|
||||
# If API key is configured, create an API key header security
|
||||
api_key_header = None
|
||||
if api_key_configured:
|
||||
api_key_header = APIKeyHeader(
|
||||
name="X-API-Key", auto_error=False, description="API Key Authentication"
|
||||
)
|
||||
|
||||
async def combined_dependency(
|
||||
request: Request,
|
||||
token: str = Security(oauth2_scheme),
|
||||
api_key_header_value: Optional[str] = None
|
||||
if api_key_header is None
|
||||
else Security(api_key_header),
|
||||
):
|
||||
# 1. Check if path is in whitelist
|
||||
path = request.url.path
|
||||
for pattern, is_prefix in whitelist_patterns:
|
||||
if (is_prefix and path.startswith(pattern)) or (
|
||||
not is_prefix and path == pattern
|
||||
):
|
||||
return # Whitelist path, allow access
|
||||
|
||||
# 2. Validate token first if provided in the request (Ensure 401 error if token is invalid)
|
||||
if token:
|
||||
try:
|
||||
token_info = auth_handler.validate_token(token)
|
||||
# Accept guest token if no auth is configured
|
||||
if not auth_configured and token_info.get("role") == "guest":
|
||||
return
|
||||
# Accept non-guest token if auth is configured
|
||||
if auth_configured and token_info.get("role") != "guest":
|
||||
return
|
||||
|
||||
# Token validation failed, immediately return 401 error
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid token. Please login again.",
|
||||
)
|
||||
except HTTPException as e:
|
||||
# If already a 401 error, re-raise it
|
||||
if e.status_code == status.HTTP_401_UNAUTHORIZED:
|
||||
raise
|
||||
# For other exceptions, continue processing
|
||||
|
||||
# 3. Acept all request if no API protection needed
|
||||
if not auth_configured and not api_key_configured:
|
||||
return
|
||||
|
||||
# 4. Validate API key if provided and API-Key authentication is configured
|
||||
if (
|
||||
api_key_configured
|
||||
and api_key_header_value
|
||||
and api_key_header_value == api_key
|
||||
):
|
||||
return # API key validation successful
|
||||
|
||||
### Authentication failed ####
|
||||
|
||||
# if password authentication is configured but not provided, ensure 401 error if auth_configured
|
||||
if auth_configured and not token:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="No credentials provided. Please login.",
|
||||
)
|
||||
|
||||
# if api key is provided but validation failed
|
||||
if api_key_header_value:
|
||||
raise HTTPException(
|
||||
status_code=HTTP_403_FORBIDDEN,
|
||||
detail="Invalid API Key",
|
||||
)
|
||||
|
||||
# if api_key_configured but not provided
|
||||
if api_key_configured and not api_key_header_value:
|
||||
raise HTTPException(
|
||||
status_code=HTTP_403_FORBIDDEN,
|
||||
detail="API Key required",
|
||||
)
|
||||
|
||||
# Otherwise: refuse access and return 403 error
|
||||
raise HTTPException(
|
||||
status_code=HTTP_403_FORBIDDEN,
|
||||
detail="API Key required or login authentication required.",
|
||||
)
|
||||
|
||||
return combined_dependency
|
||||
|
||||
|
||||
def display_splash_screen(args: argparse.Namespace) -> None:
|
||||
"""
|
||||
Display a colorful splash screen showing LightRAG server configuration
|
||||
|
||||
Args:
|
||||
args: Parsed command line arguments
|
||||
"""
|
||||
# Banner
|
||||
# Banner
|
||||
top_border = "╔══════════════════════════════════════════════════════════════╗"
|
||||
bottom_border = "╚══════════════════════════════════════════════════════════════╝"
|
||||
width = len(top_border) - 4 # width inside the borders
|
||||
|
||||
line1_text = f"LightRAG Server v{core_version}/{api_version}"
|
||||
line2_text = "Fast, Lightweight RAG Server Implementation"
|
||||
|
||||
line1 = f"║ {line1_text.center(width)} ║"
|
||||
line2 = f"║ {line2_text.center(width)} ║"
|
||||
|
||||
banner = f"""
|
||||
{top_border}
|
||||
{line1}
|
||||
{line2}
|
||||
{bottom_border}
|
||||
"""
|
||||
ASCIIColors.cyan(banner)
|
||||
|
||||
# Server Configuration
|
||||
ASCIIColors.magenta("\n📡 Server Configuration:")
|
||||
ASCIIColors.white(" ├─ Host: ", end="")
|
||||
ASCIIColors.yellow(f"{args.host}")
|
||||
ASCIIColors.white(" ├─ Port: ", end="")
|
||||
ASCIIColors.yellow(f"{args.port}")
|
||||
ASCIIColors.white(" ├─ Workers: ", end="")
|
||||
ASCIIColors.yellow(f"{args.workers}")
|
||||
ASCIIColors.white(" ├─ Timeout: ", end="")
|
||||
ASCIIColors.yellow(f"{args.timeout}")
|
||||
ASCIIColors.white(" ├─ CORS Origins: ", end="")
|
||||
ASCIIColors.yellow(f"{args.cors_origins}")
|
||||
ASCIIColors.white(" ├─ SSL Enabled: ", end="")
|
||||
ASCIIColors.yellow(f"{args.ssl}")
|
||||
if args.ssl:
|
||||
ASCIIColors.white(" ├─ SSL Cert: ", end="")
|
||||
ASCIIColors.yellow(f"{args.ssl_certfile}")
|
||||
ASCIIColors.white(" ├─ SSL Key: ", end="")
|
||||
ASCIIColors.yellow(f"{args.ssl_keyfile}")
|
||||
ASCIIColors.white(" ├─ Ollama Emulating Model: ", end="")
|
||||
ASCIIColors.yellow(f"{ollama_server_infos.LIGHTRAG_MODEL}")
|
||||
ASCIIColors.white(" ├─ Log Level: ", end="")
|
||||
ASCIIColors.yellow(f"{args.log_level}")
|
||||
ASCIIColors.white(" ├─ Verbose Debug: ", end="")
|
||||
ASCIIColors.yellow(f"{args.verbose}")
|
||||
ASCIIColors.white(" ├─ API Key: ", end="")
|
||||
ASCIIColors.yellow("Set" if args.key else "Not Set")
|
||||
ASCIIColors.white(" └─ JWT Auth: ", end="")
|
||||
ASCIIColors.yellow("Enabled" if args.auth_accounts else "Disabled")
|
||||
|
||||
# Directory Configuration
|
||||
ASCIIColors.magenta("\n📂 Directory Configuration:")
|
||||
ASCIIColors.white(" ├─ Working Directory: ", end="")
|
||||
ASCIIColors.yellow(f"{args.working_dir}")
|
||||
ASCIIColors.white(" └─ Input Directory: ", end="")
|
||||
ASCIIColors.yellow(f"{args.input_dir}")
|
||||
|
||||
# LLM Configuration
|
||||
ASCIIColors.magenta("\n🤖 LLM Configuration:")
|
||||
ASCIIColors.white(" ├─ Binding: ", end="")
|
||||
ASCIIColors.yellow(f"{args.llm_binding}")
|
||||
ASCIIColors.white(" ├─ Host: ", end="")
|
||||
ASCIIColors.yellow(f"{args.llm_binding_host}")
|
||||
ASCIIColors.white(" ├─ Model: ", end="")
|
||||
ASCIIColors.yellow(f"{args.llm_model}")
|
||||
ASCIIColors.white(" ├─ Max Async for LLM: ", end="")
|
||||
ASCIIColors.yellow(f"{args.max_async}")
|
||||
ASCIIColors.white(" ├─ Summary Context Size: ", end="")
|
||||
ASCIIColors.yellow(f"{args.summary_context_size}")
|
||||
ASCIIColors.white(" ├─ LLM Cache Enabled: ", end="")
|
||||
ASCIIColors.yellow(f"{args.enable_llm_cache}")
|
||||
ASCIIColors.white(" └─ LLM Cache for Extraction Enabled: ", end="")
|
||||
ASCIIColors.yellow(f"{args.enable_llm_cache_for_extract}")
|
||||
|
||||
# Embedding Configuration
|
||||
ASCIIColors.magenta("\n📊 Embedding Configuration:")
|
||||
ASCIIColors.white(" ├─ Binding: ", end="")
|
||||
ASCIIColors.yellow(f"{args.embedding_binding}")
|
||||
ASCIIColors.white(" ├─ Host: ", end="")
|
||||
ASCIIColors.yellow(f"{args.embedding_binding_host}")
|
||||
ASCIIColors.white(" ├─ Model: ", end="")
|
||||
ASCIIColors.yellow(f"{args.embedding_model}")
|
||||
ASCIIColors.white(" └─ Dimensions: ", end="")
|
||||
ASCIIColors.yellow(f"{args.embedding_dim}")
|
||||
|
||||
# RAG Configuration
|
||||
ASCIIColors.magenta("\n⚙️ RAG Configuration:")
|
||||
ASCIIColors.white(" ├─ Summary Language: ", end="")
|
||||
ASCIIColors.yellow(f"{args.summary_language}")
|
||||
ASCIIColors.white(" ├─ Entity Types: ", end="")
|
||||
ASCIIColors.yellow(f"{args.entity_types}")
|
||||
ASCIIColors.white(" ├─ Max Parallel Insert: ", end="")
|
||||
ASCIIColors.yellow(f"{args.max_parallel_insert}")
|
||||
ASCIIColors.white(" ├─ Chunk Size: ", end="")
|
||||
ASCIIColors.yellow(f"{args.chunk_size}")
|
||||
ASCIIColors.white(" ├─ Chunk Overlap Size: ", end="")
|
||||
ASCIIColors.yellow(f"{args.chunk_overlap_size}")
|
||||
ASCIIColors.white(" ├─ Cosine Threshold: ", end="")
|
||||
ASCIIColors.yellow(f"{args.cosine_threshold}")
|
||||
ASCIIColors.white(" ├─ Top-K: ", end="")
|
||||
ASCIIColors.yellow(f"{args.top_k}")
|
||||
ASCIIColors.white(" └─ Force LLM Summary on Merge: ", end="")
|
||||
ASCIIColors.yellow(
|
||||
f"{get_env_value('FORCE_LLM_SUMMARY_ON_MERGE', DEFAULT_FORCE_LLM_SUMMARY_ON_MERGE, int)}"
|
||||
)
|
||||
|
||||
# System Configuration
|
||||
ASCIIColors.magenta("\n💾 Storage Configuration:")
|
||||
ASCIIColors.white(" ├─ KV Storage: ", end="")
|
||||
ASCIIColors.yellow(f"{args.kv_storage}")
|
||||
ASCIIColors.white(" ├─ Vector Storage: ", end="")
|
||||
ASCIIColors.yellow(f"{args.vector_storage}")
|
||||
ASCIIColors.white(" ├─ Graph Storage: ", end="")
|
||||
ASCIIColors.yellow(f"{args.graph_storage}")
|
||||
ASCIIColors.white(" ├─ Document Status Storage: ", end="")
|
||||
ASCIIColors.yellow(f"{args.doc_status_storage}")
|
||||
ASCIIColors.white(" └─ Workspace: ", end="")
|
||||
ASCIIColors.yellow(f"{args.workspace if args.workspace else '-'}")
|
||||
|
||||
# Server Status
|
||||
ASCIIColors.green("\n✨ Server starting up...\n")
|
||||
|
||||
# Server Access Information
|
||||
protocol = "https" if args.ssl else "http"
|
||||
if args.host == "0.0.0.0":
|
||||
ASCIIColors.magenta("\n🌐 Server Access Information:")
|
||||
ASCIIColors.white(" ├─ WebUI (local): ", end="")
|
||||
ASCIIColors.yellow(f"{protocol}://localhost:{args.port}")
|
||||
ASCIIColors.white(" ├─ Remote Access: ", end="")
|
||||
ASCIIColors.yellow(f"{protocol}://<your-ip-address>:{args.port}")
|
||||
ASCIIColors.white(" ├─ API Documentation (local): ", end="")
|
||||
ASCIIColors.yellow(f"{protocol}://localhost:{args.port}/docs")
|
||||
ASCIIColors.white(" └─ Alternative Documentation (local): ", end="")
|
||||
ASCIIColors.yellow(f"{protocol}://localhost:{args.port}/redoc")
|
||||
|
||||
ASCIIColors.magenta("\n📝 Note:")
|
||||
ASCIIColors.cyan(""" Since the server is running on 0.0.0.0:
|
||||
- Use 'localhost' or '127.0.0.1' for local access
|
||||
- Use your machine's IP address for remote access
|
||||
- To find your IP address:
|
||||
• Windows: Run 'ipconfig' in terminal
|
||||
• Linux/Mac: Run 'ifconfig' or 'ip addr' in terminal
|
||||
""")
|
||||
else:
|
||||
base_url = f"{protocol}://{args.host}:{args.port}"
|
||||
ASCIIColors.magenta("\n🌐 Server Access Information:")
|
||||
ASCIIColors.white(" ├─ WebUI (local): ", end="")
|
||||
ASCIIColors.yellow(f"{base_url}")
|
||||
ASCIIColors.white(" ├─ API Documentation: ", end="")
|
||||
ASCIIColors.yellow(f"{base_url}/docs")
|
||||
ASCIIColors.white(" └─ Alternative Documentation: ", end="")
|
||||
ASCIIColors.yellow(f"{base_url}/redoc")
|
||||
|
||||
# Security Notice
|
||||
if args.key:
|
||||
ASCIIColors.yellow("\n⚠️ Security Notice:")
|
||||
ASCIIColors.white(""" API Key authentication is enabled.
|
||||
Make sure to include the X-API-Key header in all your requests.
|
||||
""")
|
||||
if args.auth_accounts:
|
||||
ASCIIColors.yellow("\n⚠️ Security Notice:")
|
||||
ASCIIColors.white(""" JWT authentication is enabled.
|
||||
Make sure to login before making the request, and include the 'Authorization' in the header.
|
||||
""")
|
||||
|
||||
# Ensure splash output flush to system log
|
||||
sys.stdout.flush()
|
||||
Reference in New Issue
Block a user