mnote/docs/superpowers/plans/2026-05-29-local-folder-mineru-ocr-sidecar.md

# Local Folder MinerU OCR Sidecar Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** 为 mnote local-folder 图片和图片型 PDF 增加手动 MinerU OCR，并把 OCR Markdown 保存到 owner 页面同目录的 `{pageStem}.ocr/` 文件夹。

**Architecture:** 后端负责 MinerU API、权限校验、OCR active job store、OCR sidecar 写入和 `.mnote/ocr-index.json` 状态缓存；前端触发本地 OCR API，并通过资源局部状态和全局 OCR 任务栏展示阶段型进度。OCR 文本默认作为资源 sidecar 被搜索、resource tab 和后续 AI 消费，只有用户显式操作时才插入正文。

**Tech Stack:** Rust `mnote-web` routes/services、local-folder metadata、MinerU HTTP API、browser runtime JS、Node smoke、Rust cargo tests。

---

## File Structure

- Create: `rust/crates/mnote-web/src/routes/local_ocr.rs`
  - OCR API request/response types、路径规划、sidecar frontmatter、active job store、`.mnote/ocr-index.json` 读写、MinerU client facade。
- Modify: `rust/crates/mnote-web/src/routes/mod.rs`
  - 注册 `/api/local-folder/ocr/jobs`、`/jobs/:id`、`/status`、`/read`、`/insert`。
- Modify: `rust/crates/mnote-web/src/routes/local_folder_source.rs`
  - 暴露必要的 local-folder path helper，识别 `.ocr/*.ocr.md` 为 OCR resource 而不是普通页面。
- Modify: `rust/crates/mnote-web/src/routes/local_search_index.rs`
  - 读取 OCR index/sidecar，`includeOcr=true` 时搜索 OCR 文本并返回 owner page。
- Modify: `rust/crates/mnote-web/src/routes/search.rs`
  - local-folder 分支把 `filters.include_ocr` 传给 local search index。
- Modify: `rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js`
  - 附件菜单展示 OCR 识别、查看 OCR、插入 OCR 链接。
- Modify: `rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js`
  - File Tree 图片/PDF 资源右键菜单接入 OCR 操作。
- Modify: `rust/crates/mnote-web/browser/document-resource-tab-runtime.js`
  - image/PDF resource tab 展示 OCR 状态和 OCR Markdown。
- Create: `rust/crates/mnote-web/browser/local-ocr-task-runtime.js`
  - 全局 OCR 任务栏 / 任务抽屉，消费 OCR jobs API 和 `local_ocr.job.updated` 事件。
- Modify: `rust/crates/mnote-web/src/ssr/pages/layout.rs`
  - 挂载全局 OCR task runtime，并断言 runtime 依赖显式注入。
- Test: `rust/crates/mnote-web/src/routes/local_ocr.rs` inline tests
- Test: `rust/crates/mnote-web/src/routes/local_search_index.rs` inline tests
- Create: `scripts/task506-local-folder-mineru-ocr-smoke.js`
  - 浏览器 smoke，使用 mock MinerU 或 test-only route fixture。

## Task 1: OCR Sidecar Path And Metadata Contract

**Files:**
- Create: `rust/crates/mnote-web/src/routes/local_ocr.rs`

- [ ] **Step 1: Add failing unit tests for sidecar path planning**

Add tests that assert:

- owner `docs/Page.md` creates OCR dir `docs/Page.ocr/`
- source `docs/Page.assets/photo.png` creates `docs/Page.ocr/photo.png.ocr.md`
- duplicate source leaf names get a short hash suffix
- OCR frontmatter contains `owner_document`, `source_path`, `source_root_relative_path`, `provider`, `model_version`, `source_size`, `source_mtime_ms`

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_sidecar -- --test-threads=1
```

Expected: FAIL because `local_ocr` does not exist yet.

- [ ] **Step 2: Implement path planning and frontmatter builder**

Implement focused helpers in `local_ocr.rs`:

- `plan_ocr_sidecar_path(root, owner_document_path, source_root_relative_path, source_metadata)`
- `build_ocr_markdown(frontmatter, mineru_markdown)`
- `parse_ocr_frontmatter(markdown)`

Keep these helpers pure and unit-testable.

- [ ] **Step 3: Verify tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_sidecar -- --test-threads=1
```

Expected: PASS.

## Task 2: OCR Index Cache

**Files:**
- Modify: `rust/crates/mnote-web/src/routes/local_ocr.rs`

- [ ] **Step 1: Add failing tests for `.mnote/ocr-index.json`**

Cover:

- write/read one entry
- `stale=true` when source size or mtime changed
- missing source returns `stale=true` with no panic
- failed entry stores only a short sanitized error

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_index -- --test-threads=1
```

Expected: FAIL.

- [ ] **Step 2: Implement index read/write and stale detection**

Use atomic JSON write, following existing `write_json_atomic` style in `local_folder_source.rs`.

Index path:

```text
<root>/.mnote/ocr-index.json
```

Entry key:

```text
sourceRootRelativePath
```

- [ ] **Step 3: Verify tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_index -- --test-threads=1
```

Expected: PASS.

## Task 3: MinerU Client And OCR Job API

**Files:**
- Modify: `rust/crates/mnote-web/src/routes/local_ocr.rs`
- Modify: `rust/crates/mnote-web/src/routes/mod.rs`

- [ ] **Step 1: Add route tests with a mock MinerU client**

Cover:

- missing token returns `mineru_token_missing`
- root escape source path is rejected
- successful mock OCR writes `{pageStem}.ocr/*.ocr.md`
- successful mock OCR exposes queued/running/done states through jobs list
- each mock OCR state change emits `local_ocr.job.updated`
- failed mock OCR writes failed index entry without token or upload URL

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_jobs -- --test-threads=1
```

Expected: FAIL.

- [ ] **Step 2: Implement API routes**

Routes:

- `POST /api/local-folder/ocr/jobs`
- `GET /api/local-folder/ocr/jobs`
- `GET /api/local-folder/ocr/jobs/{jobId}`
- `GET /api/local-folder/ocr/status`
- `GET /api/local-folder/ocr/read`

Token lookup order:

1. `MNOTE_MINERU_API_TOKEN`
2. `MINERU_API_TOKEN`
3. `~/.hermes/.env` `MINERU_API_TOKEN` for local development

- [ ] **Step 3: Implement MinerU local file flow**

Use MinerU API:

- `POST /api/v4/file-urls/batch`
- `PUT` local bytes to pre-signed URL
- `GET /api/v4/extract-results/batch/{batch_id}`
- download zip
- extract first Markdown file

Never log token or pre-signed URL.

- [ ] **Step 4: Implement active job store and status stages**

Add in-process active job state with these statuses:

```text
queued
uploading
mineru_processing
downloading
writing_sidecar
done
failed
interrupted
stale
```

Each state change must update `.mnote/ocr-index.json` and return a redacted job payload. On process restart, any persisted `queued/uploading/mineru_processing/downloading/writing_sidecar` entry without active in-memory job is exposed as `interrupted`.

- [ ] **Step 5: Broadcast OCR job events**

Emit a local realtime event for every state change:

```json
{
  "type": "local_ocr.job.updated",
  "jobId": "ocr_1760000000000_abcd",
  "rootUri": "file:///tmp/mnote-ocr",
  "ownerDocumentId": "local-md:docs~2FPage.md",
  "sourceRootRelativePath": "docs/Page.assets/spec.pdf",
  "ocrRootRelativePath": "docs/Page.ocr/spec.pdf.ocr.md",
  "status": "mineru_processing",
  "stageLabel": "识别中",
  "updatedAtMs": 1760000000000
}
```

Use the existing realtime / WS / SSE event path. Do not add a foreground `setInterval` polling loop as the primary update mechanism.

- [ ] **Step 6: Register routes**

Add route registrations in `routes/mod.rs`.

- [ ] **Step 7: Verify route tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_jobs -- --test-threads=1
```

Expected: PASS.

## Task 4: Keep OCR Markdown Out Of Page Tree

**Files:**
- Modify: `rust/crates/mnote-web/src/routes/local_folder_source.rs`
- Modify: `rust/crates/mnote-web/src/routes/local_search_index.rs`

- [ ] **Step 1: Add failing tests for `.ocr/*.ocr.md` classification**

Cover:

- `docs/Page.ocr/photo.png.ocr.md` is not a normal local Markdown document
- File Tree can still expose it as resource
- Page Tree projection excludes it

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_resource_classification -- --test-threads=1
```

Expected: FAIL.

- [ ] **Step 2: Implement OCR sidecar detection**

Add a helper such as:

```rust
fn is_local_ocr_sidecar_path(path: &Path) -> bool
```

Rules:

- parent directory name ends with `.ocr`
- file name ends with `.ocr.md`
- file has `mnote_ocr_version` frontmatter when content is available

Use path-only detection for tree projection and frontmatter-backed detection for search/index.

- [ ] **Step 3: Verify classification tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_resource_classification -- --test-threads=1
```

Expected: PASS.

## Task 5: Search OCR Text

**Files:**
- Modify: `rust/crates/mnote-web/src/routes/local_search_index.rs`
- Modify: `rust/crates/mnote-web/src/routes/search.rs`

- [ ] **Step 1: Add failing local search tests**

Cover:

- `includeOcr=false` does not match OCR sidecar text
- `includeOcr=true` matches OCR sidecar text
- result `documentId` is owner page id
- result has `hasOcr=true`
- evidence contains source and sidecar paths

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1
```

Expected: FAIL.

- [ ] **Step 2: Pass includeOcr into local search**

Update `search.rs` local-folder branch to pass `filters.include_ocr.unwrap_or(false)`.

- [ ] **Step 3: Extend local search index**

Add OCR sidecar collection separate from normal documents. Search OCR text only when `include_ocr=true`, then project owner page result.

- [ ] **Step 4: Verify search tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1
```

Expected: PASS.

## Task 6: Browser Runtime OCR Actions

**Files:**
- Modify: `rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js`
- Modify: `rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js`
- Modify: `rust/crates/mnote-web/browser/document-resource-tab-runtime.js`
- Create: `rust/crates/mnote-web/browser/local-ocr-task-runtime.js`
- Modify: `rust/crates/mnote-web/src/ssr/pages/layout.rs`

- [ ] **Step 1: Add JS syntax and SSR wiring tests**

Add or extend tests that assert OCR runtime dependencies are mounted and no `ReferenceError`-prone implicit globals are used.

Run:

```bash
node --check rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js
node --check rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js
node --check rust/crates/mnote-web/browser/document-resource-tab-runtime.js
node --check rust/crates/mnote-web/browser/local-ocr-task-runtime.js
cargo test --manifest-path rust/Cargo.toml -p mnote-web ocr_runtime -- --test-threads=1
```

Expected before implementation: Rust OCR runtime assertions fail.

- [ ] **Step 2: Add OCR actions**

Add actions:

- `OCR 识别`
- `查看 OCR`
- `插入 OCR 链接`
- `重新识别`

Only show OCR actions for image and PDF resources.

- [ ] **Step 3: Add OCR status fetch/render**

Use `/api/local-folder/ocr/status` and `/read`. Show states:

- 未识别
- 排队中
- 上传中
- 识别中
- 写入中
- 已识别
- 来源已变化
- 识别失败
- 已中断

- [ ] **Step 4: Add global OCR task bar and drawer**

Create a global OCR task entry in the shell:

- collapsed label: `OCR 2` or `1 个 OCR 任务进行中`
- drawer rows: file name, owner page, stage label, start time, completion/failure state
- row actions: `打开 OCR`, `重试`
- state source: initial `GET /api/local-folder/ocr/jobs?rootUri=...` plus `local_ocr.job.updated` events

The runtime must update from events and explicit command results. It must not rely on `setInterval` polling as the main update path.

- [ ] **Step 5: Verify JS and Rust wiring**

Run the commands from Step 1.

Expected: PASS.

## Task 7: Insert OCR Link Into Markdown

**Files:**
- Modify: `rust/crates/mnote-web/src/routes/local_ocr.rs`
- Modify: browser runtime files from Task 6

- [ ] **Step 1: Add route tests for insert**

Cover:

- inserting link appends `[OCR：photo.png](./Page.ocr/photo.png.ocr.md)`
- insert requires explicit OCR sidecar path
- insert respects local resource conflict detection
- `inlineSection` inserts heading, source comment and OCR body

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_insert -- --test-threads=1
```

Expected: FAIL.

- [ ] **Step 2: Implement `POST /api/local-folder/ocr/insert`**

Use existing local resource write patterns and keep 正文 mutation explicit.

- [ ] **Step 3: Wire UI insert actions**

After successful insert, refresh current document through existing save/refresh event path, not polling.

- [ ] **Step 4: Verify insert tests pass**

Run:

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_insert -- --test-threads=1
```

Expected: PASS.

## Task 8: Browser Smoke

**Files:**
- Create: `scripts/task506-local-folder-mineru-ocr-smoke.js`
- Modify: `scripts/TESTING_REFERENCE.md`

- [ ] **Step 1: Write smoke script**

The smoke should:

- create a temp local-folder workspace
- create `Page.md`
- place one PNG fixture and one PDF fixture next to it
- use mock MinerU mode or a test-only fixture response
- trigger OCR from UI
- assert `Page.ocr/*.ocr.md` exists
- assert the global OCR task bar shows running state after submission
- assert the global OCR task drawer moves to done after the mocked OCR event
- assert OCR resource tab opens
- assert search with `includeOcr=true` returns owner page
- assert insert link mutates `Page.md` only after explicit click

- [ ] **Step 2: Run smoke**

Run:

```bash
MNOTE_WEB_SMOKE_BASE_URL=http://127.0.0.1:3000 NODE_PATH=/mnt/Data1T/mnote/node_modules node scripts/task506-local-folder-mineru-ocr-smoke.js
```

Expected: `ok=true` and no browser `pageErrors`.

- [ ] **Step 3: Update testing reference**

Add the smoke to `scripts/TESTING_REFERENCE.md` under local-folder resource/browser smoke.

## Task 9: Final Verification

- [ ] **Step 1: Run focused Rust tests**

```bash
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr -- --test-threads=1
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1
```

- [ ] **Step 2: Run browser runtime checks**

```bash
node --check rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js
node --check rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js
node --check rust/crates/mnote-web/browser/document-resource-tab-runtime.js
node --check rust/crates/mnote-web/browser/local-ocr-task-runtime.js
```

- [ ] **Step 3: Run smoke**

```bash
MNOTE_WEB_SMOKE_BASE_URL=http://127.0.0.1:3000 NODE_PATH=/mnt/Data1T/mnote/node_modules node scripts/task506-local-folder-mineru-ocr-smoke.js
```

- [ ] **Step 4: Check diff hygiene**

```bash
git diff --check -- rust/crates/mnote-web/src/routes/local_ocr.rs rust/crates/mnote-web/src/routes/mod.rs rust/crates/mnote-web/src/routes/local_folder_source.rs rust/crates/mnote-web/src/routes/local_search_index.rs rust/crates/mnote-web/src/routes/search.rs rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js rust/crates/mnote-web/browser/document-resource-tab-runtime.js rust/crates/mnote-web/browser/local-ocr-task-runtime.js scripts/task506-local-folder-mineru-ocr-smoke.js scripts/TESTING_REFERENCE.md
```

- [ ] **Step 5: Sync CodeGraph before commit**

```bash
codegraph sync .
codegraph status .
```

Expected: no pending CodeGraph changes.