mnote/docs/superpowers/plans/2026-05-29-local-folder-mineru-ocr-sidecar.md

Local Folder MinerU OCR Sidecar Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 为 mnote local-folder 图片和图片型 PDF 增加手动 MinerU OCR，并把 OCR Markdown 保存到 owner 页面同目录的 {pageStem}.ocr/ 文件夹。

Architecture: 后端负责 MinerU API、权限校验、OCR active job store、OCR sidecar 写入和 .mnote/ocr-index.json 状态缓存；前端触发本地 OCR API，并通过资源局部状态和全局 OCR 任务栏展示阶段型进度。OCR 文本默认作为资源 sidecar 被搜索、resource tab 和后续 AI 消费，只有用户显式操作时才插入正文。

Tech Stack: Rust mnote-web routes/services、local-folder metadata、MinerU HTTP API、browser runtime JS、Node smoke、Rust cargo tests。

---

File Structure

• Create: rust/crates/mnote-web/src/routes/local_ocr.rs
  • OCR API request/response types、路径规划、sidecar frontmatter、active job store、.mnote/ocr-index.json 读写、MinerU client facade。
• Modify: rust/crates/mnote-web/src/routes/mod.rs
  • 注册 /api/local-folder/ocr/jobs、/jobs/:id、/status、/read、/insert。
• Modify: rust/crates/mnote-web/src/routes/local_folder_source.rs
  • 暴露必要的 local-folder path helper，识别 .ocr/*.ocr.md 为 OCR resource 而不是普通页面。
• Modify: rust/crates/mnote-web/src/routes/local_search_index.rs
  • 读取 OCR index/sidecar，includeOcr=true 时搜索 OCR 文本并返回 owner page。
• Modify: rust/crates/mnote-web/src/routes/search.rs
  • local-folder 分支把 filters.include_ocr 传给 local search index。
• Modify: rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js
  • 附件菜单展示 OCR 识别、查看 OCR、插入 OCR 链接。
• Modify: rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js
  • File Tree 图片/PDF 资源右键菜单接入 OCR 操作。
• Modify: rust/crates/mnote-web/browser/document-resource-tab-runtime.js
  • image/PDF resource tab 展示 OCR 状态和 OCR Markdown。
• Create: rust/crates/mnote-web/browser/local-ocr-task-runtime.js
  • 全局 OCR 任务栏 / 任务抽屉，消费 OCR jobs API 和 local_ocr.job.updated 事件。
• Modify: rust/crates/mnote-web/src/ssr/pages/layout.rs
  • 挂载全局 OCR task runtime，并断言 runtime 依赖显式注入。
• Test: rust/crates/mnote-web/src/routes/local_ocr.rs inline tests
• Test: rust/crates/mnote-web/src/routes/local_search_index.rs inline tests
• Create: scripts/task506-local-folder-mineru-ocr-smoke.js
  • 浏览器 smoke，使用 mock MinerU 或 test-only route fixture。

Task 1: OCR Sidecar Path And Metadata Contract

Files:

• Create: rust/crates/mnote-web/src/routes/local_ocr.rs

• Step 1: Add failing unit tests for sidecar path planning

Add tests that assert:

• owner docs/Page.md creates OCR dir docs/Page.ocr/
• source docs/Page.assets/photo.png creates docs/Page.ocr/photo.png.ocr.md
• duplicate source leaf names get a short hash suffix
• OCR frontmatter contains owner_document, source_path, source_root_relative_path, provider, model_version, source_size, source_mtime_ms

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_sidecar -- --test-threads=1

Expected: FAIL because local_ocr does not exist yet.

• Step 2: Implement path planning and frontmatter builder

Implement focused helpers in local_ocr.rs:

• plan_ocr_sidecar_path(root, owner_document_path, source_root_relative_path, source_metadata)
• build_ocr_markdown(frontmatter, mineru_markdown)
• parse_ocr_frontmatter(markdown)

Keep these helpers pure and unit-testable.

• Step 3: Verify tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_sidecar -- --test-threads=1

Expected: PASS.

Task 2: OCR Index Cache

Files:

• Modify: rust/crates/mnote-web/src/routes/local_ocr.rs

• Step 1: Add failing tests for .mnote/ocr-index.json

Cover:

• write/read one entry
• stale=true when source size or mtime changed
• missing source returns stale=true with no panic
• failed entry stores only a short sanitized error

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_index -- --test-threads=1

Expected: FAIL.

• Step 2: Implement index read/write and stale detection

Use atomic JSON write, following existing write_json_atomic style in local_folder_source.rs.

Index path:

<root>/.mnote/ocr-index.json

Entry key:

sourceRootRelativePath

• Step 3: Verify tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_index -- --test-threads=1

Expected: PASS.

Task 3: MinerU Client And OCR Job API

Files:

• Modify: rust/crates/mnote-web/src/routes/local_ocr.rs

• Modify: rust/crates/mnote-web/src/routes/mod.rs

• Step 1: Add route tests with a mock MinerU client

Cover:

• missing token returns mineru_token_missing
• root escape source path is rejected
• successful mock OCR writes {pageStem}.ocr/*.ocr.md
• successful mock OCR exposes queued/running/done states through jobs list
• each mock OCR state change emits local_ocr.job.updated
• failed mock OCR writes failed index entry without token or upload URL

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_jobs -- --test-threads=1

Expected: FAIL.

• Step 2: Implement API routes

Routes:

• POST /api/local-folder/ocr/jobs
• GET /api/local-folder/ocr/jobs
• GET /api/local-folder/ocr/jobs/{jobId}
• GET /api/local-folder/ocr/status
• GET /api/local-folder/ocr/read

Token lookup order:

1. MNOTE_MINERU_API_TOKEN
2. MINERU_API_TOKEN
3. ~/.hermes/.env MINERU_API_TOKEN for local development

• Step 3: Implement MinerU local file flow

Use MinerU API:

• POST /api/v4/file-urls/batch
• PUT local bytes to pre-signed URL
• GET /api/v4/extract-results/batch/{batch_id}
• download zip
• extract first Markdown file

Never log token or pre-signed URL.

• Step 4: Implement active job store and status stages

Add in-process active job state with these statuses:

queued
uploading
mineru_processing
downloading
writing_sidecar
done
failed
interrupted
stale

Each state change must update .mnote/ocr-index.json and return a redacted job payload. On process restart, any persisted queued/uploading/mineru_processing/downloading/writing_sidecar entry without active in-memory job is exposed as interrupted.

• Step 5: Broadcast OCR job events

Emit a local realtime event for every state change:

{
  "type": "local_ocr.job.updated",
  "jobId": "ocr_1760000000000_abcd",
  "rootUri": "file:///tmp/mnote-ocr",
  "ownerDocumentId": "local-md:docs~2FPage.md",
  "sourceRootRelativePath": "docs/Page.assets/spec.pdf",
  "ocrRootRelativePath": "docs/Page.ocr/spec.pdf.ocr.md",
  "status": "mineru_processing",
  "stageLabel": "识别中",
  "updatedAtMs": 1760000000000
}

Use the existing realtime / WS / SSE event path. Do not add a foreground setInterval polling loop as the primary update mechanism.

• Step 6: Register routes

Add route registrations in routes/mod.rs.

• Step 7: Verify route tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_jobs -- --test-threads=1

Expected: PASS.

Task 4: Keep OCR Markdown Out Of Page Tree

Files:

• Modify: rust/crates/mnote-web/src/routes/local_folder_source.rs

• Modify: rust/crates/mnote-web/src/routes/local_search_index.rs

• Step 1: Add failing tests for .ocr/*.ocr.md classification

Cover:

• docs/Page.ocr/photo.png.ocr.md is not a normal local Markdown document
• File Tree can still expose it as resource
• Page Tree projection excludes it

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_resource_classification -- --test-threads=1

Expected: FAIL.

• Step 2: Implement OCR sidecar detection

Add a helper such as:

fn is_local_ocr_sidecar_path(path: &Path) -> bool

Rules:

• parent directory name ends with .ocr
• file name ends with .ocr.md
• file has mnote_ocr_version frontmatter when content is available

Use path-only detection for tree projection and frontmatter-backed detection for search/index.

• Step 3: Verify classification tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_resource_classification -- --test-threads=1

Expected: PASS.

Task 5: Search OCR Text

Files:

• Modify: rust/crates/mnote-web/src/routes/local_search_index.rs

• Modify: rust/crates/mnote-web/src/routes/search.rs

• Step 1: Add failing local search tests

Cover:

• includeOcr=false does not match OCR sidecar text
• includeOcr=true matches OCR sidecar text
• result documentId is owner page id
• result has hasOcr=true
• evidence contains source and sidecar paths

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1

Expected: FAIL.

• Step 2: Pass includeOcr into local search

Update search.rs local-folder branch to pass filters.include_ocr.unwrap_or(false).

• Step 3: Extend local search index

Add OCR sidecar collection separate from normal documents. Search OCR text only when include_ocr=true, then project owner page result.

• Step 4: Verify search tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1

Expected: PASS.

Task 6: Browser Runtime OCR Actions

Files:

• Modify: rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js

• Modify: rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js

• Modify: rust/crates/mnote-web/browser/document-resource-tab-runtime.js

• Create: rust/crates/mnote-web/browser/local-ocr-task-runtime.js

• Modify: rust/crates/mnote-web/src/ssr/pages/layout.rs

• Step 1: Add JS syntax and SSR wiring tests

Add or extend tests that assert OCR runtime dependencies are mounted and no ReferenceError-prone implicit globals are used.

Run:

node --check rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js
node --check rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js
node --check rust/crates/mnote-web/browser/document-resource-tab-runtime.js
node --check rust/crates/mnote-web/browser/local-ocr-task-runtime.js
cargo test --manifest-path rust/Cargo.toml -p mnote-web ocr_runtime -- --test-threads=1

Expected before implementation: Rust OCR runtime assertions fail.

• Step 2: Add OCR actions

Add actions:

• OCR 识别
• 查看 OCR
• 插入 OCR 链接
• 重新识别

Only show OCR actions for image and PDF resources.

• Step 3: Add OCR status fetch/render

Use /api/local-folder/ocr/status and /read. Show states:

• 未识别

• 排队中

• 上传中

• 识别中

• 写入中

• 已识别

• 来源已变化

• 识别失败

• 已中断

• Step 4: Add global OCR task bar and drawer

Create a global OCR task entry in the shell:

• collapsed label: OCR 2 or 1 个 OCR 任务进行中
• drawer rows: file name, owner page, stage label, start time, completion/failure state
• row actions: 打开 OCR, 重试
• state source: initial GET /api/local-folder/ocr/jobs?rootUri=... plus local_ocr.job.updated events

The runtime must update from events and explicit command results. It must not rely on setInterval polling as the main update path.

• Step 5: Verify JS and Rust wiring

Run the commands from Step 1.

Expected: PASS.

Task 7: Insert OCR Link Into Markdown

Files:

• Modify: rust/crates/mnote-web/src/routes/local_ocr.rs

• Modify: browser runtime files from Task 6

• Step 1: Add route tests for insert

Cover:

• inserting link appends [OCR：photo.png](./Page.ocr/photo.png.ocr.md)
• insert requires explicit OCR sidecar path
• insert respects local resource conflict detection
• inlineSection inserts heading, source comment and OCR body

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_insert -- --test-threads=1

Expected: FAIL.

• Step 2: Implement POST /api/local-folder/ocr/insert

Use existing local resource write patterns and keep 正文 mutation explicit.

• Step 3: Wire UI insert actions

After successful insert, refresh current document through existing save/refresh event path, not polling.

• Step 4: Verify insert tests pass

Run:

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr_insert -- --test-threads=1

Expected: PASS.

Task 8: Browser Smoke

Files:

• Create: scripts/task506-local-folder-mineru-ocr-smoke.js

• Modify: scripts/TESTING_REFERENCE.md

• Step 1: Write smoke script

The smoke should:

• create a temp local-folder workspace

• create Page.md

• place one PNG fixture and one PDF fixture next to it

• use mock MinerU mode or a test-only fixture response

• trigger OCR from UI

• assert Page.ocr/*.ocr.md exists

• assert the global OCR task bar shows running state after submission

• assert the global OCR task drawer moves to done after the mocked OCR event

• assert OCR resource tab opens

• assert search with includeOcr=true returns owner page

• assert insert link mutates Page.md only after explicit click

• Step 2: Run smoke

Run:

MNOTE_WEB_SMOKE_BASE_URL=http://127.0.0.1:3000 NODE_PATH=/mnt/Data1T/mnote/node_modules node scripts/task506-local-folder-mineru-ocr-smoke.js

Expected: ok=true and no browser pageErrors.

• Step 3: Update testing reference

Add the smoke to scripts/TESTING_REFERENCE.md under local-folder resource/browser smoke.

Task 9: Final Verification

• Step 1: Run focused Rust tests

cargo test --manifest-path rust/Cargo.toml -p mnote-web local_ocr -- --test-threads=1
cargo test --manifest-path rust/Cargo.toml -p mnote-web local_search_ocr -- --test-threads=1

• Step 2: Run browser runtime checks

node --check rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js
node --check rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js
node --check rust/crates/mnote-web/browser/document-resource-tab-runtime.js
node --check rust/crates/mnote-web/browser/local-ocr-task-runtime.js

• Step 3: Run smoke

MNOTE_WEB_SMOKE_BASE_URL=http://127.0.0.1:3000 NODE_PATH=/mnt/Data1T/mnote/node_modules node scripts/task506-local-folder-mineru-ocr-smoke.js

• Step 4: Check diff hygiene

git diff --check -- rust/crates/mnote-web/src/routes/local_ocr.rs rust/crates/mnote-web/src/routes/mod.rs rust/crates/mnote-web/src/routes/local_folder_source.rs rust/crates/mnote-web/src/routes/local_search_index.rs rust/crates/mnote-web/src/routes/search.rs rust/crates/mnote-web/browser/sidebar-attachment-open-runtime.js rust/crates/mnote-web/browser/sidebar-filetree-open-runtime.js rust/crates/mnote-web/browser/document-resource-tab-runtime.js rust/crates/mnote-web/browser/local-ocr-task-runtime.js scripts/task506-local-folder-mineru-ocr-smoke.js scripts/TESTING_REFERENCE.md

• Step 5: Sync CodeGraph before commit

codegraph sync .
codegraph status .

Expected: no pending CodeGraph changes.