diff --git a/bugs/05-editor-mainline/done/5-42-local-image-angle-link-and-ocr-task-topbar-v1.md b/bugs/05-editor-mainline/done/5-42-local-image-angle-link-and-ocr-task-topbar-v1.md new file mode 100644 index 00000000..767d82e3 --- /dev/null +++ b/bugs/05-editor-mainline/done/5-42-local-image-angle-link-and-ocr-task-topbar-v1.md @@ -0,0 +1,42 @@ +# 5-42 local Markdown 图片尖括号链接与 OCR 任务入口回归 + +## 状态 + +- 状态:done +- owner:05-editor-mainline +- 发现时间:2026-06-01 +- 修复时间:2026-06-01 + +## 现象 + +默认测试账号的 local-folder 页面中,上传图片后 Markdown 正文出现 `![name](<./name.png>)` 形式的 CommonMark 链接目标,页面内图片显示为破损图。点击资源 tab 的“生成 OCR”后,如果 MinerU token 缺失导致 `/api/local-folder/ocr/jobs` 返回 401,右下角/任务入口没有任何可见任务,只能在资源 tab 状态里看到失败文本。 + +同时用户期望 OCR 任务入口移到右上角 topbar 固定图标区,避免继续使用右下角悬浮入口。 + +## 根因 + +- 浏览器端 `document-tiptap-conversion-runtime.js` 在把 local Markdown 图片 src 转成 `/api/local-folder/files/open` 时,没有解包 CommonMark `<...>` 链接目标,导致请求 path 携带 `%3C` / `%3E`。 +- `document-resource-tab-runtime.js` 的 OCR 创建失败分支只更新当前资源面板状态,没有构造失败 job 并写入全局 OCR 任务列表。 +- OCR 任务入口由运行时动态创建在右下角 dock,未接入 Wolai topbar action 区。 + +## 修复 + +- 本地资源路径规范化前统一解包 `<...>` Markdown 链接目标。 +- OCR 创建开始时写入 running 任务,请求失败时写入 failed 任务,任务抽屉可见错误信息与重试入口。 +- 在 topbar 增加 OCR 任务图标,运行时复用该按钮切换任务抽屉;右下角悬浮按钮移除,仅保留右上角抽屉。 + +## 验证 + +- `node --check rust/crates/mnote-web/browser/document-tiptap-conversion-runtime.js` +- `node --check rust/crates/mnote-web/browser/document-resource-tab-runtime.js` +- `node --check scripts/task526-local-folder-ocr-api-smoke.js` +- `cargo fmt --check --all` +- `cargo test --manifest-path rust/Cargo.toml -p bridge-runtime local_markdown_image_legacy_projection_exposes_image_attrs -- --nocapture` +- `cargo test --manifest-path rust/Cargo.toml -p mnote-web markdown_image_parses_as_image_block -- --nocapture` +- `cargo test --manifest-path rust/Cargo.toml -p mnote-web local_markdown_save_rewrites_local_file_open_url_to_markdown_relative_path -- --nocapture` +- `scripts/task526-local-folder-ocr-api-smoke.js` 覆盖: + - `![photo](<./Page.assets/photo.png>)` 渲染出的图片 `naturalWidth > 0`,且请求 URL 不含 `%3C` / `%3E`。 + - OCR 任务入口位于 `.wolai-topbar-actions`。 + - mock OCR 成功任务可在任务抽屉中显示。 + - 模拟 401 失败后,任务抽屉显示 failed row 与 `local_ocr_job_failed_401`。 + - OCR sidecar 可作为 Markdown resource tab 打开。 diff --git a/design/07-ai/done/7-45-chatonly-api-provider-runtime-v1.md b/design/07-ai/done/7-45-chatonly-api-provider-runtime-v1.md new file mode 100644 index 00000000..119d0a5d --- /dev/null +++ b/design/07-ai/done/7-45-chatonly-api-provider-runtime-v1.md @@ -0,0 +1,408 @@ +# 7-45 ChatOnly API provider runtime v1 + +> 创建时间:2026-06-02 +> +> 状态:`done` +> +> Owner:Page AI ChatOnly / mnote-web / SQLite control-plane / OmniRoute API provider +> +> 上位依据: +> - `design/07-ai/done/7-15-page-ai-acp-agent-runtime-unified-layer-v1.md` +> - `design/07-ai/done/7-25-acp-session-runtime-enhancement-plan-v1.md` +> - `design/07-ai/done/7-30-acp-session-load-resume-checklist-v1.md` +> - `design/07-ai/done/7-39-page-ai-agent-selector-context-authorization-settings-v1.md` +> - `design/07-ai/done/7-44-chatonly-doubao-session-binding-v1.md` + +## 1. 背景 + +当前 Page AI `Chat-only` 已接入豆包、DeepSeek、Gemini 等网页 provider。网页 provider 的价值是复用网页账号、网页历史和远端会话删除,但它也带来浏览器、登录态、验证码、前台窗口、OpenClaw provider prompt 和 provider-specific 会话绑定复杂度。 + +现在 OmniRoute 已能提供 OpenAI-compatible API endpoint: + +```text +http://127.0.0.1:20128/v1/chat/completions +``` + +并已验证以下模型可用: + +- `aisz-chat/grok-4.3` +- `aisz-chat/gemini-3.1-pro` +- `aisz-chat/gpt-5.5-extra-high-fast` +- `aisz-chat/kimi-k2.5` +- DeepSeek pro / flash 组合模型 + +这些模型不需要浏览器,也不需要 OpenClaw。对于 MNote 当前需求,它们只是最简 ChatOnly provider:用户选一个 provider,发送消息,MNote 本地保存历史,删除 MNote 会话时删除本地历史。 + +## 2. 第一结论 + +API ChatOnly 不走 OpenClaw。 + +原因: + +- OpenClaw 是 agent runtime,包含 agent prompt、skills、workspace、compaction、浏览器 profile 等能力;最简聊天窗口不需要这些能力。 +- API provider 没有网页远端历史对齐语义,不应复用 `CHATONLY_PROVIDER_CONFIGS` 的网页 conversation binding / remote delete 链路。 +- MNote 已有 ChatOnly UI、SSE 消费、SQLite runtime session、删除入口和 `reqwest` streaming 依赖;新增一个薄的 OpenAI-compatible adapter 比引入完整第三方 chat app 更可控。 + +最终分层: + +| Provider 类别 | 默认状态 | 运行方式 | 历史删除语义 | +|---|---|---|---| +| 豆包网页 | 保留默认可用 | OpenClaw `doubao-web` | MNote 本地删除 + 豆包远端 best effort 删除 | +| DeepSeek 网页 | fallback | OpenClaw `deepseek-web` | 网页远端 best effort 删除 | +| Gemini 网页 | fallback | OpenClaw `gemini-web` | 网页远端 best effort 删除 | +| Grok 网页 | fallback | OpenClaw `grok-web` | 暂不作为默认 | +| OmniRoute API Chat | 新默认候选 | mnote-web 直连 `/v1/chat/completions` | 仅 MNote 本地删除 | + +## 3. 目标 + +- 新增 `api-chat` provider kind,直接调用 OpenAI-compatible `/v1/chat/completions`。 +- 支持 DeepSeek、Gemini、Grok、GPT、Kimi API ChatOnly profiles。 +- 复用现有 Page AI ChatOnly UI、session list、session detail、rename、delete、search 和 SSE 消费模型。 +- API provider 不启动 ACP runtime,不启动 OpenClaw,不注入网页 provider metadata prompt。 +- API provider 的会话和消息以 SQLite/control-plane 为准,按用户隔离。 +- 网页 DeepSeek/Gemini/Grok 保留为 fallback,后续稳定后再决定是否从默认 UI 中隐藏。 + +非目标: + +- 不实现完整 ChatGPT / Open WebUI / LibreChat 类产品。 +- 不导入第三方 chat app 的数据库、用户系统或前端壳。 +- 不同步 Gemini/Grok/DeepSeek 网页历史。 +- 不把 API ChatOnly 暴露为可写文件 agent。 +- 不支持 tool calling、function calling、vision、文件上传、多模态附件。 + +## 4. Provider 配置合同 + +新增一个小型静态 registry,建议先放在 mnote-web 后端,前端只消费 profile 列表: + +```text +api_chat_profiles +- profile_id +- label +- provider_kind = api-chat +- base_url +- model +- api_key_env +- fallback_profile_id +- status = active | fallback | hidden +``` + +第一批 profiles: + +| profile_id | label | model | 默认状态 | +|---|---|---|---| +| `shared_api_deepseek_flash_chat` | DeepSeek Flash | DeepSeek flash 组合模型 | active | +| `shared_api_deepseek_pro_chat` | DeepSeek Pro | DeepSeek pro 组合模型 | active | +| `shared_api_gpt_chat` | GPT | `aisz-chat/gpt-5.5-extra-high-fast` | active | +| `shared_api_kimi_chat` | Kimi | `aisz-chat/kimi-k2.5` | active | +| `shared_api_gemini_chat` | Gemini API | `aisz-chat/gemini-3.1-pro` | active | +| `shared_api_grok_chat` | Grok API | `aisz-chat/grok-4.3` | active 或 hidden,按限额策略决定 | + +默认 base URL: + +```text +http://127.0.0.1:20128/v1 +``` + +API key 优先级: + +1. `MNOTE_API_CHAT__API_KEY` +2. `MNOTE_API_CHAT_API_KEY` +3. `OPENAI_API_KEY` + +不建议直接读取 `/home/lix/.codex/auth.json` 作为长期方案。开发期可以从环境注入,避免 MNote 代码绑定 Codex 配置文件。 + +## 5. 数据模型 + +继续复用现有 `ai_runtime_runs` / `ai_runtime_events` 作为会话索引和事件审计。API ChatOnly 的 run payload 增加: + +```json +{ + "agentId": "chat_only", + "providerKind": "api-chat", + "profileId": "shared_api_gpt_chat", + "model": "aisz-chat/gpt-5.5-extra-high-fast", + "sessionId": "mnote_...", + "message": "..." +} +``` + +消息恢复来源: + +- 用户消息:从 run payload 的 `message` 恢复。 +- 助手消息:从 `ai_runtime_events` 中的 `message.delta` 聚合,或从 `run.completed` 的 final text 恢复。 +- 错误:写入 `run.failed`,用于 session detail 展示。 + +API ChatOnly 不写 `ai_external_conversation_bindings`。该表只属于网页 provider 远端会话绑定。 + +## 6. 后端数据流 + +### 6.1 创建会话 + +沿用现有: + +```text +POST /api/hermes/client/sessions +``` + +当 `agentId=chat_only` 且 profile 是 `api-chat` 时: + +1. 写入本地 session index。 +2. `persistence=local_ai_session_jsonl` 或当前 SQLite runtime persistence 口径。 +3. 不创建远端会话。 +4. 不创建 ACP session。 + +### 6.2 发送消息 + +沿用现有: + +```text +POST /api/hermes/client/runs +``` + +分流规则: + +```text +agentId == chat_only + 且 profile/profileId 命中 api-chat registry + -> api_chat_runtime +否则 + -> 当前 ACP/OpenClaw/Web provider runtime +``` + +`api_chat_runtime` 执行: + +1. 按当前 `user_id + workspace_id + session_id + profile_id` 读取最近消息。 +2. 构造 OpenAI-compatible messages: + +```json +[ + { "role": "system", "content": "你是 MNote 的简洁聊天助手。不要声称能编辑文件。" }, + { "role": "user", "content": "..." }, + { "role": "assistant", "content": "..." }, + { "role": "user", "content": "当前问题" } +] +``` + +3. 调 OmniRoute: + +```text +POST {base_url}/chat/completions +Authorization: Bearer ... +Content-Type: application/json +``` + +请求体最小字段: + +```json +{ + "model": "aisz-chat/kimi-k2.5", + "messages": [], + "stream": true +} +``` + +4. 解析 upstream SSE: + +```text +data: {"choices":[{"delta":{"content":"..."}}]} +data: [DONE] +``` + +5. 转成 MNote 现有 SSE event: + +```text +event: message.delta +data: {"delta":"..."} + +event: run.completed +data: {"output":"...","model":"..."} +``` + +6. 同步写入 `ai_runtime_events`,用于历史恢复和审计。 + +### 6.3 删除会话 + +沿用现有 session delete endpoint。 + +当 session 属于 `api-chat`: + +1. 软删除本地 runs/events 或标记 session deleted。 +2. 不调用 provider conversation delete。 +3. 返回: + +```json +{ + "ok": true, + "remoteDelete": { + "attempted": false, + "reason": "api_chat_has_no_remote_conversation" + } +} +``` + +UI 文案必须避免暗示已删除网页历史。 + +## 7. 代码边界 + +建议新增模块: + +```text +rust/crates/mnote-web/src/api_chat.rs +``` + +职责: + +- profile registry。 +- provider config resolution。 +- OpenAI-compatible request construction。 +- streaming parser。 +- upstream error normalization。 +- 将 upstream chunk 映射为 `message.delta` / `run.completed` / `run.failed`。 + +`hermes_client.rs` 只做分流和现有 session/run 持久化衔接,不继续塞大量 provider 细节。 + +前端改动控制在: + +```text +rust/crates/mnote-web/browser/sidebar-page-ai-runtime.js +rust/crates/mnote-web/browser/sidebar-page-ai-session-runtime.js +``` + +前端只需要: + +- 展示新增 profiles。 +- 对 API ChatOnly 不显示“网页问答”描述,改为“API 聊天”。 +- 删除成功提示区分本地删除和网页同步删除。 + +SQLite/control-plane 改动: + +```text +rust/crates/control-plane/src/sqlite.rs +``` + +新增 shared profiles provisioning。第一版可以不新增表,只复用 profile metadata;如后续要 UI 动态配置模型,再引入 `ai_chat_provider_profiles` 表。 + +## 8. 第三方参考取舍 + +已考虑的参考: + +- `async-openai`:适合参考 OpenAI-compatible 配置、SSE streaming、请求类型;如果后续字段变多,可引入依赖。 +- HuggingFace `chat-ui`、Open WebUI、LibreChat:功能完整,但带独立前端、用户、会话和数据库模型,不适合嵌入 MNote 当前 Page AI sidebar。 +- Vercel AI SDK:前端/Next 生态友好,但 MNote 当前主链是 Rust SSR + Rust Axum,不应为最小 adapter 引入 JS server 侧依赖。 + +第一版采用 `reqwest + serde_json` 的最小 adapter。理由: + +- 当前仓库已有 `reqwest` streaming。 +- 需要支持的协议子集很小。 +- 不引入大型依赖和类型迁移成本。 +- bug 面集中在一个小模块,可用 mock upstream 和真实 OmniRoute smoke 覆盖。 + +## 9. 错误处理 + +必须明确区分: + +- `api_chat_profile_unknown`:profile 未注册。 +- `api_chat_base_url_missing`:base URL 缺失。 +- `api_chat_api_key_missing`:API key 缺失。 +- `api_chat_upstream_unavailable`:OmniRoute 不可达。 +- `api_chat_upstream_error`:OmniRoute 返回非 2xx。 +- `api_chat_stream_parse_error`:SSE chunk 解析失败。 +- `api_chat_empty_response`:流结束但没有助手文本。 + +错误事件写入: + +```text +event: run.failed +data: {"code":"api_chat_upstream_error","message":"..."} +``` + +前端展示为普通 ChatOnly 失败消息,不进入文件权限或 tool call UI。 + +## 10. 验收 + +### 10.1 单元测试 + +- profile registry 能解析 DeepSeek/Gemini/Grok/GPT/Kimi。 +- API key 优先级正确。 +- OpenAI-compatible SSE chunk 能解析 `delta.content`。 +- `[DONE]` 正确结束。 +- 非 2xx upstream 映射为 `run.failed`。 +- API ChatOnly 不命中 `CHATONLY_PROVIDER_CONFIGS`。 +- API ChatOnly 删除不调用网页 provider delete。 + +### 10.2 集成测试 + +Mock upstream: + +- `/v1/chat/completions` 返回 SSE delta。 +- MNote `/runs` 返回 runId。 +- MNote `/events/{runId}` 输出 `message.delta` 和 `run.completed`。 +- session detail 可恢复 user/assistant messages。 +- 删除 session 后列表不再出现。 + +真实 OmniRoute smoke: + +- `shared_api_deepseek_flash_chat` 发一条短消息成功。 +- `shared_api_gpt_chat` 发一条短消息成功。 +- 确认没有启动或调用 OpenClaw gateway/proxy。 + +浏览器 smoke: + +- 打开 Page AI ChatOnly。 +- 选择 GPT API,发消息,只出现一条助手回复。 +- 新建会话、切换会话、刷新页面后历史仍存在。 +- 删除会话后本地列表消失。 +- 豆包网页会话链路不回归,仍能远端删除。 + +## 11. 分阶段实施 + +### Phase 1:最小 API runtime + +- 新增 `api_chat.rs`。 +- 接入 `shared_api_deepseek_flash_chat`、`shared_api_gpt_chat`。 +- 复用现有 `/runs` 和 `/events`。 +- 跑 mock upstream 和真实 OmniRoute smoke。 + +### Phase 2:扩展 provider 列表 + +- 增加 `deepseek_pro`、`gemini_api`、`kimi`、`grok_api`。 +- Grok 默认可设为 hidden 或 fallback,避免限额被普通默认选择消耗。 +- 前端区分 `API` 与 `网页 fallback` 标签。 + +### Phase 3:收口网页 fallback + +- 观察 API provider 稳定性。 +- DeepSeek/Gemini/Grok 网页降级为 fallback 或隐藏。 +- 保留豆包网页主链,直到豆包也有可替代的稳定 API provider 且不再要求网页历史对齐。 + +## 12. 风险与约束 + +- OmniRoute 模型名可能变动,profile registry 要允许环境覆盖 model。 +- 某些 upstream 可能不完全遵守 OpenAI streaming chunk 格式,parser 要容忍 `content` / `text` / `delta` 变体,但不要吞掉错误。 +- API ChatOnly 没有网页远端删除,UI 和返回值必须说清楚。 +- 不要让 API ChatOnly 继承 Page AI 文件写权限;它只能读已显式加入 prompt 的上下文。 +- 不要为第一版引入 tool calling,否则会重新变成 agent runtime。 + +## 13. 退出条件 + +本设计可归档到 `done/` 的条件: + +- API ChatOnly 至少两个 provider 完成真实浏览器 smoke。 +- 本地历史创建、恢复、删除通过。 +- API ChatOnly 确认不启动 OpenClaw。 +- 豆包网页删除同步回归通过。 +- DeepSeek/Gemini/Grok 网页 fallback 没有被误删或误改。 + +## 14. 完成验证 + +完成时间:2026-06-02 + +已完成: + +- `api-chat` runtime 已接入 MNote ChatOnly,直接调用 OmniRoute OpenAI-compatible `/v1/chat/completions`,不启动 ACP/OpenClaw。 +- SQLite profile provisioning 已包含 DeepSeek Flash / DeepSeek Pro / GPT / Kimi / Gemini API / Grok API 六个 shared API ChatOnly profiles。 +- 后端 SSE 已将 OpenAI-compatible streaming 映射为 `message.delta` / `run.completed` / `run.failed`,并写入 `ai_runtime_events` 用于历史恢复。 +- API ChatOnly 删除只删除 MNote 本地会话,返回 `api_chat_has_no_remote_conversation`,不进入网页 provider 远端删除链路。 +- 真实 OmniRoute smoke 已验证六个模型均返回 200。 +- 真实浏览器 smoke 已验证 GPT API 与 DeepSeek Flash API:发送后只有一条助手回复,刷新后历史恢复,删除后本地会话消失,且没有写入 `ai_external_conversation_bindings`。 +- 豆包网页主链回归已验证:豆包端 user/assistant marker 各一条,未重复发送助手回复,删除走侧栏三点菜单并返回 `remote_deleted`。 diff --git a/rust/crates/bridge-runtime/src/lib.rs b/rust/crates/bridge-runtime/src/lib.rs index b1d5b7c2..2bfa0203 100644 --- a/rust/crates/bridge-runtime/src/lib.rs +++ b/rust/crates/bridge-runtime/src/lib.rs @@ -6439,6 +6439,34 @@ fn legacy_block_projection_attrs(block: &Value, block_type: &str) -> Value { } } } + if block_type == "image" { + for key in ["src", "alt", "title"] { + if let Some(value) = props + .and_then(|map| map.get(key)) + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + { + attrs.insert(key.into(), json!(value)); + } + } + if let Some(tiptap_image) = props + .and_then(|map| map.get("tiptapImage")) + .filter(|value| value.get("type").and_then(Value::as_str) == Some("image")) + .cloned() + { + attrs.insert("tiptapImage".into(), tiptap_image); + } + } + if block_type == "table" { + if let Some(tiptap_table) = props + .and_then(|map| map.get("tiptapTable")) + .filter(|value| value.get("type").and_then(Value::as_str) == Some("table")) + .cloned() + { + attrs.insert("tiptapTable".into(), tiptap_table); + } + } Value::Object(attrs) } @@ -13486,6 +13514,68 @@ mod tests { ); } + #[test] + fn local_markdown_image_legacy_projection_exposes_image_attrs() { + let legacy = json!([{ + "id": "local-block-2", + "type": "image", + "props": { + "src": "./Page.assets/photo.png", + "alt": "photo", + "title": "photo" + }, + "content": [], + "children": [] + }]); + + let projection = + project_legacy_content_to_block_document("local-md:docs~2FPage.md", &legacy, &json!(0)) + .expect("image block should project"); + let block = &projection["blocks"][0]; + assert_eq!(block["type"], json!("image")); + assert_eq!(block["attrs"]["src"], json!("./Page.assets/photo.png")); + assert_eq!(block["attrs"]["alt"], json!("photo")); + assert_eq!(block["attrs"]["title"], json!("photo")); + } + + #[test] + fn local_markdown_table_legacy_projection_exposes_tiptap_table_attrs() { + let legacy = json!([{ + "id": "local-block-3", + "type": "table", + "props": { + "tiptapTable": { + "type": "table", + "content": [{ + "type": "tableRow", + "content": [{ + "type": "tableCell", + "attrs": { "colspan": 1, "rowspan": 1, "colwidth": null }, + "content": [{ + "type": "paragraph", + "content": [{ "type": "text", "text": "Provider 类别" }] + }] + }] + }] + } + }, + "content": [], + "children": [] + }]); + + let projection = + project_legacy_content_to_block_document("local-md:docs~2FPage.md", &legacy, &json!(0)) + .expect("table block should project"); + let block = &projection["blocks"][0]; + assert_eq!(block["type"], json!("table")); + assert_eq!(block["attrs"]["tiptapTable"]["type"], json!("table")); + assert_eq!( + block["attrs"]["tiptapTable"]["content"][0]["content"][0]["content"][0]["content"][0] + ["text"], + json!("Provider 类别") + ); + } + #[test] fn legacy_content_from_editor_document_preserves_inline_link_styles() { let mut attrs = BTreeMap::new(); diff --git a/rust/crates/control-plane/src/sqlite.rs b/rust/crates/control-plane/src/sqlite.rs index b6b896f9..7c04a377 100644 --- a/rust/crates/control-plane/src/sqlite.rs +++ b/rust/crates/control-plane/src/sqlite.rs @@ -1665,6 +1665,42 @@ impl ControlPlaneStore for SqliteControlPlaneStore { "openclaw-doubao-chat", "豆包 Chat", ), + ( + "shared_api_deepseek_flash_chat", + "api-deepseek-flash-chat", + "api-deepseek-flash-chat", + "DeepSeek Flash Chat", + ), + ( + "shared_api_deepseek_pro_chat", + "api-deepseek-pro-chat", + "api-deepseek-pro-chat", + "DeepSeek Pro Chat", + ), + ( + "shared_api_gpt_chat", + "api-gpt-chat", + "api-gpt-chat", + "GPT Chat", + ), + ( + "shared_api_kimi_chat", + "api-kimi-chat", + "api-kimi-chat", + "Kimi Chat", + ), + ( + "shared_api_gemini_chat", + "api-gemini-chat", + "api-gemini-chat", + "Gemini API Chat", + ), + ( + "shared_api_grok_chat", + "api-grok-chat", + "api-grok-chat", + "Grok API Chat", + ), ]; for (profile_id, base_profile, isolated_profile, display_name) in shared_chat_profiles { conn.execute( @@ -3241,7 +3277,7 @@ mod tests { let alice_profiles = store .ensure_ai_agent_profile_policy("alice", false) .expect("alice profiles"); - assert_eq!(alice_profiles.len(), 5); + assert_eq!(alice_profiles.len(), 11); let alice_personal = alice_profiles .iter() .find(|item| item.profile.profile_kind == "personal") @@ -3265,6 +3301,24 @@ mod tests { ), ("shared_gemini_chat", "openclaw-gemini-chat", "Gemini Chat"), ("shared_doubao_chat", "openclaw-doubao-chat", "豆包 Chat"), + ( + "shared_api_deepseek_flash_chat", + "api-deepseek-flash-chat", + "DeepSeek Flash Chat", + ), + ( + "shared_api_deepseek_pro_chat", + "api-deepseek-pro-chat", + "DeepSeek Pro Chat", + ), + ("shared_api_gpt_chat", "api-gpt-chat", "GPT Chat"), + ("shared_api_kimi_chat", "api-kimi-chat", "Kimi Chat"), + ( + "shared_api_gemini_chat", + "api-gemini-chat", + "Gemini API Chat", + ), + ("shared_api_grok_chat", "api-grok-chat", "Grok API Chat"), ] { let alice_chat = alice_profiles .iter() @@ -3300,6 +3354,12 @@ mod tests { "shared_deepseek_chat", "shared_gemini_chat", "shared_doubao_chat", + "shared_api_deepseek_flash_chat", + "shared_api_deepseek_pro_chat", + "shared_api_gpt_chat", + "shared_api_kimi_chat", + "shared_api_gemini_chat", + "shared_api_grok_chat", ] { let admin_chat = store .resolve_ai_agent_profile("admin", true, profile_id) diff --git a/rust/crates/mnote-web/browser/document-resource-tab-runtime.js b/rust/crates/mnote-web/browser/document-resource-tab-runtime.js index 9951b6de..b5200376 100644 --- a/rust/crates/mnote-web/browser/document-resource-tab-runtime.js +++ b/rust/crates/mnote-web/browser/document-resource-tab-runtime.js @@ -1,4 +1,5 @@ import { + localFileOpenPathFromTiptapHref, localMarkdownDocumentIdFromRelativePath, localMarkdownRelativePathFromDocumentId, localizeTiptapAssetUrls, @@ -996,6 +997,36 @@ export const createResourceTabRuntime = (dependencies = {}) => { return override === 'mock' ? 'mock' : 'mineru'; }; + const localOcrAutoEnabled = async () => { + const cached = window.__MNOTE_LOCAL_OCR_PREFERENCES; + if (cached && typeof cached === 'object' && cached['localOcr.autoEnabled'] === true) return true; + const documentId = currentWebShellDocumentId(); + const workspaceId = currentWebShellWorkspaceId(); + const sourceKind = currentWebShellSourceKind(); + const rootUri = currentWebShellRootUri(); + if (!documentId && !workspaceId) return false; + try { + const params = new URLSearchParams(); + if (documentId) params.set('documentId', documentId); + if (workspaceId) params.set('workspaceId', workspaceId); + if (sourceKind) params.set('sourceKind', sourceKind); + if (rootUri) params.set('rootUri', rootUri); + const response = await fetch('/api/ui/preferences/effective?' + params.toString(), { + cache: 'no-store', + headers: { accept: 'application/json' }, + }); + const payload = await response.json().catch(() => null); + if (!response.ok || !payload || payload.ok !== true) return false; + const preferences = payload.result?.localOcrPreferences && typeof payload.result.localOcrPreferences === 'object' + ? payload.result.localOcrPreferences + : {}; + window.__MNOTE_LOCAL_OCR_PREFERENCES = { 'localOcr.autoEnabled': false, ...preferences }; + return window.__MNOTE_LOCAL_OCR_PREFERENCES['localOcr.autoEnabled'] === true; + } catch (_) { + return false; + } + }; + const setLocalOcrStatus = (entry, status, message, job) => { if (!(entry?.panel instanceof HTMLElement)) return; const normalizedStatus = String(status || '').trim() || 'unknown'; @@ -1044,17 +1075,93 @@ export const createResourceTabRuntime = (dependencies = {}) => { jobsBySource: new Map(), drawerOpen: false, eventSource: null, + fileTreeRefreshKeys: new Set(), }; const localOcrJobKey = (job) => String(job?.sourceRootRelativePath || job?.jobId || '').trim(); + const localOcrParentRelativePath = (relativePath) => { + const normalized = String(relativePath || '').trim().replace(/\\/g, '/').replace(/^\/+|\/+$/g, ''); + if (!normalized || normalized.indexOf('/') < 0) return ''; + return normalized.split('/').slice(0, -1).join('/'); + }; + + const dispatchLocalOcrFileTreeRefresh = (job, rootUri) => { + const status = String(job?.status || '').trim(); + if (!['done', 'stale'].includes(status)) return; + const ocrPath = String(job?.ocrRootRelativePath || '').trim().replace(/\\/g, '/').replace(/^\/+/, ''); + const normalizedRootUri = String(rootUri || localOcrTaskState.rootUri || '').trim(); + if (!ocrPath || !normalizedRootUri) return; + const refreshKey = `${ocrPath}:${String(job?.updatedAtMs || job?.finishedAtMs || status)}`; + if (localOcrTaskState.fileTreeRefreshKeys.has(refreshKey)) return; + localOcrTaskState.fileTreeRefreshKeys.add(refreshKey); + const ocrParent = localOcrParentRelativePath(ocrPath); + const ocrParentParent = localOcrParentRelativePath(ocrParent); + const affectedParents = [ocrParent, ocrParentParent] + .filter((path, index, list) => index === list.indexOf(path)) + .map((relativePath) => ({ relativePath, reason: 'local-ocr-sidecar-written' })); + const changedPaths = [ + { relativePath: ocrPath, changeType: 'created' }, + ocrParent ? { relativePath: ocrParent, changeType: 'created' } : null, + ].filter(Boolean); + document.documentElement.setAttribute('data-mnote-local-ocr-filetree-refresh', ocrPath); + window.dispatchEvent(new CustomEvent('tree:local-folder-watch-batch', { + detail: { + payload: { + schema: 'mnote.local_folder.watch_batch.v1', + source: 'local_ocr.job.updated', + rootUri: normalizedRootUri, + revision: String(job?.updatedAtMs || Date.now()), + changedPaths, + affectedParents, + }, + }, + })); + }; + const updateLocalOcrTaskState = (job) => { const key = localOcrJobKey(job); if (!key) return; + if (String(job?.status || '').trim() === 'deleted') { + localOcrTaskState.jobsBySource.delete(key); + renderLocalOcrTaskDock(); + return; + } localOcrTaskState.jobsBySource.set(key, job); renderLocalOcrTaskDock(); }; + const bindLocalOcrTopbarAction = () => { + const toggle = document.querySelector('[data-testid="mnote-local-ocr-task-toggle"]'); + if (!(toggle instanceof HTMLButtonElement)) return null; + if (toggle.getAttribute('data-mnote-local-ocr-bound') === 'true') return toggle; + toggle.setAttribute('data-mnote-local-ocr-bound', 'true'); + toggle.addEventListener('click', () => { + void runManualLocalOcrForActiveTarget(toggle).catch((error) => { + console.warn('mnote local OCR 手动入口失败', error); + }); + }); + return toggle; + }; + + const localOcrJobSnapshotFromEntry = (entry, status, provider, message, timestamp = Date.now()) => { + const sourceRootRelativePath = String(entry?.path || '').trim(); + return { + jobId: `local-ocr-${status}:${sourceRootRelativePath || 'unknown'}:${timestamp}`, + ownerDocumentId: String(entry?.ownerDocumentId || entry?.documentId || currentWebShellDocumentId() || '').trim(), + sourceRootRelativePath, + rootUri: String(entry?.rootUri || localOcrTaskState.rootUri || '').trim(), + ocrRootRelativePath: '', + provider: String(provider || localOcrProvider()), + status, + stageLabel: message || statusTextForLocalOcrJob({ status }), + stale: false, + updatedAtMs: timestamp, + finishedAtMs: status === 'failed' ? timestamp : null, + error: status === 'failed' ? String(message || '') : '', + }; + }; + const statusTextForLocalOcrJob = (job) => { const status = String(job?.status || '').trim(); if (job?.stageLabel) return String(job.stageLabel); @@ -1067,20 +1174,58 @@ export const createResourceTabRuntime = (dependencies = {}) => { const ensureLocalOcrTaskDock = () => { let dock = document.querySelector('[data-testid="mnote-local-ocr-task-dock"]'); - if (dock instanceof HTMLElement) return dock; - dock = document.createElement('section'); - dock.className = 'mnote-local-ocr-task-dock'; - dock.setAttribute('data-testid', 'mnote-local-ocr-task-dock'); - dock.innerHTML = ''; - document.body.appendChild(dock); - const toggle = dock.querySelector('[data-testid="mnote-local-ocr-task-toggle"]'); - if (toggle instanceof HTMLButtonElement) { - toggle.addEventListener('click', () => { - localOcrTaskState.drawerOpen = !localOcrTaskState.drawerOpen; - renderLocalOcrTaskDock(); - }); + if (!(dock instanceof HTMLElement)) { + dock = document.createElement('section'); + dock.className = 'mnote-local-ocr-task-dock'; + dock.setAttribute('data-testid', 'mnote-local-ocr-task-dock'); + dock.innerHTML = ''; + document.body.appendChild(dock); } + let toggle = document.querySelector('[data-testid="mnote-local-ocr-task-toggle"]'); + if (!(toggle instanceof HTMLButtonElement)) { + const actions = document.querySelector('.wolai-topbar-actions'); + toggle = document.createElement('button'); + toggle.type = 'button'; + toggle.className = 'wolai-icon-button mnote-local-ocr-task-toggle'; + toggle.setAttribute('title', 'OCR 任务'); + toggle.setAttribute('aria-label', 'OCR 任务'); + toggle.setAttribute('data-testid', 'mnote-local-ocr-task-toggle'); + toggle.setAttribute('data-mnote-action', 'toggle-ocr-tasks'); + toggle.innerHTML = ''; + if (actions instanceof HTMLElement) actions.appendChild(toggle); + else document.body.appendChild(toggle); + } + bindLocalOcrTopbarAction(); + if (dock.getAttribute('data-mnote-local-ocr-bound') === 'true') return dock; + dock.setAttribute('data-mnote-local-ocr-bound', 'true'); dock.addEventListener('click', (event) => { + const closeButton = event.target instanceof Element ? event.target.closest('[data-mnote-local-ocr-task-close]') : null; + if (closeButton instanceof HTMLElement) { + localOcrTaskState.drawerOpen = false; + renderLocalOcrTaskDock(); + return; + } + const clearButton = event.target instanceof Element ? event.target.closest('[data-mnote-local-ocr-task-clear]') : null; + if (clearButton instanceof HTMLElement) { + const sourcePath = clearButton.getAttribute('data-mnote-local-ocr-task-clear') || ''; + if (sourcePath) { + localOcrTaskState.jobsBySource.delete(sourcePath); + renderLocalOcrTaskDock(); + } + return; + } + const deleteButton = event.target instanceof Element ? event.target.closest('[data-mnote-local-ocr-task-delete]') : null; + if (deleteButton instanceof HTMLElement) { + const sourcePath = deleteButton.getAttribute('data-mnote-local-ocr-task-delete') || ''; + if (sourcePath) { + void deleteLocalOcrJob(sourcePath).catch((error) => { + const message = error instanceof Error ? error.message : String(error); + document.documentElement.setAttribute('data-mnote-local-ocr-delete-error', message); + console.warn('mnote local OCR 删除失败', error); + }); + } + return; + } const openButton = event.target instanceof Element ? event.target.closest('[data-mnote-local-ocr-task-open]') : null; if (openButton instanceof HTMLElement) { const sourcePath = openButton.getAttribute('data-mnote-local-ocr-task-open') || ''; @@ -1129,10 +1274,20 @@ export const createResourceTabRuntime = (dependencies = {}) => { const jobs = Array.from(localOcrTaskState.jobsBySource.values()) .sort((left, right) => Number(right.updatedAtMs || 0) - Number(left.updatedAtMs || 0)); const runningCount = jobs.filter((job) => !['done', 'failed', 'stale'].includes(String(job?.status || ''))).length; - const toggle = dock.querySelector('[data-testid="mnote-local-ocr-task-toggle"]'); + const toggle = document.querySelector('[data-testid="mnote-local-ocr-task-toggle"]'); if (toggle instanceof HTMLButtonElement) { - toggle.textContent = runningCount > 0 ? `OCR ${runningCount} 处理中` : `OCR ${jobs.length}`; + const label = runningCount > 0 ? `${runningCount} 个 OCR 处理中` : (jobs.length > 0 ? `${jobs.length} 个 OCR 任务` : 'OCR 任务'); + toggle.setAttribute('title', label); + toggle.setAttribute('aria-label', label); toggle.setAttribute('aria-expanded', localOcrTaskState.drawerOpen ? 'true' : 'false'); + toggle.classList.toggle('has-mnote-local-ocr-tasks', jobs.length > 0); + const badge = toggle.querySelector('[data-mnote-local-ocr-task-count]'); + if (badge instanceof HTMLElement) { + badge.textContent = String(runningCount > 0 ? runningCount : jobs.length); + badge.hidden = jobs.length === 0; + } else { + toggle.textContent = runningCount > 0 ? `OCR ${runningCount} 处理中` : `OCR ${jobs.length}`; + } } const drawer = dock.querySelector('[data-testid="mnote-local-ocr-task-drawer"]'); if (drawer instanceof HTMLElement) drawer.hidden = !localOcrTaskState.drawerOpen; @@ -1152,7 +1307,7 @@ export const createResourceTabRuntime = (dependencies = {}) => { row.setAttribute('data-mnote-local-ocr-task-row', String(job.sourceRootRelativePath || '')); row.setAttribute('data-mnote-local-ocr-task-status', String(job.status || '')); const title = String(job.sourceRootRelativePath || '').split('/').filter(Boolean).pop() || 'OCR'; - row.innerHTML = '
'; + row.innerHTML = '
'; row.querySelector('strong').textContent = title; row.querySelector('span').textContent = statusTextForLocalOcrJob(job); const open = row.querySelector('[data-mnote-local-ocr-task-open]'); @@ -1165,6 +1320,15 @@ export const createResourceTabRuntime = (dependencies = {}) => { retry.setAttribute('data-mnote-local-ocr-task-retry', String(job.sourceRootRelativePath || '')); retry.hidden = !['failed', 'stale'].includes(String(job.status || '')); } + const clear = row.querySelector('[data-mnote-local-ocr-task-clear]'); + if (clear instanceof HTMLButtonElement) { + clear.setAttribute('data-mnote-local-ocr-task-clear', String(job.sourceRootRelativePath || '')); + } + const deleteOcr = row.querySelector('[data-mnote-local-ocr-task-delete]'); + if (deleteOcr instanceof HTMLButtonElement) { + deleteOcr.setAttribute('data-mnote-local-ocr-task-delete', String(job.sourceRootRelativePath || '')); + deleteOcr.hidden = !job.ocrRootRelativePath; + } list.appendChild(row); }); }; @@ -1178,7 +1342,10 @@ export const createResourceTabRuntime = (dependencies = {}) => { const response = await fetch(url.toString(), { cache: 'no-store', headers: { accept: 'application/json' } }); const payload = await response.json().catch(() => null); if (!response.ok || !payload || payload.ok !== true) return; - (Array.isArray(payload.jobs) ? payload.jobs : []).forEach(updateLocalOcrTaskState); + (Array.isArray(payload.jobs) ? payload.jobs : []).forEach((job) => { + if (job && typeof job === 'object') job.rootUri = normalizedRoot; + updateLocalOcrTaskState(job); + }); renderLocalOcrTaskDock(); }; @@ -1198,7 +1365,11 @@ export const createResourceTabRuntime = (dependencies = {}) => { eventSource.addEventListener('local_ocr.job.updated', (event) => { let payload = null; try { payload = JSON.parse(event.data || '{}'); } catch (_) {} - if (payload?.job) updateLocalOcrTaskState(payload.job); + if (payload?.job) { + if (payload.job && typeof payload.job === 'object') payload.job.rootUri = payload.rootUri || normalizedRoot; + updateLocalOcrTaskState(payload.job); + dispatchLocalOcrFileTreeRefresh(payload.job, payload.rootUri || normalizedRoot); + } }); }; @@ -1247,8 +1418,13 @@ export const createResourceTabRuntime = (dependencies = {}) => { const createLocalOcrJob = async (entry) => { if (!isLocalOcrSourceEntry(entry)) return null; - setLocalOcrStatus(entry, 'running', 'OCR 处理中', entry.localOcrJob || null); const provider = localOcrProvider(); + const entryRootUri = String(entry.rootUri || '').trim(); + if (entryRootUri) localOcrTaskState.rootUri = entryRootUri; + const startedAt = Date.now(); + const pendingJob = localOcrJobSnapshotFromEntry(entry, 'running', provider, '处理中', startedAt); + setLocalOcrStatus(entry, 'running', 'OCR 处理中', pendingJob); + updateLocalOcrTaskState(pendingJob); const body = { rootUri: String(entry.rootUri || '').trim(), documentId: String(entry.ownerDocumentId || entry.documentId || currentWebShellDocumentId() || '').trim(), @@ -1266,15 +1442,159 @@ export const createResourceTabRuntime = (dependencies = {}) => { const payload = await response.json().catch(() => null); if (!response.ok || !payload || payload.ok !== true) { const message = payload?.error?.message || `local_ocr_job_failed_${response.status}`; - setLocalOcrStatus(entry, 'failed', message, entry.localOcrJob || null); + const failedJob = { + ...pendingJob, + status: 'failed', + stageLabel: message, + updatedAtMs: Date.now(), + finishedAtMs: Date.now(), + error: message, + }; + setLocalOcrStatus(entry, 'failed', message, failedJob); + updateLocalOcrTaskState(failedJob); throw new Error(message); } - const job = payload.job || null; + const job = payload.job && typeof payload.job === 'object' ? { ...payload.job, rootUri: entryRootUri } : null; setLocalOcrStatus(entry, String(job?.status || 'done'), job?.stale ? 'OCR 需更新' : 'OCR 已完成', job); if (job) updateLocalOcrTaskState(job); return job; }; + const deleteLocalOcrJob = async (sourceRootRelativePath) => { + const sourcePath = String(sourceRootRelativePath || '').trim(); + if (!sourcePath) return false; + const job = localOcrTaskState.jobsBySource.get(sourcePath) || null; + const rootUri = String(job?.rootUri || localOcrTaskState.rootUri || currentWebShellRootUri() || '').trim(); + const response = await fetch('/api/local-folder/ocr/delete', { + method: 'POST', + headers: { 'content-type': 'application/json', accept: 'application/json' }, + body: JSON.stringify({ + rootUri, + sourceRootRelativePath: sourcePath, + }), + }); + const payload = await response.json().catch(() => null); + if (!response.ok || !payload || payload.ok !== true) { + throw new Error(payload?.error?.message || `local_ocr_delete_failed_${response.status}`); + } + if (payload.deleted !== true) { + throw new Error('local_ocr_delete_noop'); + } + localOcrTaskState.jobsBySource.delete(sourcePath); + renderLocalOcrTaskDock(); + const ocrPath = String(job?.ocrRootRelativePath || '').trim(); + if (ocrPath) { + dispatchLocalOcrFileTreeRefresh({ ...job, status: 'stale', updatedAtMs: Date.now(), ocrRootRelativePath: ocrPath }, localOcrTaskState.rootUri); + } + return true; + }; + + const maybeAutoCreateLocalOcrJob = async (entry) => { + if (!isLocalOcrSourceEntry(entry)) return; + if (!await localOcrAutoEnabled()) return; + const existing = await readLocalOcrStatus(entry); + const status = String(existing?.status || '').trim(); + if (existing && !existing.stale && ['done', 'running'].includes(status)) { + updateLocalOcrTaskState(existing); + return; + } + await createLocalOcrJob(entry); + }; + + const activeResourceTabEntry = (paneRole = 'primary') => { + const role = normalizePaneRole(paneRole); + for (const entry of resourceTabRegistry.values()) { + if (normalizePaneRole(entry?.paneRole) !== role) continue; + if (entry?.tab instanceof HTMLElement && entry.tab.getAttribute('aria-selected') === 'true') return entry; + } + return null; + }; + + const localOcrCandidatesFromActiveMarkdown = (paneRole = 'primary') => { + const role = normalizePaneRole(paneRole); + const sourceKind = currentWebShellSourceKind(); + const rootUri = currentWebShellRootUri(); + if (sourceKind !== 'local_folder' || !rootUri) return []; + const documentId = documentIdForPane(role) || currentWebShellDocumentId(); + if (!documentId) return []; + const workspaceId = currentWebShellWorkspaceId(); + const seen = new Set(); + const candidates = []; + document.querySelectorAll(`.document-pane[data-pane-role="${role}"] .ProseMirror img`).forEach((image) => { + if (!(image instanceof HTMLImageElement)) return; + const sourcePath = localFileOpenPathFromTiptapHref(image.getAttribute('src') || image.src || ''); + if (!sourcePath || seen.has(sourcePath)) return; + seen.add(sourcePath); + const title = sourcePath.split('/').filter(Boolean).pop() || sourcePath; + candidates.push({ + sourceKind: 'local_folder', + rootUri, + path: sourcePath, + kind: sourcePath.toLowerCase().endsWith('.pdf') ? 'pdf' : 'image', + title, + fileName: title, + documentId, + ownerDocumentId: documentId, + workspaceId, + objectIdentity: `local-file:${sourcePath}`, + assetId: `local-file:${sourcePath}`, + }); + }); + return candidates; + }; + + const localOcrCandidatesForActiveTarget = (paneRole = 'primary') => { + const activeResource = activeResourceTabEntry(paneRole); + if (isLocalOcrSourceEntry(activeResource)) return [activeResource]; + return localOcrCandidatesFromActiveMarkdown(paneRole); + }; + + const runManualLocalOcrForActiveTarget = async (toggle) => { + ensureLocalOcrTaskDock(); + const candidates = localOcrCandidatesForActiveTarget('primary'); + if (!candidates.length) { + if (localOcrTaskState.jobsBySource.size > 0) { + localOcrTaskState.drawerOpen = !localOcrTaskState.drawerOpen; + renderLocalOcrTaskDock(); + } + return []; + } + if (toggle instanceof HTMLButtonElement) toggle.disabled = true; + const jobs = []; + let createdCount = 0; + try { + for (const entry of candidates) { + try { + ensureLocalOcrTaskEvents(entry.rootUri); + const existing = await readLocalOcrStatus(entry); + const existingStatus = String(existing?.status || '').trim(); + if (existing && !existing.stale && ['done', 'running'].includes(existingStatus)) { + updateLocalOcrTaskState(existing); + jobs.push(existing); + continue; + } + const job = await createLocalOcrJob(entry); + if (job) { + createdCount += 1; + jobs.push(job); + } + } catch (error) { + console.warn('mnote local OCR 手动任务失败', entry?.path, error); + if (localOcrTaskState.jobsBySource.has(String(entry?.path || '').trim())) { + createdCount += 1; + } + } + } + if (createdCount === 0 && localOcrTaskState.jobsBySource.size > 0) { + localOcrTaskState.drawerOpen = !localOcrTaskState.drawerOpen; + } + } finally { + if (toggle instanceof HTMLButtonElement) toggle.disabled = false; + renderLocalOcrTaskDock(); + } + return jobs; + }; + const renderLocalOcrToolbar = (entry) => { if (!isLocalOcrSourceEntry(entry) || !(entry.panel instanceof HTMLElement)) return; const toolbar = entry.panel.querySelector('[data-testid="mnote-local-ocr-toolbar"]'); @@ -1322,6 +1642,7 @@ export const createResourceTabRuntime = (dependencies = {}) => { const createResourceSession = (entry, input, readResult) => { const resourcePath = String(input.path || ''); + const resourceDocumentId = localMarkdownDocumentIdFromRelativePath(resourcePath) || entry.objectIdentity; const tiptapDocument = localizeTiptapAssetUrls( toTiptapDocument(readResult?.content, readResult?.text || ''), { @@ -1334,7 +1655,7 @@ export const createResourceTabRuntime = (dependencies = {}) => { const session = { key: `resource:${input.rootUri || ''}:${input.path || entry.objectIdentity}`, sessionKind: 'resource', - documentId: entry.objectIdentity, + documentId: resourceDocumentId, ownerDocumentId: String(input.documentId || currentWebShellDocumentId() || '').trim(), workspaceId: String(input.workspaceId || currentWebShellWorkspaceId() || ''), sourceKind: 'local_folder', @@ -1447,21 +1768,20 @@ export const createResourceTabRuntime = (dependencies = {}) => { const openPassiveResourceTab = (entry, input) => { const href = String(input.officeUrl || input.href || '').trim(); if (entry.kind === 'image') { - entry.panel.innerHTML = isLocalOcrSourceEntry(entry) - ? '
OCR 未生成
' - : ''; + entry.panel.innerHTML = ''; const img = entry.panel.querySelector('img'); if (img instanceof HTMLImageElement) { img.src = href; img.alt = entry.title; } - renderLocalOcrToolbar(entry); + ensureLocalOcrTaskDock(); + ensureLocalOcrTaskEvents(entry.rootUri); + void loadLocalOcrJobs(entry.rootUri).catch(() => undefined); + void maybeAutoCreateLocalOcrJob(entry).catch((error) => console.warn('mnote local OCR 自动任务失败', error)); installPassiveResourceWatch(entry); return; } - entry.panel.innerHTML = isLocalOcrSourceEntry(entry) - ? '
OCR 未生成
' - : ''; + entry.panel.innerHTML = ''; const frame = entry.panel.querySelector('iframe'); if (frame instanceof HTMLIFrameElement) { frame.title = entry.title; @@ -1473,7 +1793,12 @@ export const createResourceTabRuntime = (dependencies = {}) => { } frame.src = href; } - renderLocalOcrToolbar(entry); + if (isLocalOcrSourceEntry(entry)) { + ensureLocalOcrTaskDock(); + ensureLocalOcrTaskEvents(entry.rootUri); + void loadLocalOcrJobs(entry.rootUri).catch(() => undefined); + void maybeAutoCreateLocalOcrJob(entry).catch((error) => console.warn('mnote local OCR 自动任务失败', error)); + } installPassiveResourceWatch(entry); }; @@ -1610,6 +1935,10 @@ export const createResourceTabRuntime = (dependencies = {}) => { syncOpenEditorsSnapshot(); }; + try { + bindLocalOcrTopbarAction(); + } catch (_) {} + return { activateMainEditorTab, bindMainEditorPageTab, diff --git a/rust/crates/mnote-web/browser/document-tiptap-conversion-runtime.js b/rust/crates/mnote-web/browser/document-tiptap-conversion-runtime.js index 23fd4dd7..8dd74c4a 100644 --- a/rust/crates/mnote-web/browser/document-tiptap-conversion-runtime.js +++ b/rust/crates/mnote-web/browser/document-tiptap-conversion-runtime.js @@ -51,11 +51,12 @@ export const legacyStylesToTiptapMarks = (styles) => { export const legacyMarkArrayToTiptapMarks = (inlineMarks) => { if (!Array.isArray(inlineMarks)) return []; return inlineMarks.flatMap((mark) => { - if (!mark || typeof mark !== 'object') return []; - if (mark.type === 'bold' || mark.type === 'italic' || mark.type === 'underline' || mark.type === 'strike' || mark.type === 'code') { - return [{ type: mark.type }]; + const markType = typeof mark === 'string' ? mark : typeof mark?.type === 'string' ? mark.type : ''; + if (!markType) return []; + if (markType === 'bold' || markType === 'italic' || markType === 'underline' || markType === 'strike' || markType === 'code') { + return [{ type: markType }]; } - if (mark.type === 'link') { + if (markType === 'link') { const href = typeof mark?.attrs?.href === 'string' ? mark.attrs.href.trim() : ''; return href ? [{ type: 'link', attrs: { href } }] : []; } @@ -77,10 +78,19 @@ export const legacyInlineContentToTiptap = (value) => { if (typeof value === 'string') return value ? [{ type: 'text', text: value }] : []; if (Array.isArray(value)) return value.flatMap(legacyInlineContentToTiptap); if (value && typeof value === 'object') { - const text = typeof value.text === 'string' ? value.text : ''; + const payload = value.payload && typeof value.payload === 'object' ? value.payload : null; + if (payload?.type === 'hard_break') return [{ type: 'hardBreak' }]; + const text = typeof value.text === 'string' + ? value.text + : typeof payload?.text === 'string' + ? payload.text + : ''; if (text) { + const attrs = value.attrs && typeof value.attrs === 'object' ? value.attrs : {}; const marks = mergeTiptapMarks( + legacyStylesToTiptapMarks(attrs.styles), legacyStylesToTiptapMarks(value.styles), + legacyMarkArrayToTiptapMarks(payload?.marks), legacyMarkArrayToTiptapMarks(value.marks) ); return [{ type: 'text', text, ...(marks.length ? { marks } : {}) }]; @@ -182,7 +192,7 @@ export const legacyBlockToTiptap = (block, index = 0) => { return { type: 'paragraph', attrs: withTextAlign({ blockId }), content: mediaContent }; } if (type === 'table') { - const tableSnapshot = block?.props?.tiptapTable; + const tableSnapshot = block?.props?.tiptapTable || block?.attrs?.tiptapTable; if (tableSnapshot && typeof tableSnapshot === 'object' && tableSnapshot.type === 'table') return tableSnapshot; return { type: 'table', @@ -209,12 +219,13 @@ export const legacyBlockToTiptap = (block, index = 0) => { }; } if (type === 'image') { - const imageSnapshot = block?.props?.tiptapImage; + const imageSnapshot = block?.props?.tiptapImage || block?.attrs?.tiptapImage; if (imageSnapshot && typeof imageSnapshot === 'object' && imageSnapshot.type === 'image') return imageSnapshot; + const attrsSource = block?.attrs && typeof block.attrs === 'object' ? block.attrs : {}; const attrs = { - src: String(block?.props?.src || block?.src || ''), - alt: block?.props?.alt || block?.alt || null, - title: block?.props?.title || block?.title || null, + src: String(block?.props?.src || attrsSource.src || block?.src || ''), + alt: block?.props?.alt || attrsSource.alt || block?.alt || null, + title: block?.props?.title || attrsSource.title || block?.title || null, }; return attrs.src ? { type: 'image', attrs: { blockId, ...attrs } } : null; } @@ -318,7 +329,7 @@ export const localMarkdownDirectoryFromDocumentId = (documentId) => { }; export const isExternalOrSpecialUrl = (value) => { - const text = String(value || '').trim(); + const text = unwrapMarkdownLinkTarget(value); return !text || text.startsWith('#') || text.startsWith('data:') @@ -329,8 +340,16 @@ export const isExternalOrSpecialUrl = (value) => { || text.startsWith('/api/'); }; -export const normalizeLocalAssetRelativePath = (value, context) => { +export const unwrapMarkdownLinkTarget = (value) => { const text = String(value || '').trim(); + if (text.length >= 2 && text.startsWith('<') && text.endsWith('>')) { + return text.slice(1, -1).trim(); + } + return text; +}; + +export const normalizeLocalAssetRelativePath = (value, context) => { + const text = unwrapMarkdownLinkTarget(value); if (!text || isExternalOrSpecialUrl(text)) return text; if (text.startsWith('/')) return text.replace(/^\/+/, ''); const baseDir = localMarkdownDirectoryFromDocumentId(context?.documentId); diff --git a/rust/crates/mnote-web/browser/filetree-selection-runtime.js b/rust/crates/mnote-web/browser/filetree-selection-runtime.js index a83eb54d..2b7e85da 100644 --- a/rust/crates/mnote-web/browser/filetree-selection-runtime.js +++ b/rust/crates/mnote-web/browser/filetree-selection-runtime.js @@ -13,8 +13,9 @@ function visibleFileTreeRows(deps) { return Array.from(document.querySelectorAll('#sidebar-file-tree-root .tree-row[data-shell-mode="filetree"]')) .filter(function(row) { if (!(row instanceof HTMLElement)) return false; - if (row.closest('.tree-children--collapsed')) return false; - return row.offsetParent !== null || row.getClientRects().length > 0; + // 文件树展开恢复会在短时间内多次同步选择状态。这里不能读取 + // offsetParent/getClientRects,否则会和 DOM 写入交错触发强制布局。 + return !row.closest('.tree-children--collapsed'); }); } diff --git a/rust/crates/mnote-web/browser/local-upload-runtime.js b/rust/crates/mnote-web/browser/local-upload-runtime.js index 61ab0e38..cf1d7003 100644 --- a/rust/crates/mnote-web/browser/local-upload-runtime.js +++ b/rust/crates/mnote-web/browser/local-upload-runtime.js @@ -448,13 +448,14 @@ async function insertUploadedAssetIntoEditor(asset, targetRoot, deps) { var markdownHref = typeof deps.uploadedAssetMarkdownHref === 'function' ? deps.uploadedAssetMarkdownHref(asset) : uploadedAssetMarkdownHref(asset); var localOpenUrl = typeof deps.localAssetOpenUrl === 'function' ? deps.localAssetOpenUrl(asset, false) : localAssetOpenUrl(asset, false); var fallbackUrl = typeof deps.uploadedAssetUrl === 'function' ? deps.uploadedAssetUrl(asset) : uploadedAssetUrl(asset); + var imageUrl = localOpenUrl || fallbackUrl || markdownHref; var url = markdownHref || localOpenUrl || fallbackUrl; var type = typeof deps.uploadedAssetType === 'function' ? deps.uploadedAssetType(asset) : uploadedAssetType(asset); var assetId = String(asset && asset.id || '').trim(); var sizeLabel = typeof deps.uploadedFileSize === 'function' ? deps.uploadedFileSize(asset) : uploadedFileSize(asset); try { - if (type === 'image' && url) { - var imageInserted = editor.chain().focus().setImage({ src: url, alt: title, title: title }).run() === true; + if (type === 'image' && imageUrl) { + var imageInserted = editor.chain().focus().setImage({ src: imageUrl, alt: title, title: title }).run() === true; if (imageInserted) { document.documentElement.setAttribute('data-mnote-last-upload-inserted', 'true'); document.documentElement.setAttribute('data-mnote-last-upload-asset-id', assetId || ''); diff --git a/rust/crates/mnote-web/browser/sidebar-page-ai-profile-runtime.js b/rust/crates/mnote-web/browser/sidebar-page-ai-profile-runtime.js index b92f7431..fc5527e6 100644 --- a/rust/crates/mnote-web/browser/sidebar-page-ai-profile-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-page-ai-profile-runtime.js @@ -93,7 +93,7 @@ export function createSidebarPageAiProfileRuntime(context) { var baseProfile = String(profile && profile.baseProfile || '').trim(); var isolatedProfile = String(profile && profile.isolatedProfile || '').trim(); return PAGE_AI_CHAT_ONLY_PROFILE_REGISTRY.find(function(spec) { - return spec.profileId === profileId || spec.baseProfile === baseProfile || isolatedProfile.indexOf(spec.baseProfile) >= 0; + return spec.profileId === profileId || spec.baseProfile === profileId || spec.baseProfile === baseProfile || isolatedProfile.indexOf(spec.baseProfile) >= 0; }) || null; } diff --git a/rust/crates/mnote-web/browser/sidebar-page-ai-render-runtime.js b/rust/crates/mnote-web/browser/sidebar-page-ai-render-runtime.js index 492f612f..f03653e4 100644 --- a/rust/crates/mnote-web/browser/sidebar-page-ai-render-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-page-ai-render-runtime.js @@ -265,7 +265,9 @@ export function createSidebarPageAiRenderRuntime(context) { var chatOnlyOptions = pageAiChatOnlyProfileEntries().map(function(profile) { var spec = pageAiChatOnlyProfileSpec(profile); var label = profile.menuLabel || (spec && spec.label) || pageAiProfileDisplayLabel(profile, ''); - return pageAiRenderAgentProfileOption('chat_only', profile, label, '网页问答 · 不申请文件写权限'); + var providerKind = String(profile.providerKind || (spec && spec.providerKind) || '').trim(); + var detail = providerKind === 'api-chat' ? 'API 聊天 · 不申请文件写权限' : '网页问答 · 不申请文件写权限'; + return pageAiRenderAgentProfileOption('chat_only', profile, label, detail); }).join(''); var hermesOptions = pageAiHermesProfileEntries().map(function(profile) { var profileId = pageAiProfileValue(profile); diff --git a/rust/crates/mnote-web/browser/sidebar-page-ai-runtime.js b/rust/crates/mnote-web/browser/sidebar-page-ai-runtime.js index e813c0ed..efe92ff7 100644 --- a/rust/crates/mnote-web/browser/sidebar-page-ai-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-page-ai-runtime.js @@ -33,7 +33,13 @@ export function createSidebarPageAiRuntime(context) { var PAGE_AI_CHAT_ONLY_PROFILE_REGISTRY = [ { profileId: 'shared_deepseek_chat', baseProfile: 'deepseek-chat', label: 'DeepSeek' }, { profileId: 'shared_doubao_chat', baseProfile: 'doubao-chat', label: '豆包' }, - { profileId: 'shared_gemini_chat', baseProfile: 'gemini-chat', label: 'Gemini' } + { profileId: 'shared_gemini_chat', baseProfile: 'gemini-chat', label: 'Gemini' }, + { profileId: 'shared_api_deepseek_flash_chat', baseProfile: 'api-deepseek-flash-chat', label: 'DeepSeek Flash', providerKind: 'api-chat' }, + { profileId: 'shared_api_deepseek_pro_chat', baseProfile: 'api-deepseek-pro-chat', label: 'DeepSeek Pro', providerKind: 'api-chat' }, + { profileId: 'shared_api_gpt_chat', baseProfile: 'api-gpt-chat', label: 'GPT', providerKind: 'api-chat' }, + { profileId: 'shared_api_kimi_chat', baseProfile: 'api-kimi-chat', label: 'Kimi', providerKind: 'api-chat' }, + { profileId: 'shared_api_gemini_chat', baseProfile: 'api-gemini-chat', label: 'Gemini API', providerKind: 'api-chat' }, + { profileId: 'shared_api_grok_chat', baseProfile: 'api-grok-chat', label: 'Grok API', providerKind: 'api-chat' } ]; var PAGE_AI_CONTEXT_REF_REGISTRY = [ { id: 'current_page', label: '当前页' }, @@ -731,6 +737,7 @@ export function createSidebarPageAiRuntime(context) { alias: spec.label, kind: 'shared', baseProfile: spec.baseProfile, + providerKind: spec.providerKind || '', readonly: true }, { menuLabel: spec.label }); }); @@ -763,6 +770,7 @@ export function createSidebarPageAiRuntime(context) { ownerUserId: String(profile && profile.ownerUserId || '').trim(), baseProfile: String(profile && profile.baseProfile || '').trim(), isolatedProfile: String(profile && profile.isolatedProfile || '').trim(), + providerKind: String(profile && profile.providerKind || profile && profile.provider_kind || '').trim(), canRun: profile ? profile.canRun !== false : true, canManageSkills: canManageSkills, canManageConfig: profile ? profile.canManageConfig !== false : canManageSkills, diff --git a/rust/crates/mnote-web/browser/sidebar-page-settings-runtime.js b/rust/crates/mnote-web/browser/sidebar-page-settings-runtime.js index b0a0cc45..cc71cc83 100644 --- a/rust/crates/mnote-web/browser/sidebar-page-settings-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-page-settings-runtime.js @@ -67,6 +67,10 @@ export function createSidebarPageSettingsRuntime(context) { return Object.assign({}, DEFAULT_PAGE_WIDTH_PREFERENCES, pageUiState.pageWidthPreferences || {}); } + function currentLocalOcrPreferences() { + return Object.assign({ 'localOcr.autoEnabled': false }, pageUiState.localOcrPreferences || {}); + } + function pageWidthModeLabel(mode) { if (mode === 'inherit') return '继承默认'; if (mode === 'readable') return '阅读'; @@ -357,6 +361,17 @@ export function createSidebarPageSettingsRuntime(context) { ''; } + function createLocalOcrAutoRow() { + return '' + + ''; + } + function createPageWidthSelectRow(type) { var options = type === 'default' ? [ @@ -397,6 +412,10 @@ export function createSidebarPageSettingsRuntime(context) { popover.querySelectorAll('[data-global-option-checkbox="showHeadingNumbers"]').forEach(function(input) { input.checked = globalHeadingNumbers; }); + var localOcrPreferences = currentLocalOcrPreferences(); + popover.querySelectorAll('[data-local-ocr-option-checkbox="autoEnabled"]').forEach(function(input) { + input.checked = localOcrPreferences['localOcr.autoEnabled'] === true; + }); var preferences = currentPageWidthPreferences(); popover.querySelectorAll('[data-page-width-select]').forEach(function(select) { var type = select.getAttribute('data-page-width-select') || ''; @@ -579,6 +598,7 @@ export function createSidebarPageSettingsRuntime(context) { '' + '' + '
' + @@ -843,11 +863,47 @@ export function createSidebarPageSettingsRuntime(context) { var payload = await response.json().catch(function(){ return null; }); if (!response.ok || !payload || payload.ok !== true) return; pageUiState.pageWidthPreferences = normalizePageWidthPreferences(payload.result && payload.result.pageWidthPreferences); + pageUiState.localOcrPreferences = Object.assign({ 'localOcr.autoEnabled': false }, payload.result && payload.result.localOcrPreferences || {}); + window.__MNOTE_LOCAL_OCR_PREFERENCES = Object.assign({}, pageUiState.localOcrPreferences); + document.documentElement.setAttribute('data-mnote-local-ocr-auto-enabled', String(pageUiState.localOcrPreferences['localOcr.autoEnabled'] === true)); applyPageOptionsToShell(); if (isPageSettingsOpen()) renderPageSettingsPopover(); } catch (_) {} } + async function persistLocalOcrAutoPreference(enabled) { + var previous = currentLocalOcrPreferences(); + var next = Object.assign({}, previous, { 'localOcr.autoEnabled': Boolean(enabled) }); + pageUiState.localOcrPreferences = next; + window.__MNOTE_LOCAL_OCR_PREFERENCES = Object.assign({}, next); + renderPageSettingsPopover(); + try { + var response = await fetch('/api/ui/preferences', { + method: 'PUT', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ + documentId: currentDocumentId(), + workspaceId: resolveWorkspaceId(document.body), + ...currentWorkspaceSourcePayload(), + updates: { 'localOcr.autoEnabled': Boolean(enabled) } + }) + }); + var payload = await response.json().catch(function(){ return null; }); + if (!response.ok || !payload || payload.ok !== true) { + throw new Error(payload && payload.error && payload.error.message ? payload.error.message : 'local_ocr_preference_save_failed_' + response.status); + } + pageUiState.localOcrPreferences = Object.assign({ 'localOcr.autoEnabled': false }, payload.result && payload.result.localOcrPreferences || {}); + window.__MNOTE_LOCAL_OCR_PREFERENCES = Object.assign({}, pageUiState.localOcrPreferences); + document.documentElement.setAttribute('data-mnote-local-ocr-auto-enabled', String(pageUiState.localOcrPreferences['localOcr.autoEnabled'] === true)); + renderPageSettingsPopover(); + } catch (error) { + pageUiState.localOcrPreferences = previous; + window.__MNOTE_LOCAL_OCR_PREFERENCES = Object.assign({}, previous); + renderPageSettingsPopover(); + document.documentElement.setAttribute('data-mnote-local-ocr-preference-error', error instanceof Error ? error.message : String(error)); + } + } + async function persistPageWidthPreference(type, mode) { if (PAGE_WIDTH_TYPES.indexOf(type) < 0) return; var previous = pageUiState.pageWidthPreferences; @@ -928,6 +984,7 @@ export function createSidebarPageSettingsRuntime(context) { closePageHistoryDrawer, closePageSettingsPopover, closePageShareDialog, + currentLocalOcrPreferences, currentPageOptions, ensureHistorySnapshotsSeeded, ensurePageHistoryDrawer, @@ -938,6 +995,7 @@ export function createSidebarPageSettingsRuntime(context) { openPageSettingsPopover, openPageShareDialog, pageOptionIsSupported, + persistLocalOcrAutoPreference, persistPageOptionsPatch, persistPageWidthPreference, recordPageHistorySnapshot, diff --git a/rust/crates/mnote-web/browser/sidebar-tree-live-apply-runtime.js b/rust/crates/mnote-web/browser/sidebar-tree-live-apply-runtime.js index 24321651..8e9ebd60 100644 --- a/rust/crates/mnote-web/browser/sidebar-tree-live-apply-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-tree-live-apply-runtime.js @@ -1440,12 +1440,9 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { } } if (fileTreeState.loadingParents.has(key)) { - return fileTreeState.loadingParents.get(key).then(function(rows) { - fileTreeState.dirtyParents.delete(key); - return isFileTreeRootProjectionParent(parentRelativePath) - ? renderFileProjection({ parentRelativePath: parentRelativePath, items: rows }) - : patchFileTreeParentChildren(parentRelativePath, rows); - }); + // 变更事件可能正好撞上同一 parent 的懒加载请求。 + // 旧请求结果不包含刚落盘的文件,不能直接拿它满足本次刷新。 + await fileTreeState.loadingParents.get(key).catch(function() { return []; }); } var projection = await fetchFileTreeProjection(parentRelativePath, { childrenOnly: false }); if (!projection) return false; @@ -1556,13 +1553,14 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { return ''; } - function setTreeRowExpanded(row, button, expanded) { + function setTreeRowExpanded(row, button, expanded, options) { row.setAttribute('aria-expanded', expanded ? 'true' : 'false'); var shellMode = row.getAttribute('data-shell-mode') || ''; if (button) { button.setAttribute('aria-expanded', expanded ? 'true' : 'false'); if (shellMode === 'filetree') button.textContent = expanded ? '▾' : '▸'; } + var shouldPersist = !(options && options.persist === false); if (shellMode === 'page') { var nodeId = String(row.getAttribute('data-node-id') || '').trim(); if (!nodeId) return; @@ -1570,18 +1568,20 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { if (expanded) pageState.expandedIds.add(nodeId); else pageState.expandedIds.delete(nodeId); pageState.hasUserState = true; - persistSidebarTreeViewState('pagetree'); + if (shouldPersist) persistSidebarTreeViewState('pagetree'); return; } var relativePath = localFileTreeRelativePathFromRow(row); if (!relativePath) return; if (expanded) fileTreeExpandedRelativePaths.add(relativePath); else fileTreeExpandedRelativePaths.delete(relativePath); - persistFileTreeExpansionState(); - persistSidebarTreeViewState('filetree'); + if (shouldPersist) { + persistFileTreeExpansionState(); + persistSidebarTreeViewState('filetree'); + } } - function markExistingFileTreeChildrenLoaded(row, button) { + function markExistingFileTreeChildrenLoaded(row, button, options) { if (!(row instanceof HTMLElement)) return false; var node = row.closest('.tree-node'); if (!node) return false; @@ -1589,12 +1589,12 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { if (!(children instanceof HTMLElement)) return false; row.setAttribute('data-filetree-children-loaded', 'true'); children.classList.remove('tree-children--collapsed'); - setTreeRowExpanded(row, button, true); + setTreeRowExpanded(row, button, true, options); syncSidebarFileTreeSelection(); return true; } - function renderCachedFileTreeChildren(row, button, relativePath) { + function renderCachedFileTreeChildren(row, button, relativePath, options) { var key = currentFileTreeParentKey(relativePath); if (fileTreeState.staleParents.has(key) || fileTreeState.dirtyParents.has(key)) return false; var cachedRows = cachedFileTreeRows(relativePath); @@ -1612,7 +1612,7 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { children.replaceChildren(template.content.cloneNode(true)); children.classList.remove('tree-children--collapsed'); row.setAttribute('data-filetree-children-loaded', 'true'); - setTreeRowExpanded(row, button, true); + setTreeRowExpanded(row, button, true, options); syncSidebarFileTreeSelection(); return true; } @@ -1641,7 +1641,7 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { return promise; } - async function loadFileTreeChildren(row, button) { + async function loadFileTreeChildren(row, button, options) { if (!(row instanceof HTMLElement)) return false; if (row.getAttribute('data-shell-mode') !== 'filetree') return false; if (currentSourceKind() !== 'local_folder') return false; @@ -1649,21 +1649,21 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { var relativePath = localFileTreeRelativePathFromRow(row); var key = currentFileTreeParentKey(relativePath); var stale = fileTreeState.staleParents.has(key) || fileTreeState.dirtyParents.has(key); - if (!stale && markExistingFileTreeChildrenLoaded(row, button)) return true; - if (!stale && relativePath && renderCachedFileTreeChildren(row, button, relativePath)) return true; + if (!stale && markExistingFileTreeChildrenLoaded(row, button, options)) return true; + if (!stale && relativePath && renderCachedFileTreeChildren(row, button, relativePath, options)) return true; if (row.getAttribute('data-filetree-children-loaded') === 'true') return false; var rootUri = currentRootUri(); if (!rootUri || !relativePath) return false; - setTreeRowExpanded(row, button, true); + setTreeRowExpanded(row, button, true, options); row.setAttribute('data-filetree-children-loading', 'true'); try { var rows = await getFileTreeChildren(relativePath); if (!rows.length) { row.setAttribute('data-filetree-children-loaded', 'true'); - setTreeRowExpanded(row, button, true); + setTreeRowExpanded(row, button, true, options); return true; } - return renderCachedFileTreeChildren(row, button, relativePath); + return renderCachedFileTreeChildren(row, button, relativePath, options); } catch (error) { setTreeLiveApplyError(error && error.message ? error.message : '文件树子目录加载失败'); return false; @@ -1731,6 +1731,15 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { setTreeRowExpanded(row, button, !collapsed); } + function shouldAutoRestoreFileTreeExpansion(relativePath) { + var normalized = normalizeFileTreeRelativePath(relativePath); + if (!normalized) return false; + var scope = normalizeFileTreeRelativePath(currentFileTreeScope()); + if (!scope) return normalized.indexOf('/') < 0; + if (normalized.indexOf(scope + '/') !== 0) return false; + return normalized.slice(scope.length + 1).indexOf('/') < 0; + } + function restorePersistedFileTreeExpansionState() { if (currentSourceKind() !== 'local_folder') return false; ensureFileTreeLazyCacheScope(); @@ -1741,17 +1750,20 @@ export const createSidebarTreeLiveApplyRuntime = (dependencies = {}) => { if (String(row.getAttribute('data-row-kind') || '').trim() !== 'folder') return; var relativePath = localFileTreeRelativePathFromRow(row); if (!relativePath || !fileTreeExpandedRelativePaths.has(relativePath)) return; + // 首屏只自动恢复当前 scope 的直接子目录。历史 view-state 可能记录了 + // 多层 design 目录展开;一次性递归恢复会长时间占满浏览器主线程。 + if (!shouldAutoRestoreFileTreeExpansion(relativePath)) return; var button = row.querySelector('[data-rust-action="toggle"]'); - if (markExistingFileTreeChildrenLoaded(row, button)) { + if (markExistingFileTreeChildrenLoaded(row, button, { persist: false })) { restored = true; return; } if (row.getAttribute('data-filetree-children-loaded') === 'true') { - setTreeRowExpanded(row, button, true); + setTreeRowExpanded(row, button, true, { persist: false }); restored = true; return; } - void loadFileTreeChildren(row, button).then(function(loaded) { + void loadFileTreeChildren(row, button, { persist: false }).then(function(loaded) { if (loaded) restorePersistedFileTreeExpansionState(); }); restored = true; diff --git a/rust/crates/mnote-web/browser/sidebar-tree-runtime.js b/rust/crates/mnote-web/browser/sidebar-tree-runtime.js index 5c4e81fb..689ea474 100644 --- a/rust/crates/mnote-web/browser/sidebar-tree-runtime.js +++ b/rust/crates/mnote-web/browser/sidebar-tree-runtime.js @@ -208,6 +208,7 @@ import { createSidebarPageSettingsRuntime } from './sidebar-page-settings-runtim const openPageHistoryDrawer = (...args) => sidebarPageSettings.openPageHistoryDrawer(...args); const openPageShareDialog = (...args) => sidebarPageSettings.openPageShareDialog(...args); const pageOptionIsSupported = (...args) => sidebarPageSettings.pageOptionIsSupported(...args); + const persistLocalOcrAutoPreference = (...args) => sidebarPageSettings.persistLocalOcrAutoPreference(...args); const persistPageOptionsPatch = (...args) => sidebarPageSettings.persistPageOptionsPatch(...args); const persistPageWidthPreference = (...args) => sidebarPageSettings.persistPageWidthPreference(...args); const recordPageHistorySnapshot = (...args) => sidebarPageSettings.recordPageHistorySnapshot(...args); @@ -441,6 +442,11 @@ import { createSidebarPageSettingsRuntime } from './sidebar-page-settings-runtim return title && title.textContent ? title.textContent.trim() : (fallback || '文件夹'); } + function isLocalOcrMarkdownPath(relativePath) { + var normalized = String(relativePath || '').trim().replace(/\\/g, '/').replace(/^\/+/, ''); + return /(^|\/)[^/]+\.ocr\/[^/]+\.ocr\.md$/i.test(normalized); + } + function sidebarShortcutRows() { return Array.from(document.querySelectorAll('.wolai-starred-section [data-mnote-shortcut-kind]')); } @@ -1714,13 +1720,16 @@ import { createSidebarPageSettingsRuntime } from './sidebar-page-settings-runtim } var title = uploadedAssetTitle(asset); var markdownHref = uploadedAssetMarkdownHref(asset); - var url = markdownHref || localAssetOpenUrl(asset, false) || uploadedAssetUrl(asset); + var localOpenUrl = localAssetOpenUrl(asset, false); + var fallbackUrl = uploadedAssetUrl(asset); + var imageUrl = localOpenUrl || fallbackUrl || markdownHref; + var url = markdownHref || localOpenUrl || fallbackUrl; var type = uploadedAssetType(asset); var assetId = String(asset && asset.id || '').trim(); var sizeLabel = uploadedFileSize(asset); try { - if (type === 'image' && url) { - return editor.chain().focus().setImage({ src: url, alt: title, title: title }).run() === true; + if (type === 'image' && imageUrl) { + return editor.chain().focus().setImage({ src: imageUrl, alt: title, title: title }).run() === true; } var href = markdownHref || url; if (href) { @@ -2681,10 +2690,34 @@ import { createSidebarPageSettingsRuntime } from './sidebar-page-settings-runtim selectSidebarFileTreeRow(fileRow, { ctrlKey: e.ctrlKey, metaKey: e.metaKey, shiftKey: e.shiftKey }); var objectIdentity = readFileTreeObjectIdentity(fileRow); var workspacePath = readWorkspacePathFromRow(fileRow); + var localRelativePath = fileTreeRowLocalRelativePath(fileRow) || String(workspacePath && (workspacePath.relativePath || workspacePath.localRelativePath) || '').trim(); dispatchSidebarEvent('tree.filetree.open', { rowId: rowId, rowKind: rowKind, documentId: ownerDocumentId || null, assetId: assetId || null, assetType: assetType || null, objectIdentity: objectIdentity, workspacePath: workspacePath, workspaceId: resolveWorkspaceId(fileRow) }); if (e.shiftKey || e.ctrlKey || e.metaKey) { return; } + if ((rowKind === 'document' || rowKind === 'index' || rowKind === 'markdown') && isLocalOcrMarkdownPath(localRelativePath)) { + document.documentElement.setAttribute('data-mnote-local-ocr-filetree-open', 'resource-tab'); + var ocrResourceInput = { + path: localRelativePath, + title: fileTreeRowTitleForShortcut(fileRow, localRelativePath), + kind: 'markdown', + objectIdentity: 'local-ocr:' + localRelativePath, + assetId: 'local-ocr:' + localRelativePath, + documentId: documentId || ownerDocumentId || null, + workspaceId: resolveWorkspaceId(fileRow), + sourceKind: 'local_folder', + rootUri: currentRootUri(), + resourceKind: 'markdown', + workspacePath: workspacePath, + paneRole: 'primary' + }; + if (typeof window.__mnoteDocumentPaneRuntime?.openResourceInActiveTab === 'function') { + void window.__mnoteDocumentPaneRuntime.openResourceInActiveTab(ocrResourceInput); + } else { + void openLocalResourceInActiveTab(ocrResourceInput); + } + return; + } if ((rowKind === 'document' || rowKind === 'index' || rowKind === 'markdown') && documentId) { void recordNavigationRecent({ kind: 'page', @@ -2839,6 +2872,11 @@ import { createSidebarPageSettingsRuntime } from './sidebar-page-settings-runtim renderPageSettingsPopover(); return; } + var localOcrCheckbox = closestAction(event.target, '[data-local-ocr-option-checkbox="autoEnabled"]'); + if (localOcrCheckbox instanceof HTMLInputElement) { + void persistLocalOcrAutoPreference(localOcrCheckbox.checked); + return; + } var checkbox = closestAction(event.target, '[data-page-option-checkbox]'); if (checkbox instanceof HTMLInputElement) { var key = checkbox.getAttribute('data-page-option-checkbox') || ''; diff --git a/rust/crates/mnote-web/src/api_chat.rs b/rust/crates/mnote-web/src/api_chat.rs new file mode 100644 index 00000000..6be463f0 --- /dev/null +++ b/rust/crates/mnote-web/src/api_chat.rs @@ -0,0 +1,368 @@ +use crate::acp_bridge::SseEvent; +use serde_json::{json, Value}; +use std::env; + +const DEFAULT_API_CHAT_BASE_URL: &str = "http://127.0.0.1:20128/v1"; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ApiChatProfile { + pub profile_id: &'static str, + pub base_profile: &'static str, + pub isolated_profile: &'static str, + pub label: &'static str, + pub model: &'static str, + pub provider_kind: &'static str, + pub status: &'static str, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ResolvedApiChatProfile { + pub profile_id: String, + pub base_profile: String, + pub isolated_profile: String, + pub label: String, + pub model: String, + pub provider_kind: String, + pub status: String, + pub base_url: String, + pub api_key: Option, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ApiChatError { + pub code: &'static str, + pub message: String, +} + +impl ApiChatError { + pub fn new(code: &'static str, message: impl Into) -> Self { + Self { + code, + message: message.into(), + } + } +} + +impl std::fmt::Display for ApiChatError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}: {}", self.code, self.message) + } +} + +impl std::error::Error for ApiChatError {} + +pub const API_CHAT_PROFILES: &[ApiChatProfile] = &[ + ApiChatProfile { + profile_id: "shared_api_deepseek_flash_chat", + base_profile: "api-deepseek-flash-chat", + isolated_profile: "api-deepseek-flash-chat", + label: "DeepSeek Flash Chat", + model: "deepseek-v4-flash", + provider_kind: "api-chat", + status: "active", + }, + ApiChatProfile { + profile_id: "shared_api_deepseek_pro_chat", + base_profile: "api-deepseek-pro-chat", + isolated_profile: "api-deepseek-pro-chat", + label: "DeepSeek Pro Chat", + model: "deepseek-v4-pro", + provider_kind: "api-chat", + status: "active", + }, + ApiChatProfile { + profile_id: "shared_api_gpt_chat", + base_profile: "api-gpt-chat", + isolated_profile: "api-gpt-chat", + label: "GPT Chat", + model: "aisz-chat/gpt-5.5-extra-high-fast", + provider_kind: "api-chat", + status: "active", + }, + ApiChatProfile { + profile_id: "shared_api_kimi_chat", + base_profile: "api-kimi-chat", + isolated_profile: "api-kimi-chat", + label: "Kimi Chat", + model: "aisz-chat/kimi-k2.5", + provider_kind: "api-chat", + status: "active", + }, + ApiChatProfile { + profile_id: "shared_api_gemini_chat", + base_profile: "api-gemini-chat", + isolated_profile: "api-gemini-chat", + label: "Gemini API Chat", + model: "aisz-chat/gemini-3.1-pro", + provider_kind: "api-chat", + status: "active", + }, + ApiChatProfile { + profile_id: "shared_api_grok_chat", + base_profile: "api-grok-chat", + isolated_profile: "api-grok-chat", + label: "Grok API Chat", + model: "aisz-chat/grok-4.3", + provider_kind: "api-chat", + status: "active", + }, +]; + +pub fn api_chat_profiles() -> &'static [ApiChatProfile] { + API_CHAT_PROFILES +} + +pub fn api_chat_profile_by_id(value: &str) -> Option { + let needle = value.trim(); + if needle.is_empty() { + return None; + } + API_CHAT_PROFILES + .iter() + .copied() + .find(|profile| profile_matches(*profile, needle)) +} + +pub fn payload_uses_api_chat_profile(payload: &Value, registration_profile: &str) -> bool { + let agent_id = payload + .get("agentId") + .and_then(Value::as_str) + .map(str::trim) + .unwrap_or_default(); + if agent_id != "chat_only" { + return false; + } + let agent_profile = payload.get("agentProfileRef"); + [ + Some(registration_profile), + payload.get("profile").and_then(Value::as_str), + payload.get("profileId").and_then(Value::as_str), + payload.get("profile_id").and_then(Value::as_str), + agent_profile + .and_then(|value| value.get("baseProfile")) + .and_then(Value::as_str), + agent_profile + .and_then(|value| value.get("isolatedProfile")) + .and_then(Value::as_str), + agent_profile + .and_then(|value| value.get("profileId")) + .and_then(Value::as_str), + ] + .into_iter() + .flatten() + .any(|candidate| api_chat_profile_by_id(candidate).is_some()) +} + +pub fn resolve_api_chat_profile(value: &str) -> Result { + let profile = api_chat_profile_by_id(value).ok_or_else(|| { + ApiChatError::new( + "api_chat_profile_unknown", + format!("未知 API Chat profile: {value}"), + ) + })?; + let env_prefix = profile_env_prefix(profile.profile_id); + let model = env_value(&format!("{env_prefix}_MODEL")) + .or_else(|| env_value("MNOTE_API_CHAT_MODEL")) + .unwrap_or_else(|| profile.model.to_string()); + let base_url = env_value(&format!("{env_prefix}_BASE_URL")) + .or_else(|| env_value("MNOTE_API_CHAT_BASE_URL")) + .unwrap_or_else(|| DEFAULT_API_CHAT_BASE_URL.to_string()); + let api_key = env_value(&format!("{env_prefix}_API_KEY")) + .or_else(|| env_value("MNOTE_API_CHAT_API_KEY")) + .or_else(|| env_value("OPENAI_API_KEY")); + Ok(ResolvedApiChatProfile { + profile_id: profile.profile_id.to_string(), + base_profile: profile.base_profile.to_string(), + isolated_profile: profile.isolated_profile.to_string(), + label: profile.label.to_string(), + model, + provider_kind: profile.provider_kind.to_string(), + status: profile.status.to_string(), + base_url: base_url.trim().trim_end_matches('/').to_string(), + api_key, + }) +} + +pub fn runtime_events_from_openai_sse_chunk( + run_id: &str, + chunk: &str, +) -> Result, ApiChatError> { + let mut decoder = OpenAiSseDecoder::default(); + decoder.push_chunk(run_id, chunk) +} + +#[derive(Debug, Default)] +pub struct OpenAiSseDecoder { + buffer: String, + output: String, + completed: bool, +} + +impl OpenAiSseDecoder { + pub fn push_chunk(&mut self, run_id: &str, chunk: &str) -> Result, ApiChatError> { + self.buffer.push_str(chunk); + let mut frames = self + .buffer + .split("\n\n") + .map(str::to_string) + .collect::>(); + self.buffer = frames.pop().unwrap_or_default(); + let mut events = Vec::new(); + for frame in frames { + events.extend(self.parse_frame(run_id, &frame)?); + } + Ok(events) + } + + pub fn finish(&mut self, run_id: &str) -> Result, ApiChatError> { + let rest = std::mem::take(&mut self.buffer); + let mut events = if rest.trim().is_empty() { + Vec::new() + } else { + self.parse_frame(run_id, &rest)? + }; + if !self.completed { + self.completed = true; + events.push(self.completed_event()); + } + Ok(events) + } + + fn parse_frame(&mut self, run_id: &str, frame: &str) -> Result, ApiChatError> { + let mut events = Vec::new(); + for data in sse_frame_data_lines(frame) { + if data == "[DONE]" { + if !self.completed { + self.completed = true; + events.push(self.completed_event()); + } + continue; + } + let payload = serde_json::from_str::(&data).map_err(|error| { + ApiChatError::new( + "api_chat_stream_parse_error", + format!("OpenAI SSE chunk 解析失败: {error}"), + ) + })?; + if let Some(error) = payload.get("error") { + self.completed = true; + events.push(SseEvent { + event: "run.failed".into(), + data: json!({ + "runId": run_id, + "code": "api_chat_upstream_error", + "message": error + .get("message") + .and_then(Value::as_str) + .unwrap_or("API Chat upstream error"), + "error": error + }), + }); + continue; + } + for delta in extract_delta_texts(&payload) { + self.output.push_str(&delta); + events.push(SseEvent { + event: "message.delta".into(), + data: json!({ + "runId": run_id, + "delta": delta + }), + }); + } + } + Ok(events) + } + + fn completed_event(&self) -> SseEvent { + SseEvent { + event: "run.completed".into(), + data: json!({ + "output": self.output + }), + } + } +} + +fn profile_matches(profile: ApiChatProfile, value: &str) -> bool { + profile.profile_id == value + || profile.base_profile == value + || profile.isolated_profile == value + || profile.label == value +} + +fn profile_env_prefix(profile_id: &str) -> String { + let suffix = profile_id + .trim() + .strip_prefix("shared_api_") + .unwrap_or(profile_id) + .chars() + .map(|ch| { + if ch.is_ascii_alphanumeric() { + ch.to_ascii_uppercase() + } else { + '_' + } + }) + .collect::(); + format!("MNOTE_API_CHAT_{suffix}") +} + +fn env_value(key: &str) -> Option { + env::var(key) + .ok() + .map(|value| value.trim().to_string()) + .filter(|value| !value.is_empty()) +} + +fn sse_frame_data_lines(frame: &str) -> Vec { + let mut lines = Vec::new(); + for line in frame.lines() { + let trimmed = line.trim(); + if let Some(data) = trimmed.strip_prefix("data:") { + lines.push(data.trim().to_string()); + } + } + lines +} + +fn extract_delta_texts(payload: &Value) -> Vec { + let mut texts = Vec::new(); + if let Some(choices) = payload.get("choices").and_then(Value::as_array) { + for choice in choices { + for value in [ + choice + .get("delta") + .and_then(|delta| delta.get("content")) + .and_then(Value::as_str), + choice + .get("message") + .and_then(|message| message.get("content")) + .and_then(Value::as_str), + choice.get("text").and_then(Value::as_str), + ] + .into_iter() + .flatten() + { + if !value.is_empty() { + texts.push(value.to_string()); + } + } + } + } + if texts.is_empty() { + for value in [ + payload.get("delta").and_then(Value::as_str), + payload.get("text").and_then(Value::as_str), + payload.get("content").and_then(Value::as_str), + ] + .into_iter() + .flatten() + { + if !value.is_empty() { + texts.push(value.to_string()); + } + } + } + texts +} diff --git a/rust/crates/mnote-web/src/lib.rs b/rust/crates/mnote-web/src/lib.rs index d5fe40fe..7b4d0c2b 100644 --- a/rust/crates/mnote-web/src/lib.rs +++ b/rust/crates/mnote-web/src/lib.rs @@ -5,6 +5,7 @@ pub mod acp_client; pub mod acp_runtime; pub mod acp_session_manager; pub mod acp_types; +pub mod api_chat; pub mod app; pub mod context; pub mod document_buffer_store; diff --git a/rust/crates/mnote-web/src/routes/hermes_client.rs b/rust/crates/mnote-web/src/routes/hermes_client.rs index d404b76c..a7ec4dc4 100644 --- a/rust/crates/mnote-web/src/routes/hermes_client.rs +++ b/rust/crates/mnote-web/src/routes/hermes_client.rs @@ -13,7 +13,7 @@ use control_plane::{ AiAgentProfileAccessRecord, AiExternalConversationBindingRecord, AppendAiRuntimeEventInput, UpsertAiExternalConversationBindingInput, UpsertAiRuntimeRunInput, UpsertUserInput, }; -use futures_util::TryStreamExt; +use futures_util::{StreamExt, TryStreamExt}; use serde::Deserialize; use serde_json::{json, Value}; use std::collections::{BTreeMap, HashMap, HashSet, VecDeque}; @@ -63,6 +63,8 @@ static HERMES_RUN_QUEUE: LazyLock>> = LazyLock::new(|| Mutex::new(HashMap::new())); +static API_CHAT_RUN_PAYLOADS: LazyLock>> = + LazyLock::new(|| Mutex::new(HashMap::new())); static ACP_ACTIVE_RUNS: LazyLock>> = LazyLock::new(|| Mutex::new(HashMap::new())); static ACP_FINISHED_RUNS: LazyLock>> = @@ -739,6 +741,68 @@ pub async fn create_session( "actorId": session_store_user_id, "actorType": "user", }); + if let Ok(api_profile) = crate::api_chat::resolve_api_chat_profile(profile) { + let registration = HermesRunRegistration { + session_id: session_id.clone(), + profile: api_profile.isolated_profile.clone(), + document_id: document_id.to_string(), + trace_id: trace_id.clone(), + }; + let session_index_run_id = format!("{}_session", session_id); + let runtime_state = json!({ + "sessionId": session_id.clone(), + "runId": session_index_run_id.clone(), + "profile": api_profile.isolated_profile.clone(), + "documentId": document_id, + "traceId": trace_id.clone(), + "status": "session.created", + "transport": "api-chat", + "providerKind": api_profile.provider_kind.clone(), + "model": api_profile.model.clone(), + "baseUrl": api_profile.base_url.clone() + }); + let mut api_runtime_payload = runtime_payload.clone(); + api_runtime_payload["profile"] = Value::String(api_profile.isolated_profile.clone()); + api_runtime_payload["profileId"] = Value::String(api_profile.profile_id.clone()); + api_runtime_payload["providerKind"] = Value::String("api-chat".into()); + api_runtime_payload["apiChatProfile"] = json!({ + "profileId": api_profile.profile_id.clone(), + "baseProfile": api_profile.base_profile.clone(), + "isolatedProfile": api_profile.isolated_profile.clone(), + "label": api_profile.label.clone(), + "model": api_profile.model.clone(), + "baseUrl": api_profile.base_url.clone(), + "providerKind": api_profile.provider_kind.clone(), + "status": api_profile.status.clone() + }); + persist_acp_runtime_run( + &state, + &context, + ®istration, + &session_index_run_id, + "api-chat", + &runtime_state, + &api_runtime_payload, + ) + .await?; + return Ok(( + StatusCode::OK, + stamp_client_headers(), + Json(json!({ + "ok": true, + "sessionId": session_id, + "workspaceId": payload.workspace_id, + "documentId": payload.document_id, + "profile": api_profile.isolated_profile.clone(), + "profileId": api_profile.profile_id.clone(), + "providerKind": "api-chat", + "title": payload.title.unwrap_or_else(|| "当前页问答".into()), + "traceId": trace_id, + "persistence": ACP_RUNTIME_SQLITE_STORE, + "sessionStorage": "sqlite_control_plane" + })), + )); + } if let Some(acp_runtime) = acp_runtime_for_payload(&runtime_payload, profile) { let registration = HermesRunRegistration { session_id: session_id.clone(), @@ -808,6 +872,43 @@ pub async fn gateway_health( .or_else(active_profile_name) .unwrap_or_else(|| "default".into()); let profile_status = profile_gateway_status(&profile); + if let Ok(api_profile) = crate::api_chat::resolve_api_chat_profile(&profile) { + let api_key_configured = api_profile.api_key.is_some(); + return Ok(( + StatusCode::OK, + stamp_client_headers(), + Json(json!({ + "ok": api_key_configured, + "traceId": context.trace.trace_id, + "profile": { + "name": api_profile.isolated_profile, + "profileId": api_profile.profile_id, + "providerKind": "api-chat", + "modelDefault": api_profile.model, + "gateway": api_profile.base_url, + "modelConfigured": true, + "apiKeyConfigured": api_key_configured, + "suggestions": if api_key_configured { + Vec::::new() + } else { + vec!["请配置 MNOTE_API_CHAT_API_KEY 或 OPENAI_API_KEY。".to_string()] + } + }, + "gateway": { + "configured": true, + "upstream": api_profile.base_url, + "ok": api_key_configured, + "status": if api_key_configured { "api-chat" } else { "api_key_missing" }, + "transport": "api-chat" + }, + "suggestions": if api_key_configured { + Vec::::new() + } else { + vec!["请配置 MNOTE_API_CHAT_API_KEY 或 OPENAI_API_KEY。".to_string()] + } + })), + )); + } if is_acp_profile(&profile) { let runtime_name = crate::acp_bridge::runtime_name_for_profile(&profile); return Ok(( @@ -1400,7 +1501,15 @@ pub async fn delete_session( "sessionId": session_id, "deleted": deleted, "providerBindingLocalDeleted": provider_binding_local_deleted, - "providerConversationDelete": provider_conversation_delete + "providerConversationDelete": provider_conversation_delete, + "remoteDelete": if provider_binding.is_some() { + provider_conversation_delete + } else { + json!({ + "attempted": false, + "reason": "api_chat_has_no_remote_conversation" + }) + } } })), )); @@ -1657,6 +1766,8 @@ async fn get_acp_session( } let runtime = serde_json::from_str::(&latest_run.runtime_json) .unwrap_or_else(|_| runtime_state_for_session(session_id)); + let session_messages = + ai_runtime_session_messages_from_runs(state, context, &user_id, &runs)?; return Ok(( StatusCode::OK, stamp_client_headers(), @@ -1669,7 +1780,7 @@ async fn get_acp_session( "sessionId": session_id, "session": { "sessionId": session_id, - "messages": [], + "messages": session_messages, "runs": runs.iter().map(ai_runtime_run_to_json).collect::>() }, "runtime": runtime, @@ -1735,6 +1846,12 @@ async fn get_acp_session( .as_ref() .and_then(|run| serde_json::from_str::(&run.runtime_json).ok()) .unwrap_or_else(|| runtime_state_for_session(session_id)); + let session_messages = + ai_runtime_session_messages_from_runs(state, context, &user_id, &runs)?; + let session_provider_kind = latest_run + .as_ref() + .and_then(ai_runtime_run_provider_kind) + .unwrap_or(Value::Null); return Ok(( StatusCode::OK, stamp_client_headers(), @@ -1745,7 +1862,8 @@ async fn get_acp_session( "sessionId": session_id, "session": { "sessionId": session_id, - "messages": [], + "messages": session_messages, + "providerKind": session_provider_kind, "runs": runs.iter().map(ai_runtime_run_to_json).collect::>() }, "runtime": runtime, @@ -1872,6 +1990,70 @@ pub async fn create_run( &mut payload, )?; + if let Some(api_profile) = api_chat_profile_for_payload(&payload, ®istration.profile) { + let run_id = payload + .get("runId") + .and_then(Value::as_str) + .filter(|value| !value.trim().is_empty()) + .map(ToOwned::to_owned) + .unwrap_or_else(|| new_acp_run_id(®istration)); + payload["runId"] = Value::String(run_id.clone()); + payload["providerKind"] = Value::String("api-chat".into()); + payload["apiChatProfile"] = json!({ + "profileId": api_profile.profile_id.clone(), + "baseProfile": api_profile.base_profile.clone(), + "isolatedProfile": api_profile.isolated_profile.clone(), + "label": api_profile.label.clone(), + "model": api_profile.model.clone(), + "baseUrl": api_profile.base_url.clone(), + "providerKind": api_profile.provider_kind.clone(), + "status": api_profile.status.clone() + }); + let runtime_state = register_api_chat_runtime(®istration, &run_id, &api_profile); + API_CHAT_RUN_PAYLOADS + .lock() + .expect("api chat run payloads") + .insert(run_id.clone(), payload.clone()); + let persistence_result = persist_acp_runtime_run( + &state, + &context, + ®istration, + &run_id, + "api-chat", + &runtime_state, + &payload, + ) + .await?; + let response = json!({ + "ok": true, + "runId": run_id, + "sessionId": registration.session_id, + "profile": registration.profile, + "profileId": api_profile.profile_id.clone(), + "providerKind": "api-chat", + "model": api_profile.model.clone(), + "traceId": context.trace.trace_id, + "runtime": runtime_state, + "persistence": persistence_result + .get("persistence") + .and_then(Value::as_str) + .unwrap_or(ACP_RUNTIME_SQLITE_STORE), + "sessionStorage": persistence_result + .get("sessionStorage") + .cloned() + .unwrap_or(Value::Null), + "legacyPersistence": persistence_result + .get("legacyPersistence") + .cloned() + .unwrap_or(Value::Null), + "legacySessionStorage": persistence_result + .get("legacySessionStorage") + .cloned() + .unwrap_or(Value::Null), + }); + return Ok((StatusCode::OK, stamp_client_headers(), Json(response))); + } + // ACP path: skip the HTTP proxy, just register and return run info if let Some(acp_runtime) = acp_runtime_for_payload(&payload, ®istration.profile) { let run_id = payload @@ -2402,6 +2584,327 @@ async fn acp_stream_events( Ok(response) } +async fn api_chat_stream_events( + state: AppState, + context: RequestContext, + run_id: &str, +) -> Result { + let payload = API_CHAT_RUN_PAYLOADS + .lock() + .expect("api chat run payloads") + .get(run_id) + .cloned() + .ok_or_else(|| { + WebError::bad_gateway_code( + "api_chat_run_payload_not_found", + format!("API Chat run payload not found for run_id={run_id}"), + ) + .with_context(&context) + })?; + let registration = run_registration_from_payload(&context, &payload); + let api_profile = + api_chat_profile_for_payload(&payload, ®istration.profile).ok_or_else(|| { + WebError::bad_request_code( + "api_chat_profile_unknown", + format!("API Chat profile 未注册: {}", registration.profile), + ) + .with_context(&context) + })?; + + let Some(api_key) = api_profile.api_key.clone() else { + let event = crate::acp_bridge::SseEvent { + event: "run.failed".into(), + data: json!({ + "runId": run_id, + "code": "api_chat_api_key_missing", + "message": "API Chat 缺少 API key,请配置 MNOTE_API_CHAT_API_KEY 或 OPENAI_API_KEY。" + }), + }; + persist_acp_runtime_event( + &state, + &context, + ®istration, + run_id, + "api-chat", + &event.event, + &event.data, + &payload, + ) + .await?; + update_runtime_by_run_id(run_id, "failed", Some("run.failed"), None); + return acp_single_event_response(&context, event); + }; + + let messages = api_chat_messages_for_run(&state, &context, ®istration, run_id, &payload)?; + let request_body = json!({ + "model": api_profile.model.clone(), + "messages": messages, + "stream": true + }); + let url = format!( + "{}/chat/completions", + api_profile.base_url.trim_end_matches('/') + ); + let client = reqwest::Client::builder() + .timeout(Duration::from_secs(1800)) + .build() + .map_err(|error| { + WebError::internal(format!("API Chat client 构造失败: {error}")).with_context(&context) + })?; + let upstream_response = client + .post(url) + .bearer_auth(api_key) + .json(&request_body) + .send() + .await + .map_err(|error| { + WebError::bad_gateway_code( + "api_chat_upstream_unavailable", + format!("API Chat upstream 连接失败: {error}"), + ) + .with_context(&context) + })?; + if !upstream_response.status().is_success() { + let status = upstream_response.status(); + let text = upstream_response.text().await.unwrap_or_default(); + let event = crate::acp_bridge::SseEvent { + event: "run.failed".into(), + data: json!({ + "runId": run_id, + "code": "api_chat_upstream_error", + "message": format!("API Chat upstream 返回 {status}: {text}"), + "httpStatus": status.as_u16() + }), + }; + persist_acp_runtime_event( + &state, + &context, + ®istration, + run_id, + "api-chat", + &event.event, + &event.data, + &payload, + ) + .await?; + update_runtime_by_run_id(run_id, "failed", Some("run.failed"), None); + return acp_single_event_response(&context, event); + } + + use tokio::sync::mpsc; + let (tx, rx) = mpsc::channel::>(256); + let state_for_events = state.clone(); + let context_for_events = context.clone(); + let registration_for_events = registration.clone(); + let payload_for_events = payload.clone(); + let run_id_for_events = run_id.to_string(); + update_runtime_by_run_id(run_id, "running", Some("api_chat.events.connected"), None); + tokio::spawn(async move { + let mut decoder = crate::api_chat::OpenAiSseDecoder::default(); + let mut stream = upstream_response.bytes_stream(); + while let Some(item) = stream.next().await { + let events = match item { + Ok(chunk) => { + let text = String::from_utf8_lossy(&chunk); + decoder.push_chunk(&run_id_for_events, text.as_ref()) + } + Err(error) => Ok(vec![crate::acp_bridge::SseEvent { + event: "run.failed".into(), + data: json!({ + "runId": run_id_for_events, + "code": "api_chat_stream_read_error", + "message": format!("API Chat stream 读取失败: {error}") + }), + }]), + }; + let should_stop = forward_api_chat_events( + &state_for_events, + &context_for_events, + ®istration_for_events, + &run_id_for_events, + &payload_for_events, + events, + &tx, + ) + .await; + if should_stop { + break; + } + } + if !matches!( + runtime_status_for_run(&run_id_for_events).as_deref(), + Some("completed" | "failed" | "aborted") + ) { + let events = decoder.finish(&run_id_for_events); + let _ = forward_api_chat_events( + &state_for_events, + &context_for_events, + ®istration_for_events, + &run_id_for_events, + &payload_for_events, + events, + &tx, + ) + .await; + } + API_CHAT_RUN_PAYLOADS + .lock() + .expect("api chat run payloads") + .remove(&run_id_for_events); + }); + + let stream = tokio_stream::wrappers::ReceiverStream::new(rx); + let mut response = Response::builder() + .status(StatusCode::OK) + .header(header::CONTENT_TYPE, "text/event-stream; charset=utf-8") + .header(header::CACHE_CONTROL, "no-cache, no-transform") + .header("x-accel-buffering", "no") + .body(Body::from_stream(stream)) + .map_err(|e| { + WebError::internal(format!("API Chat events 响应构造失败: {e}")).with_context(&context) + })?; + stamp_client_headers_into(response.headers_mut()); + Ok(response) +} + +async fn forward_api_chat_events( + state: &AppState, + context: &RequestContext, + registration: &HermesRunRegistration, + run_id: &str, + payload: &Value, + events: Result, crate::api_chat::ApiChatError>, + tx: &tokio::sync::mpsc::Sender>, +) -> bool { + let events = match events { + Ok(events) => events, + Err(error) => vec![crate::acp_bridge::SseEvent { + event: "run.failed".into(), + data: json!({ + "runId": run_id, + "code": error.code, + "message": error.message + }), + }], + }; + let mut should_stop = false; + for event in events { + if let Err(error) = persist_acp_runtime_event( + state, + context, + registration, + run_id, + "api-chat", + &event.event, + &event.data, + payload, + ) + .await + { + warn!(error = ?error, run_id = %run_id, event = %event.event, "API Chat runtime event 持久化失败"); + } + let next_status = match event.event.as_str() { + "run.completed" => { + should_stop = true; + "completed" + } + "run.failed" => { + should_stop = true; + "failed" + } + _ => "running", + }; + update_runtime_by_run_id(run_id, next_status, Some(&event.event), None); + if tx.send(Ok(acp_sse_bytes(&event))).await.is_err() { + return true; + } + } + should_stop +} + +fn api_chat_messages_for_run( + state: &AppState, + context: &RequestContext, + registration: &HermesRunRegistration, + run_id: &str, + payload: &Value, +) -> Result, WebError> { + let mut messages = vec![json!({ + "role": "system", + "content": "你是 MNote 的简洁聊天助手。不要声称能编辑文件;如果需要文件内容,只能基于用户显式提供的上下文回答。" + })]; + let user_id = runtime_store_user_id(context, payload); + let workspace_id = payload + .get("workspaceId") + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned) + .or_else(|| context.workspace.workspace_id.clone()); + let runs = state + .control_plane() + .list_ai_runtime_runs( + &user_id, + workspace_id.as_deref(), + Some(®istration.document_id), + Some(®istration.session_id), + 40, + ) + .map_err(|error| { + WebError::internal(format!("SQLite API Chat 历史读取失败: {error}")) + .with_context(context) + })?; + for run in runs.iter().rev() { + if run.run_id == run_id || run.status == "session.created" { + continue; + } + if !crate::api_chat::api_chat_profile_by_id(&run.profile).is_some() { + continue; + } + if let Ok(run_payload) = serde_json::from_str::(&run.payload_json) { + if let Some(user_message) = run_payload + .get("message") + .or_else(|| run_payload.get("input")) + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + { + messages.push(json!({"role": "user", "content": user_message})); + } + } + let events = state + .control_plane() + .list_ai_runtime_events(&user_id, &run.run_id, 200) + .map_err(|error| { + WebError::internal(format!("SQLite API Chat 历史事件读取失败: {error}")) + .with_context(context) + })?; + let mut assistant = String::new(); + for event in events { + if event.event_type != "message.delta" { + continue; + } + if let Ok(payload) = serde_json::from_str::(&event.payload_json) { + if let Some(delta) = payload.get("delta").and_then(Value::as_str) { + assistant.push_str(delta); + } + } + } + if !assistant.trim().is_empty() { + messages.push(json!({"role": "assistant", "content": assistant})); + } + } + let current_message = payload + .get("message") + .or_else(|| payload.get("input")) + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or("请继续。"); + messages.push(json!({"role": "user", "content": current_message})); + Ok(messages) +} + pub async fn stream_events( State(state): State, Extension(context): Extension, @@ -2409,6 +2912,13 @@ pub async fn stream_events( ) -> Result { ensure_authenticated(&context)?; let profile = profile_for_run(&run_id).unwrap_or_else(|| "default".into()); + if API_CHAT_RUN_PAYLOADS + .lock() + .expect("api chat run payloads") + .contains_key(&run_id) + { + return api_chat_stream_events(state, context, &run_id).await; + } // ACP path: start AcpRunBridge and return SSE stream if let Some(acp_runtime_name) = acp_runtime_for_run(&run_id, &profile) { @@ -2939,6 +3449,14 @@ fn ensure_page_ai_actor_user( } fn agent_profile_access_json(access: AiAgentProfileAccessRecord) -> Value { + let provider_kind = if crate::api_chat::api_chat_profile_by_id(&access.profile.id).is_some() + || crate::api_chat::api_chat_profile_by_id(&access.profile.base_profile_name).is_some() + || crate::api_chat::api_chat_profile_by_id(&access.profile.isolated_profile_name).is_some() + { + "api-chat" + } else { + "" + }; json!({ "profileId": access.profile.id, "agentId": access.profile.agent_id, @@ -2946,6 +3464,7 @@ fn agent_profile_access_json(access: AiAgentProfileAccessRecord) -> Value { "ownerUserId": access.profile.owner_user_id, "baseProfile": access.profile.base_profile_name, "isolatedProfile": access.profile.isolated_profile_name, + "providerKind": provider_kind, "displayName": access.profile.display_name, "canRun": access.grant.can_run, "canManageSkills": access.grant.can_manage_skills, @@ -4193,6 +4712,9 @@ fn is_acp_profile(profile: &str) -> bool { } fn acp_runtime_for_payload(payload: &Value, profile: &str) -> Option { + if crate::api_chat::payload_uses_api_chat_profile(payload, profile) { + return None; + } if let Some(runtime) = payload .get("acpRuntime") .and_then(Value::as_str) @@ -4212,6 +4734,14 @@ fn acp_runtime_for_payload(payload: &Value, profile: &str) -> Option { } fn acp_runtime_for_run(run_id: &str, profile: &str) -> Option { + if API_CHAT_RUN_PAYLOADS + .lock() + .expect("api chat run payloads") + .contains_key(run_id) + || crate::api_chat::api_chat_profile_by_id(profile).is_some() + { + return None; + } if let Some(payload) = ACP_RUN_PAYLOADS .lock() .expect("acp run payloads") @@ -6790,6 +7320,68 @@ fn register_acp_runtime(registration: &HermesRunRegistration, run_id: &str) -> V json } +fn api_chat_profile_for_payload( + payload: &Value, + profile: &str, +) -> Option { + if !crate::api_chat::payload_uses_api_chat_profile(payload, profile) { + return None; + } + [ + Some(profile), + payload.get("profile").and_then(Value::as_str), + payload.get("profileId").and_then(Value::as_str), + payload.get("profile_id").and_then(Value::as_str), + payload + .get("agentProfileRef") + .and_then(|value| value.get("profileId")) + .and_then(Value::as_str), + payload + .get("agentProfileRef") + .and_then(|value| value.get("isolatedProfile")) + .and_then(Value::as_str), + payload + .get("agentProfileRef") + .and_then(|value| value.get("baseProfile")) + .and_then(Value::as_str), + ] + .into_iter() + .flatten() + .find_map(|candidate| crate::api_chat::resolve_api_chat_profile(candidate).ok()) +} + +fn register_api_chat_runtime( + registration: &HermesRunRegistration, + run_id: &str, + profile: &crate::api_chat::ResolvedApiChatProfile, +) -> Value { + let now = now_ms(); + let state = HermesRuntimeState { + session_id: registration.session_id.clone(), + run_id: run_id.to_string(), + profile: registration.profile.clone(), + document_id: registration.document_id.clone(), + trace_id: registration.trace_id.clone(), + status: "api_chat_pending".into(), + started_at: now, + last_event_at: now, + last_event: Some("run.started".into()), + last_tool_name: None, + last_tool_call_id: None, + last_audit_id: None, + }; + let mut json = runtime_state_to_json(&state); + json["transport"] = Value::String("api-chat".into()); + json["providerKind"] = Value::String(profile.provider_kind.clone()); + json["model"] = Value::String(profile.model.clone()); + json["baseUrl"] = Value::String(profile.base_url.clone()); + HERMES_RUNTIME_REGISTRY + .lock() + .expect("hermes runtime registry") + .insert(registration.session_id.clone(), state); + json +} + fn acp_runtime_run_store_args( context: &RequestContext, registration: &HermesRunRegistration, @@ -7416,6 +8008,8 @@ async fn delete_provider_conversation_for_session( fn ai_runtime_run_to_json(record: &control_plane::AiRuntimeRunRecord) -> Value { let runtime = serde_json::from_str::(&record.runtime_json).unwrap_or(Value::Null); let payload = serde_json::from_str::(&record.payload_json).unwrap_or(Value::Null); + let provider_kind = ai_runtime_value_provider_kind(&runtime) + .or_else(|| ai_runtime_value_provider_kind(&payload)); json!({ "sessionId": record.session_id, "runId": record.run_id, @@ -7426,6 +8020,7 @@ fn ai_runtime_run_to_json(record: &control_plane::AiRuntimeRunRecord) -> Value { "acpRuntime": record.acp_runtime, "traceId": record.trace_id, "status": record.status, + "providerKind": provider_kind.unwrap_or(Value::Null), "runtime": runtime, "payload": payload, "createdAt": record.created_at, @@ -7434,6 +8029,87 @@ fn ai_runtime_run_to_json(record: &control_plane::AiRuntimeRunRecord) -> Value { }) } +fn ai_runtime_run_provider_kind(record: &control_plane::AiRuntimeRunRecord) -> Option { + let runtime = serde_json::from_str::(&record.runtime_json).ok(); + let payload = serde_json::from_str::(&record.payload_json).ok(); + runtime + .as_ref() + .and_then(ai_runtime_value_provider_kind) + .or_else(|| payload.as_ref().and_then(ai_runtime_value_provider_kind)) +} + +fn ai_runtime_value_provider_kind(value: &Value) -> Option { + value + .get("providerKind") + .and_then(Value::as_str) + .map(str::trim) + .filter(|provider| !provider.is_empty()) + .map(|provider| Value::String(provider.to_string())) +} + +fn ai_runtime_session_messages_from_runs( + state: &AppState, + context: &RequestContext, + user_id: &str, + runs: &[control_plane::AiRuntimeRunRecord], +) -> Result, WebError> { + let mut messages = Vec::new(); + for run in runs.iter().rev() { + if run.status == "session.created" { + continue; + } + if let Ok(payload) = serde_json::from_str::(&run.payload_json) { + if let Some(user_message) = payload + .get("message") + .or_else(|| payload.get("input")) + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + { + messages.push(json!({"role": "user", "content": user_message})); + } + } + let events = state + .control_plane() + .list_ai_runtime_events(user_id, &run.run_id, 200) + .map_err(|error| { + WebError::internal(format!("SQLite runtime session 消息事件读取失败: {error}")) + .with_context(context) + })?; + let mut assistant = String::new(); + let mut completed_output = None; + for event in events { + if let Ok(payload) = serde_json::from_str::(&event.payload_json) { + match event.event_type.as_str() { + "message.delta" => { + if let Some(delta) = payload.get("delta").and_then(Value::as_str) { + assistant.push_str(delta); + } + } + "run.completed" => { + completed_output = payload + .get("output") + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned); + } + _ => {} + } + } + } + let assistant_content = if assistant.trim().is_empty() { + completed_output.unwrap_or_default() + } else { + assistant + }; + if !assistant_content.trim().is_empty() { + messages.push(json!({"role": "assistant", "content": assistant_content})); + } + } + Ok(messages) +} + fn ai_runtime_event_to_json(record: &control_plane::AiRuntimeEventRecord) -> Value { let payload = serde_json::from_str::(&record.payload_json).unwrap_or(Value::Null); json!({ @@ -9923,6 +10599,446 @@ mod tests { } } + #[test] + fn api_chat_profiles_do_not_match_web_provider_binding() { + let api_profile = crate::api_chat::api_chat_profile_by_id("shared_api_gpt_chat") + .expect("api chat profile"); + assert_eq!(api_profile.provider_kind, "api-chat"); + let headers = HeaderMap::from_iter([( + "x-mnote-actor-id".parse().expect("header name"), + "api_chat_user".parse().expect("header value"), + )]); + let context = RequestContext::from_http_parts( + &axum::http::Method::POST, + &"/api/hermes/client/runs".parse().expect("uri"), + &headers, + ); + let payload = json!({ + "agentId": "chat_only", + "profileId": "shared_api_gpt_chat", + "profile": api_profile.isolated_profile, + "agentProfileRef": { + "profileId": "shared_api_gpt_chat", + "baseProfile": api_profile.base_profile, + "isolatedProfile": api_profile.isolated_profile + }, + "sessionId": "api-chat-session", + "documentId": "doc-api-chat", + "traceId": "trace-api-chat" + }); + let registration = run_registration_from_payload(&context, &payload); + assert!(crate::api_chat::payload_uses_api_chat_profile( + &payload, + ®istration.profile + )); + assert!( + chatonly_provider_for_run(&payload, ®istration).is_none(), + "api-chat profiles must not enter web provider conversation binding/delete" + ); + } + + #[test] + fn api_chat_openai_sse_chunks_map_to_runtime_events() { + let events = crate::api_chat::runtime_events_from_openai_sse_chunk( + "run_api_chat_1", + "data: {\"choices\":[{\"delta\":{\"content\":\"你好\"}}]}\n\n\ + data: {\"choices\":[{\"delta\":{\"content\":\",MNote\"}}]}\n\n\ + data: [DONE]\n\n", + ) + .expect("parse openai sse"); + assert_eq!(events.len(), 3); + assert_eq!(events[0].event, "message.delta"); + assert_eq!(events[0].data["delta"], "你好"); + assert_eq!(events[1].event, "message.delta"); + assert_eq!(events[1].data["delta"], ",MNote"); + assert_eq!(events[2].event, "run.completed"); + assert_eq!(events[2].data["output"], "你好,MNote"); + } + + #[tokio::test] + async fn api_chat_create_run_registers_without_acp_payload() { + let _env_guard = env_lock().lock().expect("env lock"); + let _runtime_guard = runtime_lock().lock().expect("runtime lock"); + clear_runtime_registry(); + clear_acp_run_payloads(); + let response = app() + .oneshot( + Request::builder() + .method("POST") + .uri("/api/hermes/client/runs") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "api_chat_user") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "agentId": "chat_only", + "profileId": "shared_api_gpt_chat", + "profile": "shared_api_gpt_chat", + "sessionId": "mnote_api_chat_session", + "workspaceId": "ws_api_chat", + "documentId": "doc_api_chat", + "traceId": "trace_api_chat", + "message": "请只回复 API_CHAT_SMOKE" + }) + .to_string(), + )) + .expect("request"), + ) + .await + .expect("response"); + assert_eq!(response.status(), StatusCode::OK); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json"); + let run_id = payload["runId"].as_str().expect("run id"); + assert_eq!(payload["runtime"]["transport"], "api-chat"); + assert_eq!(payload["profile"], "api-gpt-chat"); + assert!( + !ACP_RUN_PAYLOADS + .lock() + .expect("acp run payloads") + .contains_key(run_id), + "api-chat run must not be stored as ACP/OpenClaw payload" + ); + } + + #[tokio::test] + async fn api_chat_create_session_persists_sqlite_without_acp_session() { + let _runtime_guard = runtime_lock().lock().expect("runtime lock"); + clear_runtime_registry(); + clear_acp_run_payloads(); + let response = app() + .oneshot( + Request::builder() + .method("POST") + .uri("/api/hermes/client/sessions") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "api_chat_user") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "workspaceId": "ws_api_chat", + "documentId": "doc_api_chat", + "traceId": "trace_api_chat_session", + "title": "API Chat", + "profile": "shared_api_gpt_chat" + }) + .to_string(), + )) + .expect("request"), + ) + .await + .expect("response"); + assert_eq!(response.status(), StatusCode::OK); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json"); + assert_eq!(payload["persistence"], ACP_RUNTIME_SQLITE_STORE); + assert_eq!(payload["sessionStorage"], "sqlite_control_plane"); + assert_eq!(payload["providerKind"], "api-chat"); + assert_eq!(payload["profile"], "api-gpt-chat"); + } + + #[tokio::test] + async fn api_chat_events_stream_from_openai_compatible_upstream() { + let _env_guard = env_lock().lock().expect("env lock"); + let _runtime_guard = runtime_lock().lock().expect("runtime lock"); + clear_runtime_registry(); + clear_acp_run_payloads(); + let captured_body = Arc::new(Mutex::new(Value::Null)); + let captured_for_route = Arc::clone(&captured_body); + let mock = axum::Router::new().route( + "/v1/chat/completions", + post(move |Json(body): Json| { + let captured = Arc::clone(&captured_for_route); + async move { + *captured.lock().expect("captured api body") = body; + ( + StatusCode::OK, + [("content-type", "text/event-stream; charset=utf-8")], + "data: {\"choices\":[{\"delta\":{\"content\":\"API\"}}]}\n\n\ + data: {\"choices\":[{\"delta\":{\"content\":\" Chat\"}}]}\n\n\ + data: [DONE]\n\n", + ) + } + }), + ); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("listener"); + let upstream = format!("http://{}", listener.local_addr().expect("addr")); + tokio::spawn(async move { + axum::serve(listener, mock).await.expect("mock api chat"); + }); + std::env::set_var( + "MNOTE_API_CHAT_GPT_CHAT_BASE_URL", + &format!("{upstream}/v1"), + ); + std::env::set_var("MNOTE_API_CHAT_GPT_CHAT_API_KEY", "test-api-chat-key"); + + let app = app(); + let run_response = app + .clone() + .oneshot( + Request::builder() + .method("POST") + .uri("/api/hermes/client/runs") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "api_chat_user") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "agentId": "chat_only", + "profileId": "shared_api_gpt_chat", + "profile": "shared_api_gpt_chat", + "sessionId": "mnote_api_chat_stream", + "workspaceId": "ws_api_chat", + "documentId": "doc_api_chat", + "traceId": "trace_api_chat_stream", + "message": "请只回复 API Chat" + }) + .to_string(), + )) + .expect("request"), + ) + .await + .expect("run response"); + assert_eq!(run_response.status(), StatusCode::OK); + let body = to_bytes(run_response.into_body(), usize::MAX) + .await + .expect("run body"); + let payload: Value = serde_json::from_slice(&body).expect("run json"); + let run_id = payload["runId"].as_str().expect("run id").to_string(); + + let events_response = app + .oneshot( + Request::builder() + .method("GET") + .uri(format!("/api/hermes/client/events/{run_id}")) + .header("accept", "text/event-stream") + .header("x-mnote-actor-id", "api_chat_user") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("events request"), + ) + .await + .expect("events response"); + assert_eq!(events_response.status(), StatusCode::OK); + let body = to_bytes(events_response.into_body(), usize::MAX) + .await + .expect("events body"); + let text = String::from_utf8(body.to_vec()).expect("events utf8"); + assert!(text.contains("event: message.delta"), "{text}"); + assert!(text.contains("\"delta\":\"API\""), "{text}"); + assert!(text.contains("\"delta\":\" Chat\""), "{text}"); + assert!(text.contains("event: run.completed"), "{text}"); + assert_eq!( + captured_body.lock().expect("captured api body")["model"], + "aisz-chat/gpt-5.5-extra-high-fast" + ); + + std::env::remove_var("MNOTE_API_CHAT_GPT_CHAT_BASE_URL"); + std::env::remove_var("MNOTE_API_CHAT_GPT_CHAT_API_KEY"); + } + + #[tokio::test] + async fn api_chat_session_detail_restores_messages_from_sqlite_events() { + let _env_guard = env_lock().lock().expect("env lock"); + let _runtime_guard = runtime_lock().lock().expect("runtime lock"); + clear_runtime_registry(); + clear_acp_run_payloads(); + let mock = axum::Router::new().route( + "/v1/chat/completions", + post(|| async { + ( + StatusCode::OK, + [("content-type", "text/event-stream; charset=utf-8")], + "data: {\"choices\":[{\"delta\":{\"content\":\"历史\"}}]}\n\n\ + data: {\"choices\":[{\"delta\":{\"content\":\"恢复\"}}]}\n\n\ + data: [DONE]\n\n", + ) + }), + ); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("listener"); + let upstream = format!("http://{}", listener.local_addr().expect("addr")); + tokio::spawn(async move { + axum::serve(listener, mock).await.expect("mock api chat"); + }); + std::env::set_var( + "MNOTE_API_CHAT_GPT_CHAT_BASE_URL", + &format!("{upstream}/v1"), + ); + std::env::set_var("MNOTE_API_CHAT_GPT_CHAT_API_KEY", "test-api-chat-key"); + + let app = app(); + let run_response = app + .clone() + .oneshot( + Request::builder() + .method("POST") + .uri("/api/hermes/client/runs") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "api_chat_history_user") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "agentId": "chat_only", + "profileId": "shared_api_gpt_chat", + "profile": "shared_api_gpt_chat", + "sessionId": "mnote_api_chat_history", + "workspaceId": "ws_api_chat_history", + "documentId": "doc_api_chat_history", + "traceId": "trace_api_chat_history", + "message": "请恢复历史" + }) + .to_string(), + )) + .expect("request"), + ) + .await + .expect("run response"); + assert_eq!(run_response.status(), StatusCode::OK); + let body = to_bytes(run_response.into_body(), usize::MAX) + .await + .expect("run body"); + let payload: Value = serde_json::from_slice(&body).expect("run json"); + let run_id = payload["runId"].as_str().expect("run id").to_string(); + let events_response = app + .clone() + .oneshot( + Request::builder() + .method("GET") + .uri(format!("/api/hermes/client/events/{run_id}")) + .header("accept", "text/event-stream") + .header("x-mnote-actor-id", "api_chat_history_user") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("events request"), + ) + .await + .expect("events response"); + assert_eq!(events_response.status(), StatusCode::OK); + let _ = to_bytes(events_response.into_body(), usize::MAX) + .await + .expect("events body"); + + let detail_response = app + .oneshot( + Request::builder() + .method("GET") + .uri("/api/hermes/client/sessions/mnote_api_chat_history?source=acp&workspaceId=ws_api_chat_history&documentId=doc_api_chat_history") + .header("x-mnote-actor-id", "api_chat_history_user") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("detail request"), + ) + .await + .expect("detail response"); + assert_eq!(detail_response.status(), StatusCode::OK); + let body = to_bytes(detail_response.into_body(), usize::MAX) + .await + .expect("detail body"); + let detail: Value = serde_json::from_slice(&body).expect("detail json"); + assert_eq!(detail["session"]["providerKind"], "api-chat"); + assert_eq!(detail["session"]["messages"][0]["role"], "user"); + assert_eq!(detail["session"]["messages"][0]["content"], "请恢复历史"); + assert_eq!(detail["session"]["messages"][1]["role"], "assistant"); + assert_eq!(detail["session"]["messages"][1]["content"], "历史恢复"); + + std::env::remove_var("MNOTE_API_CHAT_GPT_CHAT_BASE_URL"); + std::env::remove_var("MNOTE_API_CHAT_GPT_CHAT_API_KEY"); + } + + #[tokio::test] + async fn api_chat_delete_session_skips_remote_provider_delete() { + let _runtime_guard = runtime_lock().lock().expect("runtime lock"); + clear_runtime_registry(); + clear_acp_run_payloads(); + let app = app(); + let create_response = app + .clone() + .oneshot( + Request::builder() + .method("POST") + .uri("/api/hermes/client/sessions") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "api_chat_delete_user") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "workspaceId": "ws_api_chat_delete", + "documentId": "doc_api_chat_delete", + "traceId": "trace_api_chat_delete", + "title": "API Chat Delete", + "profile": "shared_api_gpt_chat" + }) + .to_string(), + )) + .expect("create request"), + ) + .await + .expect("create response"); + assert_eq!(create_response.status(), StatusCode::OK); + let body = to_bytes(create_response.into_body(), usize::MAX) + .await + .expect("create body"); + let create_payload: Value = serde_json::from_slice(&body).expect("create json"); + let session_id = create_payload["sessionId"].as_str().expect("session id"); + let delete_response = app + .clone() + .oneshot( + Request::builder() + .method("DELETE") + .uri(format!("/api/hermes/client/sessions/{session_id}?source=acp&workspaceId=ws_api_chat_delete&documentId=doc_api_chat_delete")) + .header("x-mnote-actor-id", "api_chat_delete_user") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("delete request"), + ) + .await + .expect("delete response"); + assert_eq!(delete_response.status(), StatusCode::OK); + let body = to_bytes(delete_response.into_body(), usize::MAX) + .await + .expect("delete body"); + let payload: Value = serde_json::from_slice(&body).expect("delete json"); + assert_eq!(payload["result"]["remoteDelete"]["attempted"], false); + assert_eq!( + payload["result"]["remoteDelete"]["reason"], + "api_chat_has_no_remote_conversation" + ); + + let list_response = app + .oneshot( + Request::builder() + .method("GET") + .uri("/api/hermes/client/sessions?source=acp&workspaceId=ws_api_chat_delete&documentId=doc_api_chat_delete") + .header("x-mnote-actor-id", "api_chat_delete_user") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("list request"), + ) + .await + .expect("list response"); + assert_eq!(list_response.status(), StatusCode::OK); + let body = to_bytes(list_response.into_body(), usize::MAX) + .await + .expect("list body"); + let list: Value = serde_json::from_slice(&body).expect("list json"); + assert!( + list["sessions"] + .as_array() + .expect("sessions") + .iter() + .all(|session| session["sessionId"] != session_id), + "{list}" + ); + } + #[tokio::test] async fn provider_conversation_bound_event_persists_doubao_binding() { let state = test_state(); diff --git a/rust/crates/mnote-web/src/routes/local_folder_source.rs b/rust/crates/mnote-web/src/routes/local_folder_source.rs index f0ab6f0e..b8a3891e 100644 --- a/rust/crates/mnote-web/src/routes/local_folder_source.rs +++ b/rust/crates/mnote-web/src/routes/local_folder_source.rs @@ -2838,7 +2838,11 @@ fn load_local_folder_page_tree_snapshot_for_scope( "PageTree scope parentRelativePath 必须指向目录", )); } - let watch_revision = local_folder_watch_revision_for_root(&canonical_root, &root_source_uri)?; + let watch_revision = if parent_relative_path.is_empty() { + local_folder_watch_revision_for_root(&canonical_root, &root_source_uri)? + } else { + local_folder_watch_revision_for_directory(&canonical_root, &scan_root, &root_source_uri)? + }; let cache_key = format!("{root_source_uri}\n{parent_relative_path}"); if let Ok(cache) = local_page_tree_snapshot_cache().lock() { if let Some(entry) = cache.get(&cache_key) { @@ -7046,7 +7050,45 @@ fn should_ignore_entry(relative_path: &str, file_name: &str) -> bool { if matches!(file_name, ".git" | "node_modules" | ".mnote") { return true; } - relative_path == ".mnote/trash" || relative_path.starts_with(".mnote/trash/") + relative_path == ".mnote/trash" + || relative_path.starts_with(".mnote/trash/") + || is_local_ocr_intermediate_entry(relative_path, file_name) +} + +fn is_local_ocr_intermediate_entry(relative_path: &str, file_name: &str) -> bool { + let normalized = relative_path.trim().trim_matches('/').replace('\\', "/"); + if normalized.is_empty() { + return false; + } + let segments = normalized + .split('/') + .filter(|segment| !segment.trim().is_empty()) + .collect::>(); + if segments.is_empty() { + return false; + } + let in_ocr_dir = segments + .len() + .checked_sub(2) + .and_then(|index| segments.get(index)) + .map(|segment| segment.ends_with(".ocr")) + .unwrap_or(false); + let in_ocr_images_dir = segments + .windows(2) + .any(|window| window[0].ends_with(".ocr") && window[1] == "images"); + if in_ocr_images_dir { + return true; + } + if !in_ocr_dir { + return false; + } + let lower = file_name.to_ascii_lowercase(); + lower == "layout.json" + || lower == "images" + || lower.ends_with("_content_list.json") + || lower.ends_with("_content_list_v2.json") + || lower.ends_with("_model.json") + || lower.ends_with("_origin.pdf") } fn compare_local_entries(a: &LocalFolderEntry, b: &LocalFolderEntry) -> Ordering { @@ -7723,6 +7765,14 @@ pub(crate) fn local_workspace_id(root: &Path) -> String { fn local_folder_watch_revision_for_root( root: &Path, root_source_uri: &str, +) -> Result { + local_folder_watch_revision_for_directory(root, root, root_source_uri) +} + +fn local_folder_watch_revision_for_directory( + root: &Path, + directory: &Path, + root_source_uri: &str, ) -> Result { let mut hasher = DefaultHasher::new(); let mut entry_count = 0usize; @@ -7763,7 +7813,7 @@ fn local_folder_watch_revision_for_root( visit( root, - root, + directory, &mut hasher, &mut entry_count, &mut latest_modified_ms, @@ -9282,9 +9332,10 @@ mod tests { get_share_links, get_user_access_policy, initialize_local_page_id, initialize_local_workspace_for_actor, load_local_folder_file_tree_children_snapshot, load_local_folder_file_tree_snapshot, load_local_folder_file_tree_snapshot_with_reveal, - load_local_folder_page_tree_snapshot, local_folder_watch_revision, - local_markdown_path_page_id, local_resource_write_editor_blocks, local_workspace_id, - open_local_file, read_local_resource, record_shared_cache, record_sync_pending_change, + load_local_folder_page_tree_scope_snapshot, load_local_folder_page_tree_snapshot, + local_folder_watch_revision, local_markdown_path_page_id, + local_resource_write_editor_blocks, local_workspace_id, open_local_file, + read_local_resource, record_shared_cache, record_sync_pending_change, resolve_local_markdown_page_aggregate, save_local_markdown_page, update_local_markdown_title, validate_local_access_root, write_local_markdown_asset, write_local_markdown_page_body, write_local_mindmap_data, write_local_resource, @@ -9424,6 +9475,30 @@ mod tests { let _ = std::fs::remove_dir_all(&root); } + #[test] + fn local_folder_page_tree_scope_cache_ignores_outside_changes() { + let root = temp_root("mnote-page-tree-scope-cache"); + init_workspace(&root); + std::fs::create_dir_all(root.join("design")).expect("create design"); + std::fs::create_dir_all(root.join("notes")).expect("create notes"); + std::fs::write(root.join("design").join("page.md"), "# Design\n").expect("write design"); + std::fs::write(root.join("notes").join("outside.md"), "# Outside\n") + .expect("write outside"); + let root_uri = format!("file://{}", root.display()); + super::reset_local_page_tree_snapshot_test_loads(); + + let first = + load_local_folder_page_tree_scope_snapshot(&root_uri, "design").expect("first scope"); + std::fs::write(root.join("notes").join("next.md"), "# Next\n").expect("write next"); + let second = + load_local_folder_page_tree_scope_snapshot(&root_uri, "design").expect("second scope"); + + assert_eq!(first.projection, second.projection); + assert_eq!(super::local_page_tree_snapshot_scan_test_loads(), 1); + + let _ = std::fs::remove_dir_all(&root); + } + #[test] fn local_markdown_identity_uses_path_even_when_frontmatter_has_mnote_id() { let root = temp_root("mnote-local-frontmatter-path-id"); @@ -13320,12 +13395,41 @@ fn main() {} init_workspace(&root); let root_uri = format!("file://{}", root.display()); std::fs::create_dir_all(root.join("docs").join("Page.ocr")).expect("ocr dir"); + std::fs::create_dir_all(root.join("docs").join("Page.ocr").join("images")) + .expect("ocr images dir"); std::fs::write(root.join("docs").join("Page.md"), "# Page\n").expect("page"); std::fs::write( root.join("docs").join("Page.ocr").join("photo.png.ocr.md"), "---\nmnote_ocr_version: 1\nprovider: mock\nmodel_version: vlm\nowner_document: ../Page.md\nsource_path: ./Page.assets/photo.png\nsource_root_relative_path: docs/Page.assets/photo.png\nsource_size: 3\nsource_mtime_ms: 1\nstatus: done\ncreated_at: 1\nupdated_at: 1\n---\n\nOCR-only-token\n", ) .expect("ocr markdown"); + std::fs::write(root.join("docs").join("Page.ocr").join("layout.json"), "{}") + .expect("ocr layout"); + std::fs::write( + root.join("docs") + .join("Page.ocr") + .join("abc_content_list.json"), + "[]", + ) + .expect("ocr content list"); + std::fs::write( + root.join("docs").join("Page.ocr").join("abc_model.json"), + "{}", + ) + .expect("ocr model"); + std::fs::write( + root.join("docs").join("Page.ocr").join("abc_origin.pdf"), + b"%PDF-1.4\n", + ) + .expect("ocr origin pdf"); + std::fs::write( + root.join("docs") + .join("Page.ocr") + .join("images") + .join("page_1.jpg"), + b"jpg", + ) + .expect("ocr image asset"); let file_tree = load_local_folder_file_tree_children_snapshot(&root_uri, "docs/Page.ocr") .expect("file tree"); @@ -13335,6 +13439,20 @@ fn main() {} assert!(file_items .iter() .any(|item| item["title"].as_str() == Some("photo.png.ocr.md"))); + for hidden_title in [ + "layout.json", + "abc_content_list.json", + "abc_model.json", + "abc_origin.pdf", + "images", + ] { + assert!( + !file_items + .iter() + .any(|item| item["title"].as_str() == Some(hidden_title)), + "OCR 中间产物不应出现在 FileTree: {hidden_title}" + ); + } let page_tree = load_local_folder_page_tree_snapshot(&root_uri).expect("page tree"); let page_items = page_tree.projection["items"] diff --git a/rust/crates/mnote-web/src/routes/local_ocr.rs b/rust/crates/mnote-web/src/routes/local_ocr.rs index e1980eb4..70e35651 100644 --- a/rust/crates/mnote-web/src/routes/local_ocr.rs +++ b/rust/crates/mnote-web/src/routes/local_ocr.rs @@ -64,6 +64,13 @@ pub(crate) struct OcrInsertRequest { mode: Option, } +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub(crate) struct OcrDeleteRequest { + root_uri: String, + source_root_relative_path: String, +} + #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] struct OcrIndex { @@ -108,6 +115,18 @@ struct MineruClientConfig { max_polls: usize, } +#[derive(Debug)] +struct MineruZipAsset { + relative_path: PathBuf, + bytes: Vec, +} + +#[derive(Debug)] +struct MineruZipExtraction { + markdown: String, + assets: Vec, +} + #[derive(Debug, Clone)] struct OcrSidecarPlan { owner_document_path: String, @@ -284,6 +303,47 @@ pub(crate) async fn status( Ok(ok_json(&context, json!({ "ok": true, "job": job }))) } +pub(crate) async fn delete_job( + State(state): State, + Extension(context): Extension, + Json(request): Json, +) -> Result<(StatusCode, HeaderMap, Json), WebError> { + let root = + ensure_local_workspace_write_access_with_state(&state, &context, request.root_uri.trim()) + .map_err(|error| error.with_context(&context))?; + let source = normalize_relative_path(&request.source_root_relative_path)?; + let mut index = read_ocr_index(&root)?; + let removed = index.entries.remove(&source); + if let Some(entry) = &removed { + let sidecar = root.join(&entry.ocr_root_relative_path); + ensure_target_under_root(&root, &sidecar, "local_ocr_delete_root_escape")?; + if sidecar.exists() { + fs::remove_file(&sidecar).map_err(|error| { + WebError::bad_request_code( + "local_ocr_delete_failed", + format!("无法删除 OCR Markdown {}: {error}", sidecar.display()), + ) + .with_context(&context) + })?; + } + cleanup_empty_ocr_sidecar_dir(&root, &sidecar)?; + } + write_ocr_index(&root, &index)?; + let key = format!("{}:{source}", request.root_uri.trim()); + if let Ok(mut jobs) = state.local_ocr_active_jobs.write() { + jobs.remove(&key); + } + broadcast_ocr_job_deleted(&state, request.root_uri.trim(), &source, removed.as_ref()); + Ok(ok_json( + &context, + json!({ + "ok": true, + "deleted": removed.is_some(), + "sourceRootRelativePath": source, + }), + )) +} + pub(crate) async fn read( State(state): State, Extension(context): Extension, @@ -519,7 +579,9 @@ async fn run_mineru_ocr( entry = advance_ocr_entry(&entry, "downloading", now_ms(), "", None); upsert_and_broadcast_ocr_index_entry(state, root, root_uri, entry)?; let zip_bytes = download_mineru_result_zip(&client, &zip_url).await?; - extract_mineru_markdown_from_zip(&zip_bytes) + let extraction = extract_mineru_markdown_and_assets_from_zip(&zip_bytes)?; + write_mineru_zip_assets(plan, &extraction.assets)?; + Ok(extraction.markdown) } #[derive(Debug)] @@ -704,7 +766,9 @@ async fn download_mineru_result_zip( }) } -fn extract_mineru_markdown_from_zip(bytes: &[u8]) -> Result { +fn extract_mineru_markdown_and_assets_from_zip( + bytes: &[u8], +) -> Result { let cursor = Cursor::new(bytes); let mut archive = zip::ZipArchive::new(cursor).map_err(|error| { WebError::bad_gateway_code( @@ -713,6 +777,7 @@ fn extract_mineru_markdown_from_zip(bytes: &[u8]) -> Result { ) })?; let mut candidates = Vec::<(String, String)>::new(); + let mut assets = Vec::::new(); for index in 0..archive.len() { let mut file = archive.by_index(index).map_err(|error| { WebError::bad_gateway_code( @@ -721,19 +786,36 @@ fn extract_mineru_markdown_from_zip(bytes: &[u8]) -> Result { ) })?; let name = file.name().replace('\\', "/"); - if !name.to_ascii_lowercase().ends_with(".md") || name.contains("/.") { + if file.is_dir() || name.contains("/.") { continue; } - let mut markdown = String::new(); - file.read_to_string(&mut markdown).map_err(|error| { + if name.to_ascii_lowercase().ends_with(".md") { + let mut markdown = String::new(); + file.read_to_string(&mut markdown).map_err(|error| { + WebError::bad_gateway_code( + "mineru_result_markdown_read_failed", + format!("MinerU Markdown 读取失败: {error}"), + ) + })?; + candidates.push((name, markdown)); + continue; + } + let Some(relative_path) = safe_mineru_asset_relative_path(&name) else { + continue; + }; + let mut asset_bytes = Vec::new(); + file.read_to_end(&mut asset_bytes).map_err(|error| { WebError::bad_gateway_code( - "mineru_result_markdown_read_failed", - format!("MinerU Markdown 读取失败: {error}"), + "mineru_result_asset_read_failed", + format!("MinerU 资源读取失败: {error}"), ) })?; - candidates.push((name, markdown)); + assets.push(MineruZipAsset { + relative_path, + bytes: asset_bytes, + }); } - candidates + let markdown = candidates .into_iter() .max_by_key(|(name, markdown)| { let preferred = @@ -747,7 +829,55 @@ fn extract_mineru_markdown_from_zip(bytes: &[u8]) -> Result { "mineru_result_markdown_missing", "MinerU 结果包中缺少 Markdown 文件", ) - }) + })?; + Ok(MineruZipExtraction { markdown, assets }) +} + +fn safe_mineru_asset_relative_path(name: &str) -> Option { + let normalized = name.trim().trim_start_matches('/').replace('\\', "/"); + if normalized.is_empty() { + return None; + } + let mut path = PathBuf::new(); + for component in Path::new(&normalized).components() { + match component { + Component::Normal(value) => { + let text = value.to_str()?.trim(); + if text.is_empty() || text == "." || text == ".." || text.starts_with('.') { + return None; + } + path.push(text); + } + _ => return None, + } + } + (!path.as_os_str().is_empty()).then_some(path) +} + +fn write_mineru_zip_assets( + plan: &OcrSidecarPlan, + assets: &[MineruZipAsset], +) -> Result<(), WebError> { + let sidecar_dir = plan.ocr_path.parent().unwrap_or_else(|| Path::new("")); + for asset in assets { + let target = sidecar_dir.join(&asset.relative_path); + ensure_target_under_root(sidecar_dir, &target, "local_ocr_asset_root_escape")?; + if let Some(parent) = target.parent() { + fs::create_dir_all(parent).map_err(|error| { + WebError::bad_request_code( + "local_ocr_asset_create_failed", + format!("无法创建 OCR 资源目录 {}: {error}", parent.display()), + ) + })?; + } + fs::write(&target, &asset.bytes).map_err(|error| { + WebError::bad_request_code( + "local_ocr_asset_write_failed", + format!("无法写入 OCR 资源 {}: {error}", target.display()), + ) + })?; + } + Ok(()) } fn parse_mineru_json(body: &str, code: &'static str) -> Result { @@ -760,7 +890,52 @@ fn find_upload_url(value: &Value) -> Option { if let Some(url) = find_json_string_by_keys(value, &["upload_url", "uploadUrl"]) { return Some(url); } - None + find_json_url_array_item_by_keys( + value, + &[ + "file_urls", + "fileUrls", + "file_url", + "fileUrl", + "urls", + "upload_urls", + "uploadUrls", + ], + ) +} + +fn find_json_url_array_item_by_keys(value: &Value, keys: &[&str]) -> Option { + match value { + Value::Object(map) => { + for key in keys { + if let Some(found) = map.get(*key).and_then(find_first_non_empty_json_string) { + return Some(found); + } + } + for nested in map.values() { + if let Some(found) = find_json_url_array_item_by_keys(nested, keys) { + return Some(found); + } + } + None + } + Value::Array(items) => items + .iter() + .find_map(|item| find_json_url_array_item_by_keys(item, keys)), + _ => None, + } +} + +fn find_first_non_empty_json_string(value: &Value) -> Option { + match value { + Value::String(text) => { + let trimmed = text.trim(); + (!trimmed.is_empty()).then(|| trimmed.to_string()) + } + Value::Array(items) => items.iter().find_map(find_first_non_empty_json_string), + Value::Object(map) => map.values().find_map(find_first_non_empty_json_string), + _ => None, + } } fn find_json_string_by_keys(value: &Value, keys: &[&str]) -> Option { @@ -1065,6 +1240,70 @@ fn upsert_and_broadcast_ocr_index_entry( Ok(()) } +fn cleanup_empty_ocr_sidecar_dir(root: &Path, sidecar: &Path) -> Result<(), WebError> { + let Some(parent) = sidecar.parent() else { + return Ok(()); + }; + ensure_target_under_root(root, parent, "local_ocr_delete_root_escape")?; + let Ok(entries) = fs::read_dir(parent) else { + return Ok(()); + }; + let has_other_sidecars = entries.filter_map(Result::ok).any(|entry| { + entry + .file_name() + .to_string_lossy() + .to_ascii_lowercase() + .ends_with(".ocr.md") + }); + if !has_other_sidecars { + fs::remove_dir_all(parent).map_err(|error| { + WebError::bad_request_code( + "local_ocr_delete_failed", + format!("无法删除 OCR sidecar 目录 {}: {error}", parent.display()), + ) + })?; + } + Ok(()) +} + +fn broadcast_ocr_job_deleted( + state: &AppState, + root_uri: &str, + source_root_relative_path: &str, + removed: Option<&OcrIndexEntry>, +) { + let now = now_ms(); + let job = json!({ + "jobId": removed.map(|entry| entry.job_id.as_str()).unwrap_or(""), + "ownerDocumentId": removed.map(|entry| entry.owner_document_id.as_str()).unwrap_or(""), + "ownerDocumentPath": removed.map(|entry| entry.owner_document_path.as_str()).unwrap_or(""), + "sourceRootRelativePath": source_root_relative_path, + "ocrRootRelativePath": removed.map(|entry| entry.ocr_root_relative_path.as_str()).unwrap_or(""), + "provider": removed.map(|entry| entry.provider.as_str()).unwrap_or(""), + "modelVersion": removed.map(|entry| entry.model_version.as_str()).unwrap_or(""), + "status": "deleted", + "stageLabel": "已删除", + "stale": false, + "updatedAtMs": now, + "finishedAtMs": now, + "plainTextPreview": "", + "error": null, + }); + let payload = json!({ + "schema": "mnote.local_ocr.job.updated.v1", + "kind": "local_ocr_job_updated", + "eventType": "local_ocr.job.updated", + "sourceKind": "local_folder", + "rootUri": root_uri, + "relativePath": source_root_relative_path, + "documentId": removed.map(|entry| entry.owner_document_id.as_str()).unwrap_or(""), + "revision": now.to_string(), + "job": job, + }); + let _ = state.local_ocr_job_tx.send(payload.clone()); + let _ = state.stream_delta_tx.send(payload); +} + fn broadcast_ocr_job_update(state: &AppState, root: &Path, root_uri: &str, entry: &OcrIndexEntry) { let job = ocr_job_payload(root, entry); let key = format!("{root_uri}:{}", entry.source_root_relative_path); @@ -1420,7 +1659,7 @@ mod tests { .collect() } - fn build_test_mineru_zip(markdown: &str) -> Vec { + fn build_test_mineru_zip_with_files(markdown: &str, files: &[(&str, &[u8])]) -> Vec { let mut bytes = Cursor::new(Vec::::new()); { let mut writer = zip::ZipWriter::new(&mut bytes); @@ -1428,6 +1667,12 @@ mod tests { .start_file("full.md", zip::write::SimpleFileOptions::default()) .expect("zip start file"); writer.write_all(markdown.as_bytes()).expect("zip markdown"); + for (name, content) in files { + writer + .start_file(*name, zip::write::SimpleFileOptions::default()) + .expect("zip asset start file"); + writer.write_all(content).expect("zip asset"); + } writer.finish().expect("zip finish"); } bytes.into_inner() @@ -1585,6 +1830,35 @@ mod tests { assert!(read_payload["markdown"] .as_str() .is_some_and(|markdown| markdown.contains("Route OCR Token"))); + let delete_response = app() + .oneshot( + Request::builder() + .method("DELETE") + .uri("/api/local-folder/ocr/jobs") + .header("content-type", "application/json") + .header("x-mnote-actor-id", "user_test") + .header("x-mnote-actor-type", "user") + .body(Body::from( + json!({ + "rootUri": root_uri, + "sourceRootRelativePath": "docs/Page.assets/photo.png" + }) + .to_string(), + )) + .expect("delete request"), + ) + .await + .expect("delete response"); + assert_eq!(delete_response.status(), StatusCode::OK); + assert!(!root + .join("docs") + .join("Page.ocr") + .join("photo.png.ocr.md") + .exists()); + assert!(read_ocr_index(&root) + .expect("index after delete") + .entries + .is_empty()); let _ = fs::remove_dir_all(root); } @@ -1690,7 +1964,10 @@ mod tests { let base_url = format!("http://{}", listener.local_addr().expect("mock addr")); let upload_count = Arc::new(AtomicUsize::new(0)); let poll_count = Arc::new(AtomicUsize::new(0)); - let zip_bytes = Arc::new(build_test_mineru_zip("# MinerU Result\n\n识别文本")); + let zip_bytes = Arc::new(build_test_mineru_zip_with_files( + "# MinerU Result\n\n![OCR 图](images/ocr.png)\n\n识别文本", + &[("images/ocr.png", b"png-bytes")], + )); let mock_mineru = axum::Router::new() .route( @@ -1699,8 +1976,12 @@ mod tests { let base_url = base_url.clone(); || async move { Json(json!({ - "batch_id": "batch_1", - "file_urls": [{ "upload_url": format!("{base_url}/upload/source") }] + "code": 0, + "msg": "ok", + "data": { + "batch_id": "batch_1", + "file_urls": [format!("{base_url}/upload/source")] + } })) } }), @@ -1831,7 +2112,18 @@ mod tests { fs::read_to_string(root.join("docs").join("Page.ocr").join("photo.png.ocr.md")) .expect("sidecar"); assert!(sidecar.contains("provider: mineru")); + assert!(sidecar.contains("![OCR 图](images/ocr.png)")); assert!(sidecar.contains("识别文本")); + assert_eq!( + fs::read( + root.join("docs") + .join("Page.ocr") + .join("images") + .join("ocr.png") + ) + .expect("sidecar image"), + b"png-bytes" + ); mock_handle.abort(); let _ = fs::remove_dir_all(root); diff --git a/rust/crates/mnote-web/src/routes/mod.rs b/rust/crates/mnote-web/src/routes/mod.rs index 3de1b8d2..a0655465 100644 --- a/rust/crates/mnote-web/src/routes/mod.rs +++ b/rust/crates/mnote-web/src/routes/mod.rs @@ -536,11 +536,14 @@ pub fn build_router(state: AppState) -> Router { ) .route( "/api/local-folder/ocr/jobs", - get(local_ocr::list_jobs).post(local_ocr::create_job), + get(local_ocr::list_jobs) + .post(local_ocr::create_job) + .delete(local_ocr::delete_job), ) .route("/api/local-folder/ocr/status", get(local_ocr::status)) .route("/api/local-folder/ocr/read", get(local_ocr::read)) .route("/api/local-folder/ocr/insert", post(local_ocr::insert)) + .route("/api/local-folder/ocr/delete", post(local_ocr::delete_job)) .route( "/api/local-folder/workspaces/default", post(local_folder_source::create_default_local_workspace), @@ -1015,7 +1018,8 @@ mod tests { "current_page": true, "folder": true }, - "ai.agent.hermes.profile_id": "mnoteai" + "ai.agent.hermes.profile_id": "mnoteai", + "localOcr.autoEnabled": true } }) .to_string(), @@ -1074,6 +1078,10 @@ mod tests { alice_payload["result"]["aiPreferences"]["ai.agent.hermes.profile_id"], "mnoteai" ); + assert_eq!( + alice_payload["result"]["localOcrPreferences"]["localOcr.autoEnabled"], + true + ); let mut bob_get = Request::builder() .uri(format!("/api/ui/preferences/effective?workspaceId=workspace-ai-a&sourceKind=local_folder&rootUri={root_uri}&documentId=local-md:ai.md")) @@ -1094,6 +1102,10 @@ mod tests { .as_object() .map(|value| value.is_empty()) .unwrap_or(false)); + assert_eq!( + bob_payload["result"]["localOcrPreferences"]["localOcr.autoEnabled"], + false + ); let _ = fs::remove_dir_all(&root); } diff --git a/rust/crates/mnote-web/src/routes/ui_preferences.rs b/rust/crates/mnote-web/src/routes/ui_preferences.rs index 777928e4..ce4bad4f 100644 --- a/rust/crates/mnote-web/src/routes/ui_preferences.rs +++ b/rust/crates/mnote-web/src/routes/ui_preferences.rs @@ -60,6 +60,7 @@ struct EffectivePagePreferences { page_options: PageOptions, page_width_preferences: BTreeMap, ai_preferences: BTreeMap, + local_ocr_preferences: BTreeMap, sources: BTreeMap, } @@ -246,10 +247,12 @@ fn resolve_effective_page_preferences( let mut sources = BTreeMap::new(); let mut page_width_preferences = default_page_width_preferences(); let mut ai_preferences = BTreeMap::new(); + let mut local_ocr_preferences = default_local_ocr_preferences(); apply_preference_records( &mut page_options, &mut page_width_preferences, &mut ai_preferences, + &mut local_ocr_preferences, &mut sources, &scope, &preferences, @@ -260,6 +263,7 @@ fn resolve_effective_page_preferences( page_options, page_width_preferences, ai_preferences, + local_ocr_preferences, sources, }) } @@ -268,6 +272,7 @@ fn apply_preference_records( page_options: &mut PageOptions, page_width_preferences: &mut BTreeMap, ai_preferences: &mut BTreeMap, + local_ocr_preferences: &mut BTreeMap, sources: &mut BTreeMap, scope: &PagePreferenceScope, preferences: &[UserUiPreferenceRecord], @@ -277,6 +282,7 @@ fn apply_preference_records( "source_family".to_string(), "workspace".to_string(), "document".to_string(), + "localOcr".to_string(), ]; for preference in preferences { if preference.scope_kind.starts_with("ai.") && !scope_kinds.contains(&preference.scope_kind) @@ -295,6 +301,7 @@ fn apply_preference_records( "workspace" => preference.scope_id.trim() == scope.workspace_id, "document" => preference.scope_id.trim() == scope.document_id, kind if kind.starts_with("ai.") => preference.scope_id.trim() == scope.workspace_id, + "localOcr" => preference.scope_id.trim() == scope.workspace_id, _ => false, }; if !scope_matches { @@ -311,6 +318,11 @@ fn apply_preference_records( sources.insert(preference.key.clone(), scope_kind.to_string()); continue; } + if preference.key.starts_with("localOcr.") { + local_ocr_preferences.insert(preference.key.clone(), value); + sources.insert(preference.key.clone(), scope_kind.to_string()); + continue; + } if let Some(content_type) = page_width_content_type_for_key(&preference.key) { if let Some(normalized) = normalize_page_width_preference(content_type, &value) { if let Some(preference_value) = page_width_preferences.get_mut(content_type) { @@ -368,6 +380,9 @@ fn preference_scope_for_key(key: &str, scope: &PagePreferenceScope) -> Option<(S return Some((format!("ai.agent.{agent}"), scope.workspace_id.clone())); } } + if trimmed.starts_with("localOcr.") { + return Some(("localOcr".to_string(), scope.workspace_id.clone())); + } if page_width_content_type_for_key(key).is_some() { return Some(("global".to_string(), "default".to_string())); } @@ -401,7 +416,11 @@ fn preference_scope_for_key(key: &str, scope: &PagePreferenceScope) -> Option<(S } fn preference_workspace_for_scope(workspace_id: &str, scope_kind: &str) -> Option { - if scope_kind == "workspace" || scope_kind == "document" || scope_kind.starts_with("ai.") { + if scope_kind == "workspace" + || scope_kind == "document" + || scope_kind.starts_with("ai.") + || scope_kind == "localOcr" + { Some(workspace_id.to_string()) } else { None @@ -409,7 +428,11 @@ fn preference_workspace_for_scope(workspace_id: &str, scope_kind: &str) -> Optio } fn preference_source_kind_for_scope(source_kind: &str, scope_kind: &str) -> Option { - if scope_kind == "workspace" || scope_kind == "document" || scope_kind.starts_with("ai.") { + if scope_kind == "workspace" + || scope_kind == "document" + || scope_kind.starts_with("ai.") + || scope_kind == "localOcr" + { Some(source_kind.to_string()) } else { None @@ -422,6 +445,8 @@ fn effective_preferences_payload(effective: EffectivePagePreferences) -> Value { serde_json::to_value(&effective.page_width_preferences).unwrap_or_else(|_| json!({})); let ai_preferences = serde_json::to_value(&effective.ai_preferences).unwrap_or_else(|_| json!({})); + let local_ocr_preferences = + serde_json::to_value(&effective.local_ocr_preferences).unwrap_or_else(|_| json!({})); let sources = effective .sources .into_iter() @@ -440,6 +465,7 @@ fn effective_preferences_payload(effective: EffectivePagePreferences) -> Value { "pageOptions": page_options, "pageWidthPreferences": page_width_preferences, "aiPreferences": ai_preferences, + "localOcrPreferences": local_ocr_preferences, "sources": Value::Object(sources), } }) @@ -594,6 +620,10 @@ fn default_page_width_preferences() -> BTreeMap BTreeMap { + BTreeMap::from([("localOcr.autoEnabled".to_string(), Value::Bool(false))]) +} + fn normalize_page_width_preference( content_type: &str, value: &Value, diff --git a/rust/crates/mnote-web/src/routes/web_shell.rs b/rust/crates/mnote-web/src/routes/web_shell.rs index 6dbffa49..bc7e5ed9 100644 --- a/rust/crates/mnote-web/src/routes/web_shell.rs +++ b/rust/crates/mnote-web/src/routes/web_shell.rs @@ -11,8 +11,8 @@ use crate::routes::gateway::default_workspace_name_for_context; use crate::routes::local_folder_source::{ ensure_local_workspace_read_access_with_state, is_local_access_policy_admin_context, load_local_folder_file_tree_children_snapshot, - load_local_folder_file_tree_snapshot_with_reveal, load_local_folder_page_tree_snapshot, - resolve_local_markdown_page_aggregate, + load_local_folder_file_tree_snapshot_with_reveal, load_local_folder_page_tree_scope_snapshot, + load_local_folder_page_tree_snapshot, resolve_local_markdown_page_aggregate, }; use crate::routes::query_support::execute_runtime_query_against_data; use crate::routes::snapshot_support::{ @@ -215,7 +215,8 @@ pub async fn document_page_shell( let (sidebar_tree_html, file_tree_html) = if is_local_folder { let root_uri = query.root_uri.as_deref().unwrap_or_default(); ( - render_local_sidebar_tree_html(root_uri, Some(&document_id)).unwrap_or_default(), + render_local_sidebar_tree_html_scoped(root_uri, Some(&document_id), file_tree_scope) + .unwrap_or_default(), render_local_file_tree_html_scoped(root_uri, Some(&document_id), None, file_tree_scope) .unwrap_or_default(), ) @@ -2659,7 +2660,22 @@ pub(crate) fn render_local_sidebar_tree_html( root_uri: &str, active_document_id: Option<&str>, ) -> Result { - let snapshot = load_local_folder_page_tree_snapshot(root_uri)?; + render_local_sidebar_tree_html_scoped(root_uri, active_document_id, None) +} + +pub(crate) fn render_local_sidebar_tree_html_scoped( + root_uri: &str, + active_document_id: Option<&str>, + file_tree_scope: Option<&str>, +) -> Result { + let snapshot = if let Some(scope) = file_tree_scope + .map(str::trim) + .filter(|value| !value.is_empty()) + { + load_local_folder_page_tree_scope_snapshot(root_uri, scope)? + } else { + load_local_folder_page_tree_snapshot(root_uri)? + }; Ok(render_local_sidebar_tree_html_from_snapshot( &snapshot, active_document_id, @@ -4277,6 +4293,51 @@ mod tests { )); } + #[tokio::test] + async fn document_shell_local_folder_filetree_scope_renders_scoped_page_tree() { + let root = std::env::temp_dir().join(format!( + "mnote-local-document-shell-scoped-{}", + std::process::id() + )); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join("design").join("done")).expect("create design done"); + std::fs::write(root.join("Home.md"), "# Home\n").expect("write home"); + std::fs::write( + root.join("design").join("done").join("Target.md"), + "# Target\n", + ) + .expect("write target"); + + let root_uri = format!("file://{}", root.display()); + init_local_workspace(&root, "user_test"); + let response = app() + .oneshot( + Request::builder() + .uri(format!( + "/documents/local-md:design~2Fdone~2FTarget.md?sourceKind=local_folder&rootUri={root_uri}&treeView=filetree&fileTreeScope=design" + )) + .header("x-mnote-actor-id", "user_test") + .header("x-mnote-actor-type", "user") + .body(Body::empty()) + .expect("request"), + ) + .await + .expect("response"); + + let _ = std::fs::remove_dir_all(&root); + + assert_eq!(response.status(), StatusCode::OK); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let html = String::from_utf8(body.to_vec()).expect("html"); + assert!(html.contains(r#"data-node-id="local-md:design~2Fdone~2FTarget.md""#)); + assert!( + !html.contains(r#"data-node-id="local-md:Home.md""#), + "文档页带 fileTreeScope 时 PageTree 不应回退到 workspace root 全量扫描" + ); + } + #[test] fn document_conflict_panel_runtime_contains_dom_helpers() { const DOCUMENT_CONFLICT_PANEL_RUNTIME_JS: &str = diff --git a/rust/crates/mnote-web/src/ssr/pages/layout.rs b/rust/crates/mnote-web/src/ssr/pages/layout.rs index a9d62d9f..c2bbbd31 100644 --- a/rust/crates/mnote-web/src/ssr/pages/layout.rs +++ b/rust/crates/mnote-web/src/ssr/pages/layout.rs @@ -188,6 +188,7 @@ pub fn PageLayout( + @@ -815,6 +816,14 @@ mod tests { assert!(LOCAL_UPLOAD_RUNTIME_JS.contains("uploadFileToMediaAsset: uploadFileToMediaAsset")); assert!(LOCAL_UPLOAD_RUNTIME_JS.contains("uploadFilesWithResolvedTarget")); assert!(LOCAL_UPLOAD_RUNTIME_JS.contains("__mnoteLastEditorUploadRoot")); + assert!( + LOCAL_UPLOAD_RUNTIME_JS.contains("var imageUrl = localOpenUrl || fallbackUrl || markdownHref;"), + "图片上传首屏必须使用本地 open URL 渲染,避免相对 href 在 /documents 下命中退役 Convex 兼容路径" + ); + assert!( + LOCAL_UPLOAD_RUNTIME_JS.contains("setImage({ src: imageUrl"), + "插入图片时不能直接把 ./asset.png 作为 img.src,否则刷新前会显示破损" + ); } #[test] diff --git a/rust/crates/mnote-web/src/ssr/styles.rs b/rust/crates/mnote-web/src/ssr/styles.rs index 6e764689..c957bd7e 100644 --- a/rust/crates/mnote-web/src/ssr/styles.rs +++ b/rust/crates/mnote-web/src/ssr/styles.rs @@ -182,6 +182,7 @@ a:hover { .material-symbols-outlined[data-icon="mode_comment"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M5 5h14v10H9l-4 4V5Z' fill='none' stroke='black' stroke-width='2' stroke-linejoin='round'/%3E%3C/svg%3E"); } .material-symbols-outlined[data-icon="person_add"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Ccircle cx='9' cy='8' r='3.2' fill='none' stroke='black' stroke-width='2'/%3E%3Cpath d='M3.5 20a5.5 5.5 0 0 1 11 0M18 8v6M15 11h6' fill='none' stroke='black' stroke-width='2' stroke-linecap='round'/%3E%3C/svg%3E"); } .material-symbols-outlined[data-icon="slideshow"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M4 5h16v12H4z' fill='none' stroke='black' stroke-width='2' stroke-linejoin='round'/%3E%3Cpath d='m10 9 5 2-5 2zM9 21h6' fill='none' stroke='black' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"); } +.material-symbols-outlined[data-icon="document_scanner"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M7 3H5a2 2 0 0 0-2 2v2M17 3h2a2 2 0 0 1 2 2v2M7 21H5a2 2 0 0 1-2-2v-2M17 21h2a2 2 0 0 0 2-2v-2M7 8h10M7 12h10M7 16h7' fill='none' stroke='black' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"); } .material-symbols-outlined[data-icon="article"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M6 4h12v16H6zM9 8h6M9 12h6M9 16h4' fill='none' stroke='black' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"); } .material-symbols-outlined[data-icon="sync"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M20 6v5h-5M4 18v-5h5M7.5 9A6 6 0 0 1 18 6M16.5 15A6 6 0 0 1 6 18' fill='none' stroke='black' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"); } .material-symbols-outlined[data-icon="drive_file_rename_outline"] { --mnote-icon-mask: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='m4 17-.5 3.5L7 20l11-11-3-3zM13 8l3 3M5 6h7' fill='none' stroke='black' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"); } @@ -2780,28 +2781,36 @@ body { .mnote-local-ocr-task-dock { position: fixed; - right: 18px; - bottom: 18px; - z-index: 80; + right: 16px; + top: 48px; + z-index: 90; color: #37352f; font-size: 13px; + pointer-events: none; } -.mnote-local-ocr-task-button { - height: 32px; - border: 1px solid rgba(55, 53, 47, 0.16); - border-radius: 4px; - padding: 0 11px; - background: #FFF; - color: #37352f; - box-shadow: 0 8px 24px rgba(15, 23, 42, 0.12); - cursor: pointer; +.mnote-local-ocr-task-toggle { + position: relative; +} + +.mnote-local-ocr-task-badge { + position: absolute; + top: 2px; + right: 2px; + min-width: 14px; + height: 14px; + padding: 0 3px; + border-radius: 999px; + background: #d1453b; + color: #fff; + font-size: 10px; + line-height: 14px; + text-align: center; + box-sizing: border-box; + pointer-events: none; } .mnote-local-ocr-task-drawer { - position: absolute; - right: 0; - bottom: 40px; width: min(360px, calc(100vw - 36px)); max-height: min(420px, calc(100vh - 120px)); overflow: auto; @@ -2809,13 +2818,33 @@ body { border-radius: 6px; background: #FFF; box-shadow: 0 14px 36px rgba(15, 23, 42, 0.16); + pointer-events: auto; } .mnote-local-ocr-task-head { + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; padding: 10px 12px; border-bottom: 1px solid rgba(55, 53, 47, 0.1); } +.mnote-local-ocr-task-close { + width: 24px; + height: 24px; + border: 0; + border-radius: 4px; + background: transparent; + color: #787774; + cursor: pointer; +} + +.mnote-local-ocr-task-close:hover { + background: rgba(55, 53, 47, 0.08); + color: #37352f; +} + .mnote-local-ocr-task-row { display: flex; align-items: center; @@ -2861,6 +2890,10 @@ body { cursor: pointer; } +.mnote-local-ocr-task-actions button[data-mnote-local-ocr-task-delete] { + color: #b3261e; +} + .mnote-resource-tab-frame, .mnote-resource-tab-image, .mnote-resource-tab-text-shell { diff --git a/scripts/task522-page-aggregate-compat-fallback-contract.js b/scripts/task522-page-aggregate-compat-fallback-contract.js index e135835c..6998054a 100644 --- a/scripts/task522-page-aggregate-compat-fallback-contract.js +++ b/scripts/task522-page-aggregate-compat-fallback-contract.js @@ -28,6 +28,10 @@ function paragraphText(doc) { return doc?.content?.[0]?.content?.[0]?.text || ""; } +function firstTableCellText(doc) { + return doc?.content?.[0]?.content?.[0]?.content?.[0]?.content?.[0]?.content?.[0]?.text || ""; +} + const { pageBodyTiptapDocumentSource, pageBodyTiptapDocument, @@ -62,6 +66,39 @@ assert.equal( "Block truth", ); +const localWithProjectedTable = { + projectionSource: "local_markdown.content", + blockDocument: { + documentId: "local-md:design~2F07-ai~2Fdone~2F7-45-chatonly-api-provider-runtime-v1.md", + rootBlockIds: ["table-1"], + blocks: [{ + blockId: "table-1", + type: "table", + attrs: { + tiptapTable: { + type: "table", + content: [{ + type: "tableRow", + content: [{ + type: "tableCell", + attrs: { colspan: 1, rowspan: 1, colwidth: null }, + content: [{ + type: "paragraph", + content: [{ type: "text", text: "Provider 类别" }], + }], + }], + }], + }, + }, + contentNodes: [], + }], + }, +}; +assert.equal( + firstTableCellText(pageBodyTiptapDocument(localWithProjectedTable)), + "Provider 类别", +); + const localLegacyOnly = { projectionSource: "local_markdown.content", content: legacyContent, diff --git a/scripts/task526-local-folder-ocr-api-smoke.js b/scripts/task526-local-folder-ocr-api-smoke.js index 1f2b8474..81fc6101 100644 --- a/scripts/task526-local-folder-ocr-api-smoke.js +++ b/scripts/task526-local-folder-ocr-api-smoke.js @@ -80,11 +80,17 @@ async function main() { const relativePath = "docs/Page.md"; const documentId = localMdDocumentId(relativePath); const sourceRootRelativePath = "docs/Page.assets/photo.png"; + const failedSourceRootRelativePath = "docs/Page.assets/photo-failed.png"; const ocrToken = "TASK526_OCR_TOKEN"; writeWorkspaceManifest(root, actorId); fs.mkdirSync(path.join(root, "docs", "Page.assets"), { recursive: true }); - fs.writeFileSync(path.join(root, relativePath), "# OCR Page\n\n![photo](./Page.assets/photo.png)\n", "utf8"); - fs.writeFileSync(path.join(root, sourceRootRelativePath), Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a])); + fs.writeFileSync(path.join(root, relativePath), "# OCR Page\n\n![photo](<./Page.assets/photo.png>)\n", "utf8"); + const tinyPng = Buffer.from( + "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAFgwJ/l8FQOQAAAABJRU5ErkJggg==", + "base64", + ); + fs.writeFileSync(path.join(root, sourceRootRelativePath), tinyPng); + fs.writeFileSync(path.join(root, failedSourceRootRelativePath), tinyPng); const browser = await chromium.launch({ headless: true }); const context = await browser.newContext(); @@ -93,6 +99,15 @@ async function main() { try { await ensureAuthenticated(page, context.request); await ensureDocumentVisible(page, root, relativePath); + await page.waitForFunction(() => { + const image = document.querySelector('.document-pane[data-pane-role="primary"] .editor-surface .ProseMirror img'); + return image instanceof HTMLImageElement + && image.complete + && image.naturalWidth > 0 + && image.src.includes("Page.assets%2Fphoto.png") + && !image.src.includes("%3C") + && !image.src.includes("%3E"); + }, null, { timeout: UI_TIMEOUT_MS }); screenshots.page = path.join(OUTPUT_DIR, "01-page.png"); await page.screenshot({ path: screenshots.page, fullPage: true }); @@ -126,17 +141,23 @@ async function main() { sourceRootRelativePath, workspaceId, }); - await page.locator('[data-testid="mnote-local-ocr-toolbar"]').first().waitFor({ - state: "visible", - timeout: UI_TIMEOUT_MS, - }); - await page.getByTestId("mnote-local-ocr-run").click({ timeout: UI_TIMEOUT_MS }); - await page.waitForFunction(() => document.querySelector('[data-mnote-local-ocr-status="done"]'), null, { + await page.locator('[data-mnote-resource-tab-panel]:not([hidden]) [data-testid="mnote-local-ocr-toolbar"]').first().waitFor({ + state: "detached", timeout: UI_TIMEOUT_MS, }); + const watchBatchBeforeOcr = await page.evaluate(() => document.documentElement.getAttribute("data-mnote-local-folder-watch-batch-applied") || ""); await page.getByTestId("mnote-local-ocr-task-toggle").waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); await page.getByTestId("mnote-local-ocr-task-toggle").click({ timeout: UI_TIMEOUT_MS }); - await page.getByTestId("mnote-local-ocr-task-drawer").waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + await page.getByTestId("mnote-local-ocr-task-drawer").waitFor({ state: "hidden", timeout: UI_TIMEOUT_MS }); + const topbarOcrButtonInfo = await page.getByTestId("mnote-local-ocr-task-toggle").evaluate((node) => { + const topbar = node.closest(".wolai-topbar-actions"); + return { + inTopbar: Boolean(topbar), + text: node.textContent || "", + badge: node.querySelector("[data-mnote-local-ocr-task-count]")?.textContent || "", + }; + }); + assert.equal(topbarOcrButtonInfo.inTopbar, true, `OCR 任务入口应位于右上角 topbar: ${JSON.stringify(topbarOcrButtonInfo)}`); await page.waitForFunction( (sourcePath) => { const row = document.querySelector(`[data-mnote-local-ocr-task-row="${CSS.escape(sourcePath)}"]`); @@ -145,111 +166,100 @@ async function main() { sourceRootRelativePath, { timeout: UI_TIMEOUT_MS }, ); - const uiOcrPath = await page.locator('[data-mnote-local-ocr-path]').first().getAttribute("data-mnote-local-ocr-path"); + await page.getByTestId("mnote-local-ocr-task-toggle").click({ timeout: UI_TIMEOUT_MS }); + await page.getByTestId("mnote-local-ocr-task-drawer").waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + const uiOcrPath = await page.evaluate(async ({ rootUri, sourceRootRelativePath }) => { + const url = new URL("/api/local-folder/ocr/status", window.location.origin); + url.searchParams.set("rootUri", rootUri); + url.searchParams.set("sourceRootRelativePath", sourceRootRelativePath); + const response = await fetch(url.toString(), { cache: "no-store", headers: { accept: "application/json" } }); + const payload = await response.json().catch(() => null); + return payload?.job?.ocrRootRelativePath || ""; + }, { + rootUri: fileUrl(root), + sourceRootRelativePath, + }); assert(uiOcrPath && uiOcrPath.endsWith(".ocr.md"), `UI OCR path invalid: ${uiOcrPath}`); + await page.waitForFunction( + ({ before, ocrPath }) => { + const root = document.documentElement; + const marker = root.getAttribute("data-mnote-local-ocr-filetree-refresh") || ""; + const applied = root.getAttribute("data-mnote-local-folder-watch-batch-applied") || ""; + return marker === ocrPath && applied && applied !== before; + }, + { before: watchBatchBeforeOcr, ocrPath: uiOcrPath }, + { timeout: UI_TIMEOUT_MS }, + ); screenshots.ocrToolbar = path.join(OUTPUT_DIR, "02-ocr-toolbar.png"); await page.screenshot({ path: screenshots.ocrToolbar, fullPage: true }); - await page.getByTestId("mnote-local-ocr-insert").click({ timeout: UI_TIMEOUT_MS }); - await page.getByTestId("mnote-local-ocr-status").filter({ hasText: "OCR 链接已插入正文" }).waitFor({ - state: "visible", - timeout: UI_TIMEOUT_MS, - }); - await page.getByTestId("mnote-local-ocr-open").click({ timeout: UI_TIMEOUT_MS }); - await page.locator('[data-mnote-resource-tab-panel][data-resource-kind="markdown"]:not([hidden]) [data-testid="mnote-leptos-tiptap-island-editor-root"]').first().waitFor({ - state: "visible", - timeout: UI_TIMEOUT_MS, - }); - const result = await page.evaluate(async ({ rootUri, documentId, sourceRootRelativePath, workspaceId, ocrToken }) => { - async function jsonFetch(pathname, init = {}) { - const response = await fetch(pathname, { - ...init, - headers: { - "content-type": "application/json", - ...(init.headers || {}), - }, - }); - const payload = await response.json().catch(() => null); - return { status: response.status, payload }; - } - const create = await jsonFetch("/api/local-folder/ocr/jobs", { - method: "POST", - body: JSON.stringify({ - rootUri, - documentId, - sourceRootRelativePath, - provider: "mock", - mockMarkdown: `# OCR Result\n\n${ocrToken} browser smoke text`, - }), + const failedOcrRoute = async (route) => { + if (route.request().method() !== "POST") return route.fallback(); + const body = route.request().postDataJSON(); + if (body?.sourceRootRelativePath !== failedSourceRootRelativePath) return route.fallback(); + return route.fulfill({ + status: 401, + contentType: "application/json", + body: JSON.stringify({ ok: false, error: { message: "local_ocr_job_failed_401" } }), }); - const ocrRootRelativePath = create.payload?.job?.ocrRootRelativePath || ""; - const read = await jsonFetch(`/api/local-folder/ocr/read?rootUri=${encodeURIComponent(rootUri)}&ocrRootRelativePath=${encodeURIComponent(ocrRootRelativePath)}`, { - method: "GET", - headers: {}, + }; + await page.route("**/api/local-folder/ocr/jobs", failedOcrRoute); + await page.evaluate(async ({ rootUri, documentId, sourceRootRelativePath, workspaceId }) => { + window.__MNOTE_LOCAL_OCR_PROVIDER = "mineru"; + const title = sourceRootRelativePath.split("/").filter(Boolean).pop() || "photo-failed.png"; + const fileUrl = new URL("/api/local-folder/files/open", window.location.origin); + fileUrl.searchParams.set("rootUri", rootUri); + fileUrl.searchParams.set("path", sourceRootRelativePath); + const opened = await window.__mnoteDocumentPaneRuntime.openResourceInActiveTab({ + objectIdentity: `local-file:${sourceRootRelativePath}`, + assetId: `local-file:${sourceRootRelativePath}`, + title, + fileName: title, + kind: "image", + rootUri, + path: sourceRootRelativePath, + href: fileUrl.toString(), + documentId, + ownerDocumentId: documentId, + workspaceId, + sourceKind: "local_folder", }); - const status = await jsonFetch(`/api/local-folder/ocr/status?rootUri=${encodeURIComponent(rootUri)}&sourceRootRelativePath=${encodeURIComponent(sourceRootRelativePath)}`, { - method: "GET", - headers: {}, - }); - const jobs = await jsonFetch(`/api/local-folder/ocr/jobs?rootUri=${encodeURIComponent(rootUri)}`, { - method: "GET", - headers: {}, - }); - const withoutOcr = await jsonFetch("/api/search/documents", { - method: "POST", - body: JSON.stringify({ - workspaceId, - sourceKind: "local_folder", - rootUri, - query: ocrToken, - limit: 5, - filters: { includeOcr: false }, - }), - }); - const withOcr = await jsonFetch("/api/search/documents", { - method: "POST", - body: JSON.stringify({ - workspaceId, - sourceKind: "local_folder", - rootUri, - query: ocrToken, - limit: 5, - filters: { includeOcr: true }, - }), - }); - const insert = await jsonFetch("/api/local-folder/ocr/insert", { - method: "POST", - body: JSON.stringify({ - rootUri, - documentId, - ocrRootRelativePath, - mode: "link", - }), - }); - return { create, read, status, jobs, withoutOcr, withOcr, insert, ocrRootRelativePath }; + if (!opened) throw new Error("OCR failed source image resource tab did not open"); }, { rootUri: fileUrl(root), documentId, - sourceRootRelativePath, + sourceRootRelativePath: failedSourceRootRelativePath, workspaceId, - ocrToken, }); - - assert.equal(result.create.status, 200, `OCR create failed: ${JSON.stringify(result.create)}`); - assert.equal(result.create.payload.job.status, "done", `OCR job should be done: ${JSON.stringify(result.create.payload)}`); - assert(result.ocrRootRelativePath.endsWith(".ocr.md"), `OCR path invalid: ${result.ocrRootRelativePath}`); - assert.equal(result.read.status, 200, `OCR read failed: ${JSON.stringify(result.read)}`); - assert(result.read.payload.markdown.includes(ocrToken), "OCR read should include mock OCR text"); - assert.equal(result.status.payload.job.ocrRootRelativePath, result.ocrRootRelativePath, "status should return OCR sidecar path"); - assert(result.jobs.payload.jobs.some((job) => job.ocrRootRelativePath === result.ocrRootRelativePath), "jobs list should include OCR job"); - assert.equal(result.withoutOcr.payload.results.length, 0, `includeOcr=false should not match OCR text: ${JSON.stringify(result.withoutOcr.payload)}`); - const ocrResult = result.withOcr.payload.results.find((item) => item.hasOcr === true); - assert(ocrResult, `includeOcr=true should return owner page OCR result: ${JSON.stringify(result.withOcr.payload)}`); - assert.equal(ocrResult.documentId, documentId, "OCR search result should point to owner document"); - assert.equal(ocrResult.ocrEvidence.ocrRootRelativePath, result.ocrRootRelativePath, "OCR evidence should include sidecar path"); - assert.equal(result.insert.status, 200, `OCR insert failed: ${JSON.stringify(result.insert)}`); - const ownerMarkdown = fs.readFileSync(path.join(root, relativePath), "utf8"); - assert(ownerMarkdown.includes(`[OCR:photo.png](`), `owner markdown should include explicit OCR link:\n${ownerMarkdown}`); + const failedImageTab = page.locator('.mnote-main-tab[data-mnote-tab-kind="image"]', { hasText: "photo-failed.png" }).first(); + await failedImageTab.waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + await failedImageTab.click({ timeout: UI_TIMEOUT_MS }); + await page.waitForFunction( + () => { + const active = document.querySelector('.mnote-main-tab.is-active[data-mnote-tab-kind="image"] .mnote-main-tab-title'); + return active && (active.textContent || "").includes("photo-failed.png"); + }, + null, + { timeout: UI_TIMEOUT_MS }, + ); + await page.locator('[data-mnote-resource-tab-panel]:not([hidden]) [data-testid="mnote-local-ocr-toolbar"]').first().waitFor({ + state: "detached", + timeout: UI_TIMEOUT_MS, + }); + await page.getByTestId("mnote-local-ocr-task-toggle").click({ timeout: UI_TIMEOUT_MS }); + await page.waitForFunction( + (sourcePath) => { + const row = document.querySelector(`[data-mnote-local-ocr-task-row="${CSS.escape(sourcePath)}"]`); + return row + && row.getAttribute("data-mnote-local-ocr-task-status") === "failed" + && (row.textContent || "").includes("local_ocr_job_failed_401"); + }, + failedSourceRootRelativePath, + { timeout: UI_TIMEOUT_MS }, + ); + screenshots.ocrFailedTask = path.join(OUTPUT_DIR, "04-ocr-failed-task.png"); + await page.screenshot({ path: screenshots.ocrFailedTask, fullPage: true }); + await page.unroute("**/api/local-folder/ocr/jobs", failedOcrRoute); const openResourceResult = await page.evaluate(async ({ rootUri, documentId, ocrRootRelativePath, workspaceId }) => { const runtime = window.__mnoteDocumentPaneRuntime; @@ -273,7 +283,7 @@ async function main() { }, { rootUri: fileUrl(root), documentId, - ocrRootRelativePath: result.ocrRootRelativePath, + ocrRootRelativePath: uiOcrPath, workspaceId, }); assert.equal(openResourceResult, true, "OCR sidecar resource tab should open"); @@ -281,15 +291,90 @@ async function main() { state: "visible", timeout: UI_TIMEOUT_MS, }); + await page.waitForFunction( + () => { + const active = document.querySelector('[data-mnote-resource-tab-panel][data-resource-kind="markdown"]:not([hidden]) .ProseMirror'); + return active && (active.textContent || "").includes("OCR UI smoke text"); + }, + null, + { timeout: UI_TIMEOUT_MS }, + ).catch(async (error) => { + const debug = await page.evaluate(() => ({ + activeTab: document.querySelector('.mnote-main-tab.is-active')?.outerHTML || '', + activePanels: Array.from(document.querySelectorAll('[data-mnote-resource-tab-panel]:not([hidden])')).map((panel) => ({ + kind: panel.getAttribute('data-resource-kind') || '', + objectIdentity: panel.getAttribute('data-mnote-object-identity') || '', + text: (panel.textContent || '').slice(0, 200), + html: panel.innerHTML.slice(0, 500), + })), + marker: document.documentElement.getAttribute('data-mnote-local-ocr-filetree-open') || '', + })); + throw new Error(`${error.message}; debug=${JSON.stringify(debug)}`); + }); + const fileTreeOpenResult = await page.evaluate((ocrRootRelativePath) => { + const parentPath = ocrRootRelativePath.split("/").slice(0, -1).join("/"); + const parentRow = document.querySelector(`#sidebar-file-tree-root .tree-row[data-shell-mode="filetree"][data-local-relative-path="${CSS.escape(parentPath)}"]`); + if (!(parentRow instanceof HTMLElement)) return { ok: false, reason: "ocr_parent_row_missing", parentPath }; + if (parentRow.getAttribute("aria-expanded") !== "true") { + const toggle = parentRow.querySelector('[data-rust-action="toggle"]'); + if (toggle instanceof HTMLElement) toggle.click(); + } + return { ok: true, parentPath }; + }, uiOcrPath); + assert.equal(fileTreeOpenResult.ok, true, `OCR sidecar parent should exist in filetree: ${JSON.stringify(fileTreeOpenResult)}`); + const ocrFileTreeRow = page.locator(`#sidebar-file-tree-root .tree-row[data-shell-mode="filetree"][data-local-relative-path="${uiOcrPath.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"]`).first(); + await ocrFileTreeRow.waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + await ocrFileTreeRow.click({ timeout: UI_TIMEOUT_MS }); + await page.waitForFunction( + () => document.documentElement.getAttribute("data-mnote-local-ocr-filetree-open") === "resource-tab", + null, + { timeout: UI_TIMEOUT_MS }, + ); + await page.waitForFunction( + () => { + const active = document.querySelector('[data-mnote-resource-tab-panel][data-resource-kind="markdown"]:not([hidden]) .ProseMirror'); + return active && (active.textContent || "").includes("OCR UI smoke text"); + }, + null, + { timeout: UI_TIMEOUT_MS }, + ).catch(async (error) => { + const debug = await page.evaluate(() => ({ + activeTab: document.querySelector('.mnote-main-tab.is-active')?.outerHTML || '', + activePanels: Array.from(document.querySelectorAll('[data-mnote-resource-tab-panel]:not([hidden])')).map((panel) => ({ + kind: panel.getAttribute('data-resource-kind') || '', + objectIdentity: panel.getAttribute('data-mnote-object-identity') || '', + text: (panel.textContent || '').slice(0, 200), + html: panel.innerHTML.slice(0, 500), + })), + marker: document.documentElement.getAttribute('data-mnote-local-ocr-filetree-open') || '', + })); + throw new Error(`${error.message}; debug=${JSON.stringify(debug)}`); + }); screenshots.ocrResource = path.join(OUTPUT_DIR, "03-ocr-resource-tab.png"); await page.screenshot({ path: screenshots.ocrResource, fullPage: true }); + await page.evaluate((sourcePath) => { + const row = document.querySelector(`[data-mnote-local-ocr-task-row="${CSS.escape(sourcePath)}"]`); + const clear = row && row.querySelector("[data-mnote-local-ocr-task-clear]"); + if (!(clear instanceof HTMLButtonElement)) throw new Error("missing OCR clear button"); + clear.click(); + }, failedSourceRootRelativePath); + await page.waitForFunction( + (sourcePath) => !document.querySelector(`[data-mnote-local-ocr-task-row="${CSS.escape(sourcePath)}"]`), + failedSourceRootRelativePath, + { timeout: UI_TIMEOUT_MS }, + ); + const deleteButtonVisible = await page.evaluate((sourcePath) => { + const row = document.querySelector(`[data-mnote-local-ocr-task-row="${CSS.escape(sourcePath)}"]`); + return Boolean(row && row.querySelector("[data-mnote-local-ocr-task-delete]")); + }, sourceRootRelativePath); + assert.equal(deleteButtonVisible, true, "已完成 OCR 任务应展示删除 OCR 按钮"); await writeResult({ ok: true, task: TASK, root, documentId, - ocrRootRelativePath: result.ocrRootRelativePath, + ocrRootRelativePath: uiOcrPath, screenshots, }); console.log(JSON.stringify({ @@ -297,7 +382,7 @@ async function main() { task: TASK, root, documentId, - ocrRootRelativePath: result.ocrRootRelativePath, + ocrRootRelativePath: uiOcrPath, screenshots, }, null, 2)); } catch (error) { diff --git a/scripts/task527-chatonly-api-provider-smoke.js b/scripts/task527-chatonly-api-provider-smoke.js new file mode 100644 index 00000000..4ab7e5ed --- /dev/null +++ b/scripts/task527-chatonly-api-provider-smoke.js @@ -0,0 +1,366 @@ +#!/usr/bin/env node +"use strict"; + +const assert = require("node:assert"); +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); +const { execFileSync } = require("node:child_process"); +const { chromium } = require("playwright"); +const { + BASE_URL, + UI_TIMEOUT_MS, + ensureAuthenticated, +} = require("./tree-shell-smoke-helpers"); + +const CONTROL_PLANE_DB = "/mnt/Data1T/Mnote_data/control-plane/control-plane.db"; +const CHROMIUM_EXECUTABLE_PATH = + process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH || + [ + "/usr/bin/google-chrome-stable", + "/usr/bin/google-chrome", + "/snap/bin/chromium", + "/usr/bin/chromium", + ].find((candidate) => fs.existsSync(candidate)); + +const PROVIDERS = [ + { + key: "gpt", + profileId: "shared_api_gpt_chat", + chipText: "ChatOnly / GPT", + markerPrefix: "MNOTE_API_CHAT_GPT", + expectedProfile: "api-gpt-chat", + expectedModel: "aisz-chat/gpt-5.5-extra-high-fast", + }, + { + key: "deepseek-flash", + profileId: "shared_api_deepseek_flash_chat", + chipText: "ChatOnly / DeepSeek Flash", + markerPrefix: "MNOTE_API_CHAT_DEEPSEEK_FLASH", + expectedProfile: "api-deepseek-flash-chat", + expectedModel: "deepseek-v4-flash", + }, +]; + +const OUTPUT_DIR = path.join(process.cwd(), "tmp", "task527-chatonly-api-provider-smoke"); +const RESULT_PATH = path.join(OUTPUT_DIR, "result.json"); + +function fileUrl(localPath) { + return `file://${localPath}`; +} + +function localMdDocumentId(relativePath) { + return `local-md:${relativePath.replaceAll("/", "~2F")}`; +} + +function documentUrl(root, relativePath) { + const url = new URL(`${BASE_URL}/documents/${encodeURIComponent(localMdDocumentId(relativePath))}`); + url.searchParams.set("sourceKind", "local_folder"); + url.searchParams.set("rootUri", fileUrl(root)); + url.searchParams.set("treeView", "filetree"); + return url.toString(); +} + +function writeWorkspaceManifest(root, ownerId, workspaceId) { + fs.mkdirSync(path.join(root, ".mnote"), { recursive: true }); + fs.writeFileSync( + path.join(root, ".mnote", "workspace.json"), + `${JSON.stringify( + { + workspaceId, + ownerId, + createdAt: new Date().toISOString(), + capabilities: ["local_files", "ai_sessions", "markdown_edit"], + }, + null, + 2, + )}\n`, + "utf8", + ); +} + +function sqlQuote(value) { + return `'${String(value).replaceAll("'", "''")}'`; +} + +function sqliteExec(sql) { + execFileSync("sqlite3", [CONTROL_PLANE_DB, sql], { + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }); +} + +function sqliteJson(sql, fallback = null) { + try { + const raw = execFileSync("sqlite3", ["-json", CONTROL_PLANE_DB, sql], { + encoding: "utf8", + stdio: ["ignore", "pipe", "pipe"], + }).trim(); + return raw ? JSON.parse(raw) : fallback; + } catch { + return fallback; + } +} + +function grantWorkspaceAccess({ actorId, workspaceId, root, rootUri, grantId }) { + const now = new Date().toISOString(); + sqliteExec(` + INSERT OR IGNORE INTO users (id, email, username, display_name, role, status, created_at, updated_at, revision) + VALUES (${sqlQuote(actorId)}, ${sqlQuote(`${actorId}@example.com`)}, ${sqlQuote(actorId)}, ${sqlQuote(actorId)}, 'user', 'active', ${sqlQuote(now)}, ${sqlQuote(now)}, 1); + INSERT OR REPLACE INTO workspaces (id, owner_user_id, name, kind, root_uri, root_path, source_kind, status, created_at, updated_at, revision) + VALUES (${sqlQuote(workspaceId)}, ${sqlQuote(actorId)}, ${sqlQuote(actorId)}, 'personal', ${sqlQuote(rootUri)}, ${sqlQuote(root)}, 'local_folder', 'active', ${sqlQuote(now)}, ${sqlQuote(now)}, 1); + INSERT OR REPLACE INTO directory_grants (id, user_id, workspace_id, root_uri, root_path, permission, recursive, capabilities_json, source, status, created_by, created_at, updated_at, revision) + VALUES (${sqlQuote(grantId)}, ${sqlQuote(actorId)}, ${sqlQuote(workspaceId)}, ${sqlQuote(rootUri)}, ${sqlQuote(root)}, 'write', 1, '["ai","markdown_edit"]', 'smoke', 'active', ${sqlQuote(actorId)}, ${sqlQuote(now)}, ${sqlQuote(now)}, 1); + `); +} + +async function saveScreenshot(page, name) { + const target = path.join(OUTPUT_DIR, `${name}.png`); + await page.screenshot({ path: target, fullPage: false }); + return target; +} + +async function waitForAssistantMarker(page, marker) { + await page.waitForFunction( + (expectedMarker) => { + const assistantText = Array.from(document.querySelectorAll(".wolai-page-ai-message--assistant")) + .map((node) => node.textContent || "") + .join("\n"); + return assistantText.includes(expectedMarker); + }, + marker, + { timeout: 180_000 }, + ); +} + +async function ensurePageAiDrawerOpen(page) { + const drawer = page.locator('[data-testid="wolai-page-ai-drawer"]'); + if (await drawer.isVisible().catch(() => false)) return; + await page.getByTestId("wolai-floating-ai").click({ timeout: UI_TIMEOUT_MS }); + await drawer.waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); +} + +async function selectProvider(page, provider) { + await page.locator("[data-page-ai-agent-button]").click({ timeout: UI_TIMEOUT_MS }); + await page.locator(`[data-page-ai-agent-id="chat_only"][data-page-ai-profile-id="${provider.profileId}"]`).click({ + timeout: UI_TIMEOUT_MS, + }); + await page.waitForFunction( + (expected) => document.querySelector("[data-page-ai-agent-chip]")?.textContent?.includes(expected), + provider.chipText, + { timeout: UI_TIMEOUT_MS }, + ); +} + +async function runProviderSmoke(page, provider, suffix) { + const marker = `${provider.markerPrefix}_${suffix}`; + const runRequests = []; + const runResponses = []; + const deleteResponses = []; + + const runRoute = async (route) => { + runRequests.push(JSON.parse(route.request().postData() || "{}")); + await route.continue(); + }; + await page.route("**/api/hermes/client/runs", runRoute); + const responseListener = async (response) => { + const url = response.url(); + const request = response.request(); + if (request.method() === "POST" && url.includes("/api/hermes/client/runs")) { + runResponses.push({ + url, + status: response.status(), + body: await response.text().catch(() => ""), + }); + } + if (request.method() === "DELETE" && url.includes("/api/hermes/client/sessions/")) { + deleteResponses.push({ + url, + status: response.status(), + body: await response.text().catch(() => ""), + }); + } + }; + page.on("response", responseListener); + + try { + await page.locator('.wolai-page-ai-header-actions [data-page-ai-action="new-session"]').click({ + timeout: UI_TIMEOUT_MS, + }); + await selectProvider(page, provider); + + await page.locator("[data-page-ai-input]").fill(`请只回复:${marker}`, { timeout: UI_TIMEOUT_MS }); + await page.locator('[data-page-ai-action="send"]').click({ timeout: UI_TIMEOUT_MS }); + await waitForAssistantMarker(page, marker); + await page.waitForFunction( + () => (document.querySelector("[data-page-ai-run-status]")?.textContent || "").includes("完成"), + null, + { timeout: 180_000 }, + ).catch(() => {}); + const afterMessageScreenshot = await saveScreenshot(page, `${provider.key}-after-message`); + + assert.strictEqual(runRequests.length, 1, `${provider.key} 本轮应只创建一个 run`); + const run = runRequests[0]; + assert.strictEqual(run.agentId, "chat_only", `${provider.key} 应使用 ChatOnly agent`); + assert.strictEqual(run.profileId, provider.profileId, `${provider.key} 应使用 API ChatOnly profile`); + assert(run.sessionId, `${provider.key} run payload 应包含 MNote sessionId`); + + assert.strictEqual(runResponses.length, 1, `${provider.key} 应返回一个 run response`); + assert.strictEqual(runResponses[0].status, 200, `${provider.key} run response 应成功`); + const runResponse = JSON.parse(runResponses[0].body || "{}"); + assert.strictEqual(runResponse.providerKind, "api-chat", `${provider.key} 后端应分流到 api-chat`); + assert.strictEqual(runResponse.runtime?.transport, "api-chat", `${provider.key} 不应启动 ACP/OpenClaw runtime`); + assert.strictEqual(runResponse.runtime?.model, provider.expectedModel, `${provider.key} model 应匹配 registry`); + assert.strictEqual(runResponse.profile, provider.expectedProfile, `${provider.key} 应使用 isolated API profile`); + + const assistantTexts = await page.locator(".wolai-page-ai-message--assistant").allTextContents(); + const markerAssistantCount = assistantTexts.filter((text) => text.includes(marker)).length; + assert.strictEqual(markerAssistantCount, 1, `${provider.key} 可见 API 回复应只有一条`); + + const sessionId = String(run.sessionId); + const bindingRows = sqliteJson( + `SELECT mnote_session_id, provider, status FROM ai_external_conversation_bindings WHERE user_id=${sqlQuote("mnote-e2e")} AND mnote_session_id=${sqlQuote(sessionId)} ORDER BY updated_at DESC LIMIT 5;`, + [], + ); + assert.strictEqual(bindingRows.length, 0, `${provider.key} API ChatOnly 不应写网页 provider conversation binding`); + + await page.reload({ waitUntil: "domcontentloaded", timeout: UI_TIMEOUT_MS }); + await ensurePageAiDrawerOpen(page); + await waitForAssistantMarker(page, marker); + const afterReloadScreenshot = await saveScreenshot(page, `${provider.key}-after-reload`); + const reloadedAssistantTexts = await page.locator(".wolai-page-ai-message--assistant").allTextContents(); + assert.strictEqual( + reloadedAssistantTexts.filter((text) => text.includes(marker)).length, + 1, + `${provider.key} 刷新恢复后仍应只有一条助手回复`, + ); + + await page.locator('.wolai-page-ai-header-actions [data-page-ai-action="history"]').click({ timeout: UI_TIMEOUT_MS }); + await page.locator('[data-page-ai-panel="history"]').waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + await page.locator(`[data-page-ai-session-row="${sessionId}"]`).waitFor({ state: "visible", timeout: UI_TIMEOUT_MS }); + page.once("dialog", async (dialog) => { + await dialog.accept(); + }); + await page.locator(`[data-page-ai-session-delete="${sessionId}"]`).click({ timeout: UI_TIMEOUT_MS }); + await page.waitForFunction( + (id) => !document.querySelector(`[data-page-ai-session-row="${CSS.escape(id)}"]`), + sessionId, + { timeout: UI_TIMEOUT_MS }, + ); + const afterDeleteScreenshot = await saveScreenshot(page, `${provider.key}-after-delete`); + + assert(deleteResponses.length >= 1, `${provider.key} 应发出本地 session DELETE 请求`); + assert.strictEqual(deleteResponses.at(-1).status, 200, `${provider.key} DELETE 应成功`); + const deleteBody = JSON.parse(deleteResponses.at(-1).body || "{}"); + assert.strictEqual(deleteBody?.result?.remoteDelete?.attempted, false, `${provider.key} 不应调用网页远端删除`); + assert.strictEqual( + deleteBody?.result?.remoteDelete?.reason, + "api_chat_has_no_remote_conversation", + `${provider.key} remoteDelete reason 应说明 API Chat 无远端会话`, + ); + + const remainingRows = sqliteJson( + `SELECT session_id, status FROM ai_runtime_runs WHERE user_id=${sqlQuote("mnote-e2e")} AND session_id=${sqlQuote(sessionId)} AND deleted_at IS NULL LIMIT 5;`, + [], + ); + assert.strictEqual(remainingRows.length, 0, `${provider.key} 删除后 SQLite active run 不应残留`); + + return { + provider: provider.key, + profileId: provider.profileId, + model: provider.expectedModel, + sessionId, + runId: runResponse.runId, + screenshots: { + afterMessage: afterMessageScreenshot, + afterReload: afterReloadScreenshot, + afterDelete: afterDeleteScreenshot, + }, + }; + } finally { + await page.unroute("**/api/hermes/client/runs", runRoute).catch(() => {}); + page.off("response", responseListener); + } +} + +async function main() { + fs.mkdirSync(OUTPUT_DIR, { recursive: true }); + const suffix = Date.now().toString(36); + const actorId = "mnote-e2e"; + const root = fs.mkdtempSync(path.join(os.tmpdir(), "mnote-task527-api-chat-")); + const rootUri = fileUrl(root); + const workspaceId = `local-ws:${actorId}:task527-api-chat-${suffix}`; + const relativePath = "ApiChatOnly.md"; + + writeWorkspaceManifest(root, actorId, workspaceId); + fs.writeFileSync(path.join(root, relativePath), ["# API ChatOnly", "", `MNOTE_API_CHAT_WORKSPACE_${suffix}`, ""].join("\n"), "utf8"); + grantWorkspaceAccess({ + actorId, + workspaceId, + root, + rootUri, + grantId: `grant_task527_api_chat_${suffix}`, + }); + + const browser = await chromium.launch({ + headless: true, + ...(CHROMIUM_EXECUTABLE_PATH ? { executablePath: CHROMIUM_EXECUTABLE_PATH } : {}), + }); + const context = await browser.newContext({ + viewport: { width: 1440, height: 960 }, + locale: "zh-CN", + extraHTTPHeaders: { + "x-mnote-actor-id": actorId, + "x-mnote-actor-type": "user", + }, + }); + const page = await context.newPage(); + let caughtError = null; + const results = []; + + try { + await ensureAuthenticated(page, context.request); + const response = await page.goto(documentUrl(root, relativePath), { + waitUntil: "domcontentloaded", + timeout: UI_TIMEOUT_MS, + }); + assert(response && response.status() === 200, `文档页状态码异常: ${response && response.status()}`); + await page.locator('[data-testid="mnote-leptos-tiptap-island-editor-root"] .ProseMirror').first().waitFor({ + state: "visible", + timeout: UI_TIMEOUT_MS, + }); + await ensurePageAiDrawerOpen(page); + await saveScreenshot(page, "initial-drawer"); + + for (const provider of PROVIDERS) { + results.push(await runProviderSmoke(page, provider, suffix)); + } + } catch (error) { + caughtError = error; + await saveScreenshot(page, "failure").catch(() => undefined); + } finally { + await browser.close().catch(() => {}); + } + + const resultPayload = { + ok: !caughtError, + error: caughtError ? String(caughtError && caughtError.stack || caughtError) : "", + root, + workspaceId, + relativePath, + providers: results, + outputDir: OUTPUT_DIR, + resultPath: RESULT_PATH, + }; + fs.writeFileSync(RESULT_PATH, `${JSON.stringify(resultPayload, null, 2)}\n`, "utf8"); + if (caughtError) { + console.error(JSON.stringify(resultPayload, null, 2)); + process.exit(1); + } + console.log(JSON.stringify(resultPayload, null, 2)); +} + +main().catch((error) => { + console.error(error); + process.exit(1); +});